IRC logs for #baserock for Monday, 2016-02-15

*** edcragg_ has quit IRC00:40
*** richard_maw has quit IRC04:18
*** richard_maw has joined #baserock04:19
*** ChanServ sets mode: +v richard_maw04:19
*** pedroalvarez has quit IRC05:02
*** dabukalam has quit IRC05:02
*** pedroalvarez has joined #baserock05:03
*** ChanServ sets mode: +v pedroalvarez05:03
*** gtristan has quit IRC05:25
*** gtristan has joined #baserock05:48
*** brlogger has joined #baserock07:05
*** CTtpollard has joined #baserock08:15
*** paulw has joined #baserock08:32
*** paulw has quit IRC08:34
*** bruce_ has joined #baserock08:57
*** bashrc has joined #baserock09:07
*** toscalix has joined #baserock09:15
*** ssam2 has joined #baserock09:19
*** ChanServ sets mode: +v ssam209:19
*** jonathanmaw has joined #baserock09:33
paulsherwooddo we have a shortage of reviewers, presently?09:37
ssam2I would say so, yes09:41
paulsherwoodok, i'll see what i can do :)09:42
jjardonMaybe change the current policy of 2 +1 to a single one would be a good idea09:46
paulsherwoodi was thinking of encouraging some more folks to contribute reviews09:47
ssam2jjardon: i tend to +1 anything that has a +1, so i don't think it would make much difference :-)09:55
paulsherwoodssam2: is the gap mainly on tools, definitions, trove, infrastructure, or spread across all of these09:56
ssam2well, infra is special because the main work is maintenance, not patch review09:59
ssam2other than that, the gap is wherever we have outstanding patches, really10:00
ssam2simple patches to anything get through fine, it's the big changes which get stuck10:00
paulsherwoodssam2: while you're on, i (only last night) noticed that you opted for separate schemas for each 'kind' in definitions - do you remember why you did that?10:02
paulsherwood(i'm wondering how to re-incorporate schema validations in the definitions loading step)10:02
ssam2paulsherwood: there was definitely a reason10:06
ssam2I think it was so they are tighter, in fact10:07
ssam2there's no way in json-schema to say "this element is allowed only if the kind field was set to 'chunk'"10:07
richard_mawssam2: enum10:08
ssam2ah ,ok10:08
ssam2then maybe it's just because using an enum was messy10:08
jjardonMmm, not sure i agree; trivial patches get stuck as well; the vala separation took more than 3 months, for example10:08
richard_mawno other way AIUI10:08
paulsherwoodssam2: my original schema was deliberately lax, because actual historical definitions have quite a lot of small breakages10:09
ssam2i documented my rationale in schemas/README :-)10:10
ssam2"It is possible to merge all these10:10
ssam2into one file, and use the 'oneOf' field to say that any .morph file should10:10
ssam2match exactly one of the layouts. The only issue with this approach is that10:10
ssam2the Python 'jsonschema' model will give you totally useless errors if anything10:10
ssam2is invalid (along the lines of "<dump of entire file> is not valid under any of10:10
ssam2the given schemas"). So for now they are separate."10:10
*** edcragg has joined #baserock10:11
paulsherwoodok. did you actually try validating actual definitions against the schema?10:11
paulsherwoodcool - is there code for that anywhere?10:11
* paulsherwood would like to crib, if possible :)10:12
ssam2in definitions10:12
ssam2that does a single file10:12
paulsherwoodof any 'kind' ?10:12
ssam2or does it?10:13
* tiagogomes_ points to, which produces, IMO, nice error messages10:13
ssam2no, it is not at all smart10:13
ssam2(my tool)10:13
paulsherwoodtiagogomes_: thanks for that!10:13
ssam2the best that tool can do is `schemas/tools/jsonschema-validate schemas/stratum.json-schema strata/*.morph`10:14
ssam2I did fix definitions to all be valid against the schema, but seems some have regressed (which is not surprising)10:14
*** franred_ has quit IRC10:16
*** franred has joined #baserock10:29
*** bruce_ has quit IRC11:03
*** bruce_ has joined #baserock11:18
*** gtristan has quit IRC11:30
*** gtristan has joined #baserock11:42
pedroalvarezjust in case anybody is wondering about the gcc lorry not showing up:
pedroalvarezwat indeed12:11
rjekIs the upstream repo somehow corrupt?12:12
ssam2good catch!12:13
ssam2does look like something might be wrong with their git server..12:13
ssam2searching shows that it could also be a https issue, or a proxy issue12:15
jmacsI'm getting errors from my VM too.12:21
pedroalvarezjmacs: is that somethiing that you weren't having before? or did just tested the commands that are failing for us to reproduce the error?12:24
jmacsAll I'm doing is "git clone"12:24
jmacsWhich I have done before (although I can't remember how recently)12:25
pedroalvarezI see12:25
pedroalvarez--depth 3 works12:25
pedroalvarezI was trying to check if the commits on github mirror have the same sha12:27
*** locallycompact has joined #baserock12:28
pedroalvarezand they do12:30
jmacs"git clone git://" has no such problems12:30
pedroalvarezthen we have a winner12:31
pedroalvarezjmacs: thanks!12:31
paulsherwoodhmmm... could someone pls check what happened to lc's initial attempt to clone gcc?12:32
pedroalvarezwe have been doing that since 12:07 :)12:33
paulsherwoodah, ok12:33
pedroalvarezand here the patch:
paulsherwoodi've approved :)12:38
* paulsherwood crosses fingers12:38
paulsherwoodtiagogomes_: i think will get proactive review this week12:43
tiagogomes_great :)12:45
paulsherwoodssam2: i'm hopeful we could make both ybd and morph verify schema before doing anything else. that should nip regressions in the bud? :)13:18
ssam2yes, that would be good. I think that's what Tiago's patches do in fact13:31
ssam2should probably warn to begin with rather than failing though13:31
paulsherwoodconf flag :)13:53
tiagogomes_I'd fail. if you warn instead of failing you would need to add code to handle the case when the definitions are not valid13:58
paulsherwoodactually, +113:59
tiagogomes_And then you'd lose some advantages of using the schema for validation13:59
paulsherwood(except that for ybd i still aim to support retrospective definitions)13:59
tiagogomes_They key is, ensuring that the error messages are helpful to find out the root cause of the validation failure14:00
paulsherwood+11 :)14:00
paulsherwoodWWWWWWW0000000000000000000000000000000000000000000T :)14:02
rjekCalm down, dear.14:03
paulsherwoodnow, for the avoidance of unnecessary pain, am i correct that downstream troves don't have to mirror everything by default? :)14:03
* rjek tries cloning it :D14:06
rjekIt's... taking a while14:08
jmacsgcc is a big repository.14:09
rjekIt's just saying "Cloning into git..." atm; not got to a stage where it provides progress feedback14:09
rjekDone.  the .git is 1.6GB!14:15
paulsherwoodthat's pretty fast :)14:16
paulsherwoodis trove doing the tarball trick for it?14:16
jmacsI was trying to do a binary chop between gcc 4.7 and 4.8 a while back; it was 12000 revisions14:17
rjekWell, that's interesting.  git gc makes .git grow from 1.6GB to 3.2GB14:21
paulsherwoodsounds like a bug?14:21
rjekPerhaps "gc" stands for "garbage creation" ?14:22
* rjek assumes some magic has happened here that perhaps does not interact well with a garbage collection.14:23
*** edcragg_ has joined #baserock14:23
*** edcragg has quit IRC14:24
pedroalvarezpaulsherwood: yes, trove is doing th tarball trick for it, but this tarbal is 9.7G big14:32
pedroalvarezthe gcc-tarball one is 154M14:33
pedroalvareztiny <314:33
*** tiagogomes_ has quit IRC14:34
pedroalvarezso, if Morph fetches the whole repo this is going to be really slow to build14:35
pedroalvarezactually, if Morph/ybd try to fetch the tarball before trying git clone.. this is going to be really slow14:36
richard_mawrjek: it's probably repacking with a smaller window than the repo it was fetched from14:36
richard_mawpaulsherwood: downstream troves can be configured to not fetch gcc, but the default config is to get evertying in delta/14:36
paulsherwoodrichard_maw: ack, thanks14:38
paulsherwoodis the tarball trick optional?14:39
paulsherwoodseems we'd be better turning it off for this14:39
pedroalvarezI wonder... if this was a shallow lorry...14:40
richard_mawpedroalvarez: if it was shallow then we'd have different problems14:41
ssam2tiagogomes_, paulsherwood: so you'd be ok if a 'git pull' of YBD caused some definitions to suddenly fail to build?14:42
ssam2i would find that pretty annoying14:42
paulsherwoodssam2: with a conf flag, off by default, i'd be fine14:42
ssam2yes, that would be perfect14:42
ssam2maybe we should stick with gcc-tarball.git in the reference system definitions, and leave gcc.git as an 'opt-in' for folk who want latest GCC ?14:44
paulsherwoodfine by me14:48
paulsherwoodbut still, i wonder why the tarball is so much bigger than the repo14:48
pedroalvareztarball is actually smaller, the repo is 12G ish14:48
*** tiagogomes_ has joined #baserock14:48
pedroalvarezand a checkout is from 1.6GB to 3.2GB14:49
paulsherwoodah, ok14:49
*** rdale has joined #baserock15:10
*** gtristan has quit IRC15:25
*** gtristan has joined #baserock15:34
jjardonpaulsherwood: any idea why this is failing to build, same definition works in morph?:15:57
jjardonthis is the chunk definition:15:58
*** gtristan has quit IRC16:08
paulsherwoodjjardon: at risk of sounding like a stuck record, please could you try master ybd?16:11
jjardonpaulsherwood: same problem16:13
paulsherwoodwhat kind of ref is that?16:13
jjardonpaulsherwood: seems like a normal commit:
paulsherwoodjjardon: fatal: bad object a182447a13103c5b78eefc8f742cb6b779f120eb16:14
paulsherwoodfrom a fresh git clone16:15
jjardonmmm, how on earth is working with morph then?16:16
*** gtristan has joined #baserock16:16
richard_mawalready had good objects in the morph git cache?16:16
paulsherwoodmorph doesn't check for trees in the same way?16:16
richard_mawlooks like upstream gstreamer changed a tag16:18
richard_mawand morph had fetched it before they changed it16:18
richard_mawand now that object is not referenced by any ref, hence their git server isn't sending it16:18
richard_mawand that link only works because that commit hasn't been gc'd by their git server yet16:18
jjardonthats probably, Ive complained the other they about the tag not being pushed correctly in the repo16:19
paulsherwoodsounds plausible16:19
richard_mawthis is one reason I recommended making anchor refs16:19
jjardonbut anyway, Im using sha, not tags here16:20
*** CTtpollard has quit IRC16:20
richard_mawjjardon: yeah, but the sha you used isn't referenced by anything you can clone from git://
richard_mawnot any more anyway16:21
richard_mawso we've reliably locked down what contributed to the build, but because there wasn't an anchor ref, we can no longer build it, because that commit is no longer available, except in jjardon's morph git cache16:22
*** CTtpollard has joined #baserock16:23
richard_mawso… if jjardon can dig into his morph git cache and push that to a branch on, then things will be better16:23
jjardonah, ok; I will change the patch then (still a bit surprised you can access the commit through the web interface tough). Thanks for the help!16:23
paulsherwoodrichard_maw: i assume that jjardon will want to provide a better sha than this16:23
jjardonrichard_maw: the patch is still not merged, so dont think that would be needed16:24
richard_mawok then, just change the sha1 to that of the new version of the tag, and assume they moved the tag for a good reason16:25
richard_maw(or use your local cache to see if they've made any significant change between the versions)16:25
*** jonathanmaw_ has joined #baserock16:25
*** jonathanmaw_ has quit IRC16:27
*** jonathanmaw_ has joined #baserock16:27
*** jonathanmaw has quit IRC16:29
*** gtristan has quit IRC17:00
*** gtristan has joined #baserock17:01
*** bashrc has quit IRC17:05
*** bashrc has joined #baserock17:12
*** gtristan has quit IRC17:20
*** bruce_ has quit IRC17:22
paulsherwoodlocallycompact: looking at your RFC, this will require a fix for each component with submodules?17:37
paulsherwoodso ansible is just an example you've come across?17:37
paulsherwoodi wonder how hard it would be to track down the rest?17:37
locallycompactIt would require that17:38
tiagogomes_I'd prefer the genericity of supporting multiple sources for a component17:39
locallycompacttiagogomes_, That requires updating every sha for every submodule everytime the parent repo changes, no?17:39
* paulsherwood makes a whooshing sound, as this goes over his head17:40
franredlocallycompact, you still need to modify the parent repository to point to the trove submodules. Or you lost track/reproducibility of the submodules17:41
locallycompactNo you don't, the reproducibility is tracked by definitions17:41
*** jonathanmaw has joined #baserock17:42
richard_mawif you encode submodule url rewrite rules in the cache key, you can safely have the build tool perform translation to build from a trove17:42
richard_maws/build from a trove/fetch submodule commits from repositories lorried to the trove/17:42
tiagogomes_locallycompact, that should be the default, but you should be able to fix a sha for the submodule if needed17:42
tiagogomes_locallycompact, related
locallycompactI'm confused, all I want to do is consume upstream tags for repos that have submodules17:43
locallycompactThe shas are already tracked17:43
*** jonathanmaw_ has quit IRC17:44
locallycompactSome repositories like for example concourse, are repositories that consist of more or less only submodule increments for a hundred submodules, we would be constantly curating it.17:47
*** jonathanmaw has quit IRC17:56
*** franred has quit IRC17:57
*** ssam2 has quit IRC18:04
paulsherwoodlocallycompact: would you mind creating a patch for all of the definitions, rather than just ansible?18:06
paulsherwood(i'm assuming it would be possible to use ybd or morph to iterate through the chunks (eg for ci.morph) and identify them)18:07
paulsherwoodybd no-build: True might help18:08
paulsherwood(or not, can't remember if it does all the git operations)18:08
*** bashrc has quit IRC18:09
paulsherwoodjjardon: have you tested ?18:12
jjardonpaulsherwood: Ive built a gnome system, yes; the upgrade is needed because 1.6.3 is the minimum requirement for gstreamer-vaapi18:14
jjardonpaulsherwood: thanks! mind take a look to as well?18:17
*** toscalix has quit IRC18:24
*** locallycompact has quit IRC18:33
*** edcragg_ has quit IRC18:35
*** rdale has quit IRC18:36
*** gtristan has joined #baserock19:34
*** locallycompact has joined #baserock20:14
*** gtristan has quit IRC20:18
*** gtristan has joined #baserock20:32
*** gtristan has quit IRC20:46
*** gtristan has joined #baserock20:53
*** gtristan has quit IRC21:01
*** edcragg has joined #baserock21:02
*** gtristan has joined #baserock21:25
*** gtristan has quit IRC21:35
*** gtristan has joined #baserock21:45
*** gtristan has quit IRC22:05
*** zoli_ has quit IRC22:25
*** gtristan has joined #baserock22:45
*** gtristan has quit IRC23:39

Generated by 2.15.3 by Marius Gedminas - find it at!