IRC logs for #baserock for Thursday, 2015-06-04

*** zoli__ has joined #baserock05:45
*** zoli__ has joined #baserock05:46
*** zoli__ has quit IRC06:40
*** zoli__ has joined #baserock07:05
paulsher1oodmason is 404 ???07:16
*** a1exhughe5 has joined #baserock07:16
*** bashrc_ has joined #baserock08:00
pedroalvarezpaulsher1ood: looks like that. I'll check08:04
pedroalvarezdisk full, btrfs, etc08:25
Kinnisonpoor mason :(08:25
pedroalvarezI guess next time we deploy mason, we should attach a volume08:25
* Kinnison would send it love and snuggles, but its disc is full08:26
pedroalvarezI could remove things but its disk is full :)08:26
KinnisonTo the redeploymobile!08:26
pedroalvareztananananananana ná08:27
rjekto the ext4mobile!08:28
*** gary_perkins has joined #baserock08:28
pedroalvarezthe red mason is alive again08:40
KinnisonTo the greening-mobile08:40
Kinnisontananananananana ná08:40
*** jonathanmaw has joined #baserock08:40
pedroalvarezI wonder if jjardon finished his build test of a weston system08:42
radiofreewayland 1.8!08:46
*** ssam2 has joined #baserock08:54
*** ChanServ sets mode: +v ssam208:54
*** bashrc_ has quit IRC09:06
*** bashrc_ has joined #baserock09:06
paulsher1oodlol09:06
paulsher1oodto the cloudfoundry mobile :)09:06
paulsher1oodhttps://ci.concourse.ci09:07
* paulsher1ood is hoping to hook up a ybd instance to concourse... 09:07
paulsher1oodmason is lovely, and all, but we don't have time to write all the things :-)09:08
*** Krin has joined #baserock09:09
pedroalvarezmason is not lovely09:10
*** CTtpollard has quit IRC09:10
ssam2I still think reusing Zuul is promising09:10
* SotK does too09:10
ssam2but it doesn't have a UI we can reuse09:10
KinnisonZuul's UI is gerrit is it not?09:10
ssam2beyond its "post stuff to Gerrit" hooks09:10
ssam2exactly09:11
paulsher1oodZuul's ui is gerrit, and gerrit's ui is, erm, how do i say it.... ?09:12
richard_mawpaulsher1ood: you don't need to09:12
richard_mawpaulsher1ood: we are all aware09:12
ssam2choosing Gerrit as the UI means we need to separately handle serving log files, showing what the status of the workers are and how many jobs are queued, whether 'master' builds successfully, etc09:12
paulsher1oodbut anyway, i agree zuul has some great things09:12
ssam2there is http://status.openstack.org/zuul/ but it doesn't really make sense to me09:12
*** CTtpollard has joined #baserock09:13
paulsher1oodconcourse has a really fabulous ui, i'm still trying to work out how solid their infrastructure assumptions are... they're working on caching, so once that's done i'll be giving it a proper spin09:13
*** mariaderidder has joined #baserock09:15
ssam2i can't make head or tail of the UI, but I guess this is because it's full of terms I don't understand09:15
ssam2oh I see, it's aeroplane metaphors09:16
ssam2fly, atc, baggageclaim, blackbox, testflight...09:16
paulsher1oodyup09:16
*** edcragg has joined #baserock09:17
paulsher1oodimagine how folks feel when they turn up here and we hit them with morphologies and strata :)09:17
ssam2I try to use '.morph file', instead, which I think is quite clear09:18
ssam2strata maybe clearer as layers, indeed09:18
paulsher1oodsomeone said to me the other day that they'd had feedback from newbies excited at the baserock concept, turning up at wiki.baserock.org and concluding 'there's no there, there'09:18
paulsher1oodwhich shows just how wrong folks can be, but also just how broken our messaging seems09:18
* SotK tries to parse "there's no there, there"09:19
pedroalvarezI failed :/09:19
paulsher1oodit's a well established meme, i believe09:19
DavePagepaulsher1ood: It's sensible to assume that anybody who wants to learn more about baserock is an idiot.09:19
DavePage(this also applies to non-baserock of course)09:19
ssam2DavePage: ouch09:19
* SotK googles it09:19
DavePageThe smoother the learning curve about a new project, the better (provided there's a way for more experienced people to jump ahead).09:20
paulsher1oodgertrude stein, SotK09:20
paulsher1oodi think09:20
SotKindeed09:20
*** lachlanmackenzie has joined #baserock09:31
rdale_there are some comments on baserock-dev about using spdx for license tracking in baserock - has anyone done anything on that yet?09:38
KinnisonGiven the lack of continued discussion, I doubt it has gotten any further yet09:38
rdale_ok09:38
paulsher1oodi think that's right09:51
SotKpaulsher1ood: I keep seeing builds fail with ybd when trying to mount --bind stuff in /root/.cache/ybd/ccache09:56
paulsher1oodinteresting10:01
wdutchI'm looking at gerrit.baserock.org/#/c/430 because gerrit says 'The change cannot be merged due to a path conflict', when I checkout this topic I have no trouble rebasing it to master and it won't let me run 'git review' claiming that nothing has changed, I don't understand what the problem is for gerrit10:04
pedroalvarezwdutch: well, it depends on things that don't exist10:10
franred...in master firehose10:11
pedroalvarezin gerrit10:11
franredwdutch, it requires to merge the rest of the patches before that patch can be merged10:12
wdutchoh so it will say merge conflict on all except the bottom one until that one has gone through?10:13
franredwdutch, the one which does not have the merge conflict is the first patch in the series, AFAICT10:14
richard_mawwdutch: you have to merge them in order. You can see the order by using the old gerrit UI, or set up gertty10:16
richard_mawI don't trust the new UI's related changes list10:16
straycat*generally*, the related changes section lists the changes in order, but not always10:17
wdutchIt seems odd to me that gerrit doesn't have a nicer way to communicate that a change cannot be merged because it depends on another pending change10:18
straycatthis is because gerrit is not built with series in mind10:18
wdutchso then could somebody with +2 powers review this please https://gerrit.baserock.org/#/c/15910:20
richard_mawwdutch: merged10:21
wdutchthanks10:21
richard_mawwdutch: dependent patches may need to be rebased, depending on if there were any other changes involved, but since this patch didn't need to be rebased, probably not10:21
paulsher1oodSotK: what's the fail message?10:26
paulsher1ood(can you paste the log, please?)10:27
SotKthe log as in the ybd output, or the build log it gives a path to?10:28
* SotK goes for both10:28
paulsher1oodthe pathed build-log10:29
paulsher1oodssam2: am i right you may have something for doing the sandboxing in a different way? maybe that would be of interest to SotK10:30
SotKpaulsher1ood: http://sprunge.us/SEDN10:31
paulsher1oodSotK: i think that's broken definitions. try master?10:32
richard_mawpaulsher1ood: the bind is failing before it has any input from definitions10:33
SotKpaulsher1ood: that is master10:33
paulsher1oodhmm.10:33
SotK(or at least, master when I cloned it at some point yesterday)10:33
paulsher1oodwhich target?10:34
SotKclusters/release.morph x86_6410:34
paulsher1oodi did see this a few days ago, but i've been juggling lots of things, will need to re-engage brain10:34
* paulsher1ood kicks ybd into action10:36
ssam2that error is from linux-user-chroot, either one of the source or target directories passed to --mount-readonly or --mount-bind doesn't exist10:37
paulsher1oodhmmm. so i'm already past stage2-fhs-dirs.10:37
* paulsher1ood clears cache10:38
paulsher1oodthe one thing i wonder is whether there could be some difference between stuff under /root/.ybd/ and stuff under /src ?10:39
paulsher1oodSotK: are you running on a baserock system, or something else?10:39
SotKon baserock10:39
paulsher1oodwith no /src director?10:40
SotKpaulsher1ood: correct10:41
paulsher1oodwould you mind trying again with a /src ?10:41
SotKsure10:42
paulsher1oodtvm10:42
paulsher1oodin the meantime i'm rebuilding build-essential10:42
SotKhm, so am I apparently10:44
*** pacon has joined #baserock10:44
SotKdoes YBD magically use a different cachedir if /src is present?10:44
ssam2yes10:45
ssam2check in app.py10:45
KinnisonMOAR MAGICK10:45
jjardonHi, if you have some time, I made a couple of patches to avoid unnecessary rebuilds; https://gerrit.baserock.org/#/q/topic:foundation_core10:46
Kinnisonjjardon: Have you actually tested these changes properly and thoroughly (unlike the last set of things I reviewed for you?)10:46
jjardonKinnison: the testing I did is in the comments10:47
Kinnisonjjardon: So not sufficient then10:49
KinnisonYou changed tools10:49
Kinnisonthat implies you need to revalidate the *BEHAVIOUR* of every system which includes tools, or includes a stratum which build-depends on tools10:50
KinnisonNot just build one system10:50
jjardonKinnison: put your concerns in gerrit. But is unlikely that I (or anyone else) will have time to build and test the behaviour of all the systems that include the tools stratum (I count around ~40 systems in definitions)10:54
paulsher1oodno, it's not what you think, Kinnison :)10:55
Kinnisonjjardon: Until and unless we have automated validation, we *MUST* put this effort in for this kind of change10:55
Kinnisonjjardon: Otherwise we get the kinds of hiccoughs we have when we merged your expat code10:56
Kinnisons/code/change/10:56
KinnisonI've raised my concerns here.  Interacting with gerrit on those changes will mean gerrit will mail me more crap I don't care about10:57
Kinnisonpaulsher1ood: it's always as bad as I think10:57
Kinnisonpaulsher1ood: sometimes it's worse10:57
Kinnison:)10:57
bashrc_from the talk of upstreams yesterday, if gerrit isn't very good (imho it isn't) then we should maybe put development effort into improving it10:59
KinnisonYou're leaking a bit -- but yes we have considered what it'd take to improve Gerrit10:59
bashrc_or write a better tool10:59
KinnisonIndeed10:59
KinnisonThis is, however, a very subjective arena11:00
jmacsI'm not convinced gerrit is an improvement on the previous mailing list approach.11:00
Kinnisonjmacs: For some people it very much is.  For others, not so much :-/11:00
Kinnisonjmacs: Fortunately we do still accept patches via the ML :)11:00
jjardonKinnison: I guess It's better to make changes and sometimes break things than not make changes at all11:00
Kinnisonjjardon: That's a matter of opinion and a question for policy for Baserock as a project11:01
Kinnison"Should master always work?"11:01
jmacsI agree with that - I get the impression that it's more efficient for full-time users11:01
jmacsFor infrequent contributors like me it's a pain11:01
rjek+111:01
KinnisonOh Gods, at least one person is subscribed to baserock-dev via Digest mode11:01
rjekWhy do we even let them?11:02
Kinnisonjmacs: amusingly, there are others who say that Gerrit is way better for infrequent contributors11:02
Kinnison:)11:02
KinnisonIt's a very very personal thing IMO11:02
jmacsKinnison: That is amusing.11:02
jjardonbut i'm curious if what you propose to fix this: eventually, we want to have a machine test the changes before merge them. Will this machine build and test/run ~40 systems for every change we sent to gerrit?11:02
jjardonKinnison: ^11:02
rjekjjardon: Hopefully.11:02
Kinnisonjjardon: Hopefully yes11:03
SotKthat will take a long time11:03
Kinnisonjjardon: Well, hopefully it'll build/test all the *affected* systems11:03
rjekdistbuild solves all problems11:03
Kinnisonjjardon: and we have initiatives already underway which will help to constrain that set11:03
KinnisonUntil that day, the only way to constrain the set is for humans to do *analysis*11:03
SotKrjek: not the problem of testing that `morph build` works in the resulting build and devel systems11:03
Kinnisonand my rudimentary analysis says "You edited 'tools' -- you need to validate a build system can bootstrap a new build without a cache server, and you need to verify that troves built using the new tools stratum still work"11:04
Kinnisonthat would, IMO, be the bare-minimum of testing for that change11:04
ssam2I think the Baserock project is at a point where we can't require people to manually validate everything that might be affected by a change for every change11:05
ssam2certainly I wouldn't contribute anything to definitions if I had to do that11:05
Kinnisonssam2: If you make small changes, they're more easily verified by analysis rather than by active build/run testing11:06
Kinnisonssam2: But for large changes like jjardon just proposed, I fear analysis may be insufficient11:06
Kinnisonesp. if done in a slapdash fashion11:06
ssam2we have post-merge CI, so merging is a reasonable way to test11:06
ssam2I would prefer that we had a set cadence of stabilisation11:07
jjardonKinnison: that's more reasonable, and a bit different from build and test ~40 systems. comment in gerrit and I will try to test that when I have time11:07
franredssam2, we only have post-merge builder, not tester, if we don't test some of the system we may end up thinking we are supporting some systems which are not really tested after a while - and the effort in fixing them would be huge11:08
ssam2yeah, true11:08
* pedroalvarez finally redeploys mason https://mason-x86-64.baserock.org/11:17
pedroalvarezI've put a note there so no-one asks why it's red/blank11:17
jjardonpedroalvarez: nice11:18
SotKpedroalvarez: maybe we could modify the mason script to do something similar when it starts a build? :)11:19
pedroalvarezdon't expect that nice message again :)11:19
pedroalvarezSotK: it will be really easy I thiink11:19
SotKI think so too11:19
* SotK hasn't tried though11:19
SotKpaulsher1ood: seems to be working fine with /src12:34
richard_mawperhaps it would be more appropriate for it to be configurable, than have ybd attempt to magically guess12:55
*** pacon has quit IRC13:27
*** pacon has joined #baserock13:28
*** zoli__ has quit IRC13:30
*** pacon has quit IRC13:30
pedroalvarezwow, it wasn't difficult at all to rebase my branch on top of master, even with patches for files that have been moved to subfolders13:51
* pedroalvarez hugs git13:51
Kinnisonhehe13:51
Kinnisongit's merge/rebase code is pretty clever13:51
richard_mawvery much so, given it doesn't track renames13:52
robtaylorits clever enough to figure that out ;)13:55
richard_mawdoes anyone have review bandwidth for https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/morph+branch:master+topic:strip-commands ?14:14
Kinnisonrichard_maw: is it also on a git server somewhere easy to deal with?14:15
richard_mawI don't expect any quick reviews for the sourceresolver rewrite, but there's a lot of changes before it that should be mostly independent14:15
richard_mawKinnison: `git pull https://gerrit.baserock.org/baserock/baserock/morph refs/changes/86/786/1`. Gerrit gets awfully confused if you push commits with change-ids to personal branches14:16
* Kinnison tries14:17
ssam2it does. it may be that we can alter the git.baserock.org -> gerrit.baserock.org mirroring to ignore personal branches14:17
ssam2it's just hard to know which branches are "personal"14:17
* Kinnison runs git format-patch and then reads it14:18
richard_mawhm, because gerrit doesn't have access to the gitano user list14:18
KinnisonIn theory we could write a gitano plugin to push user information across14:21
paulsher1oodSotK: so this does make me wonder if there is something special about /home/root ?14:49
paulsher1oodrichard_maw: yup. better configurable, in a default yaml file. there's lots of stuff in app.setup that should go there14:49
richard_mawthat'd make it bloody hard to have a test suite for14:50
paulsher1ood:)14:50
* pedroalvarez hits the sourceresolve rewrite patch14:51
richard_mawI don't believe a smiley face to be the appropriate response.14:51
paulsher1oodrichard_maw: you may be right, i need to think about that.14:51
paulsher1oodrichard_maw: would you prefer i get grumpy?14:51
pedroalvarezI don't want grupmy discussions in #baserock please14:52
paulsher1oodpedroalvarez: i agree :-)14:52
richard_mawit just rubbed me up the wrong way, as it implied you were gleeful about not wanting a test suite, and I'm of the opinion that without a test suite you can't prove it works14:53
paulsher1oodah, no i wasn't being gleeful, just expressing that i'm not worried by that so far, because i'm very confident in my existing test regime14:53
paulsher1oodnot that ybd master always works... just that i expect to spot breakages very quickly14:54
paulsher1oodbut i acknowledge that my current approach 'doesn't scale'14:54
richard_mawI was just about to say that it's fine for a 1 man project, but causes friction when there's other simultaneous developers.14:55
paulsher1oodagreed14:55
paulsher1oodi don't expect ybd will ever have a lot of folks working on it :)14:56
paulsher1oodi mean, the few folks who have looekd at the code so far seem to hate it :-)14:56
paulsher1oodrichard_maw: any idea why stuff under /home/root might trigger the bind error, but under /src does not?15:02
richard_mawpaulsher1ood: nope, haven't looked at that code15:02
paulsher1oodrichard_maw: i mean, is there anything special about /home/root ?15:03
richard_mawjust that usually root's home is /root15:03
paulsher1oodno strange permissions or filesystem trickery?15:03
richard_mawonly if you've got encrypted home directories separately from encrypted /home volume15:04
* SotK never mentioned /home/root? :)15:04
SotKjust /root15:04
paulsher1oodah, sorry.15:06
paulsher1oodrichard_maw: is there anything special about /root ?15:06
pedroalvarezit shouldn't be anything special about it15:07
pedroalvarezis just another folder15:07
pedroalvarezit might be smaller than /src if you have attached a disk in /src15:07
pedroalvarezs/attached/mounted/15:07
* SotK didn't attach a disk, I just created a /src directory15:08
paulsher1oodthere is no code difference for ybd between these two cases afaict - it's all just based on app.settings['base'] https://github.com/devcurmudgeon/ybd/blob/master/app.py#L9315:09
paulsher1oodso i'm intrigued. i can fix the problem, i just wonder what's going on15:10
KinnisonBase permissions on /root perhaps being less permissive15:10
Kinnisontypically it's 750 not 75515:11
Kinnisonfr.ex.15:11
Kinnison?15:11
Kinnisonrichard_maw: I have plowed my way through that epic topic15:11
Kinnisonrichard_maw:  you have my +1 across it, but with some commentary15:11
richard_mawKinnison: cool, thanks.15:11
KinnisonMy main issue surrounds the claim that we now actively support 6 variants of definitions version15:12
Kinnisonand my worry that it may not be true15:12
paulsher1oodwhich topic is this?15:12
KinnisonSecondarily my worry is that 6 versions is a very large support-surface15:12
richard_mawpaulsher1ood: https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/morph+branch:master+topic:strip-commands15:12
paulsher1oodah, ok. i'll read that if folks want input?15:13
SotKI think that most of the versions are fairly small changes15:13
richard_mawpossibly feature flags would be better than so many numbered versions there15:13
Kinnisonrichard_maw: I believe you and I discussed that over lunch recently :)15:14
paulsher1oodshall we think more deeply about this whole version thing before it really does become a nightmare?15:14
perryli'm looking at deterministic builds wrt artifacts and considering that when the tarball is generated that it may not be done deterministically, i.e. individual file permissions/file ordering may vary from build to build. where would i check this, and if necessary, patch functionality?15:16
richard_mawperryl: morphlib/bins.py I think15:17
perrylrichard_maw: thanks!15:17
paulsher1oodperryl: are you looking at this for ybd, or morph?15:18
perrylboth, if possible15:19
paulsher1ood:)15:19
paulsher1oodperryl: https://github.com/devcurmudgeon/ybd/blob/master/cache.py#L7515:20
radiofreemeta file for build ybd artefacts could contain a bit more information15:21
paulsher1oodradiofree: agreed. any ideas on what would be most useful there?15:21
radiofreerepo and sha1 at least15:21
perrylhah, was about to aay it may be easier to grep for in ybd :)15:21
paulsher1oodradiofree: ok. anything else off the top of your head?15:22
radiofreethat's usually all the information i ever need from a baserock meta file15:22
radiofree+ files of course15:22
radiofree(which are already there)15:22
paulsher1oodradiofree: thanks. https://github.com/devcurmudgeon/ybd/issues/4115:23
paulsher1oodi'll get to it before monday15:23
radiofreeheh, upstream is fast!15:23
paulsher1oodi do my best15:23
*** zoli__ has joined #baserock15:30
*** rdale_ has quit IRC15:37
*** rdale has joined #baserock15:38
*** a1exhughe5 has quit IRC15:42
paulsher1oodhttps://github.com/devcurmudgeon/ybd/commit/8e06030d41f28a15fae13bb8758cb25855089b4915:44
* paulsher1ood wonders why flush is required?, anyway... radiofree - master has a fix now :)15:45
Kinnisonbecause otherwise the .write may not be committed before the subprocess runs15:47
KinnisonBy default, python's file objects are buffered15:47
*** lachlanmackenzie has quit IRC15:48
paulsher1oodah, ok.15:51
paulsher1oodthanks, Kinnison15:51
paulsher1oodssam2: your patch looks great! a chance to get rid of loads of code from ybd! :)15:53
paulsher1oodwhat's the least work to ensure no user is harmed if i merge it?15:53
paulsher1oodwould this work (as root) on jrandom linux with pip-installed sandboxlib?15:54
* paulsher1ood is a bit concerned about pip-installing on his baserock vm15:56
KinnisonCorrect15:56
* Kinnison is concerned about pip on any platform15:56
* Kinnison thinks of pip as standing for "Provenance is passé"15:57
paulsher1ood:)15:57
ssam2paulsher1ood: I've done that in Baserock to test it15:58
ssam2i.e. I've done 'pip install sandboxlib' inside Baserock15:58
ssam2I'll also send a patch to add sandboxlib to the Baserock reference systems15:58
paulsher1oodssam2: ok, great. is there a non pip way that i could safely add this to my br vm for testing, then remove it if i don't want it?15:59
SotKgit clone and set PYTHONPATH?16:00
ssam2you can clone https://github.com/CodethinkLabs/sandboxlib/ somewhere, then do PYTHONPATH=/path/to/sandboxlib python ybd ...16:00
paulsher1oodah, ok. thanks!16:00
paulsher1oodmaybe the readme needs to be updated with this, at least temporarily?16:01
paulsher1oodi'll merge, anyway16:01
paulsher1ood2 files changed, 100 insertions(+), 238 deletions(-16:02
ssam2I can add a 'dependencies' section to the YBD readme.md file16:03
paulsher1oodssam2: i know others here will be happy, you're heloping to destroy ybd :)16:03
paulsher1oodssam2: yes please16:03
richard_mawpaulsher1ood: ok, that joke made me grin16:03
paulsher1oodlol16:04
paulsher1oodrichard_maw: i'll be delighted to see as little code in ybd as possible16:04
*** zoli___ has joined #baserock16:04
ssam2contributions to https://github.com/CodethinkLabs/sandboxlib are welcome by the way, there are rough edges16:05
* paulsher1ood will take a look16:05
richard_mawpaulsher1ood: help us rework morphlib into being an actually useful library then, and you can just import morphlib into ybd16:05
ssam2we can add collaborators from outside Codethink to the project as far as I know, don't let the 'codethinklabs' organisation put you off16:05
paulsher1oodssam2: anything you'd want me to know about my coding unstyle before i submitted patches?16:05
ssam2run 'pep8' or 'pylint' first to look for obvious style errors16:06
ssam2there's a HACKING.rst file which hopefully has all the info you need16:06
paulsher1oodssam2: i do pep8 actually, when i get round to it. pep8 does not seem to be on baserock systems by default, though16:06
rdalehow do you set up a project in CodethinkLabs?16:07
KinnisonI think you have to be a Codethink employee16:07
paulsher1oodrdale: ask nicely, or be a member of the organisation. i think you already are?16:07
paulsher1oods/organisation/github organisation/16:07
* paulsher1ood checks16:07
rdalei see - it's ages since i create a github project16:07
ssam2paulsher1ood: you can install it with 'pip' though16:08
*** zoli__ has quit IRC16:08
paulsher1oodrdale: there are too many github proejcts already, think carefully before creating yet another one :)16:08
ssam2pep8 is not a component where provenance really matters I think16:08
paulsher1oodssam2: possibly, except on the day that the pip infrastructure gets hacked, and i install perp8 and its malware friend16:09
rdalewhat does 'too many' mean? is there a numerical limit to codethink's allowance? or is it just untidy?16:10
richard_mawpaulsher1ood: you may be interested in knowing that the current version of my patches to add strip-commands support to ybd only adds 2 lines in total, as I was able to remove 18 from your build system handling code16:10
ssam2paulsher1ood: true16:10
paulsher1oodrichard_maw: you're a star!16:10
rjekI stopped mirroring LuaRocks (the Lua equiv. of pip, rubygems, cpan, etc) when they changed the policy to "allow anybody on the internet to upload a tantalisingly-named or replacement for popular module which contains a rootkit."16:10
paulsher1oodrichard_maw: i assume you're doing it in morph, too? what are the equivalent numbers?16:11
richard_mawpaulsher1ood: this is all predicated on the assumption that the reason I can't find any code for handling definitions that aren't in definitions.git, is that there isn't any code to support it, rather than you defaulting to an empty dict for nascent definitions16:11
paulsher1oodrichard_maw: correct. if it's not somwhere under os.getcwd()/*/ it's not a definition as far as ybd is concerned16:12
paulsher1ood(that assumption may not hold in future of course)16:13
SotKpaulsher1ood: how does that work for chunks which don't have a morphology?16:13
*** lachlanmackenzie has joined #baserock16:13
richard_mawpaulsher1ood: I gave those numbers yesterday. For morph: ~700 lines changed, 11 added in total excluding the ~300 line test-suite removal as I want to keep that really. It would be >1000 lines to add the strip commands to every definition in definitions.git.16:13
paulsher1oodSotK: it 'just works, tm' i think?16:13
* SotK tries to figure it out16:13
richard_mawIt's the same thing AIUI16:14
paulsher1oodi'm not being flippant, every time i need to think about definitoins it takes me 20 mins to inbound the logic to my brain16:14
richard_mawpaulsher1ood: understood16:14
paulsher1oodi have some simplifying ideas in mind, if we could work out how to make schema/structure changes trivial and bulletproof16:15
* paulsher1ood dreams on16:15
SotKI'm guessing from a quick look that for chunks without morph files, the chunk spec from the stratum is added to Definitions.__definitions16:17
richard_mawhmm, that sounds like it would make my proposed reduction incorrect then16:19
paulsher1oodit parses each definition file...16:19
SotKpaulsher1ood: some chunks don't have definition files16:20
richard_mawyes, but chunk definitions' content is spread across both the stratum they are introduced in, and the definition file that may be referred to16:20
richard_mawthe latter is optional in morph, and results in the buils system being guessed16:20
paulsher1oodfor each component (which may be stratum, chunk) it either adds what it finds here to what it already knows, or creates a new definition if this is the first info16:20
SotKso chunks without definitions will be added as just the entry in the stratum then16:22
paulsher1oodthe code is nightmarish enough that Kinnison called it difficult to understand, but i blame that more on our definitions syntax than my programming16:22
* Kinnison blames it entirely on your programming :)16:22
paulsher1oodSotK: yup. and then ybd will guess, same as moprh16:22
* Kinnison can quite confidently grok our definitions syntax16:22
paulsher1oodKinnison: patches welcome16:22
Kinnisonpaulsher1ood: I can't bring myself to believe I know how it's *meant* to function, since its external surface lacks any tests16:22
KinnisonAlso I have way more projects to contribute to than time and energy already :)16:23
richard_mawpaulsher1ood: your guessing code is based on not having "build-system" in the definition. This is a divergence in behaviour, since in morph that means use the manual build system instead.16:23
richard_mawpaulsher1ood: normally that wouldn't bite you, since build-system: manually has previously been a no-op16:23
paulsher1oodKinnison: if you can supply < 150 lines of python that do the job and meet your criteria, i'll buy you dinner16:23
richard_mawpaulsher1ood: but I want to hang the strip commands off it16:23
Kinnisonpaulsher1ood: given my criteria involve explicitly making it longer and thus more grokable, I'll buy myself an infinite improbability drive if I manage that16:24
Kinnisonsince I'll have defeated physics16:24
Kinnisonand information theory16:24
Kinnisonall in python16:24
* paulsher1ood needs to think through richard_maw's comment16:24
paulsher1oodKinnison: how much longer? i'm happy to compromise :)16:25
KinnisonI dunno16:25
KinnisonI want classes with explicit attributes and testable surfaces16:25
paulsher1oodi think i want that too16:25
paulsher1oodi just hate having more code than necessary16:26
* Kinnison feels that in your abject fear of lines-of-code you have erred into the simplistic rather than the simplified.16:27
paulsher1oodi note your feelings, Kinnison. it's not fear, on my side though, it's loathing :)16:27
* Kinnison sends paulsher1ood to Las Vegas16:27
Kinnisonthey like that kind of thing there16:27
paulsher1oodthe complexity really is due to the syntax, honest. i had to spend three whole weekends on it16:28
KinnisonI think the complexity in your code is mostly due to your desire to not transform the physical data structure into a logical one16:29
KinnisonBut I can't be sure16:29
paulsher1ood__definitions is my attempt at the logical structure, i think16:32
paulsher1oodbut this might be a similar blindspot to the 'no graph' discussion16:33
KinnisonI think it is16:34
KinnisonHowever, I have to bimble off now and deal with the fact that people have started posting about some code I wrote ages ago on things like identi.ca16:34
* SotK suddenly realises his "Fix deployment" patch may have broken running ybd with anything but a cluster and tests to find out if he's right16:34
paulsher1oodSotK: did i merge it?16:35
SotKpaulsher1ood: yes16:35
paulsher1oodwell i think it's working, so relax :)16:35
SotKhave you run it with anything other than a cluster, to completion?16:36
paulsher1oodi believe so. i'll check again16:36
paulsher1oodoh, no :)16:36
paulsher1oodinteresting... what would you expect to break?16:37
paulsher1oodSotK: ^^16:37
SotKI'm imagining it will attempt to deploy a system, stratum or chunk definition16:38
SotKit might work without breaking, there is too much recursion for me to understand at this time of day :)16:39
paulsher1oodassembly.deploy always runs, if that's what you mean16:40
paulsher1oodit did before your patch, anyway16:40
SotKI know it does, but I removed the if statement that makes it return if the definitions doesn't have a 'systems' key16:40
paulsher1oodSotK: but whhhhhhhhhyyyyyyyyy ? :)16:41
SotKI actually think everything has sensible enough defaults it won't break16:41
SotKpaulsher1ood: because it was making nothing get deployed16:42
paulsher1oodi think it'll be ok. i'm more worried about sandboxlib vs stage2-fhs-dirs16:42
paulsher1oodSotK: one thing... are you saying that ybd now deploys system files, strata files, chunk files? if so, to what?16:43
SotKpaulsher1ood: no, I think it will just do nothing16:43
richard_mawpaulsher1ood: Currently at 4 lines of code removed to make YBD handle an absent build-system meaning build-system: manual, then 20 added to make it support strip commands, most of that is the shell command formatted in a non eye-bleeding way16:44
paulsher1oodk. i don't know what a deployment would be if there's no defined location/target fro it16:44
paulsher1oodrichard_maw: cool!16:44
SotKpaulsher1ood: it wouldn't work :)16:44
richard_mawpaulsher1ood: well, there's no test suite to prove I haven't broken it in some way16:45
richard_mawso I'm going to have to wait for gcc to build16:45
paulsher1oodrichard_maw: how long does that take on your machine? 8 mins on mine16:46
paulsher1oodrichard_maw: noted about the test suite16:46
SotKpaulsher1ood: the reason that return statement was stopping deployment from happening was that only clusters have a "systems" key, so deploy(system) returned without doing anything16:47
richard_mawpaulsher1ood: show-off. I can let you know when it's built gcc, though the number is likely to be poor, as I'm suffering from poor IO currently.16:47
paulsher1oodSotK: that's correct behaviour, iiuc16:47
SotKpaulsher1ood: nope, because that means nothing is ever deployed16:48
SotKclusters don't have a 'deploy' key, so no extensions are run and nothing is extracted to be deployed16:49
paulsher1oodSotK: ok, you may be right. i had it working, briefly, until i noticed that extensions were in so many places and gave up16:49
SotKand the system specs in the clusters which do have a 'deploy' key don't have a 'system' key, meaning that the function attempting to deploy them returned before anything was extracted or any extensions were run16:50
paulsher1oodweird. that's another aspect of the definitoins syntax i still don't understand properly, then16:51
rdalethe syntax is yaml, it sounds like the semantics that are the problem16:52
paulsher1oodsorry, you're right. i get confused. thanks rdale16:53
paulsher1oodperryl: thanks for the patch!16:54
rdalethere seem to be a lot of arbitrary things which look like environment variables in the cluster definitions which i don't understand16:54
paulsher1oodrdale: could you mail the list highlighing them?16:54
richard_mawthey are passed as environment variables16:54
perrylpaulsher1ood: thanks for the speedy merge! :)16:55
paulsher1ood:-)16:55
paulsher1oodperryl: what was the elapsed time between submission and notification of merge, from your perspective?16:56
ssam2rdale: deployment is extensible, so the configuration for deployment is basically anarchy16:56
rdaleso are there a list of cluster attributes which aren't environment variables, and anything which isn't one of them is considered to be an environment variable?16:56
ssam2rdale: http://wiki.baserock.org/definitions/current/#index4h3 might help16:57
ssam2only 'type' and 'location' are standard16:57
richard_mawrdale: [type, location, upgrade-type, upgrade-location]16:57
ssam2oh, yeah16:57
richard_mawI think ssam2 was responsible for the latter 216:57
perrylpaulsher1ood: <5 minutes i think, either that or my perception of time is fudged today!16:57
*** Krin has quit IRC16:58
paulsher1oodperryl: cool. the reason i ask is that i was once looking at flynn.io, very interesting project....16:58
paulsher1oodand noticed a problem in the documentation. fixed it... and was merged within 18 minutes, start to finish16:59
perrylimpressive! :)16:59
paulsher1oodand that really amazed me... i thought, wow, that's really cool16:59
paulsher1oodand it wasn't a bot, or anthging - the upstream guy commented on someting i needed to fix, i did that, all within 18 mins16:59
* richard_maw had the same turn-around for a documentation patch to systemd17:00
paulsher1oodi can't see gerrit will ever match github for awesomness17:00
richard_mawpaulsher1ood: github vs gerrit is irrelevant in this comparison17:00
paulsher1oodis it?17:01
ssam2i've merged patches in minutes on our Gerrit17:01
richard_mawI've merged patches in minutes via the mailing list17:01
*** CTtpollard has quit IRC17:01
richard_mawit's that there's upstream developers currently reviewing patches that does it17:01
paulsher1oodi must still be missing some fundamental tricks in gerrit, then.17:01
*** bashrc_ has quit IRC17:02
paulsher1oodi'll think on this properly, i'm not fully understnding what my perception of the bottlenecks is.17:02
ssam2several people seem to think 'gertty' is the fundamental trick17:03
paulsher1oodreminds me of the blue light story somehow, sorry for the noise... http://theoryofconstraints.blogspot.co.uk/2007/06/toc-stories-2-blue-light-creating.html17:03
pedroalvarezbottleneck for reviewing? testers17:03
*** jonathanmaw has quit IRC17:04
richard_mawtesting documentation is reading it17:04
paulsher1oodpedroalvarez: depends on the nature of the patch, and the nature of the test/review/deploy/deliver pipeline17:04
paulsher1oodrichard_maw: +117:04
richard_mawhence documentation patches have fast turn around17:04
paulsher1oodso gertty can't help vs one part of what i meant, which is that github is easy to navigate for non-cli users17:05
paulsher1oodi made the patch in github, never even pulled the repo locally17:05
ssam2fair enough. Can't do that in Gerrit for sure17:06
paulsher1oodsimilarly for some ybd patches, i've merge at github, without pulling (for various reasons)17:06
pedroalvarezwow, I can deploy a Kilo openstack (one node) in 4 minutes17:06
paulsher1oodwtf!!!! w00t-and a half!17:06
pedroalvarezmaybe a couple more17:07
paulsher1oodw00ts, or minutes?17:07
pedroalvarezbut it's still great17:07
paulsher1ood:)17:07
pedroalvarezis not ready to be merged on master, but i though I had to mention it17:07
pedroalvarezI can screw it up, and redeploy one in a few minutes, and that makes me  happy17:08
franreddeploy the actual Openstack in Baserock for Juno should take the same amount of time ;-)17:08
pedroalvarezbut juno is old17:09
franredxD - franred doesn't enter in this kind of holy wars17:09
ssam2anyone seen Intel "Clear Linux" before? seems to have a familiar sort of feature set... https://clearlinux.org/features17:09
ssam2(other than the 'fast VMs as containers' thing which seems unique)17:10
paulsher1oodsimilar to what?17:10
paulsher1oodit seems to be only Intel, which is nice of them :)17:11
ssam2similar to Baserock, Ubuntu Core, Fedora Atomic, ...17:15
paulsher1oodah, ok17:15
ssam2well, similar to what Ubuntu Core pretends to be :)17:15
paulsher1oodcareful, they might hear you :-)17:16
paulsher1oodiiuc  Ubuntu Core is different thing, now?17:17
paulsher1oodssam2: btw, sandboxlib seems to be working for me17:17
ssam2cool17:17
paulsher1ood:-)17:17
paulsher1oodssam2: as you know, software normally breaks for me, so this is lovely :-)17:18
*** ssam2 has quit IRC17:20
*** mariaderidder has quit IRC17:24
tlsahow can I make ybd rebuild busybox?  I tried deleting its artifact and git repo from /src17:28
paulsher1oodchange someging in its definition file?17:30
paulsher1oodshouldn't need to delete its repo17:30
paulsher1oodthat would be weird :)17:30
tlsaI don't want to change its definition; I'm testing that subsequent builds of the same thing give bit-for-bit the same result17:31
*** edcragg has quit IRC17:32
paulsher1oodah, ok17:34
paulsher1oodtlsa: are you running ybd.py to build busybox, or some other target17:35
tlsaI was makeing it build build-essential, which contains busybox.  Doing busybox directly seems to have worked17:36
paulsher1oodtlsa: yup. it didn't rebuld busybox, because it already had build-essential17:36
tlsaright, makes sense17:37
paulsher1ood:-)17:37
* paulsher1ood goes swimming... after weeks of sitting in aeroplanes... pain is expected17:37
tlsa:)17:38
* jmacs wonders why he's suddenly getting "ERROR: Could not find extension cloud-init.configure". I'm not explicitly using that extension.17:55
*** gary_perkins has quit IRC18:04
*** CTtpollard has joined #baserock18:12
*** CTtpollard has quit IRC18:14
*** lachlanmackenzie has quit IRC18:38
*** edcragg has joined #baserock18:39
*** edcragg has quit IRC20:06
paulsher1oodjmacs: with morph, or ybd?20:13
paulsher1oodactually, can't be ybd20:15
paulsher1oodjmacs: definitions got updated recently to move all extensions into a tidy place. seems this is a case that didn't get spotted. are you using latest morph, recent definitions?20:15
*** zoli___ has quit IRC20:18
paulsher1oodrdale: are you around?20:26
* paulsher1ood is hoping to ask dumb questions about 'migrations'20:26
* paulsher1ood hopes rdale is enjoying sangria or such, rather than online20:27
*** pacon has joined #baserock21:05
*** pacon has quit IRC21:47
paulsher1oodhmmm. how to install linux-user-chroot on jrandom linux?22:04
*** SotK has quit IRC22:34
*** SotK has joined #baserock22:34
*** zoli__ has joined #baserock22:38
*** Stanto has quit IRC22:38
*** Stanto has joined #baserock22:41
*** zoli__ has quit IRC23:00

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!