| *** zoli__ has joined #baserock | 05:45 | |
| *** zoli__ has joined #baserock | 05:46 | |
| *** zoli__ has quit IRC | 06:40 | |
| *** zoli__ has joined #baserock | 07:05 | |
| paulsher1ood | mason is 404 ??? | 07:16 |
|---|---|---|
| *** a1exhughe5 has joined #baserock | 07:16 | |
| *** bashrc_ has joined #baserock | 08:00 | |
| pedroalvarez | paulsher1ood: looks like that. I'll check | 08:04 |
| pedroalvarez | disk full, btrfs, etc | 08:25 |
| Kinnison | poor mason :( | 08:25 |
| pedroalvarez | I guess next time we deploy mason, we should attach a volume | 08:25 |
| * Kinnison would send it love and snuggles, but its disc is full | 08:26 | |
| pedroalvarez | I could remove things but its disk is full :) | 08:26 |
| Kinnison | To the redeploymobile! | 08:26 |
| pedroalvarez | tananananananana ná | 08:27 |
| rjek | to the ext4mobile! | 08:28 |
| *** gary_perkins has joined #baserock | 08:28 | |
| pedroalvarez | the red mason is alive again | 08:40 |
| Kinnison | To the greening-mobile | 08:40 |
| Kinnison | tananananananana ná | 08:40 |
| *** jonathanmaw has joined #baserock | 08:40 | |
| pedroalvarez | I wonder if jjardon finished his build test of a weston system | 08:42 |
| radiofree | wayland 1.8! | 08:46 |
| *** ssam2 has joined #baserock | 08:54 | |
| *** ChanServ sets mode: +v ssam2 | 08:54 | |
| *** bashrc_ has quit IRC | 09:06 | |
| *** bashrc_ has joined #baserock | 09:06 | |
| paulsher1ood | lol | 09:06 |
| paulsher1ood | to the cloudfoundry mobile :) | 09:06 |
| paulsher1ood | https://ci.concourse.ci | 09:07 |
| * paulsher1ood is hoping to hook up a ybd instance to concourse... | 09:07 | |
| paulsher1ood | mason is lovely, and all, but we don't have time to write all the things :-) | 09:08 |
| *** Krin has joined #baserock | 09:09 | |
| pedroalvarez | mason is not lovely | 09:10 |
| *** CTtpollard has quit IRC | 09:10 | |
| ssam2 | I still think reusing Zuul is promising | 09:10 |
| * SotK does too | 09:10 | |
| ssam2 | but it doesn't have a UI we can reuse | 09:10 |
| Kinnison | Zuul's UI is gerrit is it not? | 09:10 |
| ssam2 | beyond its "post stuff to Gerrit" hooks | 09:10 |
| ssam2 | exactly | 09:11 |
| paulsher1ood | Zuul's ui is gerrit, and gerrit's ui is, erm, how do i say it.... ? | 09:12 |
| richard_maw | paulsher1ood: you don't need to | 09:12 |
| richard_maw | paulsher1ood: we are all aware | 09:12 |
| ssam2 | choosing Gerrit as the UI means we need to separately handle serving log files, showing what the status of the workers are and how many jobs are queued, whether 'master' builds successfully, etc | 09:12 |
| paulsher1ood | but anyway, i agree zuul has some great things | 09:12 |
| ssam2 | there is http://status.openstack.org/zuul/ but it doesn't really make sense to me | 09:12 |
| *** CTtpollard has joined #baserock | 09:13 | |
| paulsher1ood | concourse has a really fabulous ui, i'm still trying to work out how solid their infrastructure assumptions are... they're working on caching, so once that's done i'll be giving it a proper spin | 09:13 |
| *** mariaderidder has joined #baserock | 09:15 | |
| ssam2 | i can't make head or tail of the UI, but I guess this is because it's full of terms I don't understand | 09:15 |
| ssam2 | oh I see, it's aeroplane metaphors | 09:16 |
| ssam2 | fly, atc, baggageclaim, blackbox, testflight... | 09:16 |
| paulsher1ood | yup | 09:16 |
| *** edcragg has joined #baserock | 09:17 | |
| paulsher1ood | imagine how folks feel when they turn up here and we hit them with morphologies and strata :) | 09:17 |
| ssam2 | I try to use '.morph file', instead, which I think is quite clear | 09:18 |
| ssam2 | strata maybe clearer as layers, indeed | 09:18 |
| paulsher1ood | someone said to me the other day that they'd had feedback from newbies excited at the baserock concept, turning up at wiki.baserock.org and concluding 'there's no there, there' | 09:18 |
| paulsher1ood | which shows just how wrong folks can be, but also just how broken our messaging seems | 09:18 |
| * SotK tries to parse "there's no there, there" | 09:19 | |
| pedroalvarez | I failed :/ | 09:19 |
| paulsher1ood | it's a well established meme, i believe | 09:19 |
| DavePage | paulsher1ood: It's sensible to assume that anybody who wants to learn more about baserock is an idiot. | 09:19 |
| DavePage | (this also applies to non-baserock of course) | 09:19 |
| ssam2 | DavePage: ouch | 09:19 |
| * SotK googles it | 09:19 | |
| DavePage | The smoother the learning curve about a new project, the better (provided there's a way for more experienced people to jump ahead). | 09:20 |
| paulsher1ood | gertrude stein, SotK | 09:20 |
| paulsher1ood | i think | 09:20 |
| SotK | indeed | 09:20 |
| *** lachlanmackenzie has joined #baserock | 09:31 | |
| rdale_ | there are some comments on baserock-dev about using spdx for license tracking in baserock - has anyone done anything on that yet? | 09:38 |
| Kinnison | Given the lack of continued discussion, I doubt it has gotten any further yet | 09:38 |
| rdale_ | ok | 09:38 |
| paulsher1ood | i think that's right | 09:51 |
| SotK | paulsher1ood: I keep seeing builds fail with ybd when trying to mount --bind stuff in /root/.cache/ybd/ccache | 09:56 |
| paulsher1ood | interesting | 10:01 |
| wdutch | I'm looking at gerrit.baserock.org/#/c/430 because gerrit says 'The change cannot be merged due to a path conflict', when I checkout this topic I have no trouble rebasing it to master and it won't let me run 'git review' claiming that nothing has changed, I don't understand what the problem is for gerrit | 10:04 |
| pedroalvarez | wdutch: well, it depends on things that don't exist | 10:10 |
| franred | ...in master firehose | 10:11 |
| pedroalvarez | in gerrit | 10:11 |
| franred | wdutch, it requires to merge the rest of the patches before that patch can be merged | 10:12 |
| wdutch | oh so it will say merge conflict on all except the bottom one until that one has gone through? | 10:13 |
| franred | wdutch, the one which does not have the merge conflict is the first patch in the series, AFAICT | 10:14 |
| richard_maw | wdutch: you have to merge them in order. You can see the order by using the old gerrit UI, or set up gertty | 10:16 |
| richard_maw | I don't trust the new UI's related changes list | 10:16 |
| straycat | *generally*, the related changes section lists the changes in order, but not always | 10:17 |
| wdutch | It seems odd to me that gerrit doesn't have a nicer way to communicate that a change cannot be merged because it depends on another pending change | 10:18 |
| straycat | this is because gerrit is not built with series in mind | 10:18 |
| wdutch | so then could somebody with +2 powers review this please https://gerrit.baserock.org/#/c/159 | 10:20 |
| richard_maw | wdutch: merged | 10:21 |
| wdutch | thanks | 10:21 |
| richard_maw | wdutch: dependent patches may need to be rebased, depending on if there were any other changes involved, but since this patch didn't need to be rebased, probably not | 10:21 |
| paulsher1ood | SotK: what's the fail message? | 10:26 |
| paulsher1ood | (can you paste the log, please?) | 10:27 |
| SotK | the log as in the ybd output, or the build log it gives a path to? | 10:28 |
| * SotK goes for both | 10:28 | |
| paulsher1ood | the pathed build-log | 10:29 |
| paulsher1ood | ssam2: am i right you may have something for doing the sandboxing in a different way? maybe that would be of interest to SotK | 10:30 |
| SotK | paulsher1ood: http://sprunge.us/SEDN | 10:31 |
| paulsher1ood | SotK: i think that's broken definitions. try master? | 10:32 |
| richard_maw | paulsher1ood: the bind is failing before it has any input from definitions | 10:33 |
| SotK | paulsher1ood: that is master | 10:33 |
| paulsher1ood | hmm. | 10:33 |
| SotK | (or at least, master when I cloned it at some point yesterday) | 10:33 |
| paulsher1ood | which target? | 10:34 |
| SotK | clusters/release.morph x86_64 | 10:34 |
| paulsher1ood | i did see this a few days ago, but i've been juggling lots of things, will need to re-engage brain | 10:34 |
| * paulsher1ood kicks ybd into action | 10:36 | |
| ssam2 | that error is from linux-user-chroot, either one of the source or target directories passed to --mount-readonly or --mount-bind doesn't exist | 10:37 |
| paulsher1ood | hmmm. so i'm already past stage2-fhs-dirs. | 10:37 |
| * paulsher1ood clears cache | 10:38 | |
| paulsher1ood | the one thing i wonder is whether there could be some difference between stuff under /root/.ybd/ and stuff under /src ? | 10:39 |
| paulsher1ood | SotK: are you running on a baserock system, or something else? | 10:39 |
| SotK | on baserock | 10:39 |
| paulsher1ood | with no /src director? | 10:40 |
| SotK | paulsher1ood: correct | 10:41 |
| paulsher1ood | would you mind trying again with a /src ? | 10:41 |
| SotK | sure | 10:42 |
| paulsher1ood | tvm | 10:42 |
| paulsher1ood | in the meantime i'm rebuilding build-essential | 10:42 |
| SotK | hm, so am I apparently | 10:44 |
| *** pacon has joined #baserock | 10:44 | |
| SotK | does YBD magically use a different cachedir if /src is present? | 10:44 |
| ssam2 | yes | 10:45 |
| ssam2 | check in app.py | 10:45 |
| Kinnison | MOAR MAGICK | 10:45 |
| jjardon | Hi, if you have some time, I made a couple of patches to avoid unnecessary rebuilds; https://gerrit.baserock.org/#/q/topic:foundation_core | 10:46 |
| Kinnison | jjardon: Have you actually tested these changes properly and thoroughly (unlike the last set of things I reviewed for you?) | 10:46 |
| jjardon | Kinnison: the testing I did is in the comments | 10:47 |
| Kinnison | jjardon: So not sufficient then | 10:49 |
| Kinnison | You changed tools | 10:49 |
| Kinnison | that implies you need to revalidate the *BEHAVIOUR* of every system which includes tools, or includes a stratum which build-depends on tools | 10:50 |
| Kinnison | Not just build one system | 10:50 |
| jjardon | Kinnison: put your concerns in gerrit. But is unlikely that I (or anyone else) will have time to build and test the behaviour of all the systems that include the tools stratum (I count around ~40 systems in definitions) | 10:54 |
| paulsher1ood | no, it's not what you think, Kinnison :) | 10:55 |
| Kinnison | jjardon: Until and unless we have automated validation, we *MUST* put this effort in for this kind of change | 10:55 |
| Kinnison | jjardon: Otherwise we get the kinds of hiccoughs we have when we merged your expat code | 10:56 |
| Kinnison | s/code/change/ | 10:56 |
| Kinnison | I've raised my concerns here. Interacting with gerrit on those changes will mean gerrit will mail me more crap I don't care about | 10:57 |
| Kinnison | paulsher1ood: it's always as bad as I think | 10:57 |
| Kinnison | paulsher1ood: sometimes it's worse | 10:57 |
| Kinnison | :) | 10:57 |
| bashrc_ | from the talk of upstreams yesterday, if gerrit isn't very good (imho it isn't) then we should maybe put development effort into improving it | 10:59 |
| Kinnison | You're leaking a bit -- but yes we have considered what it'd take to improve Gerrit | 10:59 |
| bashrc_ | or write a better tool | 10:59 |
| Kinnison | Indeed | 10:59 |
| Kinnison | This is, however, a very subjective arena | 11:00 |
| jmacs | I'm not convinced gerrit is an improvement on the previous mailing list approach. | 11:00 |
| Kinnison | jmacs: For some people it very much is. For others, not so much :-/ | 11:00 |
| Kinnison | jmacs: Fortunately we do still accept patches via the ML :) | 11:00 |
| jjardon | Kinnison: I guess It's better to make changes and sometimes break things than not make changes at all | 11:00 |
| Kinnison | jjardon: That's a matter of opinion and a question for policy for Baserock as a project | 11:01 |
| Kinnison | "Should master always work?" | 11:01 |
| jmacs | I agree with that - I get the impression that it's more efficient for full-time users | 11:01 |
| jmacs | For infrequent contributors like me it's a pain | 11:01 |
| rjek | +1 | 11:01 |
| Kinnison | Oh Gods, at least one person is subscribed to baserock-dev via Digest mode | 11:01 |
| rjek | Why do we even let them? | 11:02 |
| Kinnison | jmacs: amusingly, there are others who say that Gerrit is way better for infrequent contributors | 11:02 |
| Kinnison | :) | 11:02 |
| Kinnison | It's a very very personal thing IMO | 11:02 |
| jmacs | Kinnison: That is amusing. | 11:02 |
| jjardon | but i'm curious if what you propose to fix this: eventually, we want to have a machine test the changes before merge them. Will this machine build and test/run ~40 systems for every change we sent to gerrit? | 11:02 |
| jjardon | Kinnison: ^ | 11:02 |
| rjek | jjardon: Hopefully. | 11:02 |
| Kinnison | jjardon: Hopefully yes | 11:03 |
| SotK | that will take a long time | 11:03 |
| Kinnison | jjardon: Well, hopefully it'll build/test all the *affected* systems | 11:03 |
| rjek | distbuild solves all problems | 11:03 |
| Kinnison | jjardon: and we have initiatives already underway which will help to constrain that set | 11:03 |
| Kinnison | Until that day, the only way to constrain the set is for humans to do *analysis* | 11:03 |
| SotK | rjek: not the problem of testing that `morph build` works in the resulting build and devel systems | 11:03 |
| Kinnison | and my rudimentary analysis says "You edited 'tools' -- you need to validate a build system can bootstrap a new build without a cache server, and you need to verify that troves built using the new tools stratum still work" | 11:04 |
| Kinnison | that would, IMO, be the bare-minimum of testing for that change | 11:04 |
| ssam2 | I think the Baserock project is at a point where we can't require people to manually validate everything that might be affected by a change for every change | 11:05 |
| ssam2 | certainly I wouldn't contribute anything to definitions if I had to do that | 11:05 |
| Kinnison | ssam2: If you make small changes, they're more easily verified by analysis rather than by active build/run testing | 11:06 |
| Kinnison | ssam2: But for large changes like jjardon just proposed, I fear analysis may be insufficient | 11:06 |
| Kinnison | esp. if done in a slapdash fashion | 11:06 |
| ssam2 | we have post-merge CI, so merging is a reasonable way to test | 11:06 |
| ssam2 | I would prefer that we had a set cadence of stabilisation | 11:07 |
| jjardon | Kinnison: that's more reasonable, and a bit different from build and test ~40 systems. comment in gerrit and I will try to test that when I have time | 11:07 |
| franred | ssam2, we only have post-merge builder, not tester, if we don't test some of the system we may end up thinking we are supporting some systems which are not really tested after a while - and the effort in fixing them would be huge | 11:08 |
| ssam2 | yeah, true | 11:08 |
| * pedroalvarez finally redeploys mason https://mason-x86-64.baserock.org/ | 11:17 | |
| pedroalvarez | I've put a note there so no-one asks why it's red/blank | 11:17 |
| jjardon | pedroalvarez: nice | 11:18 |
| SotK | pedroalvarez: maybe we could modify the mason script to do something similar when it starts a build? :) | 11:19 |
| pedroalvarez | don't expect that nice message again :) | 11:19 |
| pedroalvarez | SotK: it will be really easy I thiink | 11:19 |
| SotK | I think so too | 11:19 |
| * SotK hasn't tried though | 11:19 | |
| SotK | paulsher1ood: seems to be working fine with /src | 12:34 |
| richard_maw | perhaps it would be more appropriate for it to be configurable, than have ybd attempt to magically guess | 12:55 |
| *** pacon has quit IRC | 13:27 | |
| *** pacon has joined #baserock | 13:28 | |
| *** zoli__ has quit IRC | 13:30 | |
| *** pacon has quit IRC | 13:30 | |
| pedroalvarez | wow, it wasn't difficult at all to rebase my branch on top of master, even with patches for files that have been moved to subfolders | 13:51 |
| * pedroalvarez hugs git | 13:51 | |
| Kinnison | hehe | 13:51 |
| Kinnison | git's merge/rebase code is pretty clever | 13:51 |
| richard_maw | very much so, given it doesn't track renames | 13:52 |
| robtaylor | its clever enough to figure that out ;) | 13:55 |
| richard_maw | does anyone have review bandwidth for https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/morph+branch:master+topic:strip-commands ? | 14:14 |
| Kinnison | richard_maw: is it also on a git server somewhere easy to deal with? | 14:15 |
| richard_maw | I don't expect any quick reviews for the sourceresolver rewrite, but there's a lot of changes before it that should be mostly independent | 14:15 |
| richard_maw | Kinnison: `git pull https://gerrit.baserock.org/baserock/baserock/morph refs/changes/86/786/1`. Gerrit gets awfully confused if you push commits with change-ids to personal branches | 14:16 |
| * Kinnison tries | 14:17 | |
| ssam2 | it does. it may be that we can alter the git.baserock.org -> gerrit.baserock.org mirroring to ignore personal branches | 14:17 |
| ssam2 | it's just hard to know which branches are "personal" | 14:17 |
| * Kinnison runs git format-patch and then reads it | 14:18 | |
| richard_maw | hm, because gerrit doesn't have access to the gitano user list | 14:18 |
| Kinnison | In theory we could write a gitano plugin to push user information across | 14:21 |
| paulsher1ood | SotK: so this does make me wonder if there is something special about /home/root ? | 14:49 |
| paulsher1ood | richard_maw: yup. better configurable, in a default yaml file. there's lots of stuff in app.setup that should go there | 14:49 |
| richard_maw | that'd make it bloody hard to have a test suite for | 14:50 |
| paulsher1ood | :) | 14:50 |
| * pedroalvarez hits the sourceresolve rewrite patch | 14:51 | |
| richard_maw | I don't believe a smiley face to be the appropriate response. | 14:51 |
| paulsher1ood | richard_maw: you may be right, i need to think about that. | 14:51 |
| paulsher1ood | richard_maw: would you prefer i get grumpy? | 14:51 |
| pedroalvarez | I don't want grupmy discussions in #baserock please | 14:52 |
| paulsher1ood | pedroalvarez: i agree :-) | 14:52 |
| richard_maw | it just rubbed me up the wrong way, as it implied you were gleeful about not wanting a test suite, and I'm of the opinion that without a test suite you can't prove it works | 14:53 |
| paulsher1ood | ah, no i wasn't being gleeful, just expressing that i'm not worried by that so far, because i'm very confident in my existing test regime | 14:53 |
| paulsher1ood | not that ybd master always works... just that i expect to spot breakages very quickly | 14:54 |
| paulsher1ood | but i acknowledge that my current approach 'doesn't scale' | 14:54 |
| richard_maw | I was just about to say that it's fine for a 1 man project, but causes friction when there's other simultaneous developers. | 14:55 |
| paulsher1ood | agreed | 14:55 |
| paulsher1ood | i don't expect ybd will ever have a lot of folks working on it :) | 14:56 |
| paulsher1ood | i mean, the few folks who have looekd at the code so far seem to hate it :-) | 14:56 |
| paulsher1ood | richard_maw: any idea why stuff under /home/root might trigger the bind error, but under /src does not? | 15:02 |
| richard_maw | paulsher1ood: nope, haven't looked at that code | 15:02 |
| paulsher1ood | richard_maw: i mean, is there anything special about /home/root ? | 15:03 |
| richard_maw | just that usually root's home is /root | 15:03 |
| paulsher1ood | no strange permissions or filesystem trickery? | 15:03 |
| richard_maw | only if you've got encrypted home directories separately from encrypted /home volume | 15:04 |
| * SotK never mentioned /home/root? :) | 15:04 | |
| SotK | just /root | 15:04 |
| paulsher1ood | ah, sorry. | 15:06 |
| paulsher1ood | richard_maw: is there anything special about /root ? | 15:06 |
| pedroalvarez | it shouldn't be anything special about it | 15:07 |
| pedroalvarez | is just another folder | 15:07 |
| pedroalvarez | it might be smaller than /src if you have attached a disk in /src | 15:07 |
| pedroalvarez | s/attached/mounted/ | 15:07 |
| * SotK didn't attach a disk, I just created a /src directory | 15:08 | |
| paulsher1ood | there is no code difference for ybd between these two cases afaict - it's all just based on app.settings['base'] https://github.com/devcurmudgeon/ybd/blob/master/app.py#L93 | 15:09 |
| paulsher1ood | so i'm intrigued. i can fix the problem, i just wonder what's going on | 15:10 |
| Kinnison | Base permissions on /root perhaps being less permissive | 15:10 |
| Kinnison | typically it's 750 not 755 | 15:11 |
| Kinnison | fr.ex. | 15:11 |
| Kinnison | ? | 15:11 |
| Kinnison | richard_maw: I have plowed my way through that epic topic | 15:11 |
| Kinnison | richard_maw: you have my +1 across it, but with some commentary | 15:11 |
| richard_maw | Kinnison: cool, thanks. | 15:11 |
| Kinnison | My main issue surrounds the claim that we now actively support 6 variants of definitions version | 15:12 |
| Kinnison | and my worry that it may not be true | 15:12 |
| paulsher1ood | which topic is this? | 15:12 |
| Kinnison | Secondarily my worry is that 6 versions is a very large support-surface | 15:12 |
| richard_maw | paulsher1ood: https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/morph+branch:master+topic:strip-commands | 15:12 |
| paulsher1ood | ah, ok. i'll read that if folks want input? | 15:13 |
| SotK | I think that most of the versions are fairly small changes | 15:13 |
| richard_maw | possibly feature flags would be better than so many numbered versions there | 15:13 |
| Kinnison | richard_maw: I believe you and I discussed that over lunch recently :) | 15:14 |
| paulsher1ood | shall we think more deeply about this whole version thing before it really does become a nightmare? | 15:14 |
| perryl | i'm looking at deterministic builds wrt artifacts and considering that when the tarball is generated that it may not be done deterministically, i.e. individual file permissions/file ordering may vary from build to build. where would i check this, and if necessary, patch functionality? | 15:16 |
| richard_maw | perryl: morphlib/bins.py I think | 15:17 |
| perryl | richard_maw: thanks! | 15:17 |
| paulsher1ood | perryl: are you looking at this for ybd, or morph? | 15:18 |
| perryl | both, if possible | 15:19 |
| paulsher1ood | :) | 15:19 |
| paulsher1ood | perryl: https://github.com/devcurmudgeon/ybd/blob/master/cache.py#L75 | 15:20 |
| radiofree | meta file for build ybd artefacts could contain a bit more information | 15:21 |
| paulsher1ood | radiofree: agreed. any ideas on what would be most useful there? | 15:21 |
| radiofree | repo and sha1 at least | 15:21 |
| perryl | hah, was about to aay it may be easier to grep for in ybd :) | 15:21 |
| paulsher1ood | radiofree: ok. anything else off the top of your head? | 15:22 |
| radiofree | that's usually all the information i ever need from a baserock meta file | 15:22 |
| radiofree | + files of course | 15:22 |
| radiofree | (which are already there) | 15:22 |
| paulsher1ood | radiofree: thanks. https://github.com/devcurmudgeon/ybd/issues/41 | 15:23 |
| paulsher1ood | i'll get to it before monday | 15:23 |
| radiofree | heh, upstream is fast! | 15:23 |
| paulsher1ood | i do my best | 15:23 |
| *** zoli__ has joined #baserock | 15:30 | |
| *** rdale_ has quit IRC | 15:37 | |
| *** rdale has joined #baserock | 15:38 | |
| *** a1exhughe5 has quit IRC | 15:42 | |
| paulsher1ood | https://github.com/devcurmudgeon/ybd/commit/8e06030d41f28a15fae13bb8758cb25855089b49 | 15:44 |
| * paulsher1ood wonders why flush is required?, anyway... radiofree - master has a fix now :) | 15:45 | |
| Kinnison | because otherwise the .write may not be committed before the subprocess runs | 15:47 |
| Kinnison | By default, python's file objects are buffered | 15:47 |
| *** lachlanmackenzie has quit IRC | 15:48 | |
| paulsher1ood | ah, ok. | 15:51 |
| paulsher1ood | thanks, Kinnison | 15:51 |
| paulsher1ood | ssam2: your patch looks great! a chance to get rid of loads of code from ybd! :) | 15:53 |
| paulsher1ood | what's the least work to ensure no user is harmed if i merge it? | 15:53 |
| paulsher1ood | would this work (as root) on jrandom linux with pip-installed sandboxlib? | 15:54 |
| * paulsher1ood is a bit concerned about pip-installing on his baserock vm | 15:56 | |
| Kinnison | Correct | 15:56 |
| * Kinnison is concerned about pip on any platform | 15:56 | |
| * Kinnison thinks of pip as standing for "Provenance is passé" | 15:57 | |
| paulsher1ood | :) | 15:57 |
| ssam2 | paulsher1ood: I've done that in Baserock to test it | 15:58 |
| ssam2 | i.e. I've done 'pip install sandboxlib' inside Baserock | 15:58 |
| ssam2 | I'll also send a patch to add sandboxlib to the Baserock reference systems | 15:58 |
| paulsher1ood | ssam2: ok, great. is there a non pip way that i could safely add this to my br vm for testing, then remove it if i don't want it? | 15:59 |
| SotK | git clone and set PYTHONPATH? | 16:00 |
| ssam2 | you can clone https://github.com/CodethinkLabs/sandboxlib/ somewhere, then do PYTHONPATH=/path/to/sandboxlib python ybd ... | 16:00 |
| paulsher1ood | ah, ok. thanks! | 16:00 |
| paulsher1ood | maybe the readme needs to be updated with this, at least temporarily? | 16:01 |
| paulsher1ood | i'll merge, anyway | 16:01 |
| paulsher1ood | 2 files changed, 100 insertions(+), 238 deletions(- | 16:02 |
| ssam2 | I can add a 'dependencies' section to the YBD readme.md file | 16:03 |
| paulsher1ood | ssam2: i know others here will be happy, you're heloping to destroy ybd :) | 16:03 |
| paulsher1ood | ssam2: yes please | 16:03 |
| richard_maw | paulsher1ood: ok, that joke made me grin | 16:03 |
| paulsher1ood | lol | 16:04 |
| paulsher1ood | richard_maw: i'll be delighted to see as little code in ybd as possible | 16:04 |
| *** zoli___ has joined #baserock | 16:04 | |
| ssam2 | contributions to https://github.com/CodethinkLabs/sandboxlib are welcome by the way, there are rough edges | 16:05 |
| * paulsher1ood will take a look | 16:05 | |
| richard_maw | paulsher1ood: help us rework morphlib into being an actually useful library then, and you can just import morphlib into ybd | 16:05 |
| ssam2 | we can add collaborators from outside Codethink to the project as far as I know, don't let the 'codethinklabs' organisation put you off | 16:05 |
| paulsher1ood | ssam2: anything you'd want me to know about my coding unstyle before i submitted patches? | 16:05 |
| ssam2 | run 'pep8' or 'pylint' first to look for obvious style errors | 16:06 |
| ssam2 | there's a HACKING.rst file which hopefully has all the info you need | 16:06 |
| paulsher1ood | ssam2: i do pep8 actually, when i get round to it. pep8 does not seem to be on baserock systems by default, though | 16:06 |
| rdale | how do you set up a project in CodethinkLabs? | 16:07 |
| Kinnison | I think you have to be a Codethink employee | 16:07 |
| paulsher1ood | rdale: ask nicely, or be a member of the organisation. i think you already are? | 16:07 |
| paulsher1ood | s/organisation/github organisation/ | 16:07 |
| * paulsher1ood checks | 16:07 | |
| rdale | i see - it's ages since i create a github project | 16:07 |
| ssam2 | paulsher1ood: you can install it with 'pip' though | 16:08 |
| *** zoli__ has quit IRC | 16:08 | |
| paulsher1ood | rdale: there are too many github proejcts already, think carefully before creating yet another one :) | 16:08 |
| ssam2 | pep8 is not a component where provenance really matters I think | 16:08 |
| paulsher1ood | ssam2: possibly, except on the day that the pip infrastructure gets hacked, and i install perp8 and its malware friend | 16:09 |
| rdale | what does 'too many' mean? is there a numerical limit to codethink's allowance? or is it just untidy? | 16:10 |
| richard_maw | paulsher1ood: you may be interested in knowing that the current version of my patches to add strip-commands support to ybd only adds 2 lines in total, as I was able to remove 18 from your build system handling code | 16:10 |
| ssam2 | paulsher1ood: true | 16:10 |
| paulsher1ood | richard_maw: you're a star! | 16:10 |
| rjek | I stopped mirroring LuaRocks (the Lua equiv. of pip, rubygems, cpan, etc) when they changed the policy to "allow anybody on the internet to upload a tantalisingly-named or replacement for popular module which contains a rootkit." | 16:10 |
| paulsher1ood | richard_maw: i assume you're doing it in morph, too? what are the equivalent numbers? | 16:11 |
| richard_maw | paulsher1ood: this is all predicated on the assumption that the reason I can't find any code for handling definitions that aren't in definitions.git, is that there isn't any code to support it, rather than you defaulting to an empty dict for nascent definitions | 16:11 |
| paulsher1ood | richard_maw: correct. if it's not somwhere under os.getcwd()/*/ it's not a definition as far as ybd is concerned | 16:12 |
| paulsher1ood | (that assumption may not hold in future of course) | 16:13 |
| SotK | paulsher1ood: how does that work for chunks which don't have a morphology? | 16:13 |
| *** lachlanmackenzie has joined #baserock | 16:13 | |
| richard_maw | paulsher1ood: I gave those numbers yesterday. For morph: ~700 lines changed, 11 added in total excluding the ~300 line test-suite removal as I want to keep that really. It would be >1000 lines to add the strip commands to every definition in definitions.git. | 16:13 |
| paulsher1ood | SotK: it 'just works, tm' i think? | 16:13 |
| * SotK tries to figure it out | 16:13 | |
| richard_maw | It's the same thing AIUI | 16:14 |
| paulsher1ood | i'm not being flippant, every time i need to think about definitoins it takes me 20 mins to inbound the logic to my brain | 16:14 |
| richard_maw | paulsher1ood: understood | 16:14 |
| paulsher1ood | i have some simplifying ideas in mind, if we could work out how to make schema/structure changes trivial and bulletproof | 16:15 |
| * paulsher1ood dreams on | 16:15 | |
| SotK | I'm guessing from a quick look that for chunks without morph files, the chunk spec from the stratum is added to Definitions.__definitions | 16:17 |
| richard_maw | hmm, that sounds like it would make my proposed reduction incorrect then | 16:19 |
| paulsher1ood | it parses each definition file... | 16:19 |
| SotK | paulsher1ood: some chunks don't have definition files | 16:20 |
| richard_maw | yes, but chunk definitions' content is spread across both the stratum they are introduced in, and the definition file that may be referred to | 16:20 |
| richard_maw | the latter is optional in morph, and results in the buils system being guessed | 16:20 |
| paulsher1ood | for each component (which may be stratum, chunk) it either adds what it finds here to what it already knows, or creates a new definition if this is the first info | 16:20 |
| SotK | so chunks without definitions will be added as just the entry in the stratum then | 16:22 |
| paulsher1ood | the code is nightmarish enough that Kinnison called it difficult to understand, but i blame that more on our definitions syntax than my programming | 16:22 |
| * Kinnison blames it entirely on your programming :) | 16:22 | |
| paulsher1ood | SotK: yup. and then ybd will guess, same as moprh | 16:22 |
| * Kinnison can quite confidently grok our definitions syntax | 16:22 | |
| paulsher1ood | Kinnison: patches welcome | 16:22 |
| Kinnison | paulsher1ood: I can't bring myself to believe I know how it's *meant* to function, since its external surface lacks any tests | 16:22 |
| Kinnison | Also I have way more projects to contribute to than time and energy already :) | 16:23 |
| richard_maw | paulsher1ood: your guessing code is based on not having "build-system" in the definition. This is a divergence in behaviour, since in morph that means use the manual build system instead. | 16:23 |
| richard_maw | paulsher1ood: normally that wouldn't bite you, since build-system: manually has previously been a no-op | 16:23 |
| paulsher1ood | Kinnison: if you can supply < 150 lines of python that do the job and meet your criteria, i'll buy you dinner | 16:23 |
| richard_maw | paulsher1ood: but I want to hang the strip commands off it | 16:23 |
| Kinnison | paulsher1ood: given my criteria involve explicitly making it longer and thus more grokable, I'll buy myself an infinite improbability drive if I manage that | 16:24 |
| Kinnison | since I'll have defeated physics | 16:24 |
| Kinnison | and information theory | 16:24 |
| Kinnison | all in python | 16:24 |
| * paulsher1ood needs to think through richard_maw's comment | 16:24 | |
| paulsher1ood | Kinnison: how much longer? i'm happy to compromise :) | 16:25 |
| Kinnison | I dunno | 16:25 |
| Kinnison | I want classes with explicit attributes and testable surfaces | 16:25 |
| paulsher1ood | i think i want that too | 16:25 |
| paulsher1ood | i just hate having more code than necessary | 16:26 |
| * Kinnison feels that in your abject fear of lines-of-code you have erred into the simplistic rather than the simplified. | 16:27 | |
| paulsher1ood | i note your feelings, Kinnison. it's not fear, on my side though, it's loathing :) | 16:27 |
| * Kinnison sends paulsher1ood to Las Vegas | 16:27 | |
| Kinnison | they like that kind of thing there | 16:27 |
| paulsher1ood | the complexity really is due to the syntax, honest. i had to spend three whole weekends on it | 16:28 |
| Kinnison | I think the complexity in your code is mostly due to your desire to not transform the physical data structure into a logical one | 16:29 |
| Kinnison | But I can't be sure | 16:29 |
| paulsher1ood | __definitions is my attempt at the logical structure, i think | 16:32 |
| paulsher1ood | but this might be a similar blindspot to the 'no graph' discussion | 16:33 |
| Kinnison | I think it is | 16:34 |
| Kinnison | However, I have to bimble off now and deal with the fact that people have started posting about some code I wrote ages ago on things like identi.ca | 16:34 |
| * SotK suddenly realises his "Fix deployment" patch may have broken running ybd with anything but a cluster and tests to find out if he's right | 16:34 | |
| paulsher1ood | SotK: did i merge it? | 16:35 |
| SotK | paulsher1ood: yes | 16:35 |
| paulsher1ood | well i think it's working, so relax :) | 16:35 |
| SotK | have you run it with anything other than a cluster, to completion? | 16:36 |
| paulsher1ood | i believe so. i'll check again | 16:36 |
| paulsher1ood | oh, no :) | 16:36 |
| paulsher1ood | interesting... what would you expect to break? | 16:37 |
| paulsher1ood | SotK: ^^ | 16:37 |
| SotK | I'm imagining it will attempt to deploy a system, stratum or chunk definition | 16:38 |
| SotK | it might work without breaking, there is too much recursion for me to understand at this time of day :) | 16:39 |
| paulsher1ood | assembly.deploy always runs, if that's what you mean | 16:40 |
| paulsher1ood | it did before your patch, anyway | 16:40 |
| SotK | I know it does, but I removed the if statement that makes it return if the definitions doesn't have a 'systems' key | 16:40 |
| paulsher1ood | SotK: but whhhhhhhhhyyyyyyyyy ? :) | 16:41 |
| SotK | I actually think everything has sensible enough defaults it won't break | 16:41 |
| SotK | paulsher1ood: because it was making nothing get deployed | 16:42 |
| paulsher1ood | i think it'll be ok. i'm more worried about sandboxlib vs stage2-fhs-dirs | 16:42 |
| paulsher1ood | SotK: one thing... are you saying that ybd now deploys system files, strata files, chunk files? if so, to what? | 16:43 |
| SotK | paulsher1ood: no, I think it will just do nothing | 16:43 |
| richard_maw | paulsher1ood: Currently at 4 lines of code removed to make YBD handle an absent build-system meaning build-system: manual, then 20 added to make it support strip commands, most of that is the shell command formatted in a non eye-bleeding way | 16:44 |
| paulsher1ood | k. i don't know what a deployment would be if there's no defined location/target fro it | 16:44 |
| paulsher1ood | richard_maw: cool! | 16:44 |
| SotK | paulsher1ood: it wouldn't work :) | 16:44 |
| richard_maw | paulsher1ood: well, there's no test suite to prove I haven't broken it in some way | 16:45 |
| richard_maw | so I'm going to have to wait for gcc to build | 16:45 |
| paulsher1ood | richard_maw: how long does that take on your machine? 8 mins on mine | 16:46 |
| paulsher1ood | richard_maw: noted about the test suite | 16:46 |
| SotK | paulsher1ood: the reason that return statement was stopping deployment from happening was that only clusters have a "systems" key, so deploy(system) returned without doing anything | 16:47 |
| richard_maw | paulsher1ood: show-off. I can let you know when it's built gcc, though the number is likely to be poor, as I'm suffering from poor IO currently. | 16:47 |
| paulsher1ood | SotK: that's correct behaviour, iiuc | 16:47 |
| SotK | paulsher1ood: nope, because that means nothing is ever deployed | 16:48 |
| SotK | clusters don't have a 'deploy' key, so no extensions are run and nothing is extracted to be deployed | 16:49 |
| paulsher1ood | SotK: ok, you may be right. i had it working, briefly, until i noticed that extensions were in so many places and gave up | 16:49 |
| SotK | and the system specs in the clusters which do have a 'deploy' key don't have a 'system' key, meaning that the function attempting to deploy them returned before anything was extracted or any extensions were run | 16:50 |
| paulsher1ood | weird. that's another aspect of the definitoins syntax i still don't understand properly, then | 16:51 |
| rdale | the syntax is yaml, it sounds like the semantics that are the problem | 16:52 |
| paulsher1ood | sorry, you're right. i get confused. thanks rdale | 16:53 |
| paulsher1ood | perryl: thanks for the patch! | 16:54 |
| rdale | there seem to be a lot of arbitrary things which look like environment variables in the cluster definitions which i don't understand | 16:54 |
| paulsher1ood | rdale: could you mail the list highlighing them? | 16:54 |
| richard_maw | they are passed as environment variables | 16:54 |
| perryl | paulsher1ood: thanks for the speedy merge! :) | 16:55 |
| paulsher1ood | :-) | 16:55 |
| paulsher1ood | perryl: what was the elapsed time between submission and notification of merge, from your perspective? | 16:56 |
| ssam2 | rdale: deployment is extensible, so the configuration for deployment is basically anarchy | 16:56 |
| rdale | so are there a list of cluster attributes which aren't environment variables, and anything which isn't one of them is considered to be an environment variable? | 16:56 |
| ssam2 | rdale: http://wiki.baserock.org/definitions/current/#index4h3 might help | 16:57 |
| ssam2 | only 'type' and 'location' are standard | 16:57 |
| richard_maw | rdale: [type, location, upgrade-type, upgrade-location] | 16:57 |
| ssam2 | oh, yeah | 16:57 |
| richard_maw | I think ssam2 was responsible for the latter 2 | 16:57 |
| perryl | paulsher1ood: <5 minutes i think, either that or my perception of time is fudged today! | 16:57 |
| *** Krin has quit IRC | 16:58 | |
| paulsher1ood | perryl: cool. the reason i ask is that i was once looking at flynn.io, very interesting project.... | 16:58 |
| paulsher1ood | and noticed a problem in the documentation. fixed it... and was merged within 18 minutes, start to finish | 16:59 |
| perryl | impressive! :) | 16:59 |
| paulsher1ood | and that really amazed me... i thought, wow, that's really cool | 16:59 |
| paulsher1ood | and it wasn't a bot, or anthging - the upstream guy commented on someting i needed to fix, i did that, all within 18 mins | 16:59 |
| * richard_maw had the same turn-around for a documentation patch to systemd | 17:00 | |
| paulsher1ood | i can't see gerrit will ever match github for awesomness | 17:00 |
| richard_maw | paulsher1ood: github vs gerrit is irrelevant in this comparison | 17:00 |
| paulsher1ood | is it? | 17:01 |
| ssam2 | i've merged patches in minutes on our Gerrit | 17:01 |
| richard_maw | I've merged patches in minutes via the mailing list | 17:01 |
| *** CTtpollard has quit IRC | 17:01 | |
| richard_maw | it's that there's upstream developers currently reviewing patches that does it | 17:01 |
| paulsher1ood | i must still be missing some fundamental tricks in gerrit, then. | 17:01 |
| *** bashrc_ has quit IRC | 17:02 | |
| paulsher1ood | i'll think on this properly, i'm not fully understnding what my perception of the bottlenecks is. | 17:02 |
| ssam2 | several people seem to think 'gertty' is the fundamental trick | 17:03 |
| paulsher1ood | reminds me of the blue light story somehow, sorry for the noise... http://theoryofconstraints.blogspot.co.uk/2007/06/toc-stories-2-blue-light-creating.html | 17:03 |
| pedroalvarez | bottleneck for reviewing? testers | 17:03 |
| *** jonathanmaw has quit IRC | 17:04 | |
| richard_maw | testing documentation is reading it | 17:04 |
| paulsher1ood | pedroalvarez: depends on the nature of the patch, and the nature of the test/review/deploy/deliver pipeline | 17:04 |
| paulsher1ood | richard_maw: +1 | 17:04 |
| richard_maw | hence documentation patches have fast turn around | 17:04 |
| paulsher1ood | so gertty can't help vs one part of what i meant, which is that github is easy to navigate for non-cli users | 17:05 |
| paulsher1ood | i made the patch in github, never even pulled the repo locally | 17:05 |
| ssam2 | fair enough. Can't do that in Gerrit for sure | 17:06 |
| paulsher1ood | similarly for some ybd patches, i've merge at github, without pulling (for various reasons) | 17:06 |
| pedroalvarez | wow, I can deploy a Kilo openstack (one node) in 4 minutes | 17:06 |
| paulsher1ood | wtf!!!! w00t-and a half! | 17:06 |
| pedroalvarez | maybe a couple more | 17:07 |
| paulsher1ood | w00ts, or minutes? | 17:07 |
| pedroalvarez | but it's still great | 17:07 |
| paulsher1ood | :) | 17:07 |
| pedroalvarez | is not ready to be merged on master, but i though I had to mention it | 17:07 |
| pedroalvarez | I can screw it up, and redeploy one in a few minutes, and that makes me happy | 17:08 |
| franred | deploy the actual Openstack in Baserock for Juno should take the same amount of time ;-) | 17:08 |
| pedroalvarez | but juno is old | 17:09 |
| franred | xD - franred doesn't enter in this kind of holy wars | 17:09 |
| ssam2 | anyone seen Intel "Clear Linux" before? seems to have a familiar sort of feature set... https://clearlinux.org/features | 17:09 |
| ssam2 | (other than the 'fast VMs as containers' thing which seems unique) | 17:10 |
| paulsher1ood | similar to what? | 17:10 |
| paulsher1ood | it seems to be only Intel, which is nice of them :) | 17:11 |
| ssam2 | similar to Baserock, Ubuntu Core, Fedora Atomic, ... | 17:15 |
| paulsher1ood | ah, ok | 17:15 |
| ssam2 | well, similar to what Ubuntu Core pretends to be :) | 17:15 |
| paulsher1ood | careful, they might hear you :-) | 17:16 |
| paulsher1ood | iiuc Ubuntu Core is different thing, now? | 17:17 |
| paulsher1ood | ssam2: btw, sandboxlib seems to be working for me | 17:17 |
| ssam2 | cool | 17:17 |
| paulsher1ood | :-) | 17:17 |
| paulsher1ood | ssam2: as you know, software normally breaks for me, so this is lovely :-) | 17:18 |
| *** ssam2 has quit IRC | 17:20 | |
| *** mariaderidder has quit IRC | 17:24 | |
| tlsa | how can I make ybd rebuild busybox? I tried deleting its artifact and git repo from /src | 17:28 |
| paulsher1ood | change someging in its definition file? | 17:30 |
| paulsher1ood | shouldn't need to delete its repo | 17:30 |
| paulsher1ood | that would be weird :) | 17:30 |
| tlsa | I don't want to change its definition; I'm testing that subsequent builds of the same thing give bit-for-bit the same result | 17:31 |
| *** edcragg has quit IRC | 17:32 | |
| paulsher1ood | ah, ok | 17:34 |
| paulsher1ood | tlsa: are you running ybd.py to build busybox, or some other target | 17:35 |
| tlsa | I was makeing it build build-essential, which contains busybox. Doing busybox directly seems to have worked | 17:36 |
| paulsher1ood | tlsa: yup. it didn't rebuld busybox, because it already had build-essential | 17:36 |
| tlsa | right, makes sense | 17:37 |
| paulsher1ood | :-) | 17:37 |
| * paulsher1ood goes swimming... after weeks of sitting in aeroplanes... pain is expected | 17:37 | |
| tlsa | :) | 17:38 |
| * jmacs wonders why he's suddenly getting "ERROR: Could not find extension cloud-init.configure". I'm not explicitly using that extension. | 17:55 | |
| *** gary_perkins has quit IRC | 18:04 | |
| *** CTtpollard has joined #baserock | 18:12 | |
| *** CTtpollard has quit IRC | 18:14 | |
| *** lachlanmackenzie has quit IRC | 18:38 | |
| *** edcragg has joined #baserock | 18:39 | |
| *** edcragg has quit IRC | 20:06 | |
| paulsher1ood | jmacs: with morph, or ybd? | 20:13 |
| paulsher1ood | actually, can't be ybd | 20:15 |
| paulsher1ood | jmacs: definitions got updated recently to move all extensions into a tidy place. seems this is a case that didn't get spotted. are you using latest morph, recent definitions? | 20:15 |
| *** zoli___ has quit IRC | 20:18 | |
| paulsher1ood | rdale: are you around? | 20:26 |
| * paulsher1ood is hoping to ask dumb questions about 'migrations' | 20:26 | |
| * paulsher1ood hopes rdale is enjoying sangria or such, rather than online | 20:27 | |
| *** pacon has joined #baserock | 21:05 | |
| *** pacon has quit IRC | 21:47 | |
| paulsher1ood | hmmm. how to install linux-user-chroot on jrandom linux? | 22:04 |
| *** SotK has quit IRC | 22:34 | |
| *** SotK has joined #baserock | 22:34 | |
| *** zoli__ has joined #baserock | 22:38 | |
| *** Stanto has quit IRC | 22:38 | |
| *** Stanto has joined #baserock | 22:41 | |
| *** zoli__ has quit IRC | 23:00 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!