IRC logs for #baserock for Wednesday, 2015-06-17

* persia examines PEP-426, and is delighted with the nomenclature of requirements: "run_requires", "build_requires", "test_requires", and "dev_requires".00:00
*** zoli__ has joined #baserock00:54
*** zoli__ has quit IRC00:59
*** zoli__ has joined #baserock04:02
*** zoli__ has quit IRC04:22
*** petefoth has quit IRC04:32
*** tiagogomes has quit IRC04:32
*** nowster has quit IRC04:32
*** flatmush has quit IRC04:32
*** fay_ has quit IRC04:32
*** paulw has quit IRC04:32
*** bashrc has quit IRC04:32
*** sambishop has quit IRC04:32
*** CTtpollard has quit IRC04:32
*** petefoth has joined #baserock04:32
*** nowster has joined #baserock04:32
*** bashrc has joined #baserock04:32
*** paulw has joined #baserock04:32
*** fay_ has joined #baserock04:32
*** franred has quit IRC04:33
*** sambishop has joined #baserock04:33
*** tiagogomes has joined #baserock04:33
*** franred has joined #baserock04:33
*** flatmush has joined #baserock04:33
*** zoli__ has joined #baserock04:34
*** zoli__ has quit IRC06:36
*** zoli__ has joined #baserock06:37
*** zoli__ has quit IRC06:41
*** zoli__ has joined #baserock07:03
*** a1exhughe5 has joined #baserock07:30
*** edcragg has quit IRC07:37
*** mdunford has joined #baserock07:43
SotKbah, mason-x86-32 has gone kaput08:00
petefoth Is it the 'out of disk space' problem again?08:01
SotKI guess so, that was the cause of 503 last time wasn't it?08:02
petefothFrom what I gathered by lurking in here :)08:02
*** ssam2 has joined #baserock08:35
*** ChanServ sets mode: +v ssam208:35
*** jonathanmaw has joined #baserock08:36
ssam2I tried setting up a continuous builder with YBD, but it seems to have broken almost as soon as I left instead of building anything :(08:40
SotK:(08:40
ssam2with: 'OSError: [Errno 18] Invalid cross-device link'08:40
ssam2oh, it's while trying to hardlink stuff into the sandbox08:40
rjekOops08:41
ssam2seems it can't handle won't detect the case where tmp is on a different partition to the cachedir08:42
ssam2can't handle and won't detect08:42
Kinnisontracebacks FTW08:42
*** mariaderidder has joined #baserock08:43
*** jonathanmaw has quit IRC08:47
*** jonathanmaw has joined #baserock08:50
ssam2although tracebacks in the systemd journal are slightly less awesome, because journalctl strips the indentation08:52
ssam2or systemd-journald strips it. They are hard to read, anyway08:52
KinnisonMmm, I think it's to do with the stdxxx->journal processing08:52
Kinnisonthis is why morph --log was useful08:52
* ssam2 filed https://github.com/devcurmudgeon/ybd/issues/8308:58
*** edcragg has joined #baserock09:00
* straycat thank SotK for --snapshot09:06
*** gary_perkins has joined #baserock09:17
*** lachlanmackenzie has joined #baserock09:18
*** jonathanmaw has quit IRC09:38
*** jonathanmaw has joined #baserock09:39
richard_mawThe change in https://gerrit.baserock.org/#/c/850/ will work for most people, but not all. I'd prefer if another reviewer could make a judgement call on whether the change is sufficient, so I've +1d rather than +2d09:47
mwilliams_cthmm, I had thought /etc/morph.conf had to be symlinked to whatever your config file is. til! thanks richard_maw09:50
richard_mawyeah, it can also take drop-in files in /etc/morph/*.conf, or ~/.config/morph/*.conf09:51
richard_mawsorry I didn't think of this in my first review09:51
mwilliams_ctI think it should be changed, will -1 and fix up09:52
*** jonathanmaw has quit IRC09:58
*** jonathanmaw has joined #baserock09:58
richard_mawoh, one thing10:02
richard_mawI think you'll always get a value for tempdir with the output provided by morph10:03
richard_mawbut it may be an empty string10:03
richard_mawso you may need to amend your condition to check for that, rather than the option being missing10:03
mwilliams_ctyes, same as it was in earlier versions - use if -z TMPDIR10:04
*** zoli__ has quit IRC10:10
*** zoli__ has joined #baserock10:10
nowsterybd doesn't build well without linux-user-chroot10:11
radiofreeit doesn't build at all without it?10:11
nowster...it gives the crash in either linux-api-headers or fhs-dirs that I experienced yesterday afternoon10:12
ssam2radiofree: it has a chroot() backend via sandboxlib10:12
nowsterchange the kernel and reenable linux-user-chroot, and it's building again10:12
richard_mawSotK: You +1'd a previous version of https://gerrit.baserock.org/#/c/581/ , are you available to review the latest version?10:12
ssam2nowster: do you have a log of the error?10:12
SotKrichard_maw: sure10:12
radiofreessam2: ah, cool!10:12
nowsterssam2: from scrollback : http://paste.baserock.org/jiferuvuni10:13
richard_mawnowster, ssam2: That could possibly be a lack of a chdir after the chroot10:14
ssam2richard_maw: indeed. the code is there to do that, but maybe it's not working correctly10:14
nowsterI notice a lot of stuff left as /src/tmp/tmpXXXXX after a ybd run.10:15
ssam2as it doesn't use the `chroot` program, but rather a separate thread that calls os.chroot(), os.chdir() and then subprocess.Popen()10:15
*** zoli__ has quit IRC10:15
ssam2mason-x86-32.baserock.org seems to have lost network connectivity! not seen that before10:18
ssam2it has an IP, can't ping its own gateway.10:18
ssam2is there a way to forcefully disconnect and reconnect from the network using `networkctl` ?10:20
ssam2it's an OpenStack instance, so I can't just pull the wire out10:20
ssam2ifdown just says 'interface eth0 not configured'10:20
richard_mawssam2: use `ip`10:21
richard_mawtry bringing the interface down then restarting systemd-networkd10:21
ssam2just doing 'ip link set down dev eth0' then 'ip link set up dev eth0' seems to have fixed it, thanks for the pointer10:23
ssam2I guess we should remove ifup and ifdown from our busybox config if they don't actually work... although I suppose we only want to remove them in systems that use systemd-networkd, which can't really be expressed in definitions...10:24
ssam2maybe the systemd chunk could overwrite them with shell scripts that run 'ip link set down/up dev' :)10:25
richard_mawI'd rather not10:30
richard_mawifup/ifdown are explicitly for interfaces managed by /etc/network/interfaces10:30
richard_mawand theoretically you could still manage some interfaces with ifupdown and others with systemd-networkd10:30
*** zoli__ has joined #baserock10:37
perryli have a file of components in build-system-x86_64 listing non-deterministic files that i wish to add to WBO, but it's about 9000 lines. would it be best to just put the component names (e.g. ansible, cmake, etc) rather than a full break down of what isn't bit-for-bit? or is there a way of hiding it behind an expand tab to include everything (e.g. .pyo, .pyc, .gz etc)10:57
richard_mawI wouldn't worry about it being 9000 lines and add it to w.b.o anyway10:59
wdutchover 9000?!11:00
* richard_maw looks down on wdutch11:00
richard_mawwdutch: Use interrobangs11:00
richard_mawcompose-!-?11:00
richard_mawwdutch: ⸘OVER 9000‽11:00
ssam2perryl: I think add the raw data to wiki.baserock.org, but on its own page11:01
richard_mawwdutch: oh, and that's a tired old meme you should be ashamed of too11:01
ssam2we can worry about making it more readable as a later step, and people may be interested in the raw data anyway to do their own processing of it11:01
richard_mawwdutch: but you know, interrobangs!11:01
perrylssam2: seems reasonable, will do so!11:01
wdutchif only I had a compose key11:01
nowsterwdutch: Shift-AltGr11:02
Kinnisonnowster: that depends on the desktop environment11:02
richard_maw11:02
nowsterit's common11:02
Kinnisonit's just AltGr for me, and IIRC rjek has a real key marked 'compose' :)11:02
* rjek does.11:03
nowsterI've mapped capslock to compose on this laptop11:03
rjekMy keyboard has many surprising and unusual keys11:03
rjekIt also has a Compose LED which I can't work out how to have X enable11:03
richard_mawrjek: does it have an any key?11:03
rjekrichard_maw: It has an Again key.  Does that count?11:04
nowsterControl-Meta-CokeBottle?11:04
petefothperryl: yuo could attach a plain text file to an exoisting or new wiki page11:04
*** tiagogomes has quit IRC11:10
straycatrichard_maw, can haz review of https://gerrit.baserock.org/877 ?11:10
richard_mawough, how did I miss that!11:12
straycatprobably cause of some mad rush/deadline/cats falling out of the sky/etc11:12
richard_mawstraycat: thanks, merged11:12
*** sambishop has quit IRC11:13
*** sambishop has joined #baserock11:14
*** tiagogomes has joined #baserock11:21
*** nowster has quit IRC11:56
*** nowster has joined #baserock11:59
jmacsGetting a failure to build liberasurecode, using morph or ybd12:31
tiagogomessame12:32
jmacs"This is libtool 2.4.6, but the definition of this LT_INIT comes from libtool 2.4.2"12:33
*** CTtpollard has joined #baserock12:35
paulsherwoodssam2: are you around?12:37
tiagogomesjmacs, this should fix it https://gerrit.baserock.org/#/c/879/12:44
jmacsThanks tiagogomes, I was digging around with the libtool-tarball changes12:46
tiagogomesis the baserock roadmap on the wiki? I'm a bit lost with what is going on. Should we have a baserock blog where major changes are announced? Or make a better use of news on the wiki12:46
tiagogomese.g https://gerrit.baserock.org/#/c/872/, will this will remove the support for creating system branches?12:47
petefothtiagogomes: there's http://wiki.baserock.org/doing-stuff-with-baserock/ which references https://storyboard.baserock.org/ and http://wiki.baserock.org/wishlist/12:48
jmacsI think the mailing list is the best place for announcements like that12:50
franredtiagogomes, jmacs, 879 is merged12:51
tiagogomesI don't think that is enough12:51
SotKthese are my draft release notes, anyone have any comments on them? http://paste.baserock.org/opamenugun12:51
petefothtiagogomes: it's pretty easy to create a post that will appear in the news section on the home page12:51
jmacsI personally wouldn't read announcements in the 'news' section of a wiki12:51
tiagogomesIMO, I also think the wiki looks bad12:52
* petefoth gives up and goes to a meeting12:52
* SotK thinks major changes (such as removing support for system branches) should be announced on baserock-dev12:52
jmacsThanks franred12:53
franred+1 to that12:53
franredjmacs, no probs12:53
*** petefoth_ has joined #baserock12:56
*** petefoth has quit IRC12:56
*** petefoth_ is now known as petefoth12:56
ssam2paulsherwood: i am now13:03
straycattiagogomes, that topic is aiming to remove workspace/branch/checkout/edit/foreach/... etc13:03
straycat(the topic that change belongs to)13:03
tiagogomesso you can't `morph edit chunk` anymore?13:04
ssam2tiagogomes: I think of release notes as our main means of communication with the world. But  agree that we could do much more to communicate plans13:04
straycattiagogomes, once 872 is merged you won't be able to13:04
straycatbut we clearly need to sort out the docs and communicate the changes before we can do that13:05
ssam2indeed. we agreed to do that at the meetup we had in february, but it was a while ago and not everyone who is active here/ on baserock-dev was present13:05
ssam2it's also the first item on http://wiki.baserock.org/wishlist/13:06
ssam2but again, that document isn't widely publicised13:06
franredalso there are documentation for future changes in storyboard --> https://storyboard.baserock.org/#!/project/list13:07
ssam2SotK: 'A java-build stratum was added': I think you should point out that it still requires injecting a prebuild JDK binary at build time13:09
ssam2otherwise people might get excited that the Java work is totally done13:09
SotKmakes sense13:09
ssam2SotK: does binary stripping work in Morph in this release or not? I forget13:10
ssam2if so, we should mention that13:10
SotKoh, yes it does, good spot13:10
*** tiagogomes has quit IRC13:12
ssam2might be worth also mentioning *why* we moved the extensions: I think the reason was to decouple deployment from Morph13:12
ssam2as previously, every change in interface to any deployment extension in morph.git required a new version of the definitions format, which sucked13:13
ssam2the addition of /etc/inputrc is a big deal for me as well because finally now the delete key should work instead of being ~13:14
ssam2:)13:14
ssam2looks fine other than that13:14
franredwhere is listed the whishlist ? I couldn't find it until ssam2 has posted here13:14
franredit is not in the wiki index13:15
franredunless Im blind13:15
ssam2I don't think it's linked anywhere13:15
franredwould be nice to have it under community13:15
ssam2it was produced only by a couple of people, it's not something that reflects the wishes of everyone who contributes stuff to Baserock13:15
*** nowster has quit IRC13:15
ssam2however, I'd be happy to adopt a version of it as the official wishlist if we consulted with the baserock-dev list first13:16
*** nowster has joined #baserock13:18
franredssam2, I think that its content are in the same direction as what is reflected in storyboard and the baserock meeting. Im happy either ways: adopted or discussed in the mailing list13:18
persiaI seem to recall the wishlist being brought to baserock-dev before, but I don't think we had much discussion, other than that perhaps some of it belonged in storyboard.13:18
ssam2it was, I don't remember much feedback either13:20
ssam2but I don't think it was presented as "this is going to be our official wishlist", either13:21
persiaI believe it was presented as specifically not that: rather identifying the folk who wrote it, and suggesting that others consider these as potentially interesting.13:22
persiaI suspect that it might have worked better if any of the authors had followed up more to try to make it any sort of "official" wislist, or migrate things to storyboard.13:22
*** tiagogomes has joined #baserock13:25
SotKsecond draft of the release notes: http://paste.baserock.org/xiqisiwala13:33
dabukalampersia: have you sent anything yet?13:34
straycatoh cool we have lm-sensors now13:36
Zarastraycat: it doesn't include rrdtool if you need that13:36
straycatthat's no problem :)13:37
Zara:) cool, we didn't think we needed it at that point. may have some notes somewhere if you need it in the future, but they're probably just 'argh x depends on y which depends on z which depends on everything ever'13:38
straycatSotK, worth mentioning distbuild stuff?13:38
jmacsThis java work seems to be going round in circles13:39
*** tiagogomes has quit IRC13:39
straycatZara, ack :)13:39
franredSotK, s/arping/iputils/13:40
SotKstraycat: sure, what distbuild stuff in particular?13:40
straycatiirc perryl added a lot of new functionality, distbuild-start, distbuild-cancel, distbuild-list-jobs13:41
SotKI think they were in the previous release13:41
* SotK checks13:41
straycatoh13:41
perrylyeah, they were in 15.19 IIRC13:41
SotKyup, the release notes for 15.19 mention them13:42
Zarastraycat: tbf, we've still lorried it all; I just know that the prog/sensord stuff won't build without some extra rrdtool stuff.13:43
*** tiagogomes_ has joined #baserock13:43
persiajmacs: How do you mean?13:43
straycatahh okay, I thought we'd created 15.19.1 for those changes but I'm mistaken13:45
jmacspersia: Having built gcj, it now appears that it depends on 'ecj.jar' to run, which is a compiled binary from the eclipse project13:47
radiofreessam2: so about uploading those cache artifacts13:47
persiajmacs: My understanding was that ecj could be compiled by gcj13:48
ssam2radiofree: yeah. shall I add your ssh key to git.baserock.org so you can rsync them across?13:48
radiofreethat would be great, i'll e-mail you13:49
radiofreei don't need to upload the system image right?13:49
persiajmacs: But to be fair, I've never found a canonical location for the gcj-compilable source for ecj (although there is *a* source for this available from Debian, as part of thier bootstrap)13:49
*** radiofree has quit IRC13:53
*** radiofree has joined #baserock13:53
ssam2radiofree: we usually don't upload the system image13:54
persiaOh, heh.  This time I found it more easily: http://git.eclipse.org/c/jdt/eclipse.jdt.core.git13:55
persia3.10 (in platform 4.4) is known to build with gcj.13:55
straycatrichard_maw, does https://gerrit.baserock.org/#/c/878/ seem a reasonable approach for that? i ask because i'd like to base other changes on that change13:57
jmacsI don't see how that's possible if gcj can't produce any executable without ecj in the first place13:57
richard_mawstraycat: looking14:01
richard_mawstraycat: I'd prefer we had different steps for switching branches and adding definitions14:02
richard_mawstraycat: otherwise that looks fine to me14:02
* richard_maw goes for a late lunch14:03
persiajmacs: I think you only need ecj to compile to java bytecode.  If you are compiling natively, I think you can skip it, so you can use gcj to compile ecj natively, and then recompile gcj with the native ecj, and use that to compile a bytecode ecj, and use that to ...14:03
richard_mawmade a lot of progress on teasing apart some of the different parts of sourceresolver into distinct modules14:03
straycatokay you're referring to "GIVEN a new system... in definitions branch two" i guess14:04
straycatrichard_maw, also cool :)14:04
* SotK looks forward to the patch for that14:04
richard_mawSotK: what do you take me for‽ It'll be a bunch of patches forming a series!14:05
richard_maw14:05
SotK:D14:05
straycatheh14:05
jmacspersia: ok, might be worth a try14:08
paulsherwoodssam2: was wondering if you're thinking of proposing anything for ELCE? maybe on b4b, sandboxlib?14:09
*** nowster has quit IRC14:11
* richard_maw translates that as "ssam2: How much would you mind proposing b4b or sandboxlib as topics for ELCE"14:11
straycatI haven't really read devel-with before, it's interesting that the first step is changing into a directory that doesn't exist14:11
*** nowster has joined #baserock14:11
paulsherwoodrichard_maw: why do you translate it as that?14:13
rjekIt's how I read it :)14:13
paulsherwoodi'm asking, that's all. if i wanted to push the outcome i wouldn't do it in public on #baserock14:14
ssam2paulsherwood: I've decided against it, really14:14
ssam2partly due to lack of time before deadline, partly because i've never actually been to elce and don't really have an idea what it's like14:14
paulsherwoodah, ok.14:15
paulsherwoodalso maybe not enough to talk about yet?14:15
ssam2indeed14:15
paulsherwoodssam2: if one of my submissions is accepted, i may mention the work in progress14:16
ssam2cool14:17
ssam2hopefully by October there'll be more progress, too14:17
paulsherwoodssam2: any ideas on how useful sandboxlib, b4b might be for automotive? i'm involved in some prep for genivi talks there too14:18
ssam2hopefully by Friday there will be more progress :)14:18
* paulsherwood crosses fingers, toes etc14:18
ssam2i'm not sure how useful sandboxlib is outside the realm of build systems14:18
ssam2if you were trying to sandbox a full app, xdg-app is much more appropriate14:19
paulsherwoods/build systems/integration systems/14:19
ssam2ok :)14:19
ssam2bit for bit reproducibility is interesting for everyone14:19
ssam2well, mostly in the open source world I guess, where trusting build output is an issue14:20
paulsherwooddo you disagree with the distinction? i think autotools, cmake are build systems?14:20
ssam2i don't mind the distinction, but I think it's a bit more complex14:20
KinnisonI think autotools and cmake are build tooling14:20
KinnisonI think they operate on build systems14:20
KinnisonBut i appreciate that the terminology is a bit fuzzy14:21
rjekmake is a language in which one writes build systems, a Makefile is a build system?14:22
ssam2paulsherwood: is there anything else around right now that you'd call an 'integration system' ?14:22
rjekapt? :)14:22
rjek(or yum, etc)14:22
paulsherwoodno, they are packaging tools14:22
paulsherwood(imo)14:22
rjekThat would be dpkg or rpm14:23
rjekapt and yum take a requirement and calculate how to achieve it before going ahead and doing it14:23
paulsherwoodok package installers, then?14:23
rjekIsn't ybd/morph just a build system and chunk installer?14:24
paulsherwoodno - they can deploy sets of whole systems14:24
Kinnisonapt and yum are more about dependency resolution and management of recommendations/suggestions, finding and preparing package transactions etc.14:24
Kinnisondpkg and rpm actually handle installing things14:24
rjekdebootstrap, for example, can be given a list of pieces of software (and which versions you want, and where to get them from) and integrate them into a working system image14:24
Kinnisondebootstrap or FAI is probably as close to an 'integration system' for debian-type stuff14:25
paulsherwoodrjek: ok, then debootstrap does some of what i'm calling an integration system14:25
rjekapt and yum only really differ because they mutate an extant system14:25
paulsherwoodbut actually i think 'integration tool' may be better, since system has other connotations14:25
ssam2perhaps the comparison should be more with the Debian build servers14:26
ssam2they only partially integrate things, and leave the rest up to the package managers14:26
ssam2package management tooling, I mean14:26
paulsherwoodradiofree: how are you getting on with the GDP stuff? arthur seems to be struggling with the yocto instructions14:28
straycatrichard_maw, ping14:30
*** mdunford has quit IRC14:31
paulsherwoodssam2: i notice on http://wiki.baserock.org/projects/deterministic-builds/ we've said libfaketime is a bad idea for this. is there no way we could fix libfaketime to make it a good idea?14:31
radiofreepaulsherwood: uploading the artifacts to gbo now14:32
radiofreeoh god x8614:32
paulsherwoodradiofree: you're a star :)14:32
radiofreex86 won't be ready today14:32
ssam2paulsherwood: will it ever be a good idea to hack things into working vs. collaborating with upstreams?14:32
radiofreewell, the definitions will be, it just won't be cache14:32
radiofreed14:32
paulsherwoodssam2: i was thinking about collaborating with libfaketime upstream14:32
ssam2paulsherwood: we could work around the issue with busybox by making faketime ignore certain binaries, perhaps. So it'd then let 'make' continue to work14:33
paulsherwoodit's unlikely we can collaborate with all the upstreams to cause them to make their stuff b4b. there's no clear benefit for them?14:33
ssam2maybe not on our own14:33
petefothstraycat: if you follow the steps in http://wiki.baserock.org/quick-start/, specifically the 'Configure Morph' section, then the direcroy *should* exist14:33
ssam2there are ~60 other people in #debian-reproducible14:33
petefoth*directory14:33
paulsherwoodnot even with an army. most users don't require it?14:33
ssam2paulsherwood: Debian is up to 80% of packages reproducible without use of libfaketime.14:34
ssam2it really seems like barking up the wrong tree14:34
paulsherwoodby reproducible you mean b4b?14:35
ssam2yeah14:35
ssam2with a patched toolchain, build system etc14:35
paulsherwoodoh, well you're right and i'm wrong then, simple as that :)14:35
* paulsherwood would love to be right about all the things14:35
*** petefoth has quit IRC14:36
* persia reads backscroll, and has different opinons about Debian tool categorisation.14:36
paulsherwoodpersia: please elaborate?14:37
persiadpkg-buildpackage (not part of dpkg) is Debian's build sytem.  Arguably, sbuild is more properly that, as the build servers use sbuild.14:37
persialive-build is Debian's integration system.14:37
Zarapetefoth: I'm working on making that aspect of wiki-navigation clearer, because I don't think the current order of the steps is obvious unless you happen to start on the right page. (though it's certainly better than it used to be)14:37
persiaThat said, I agree with ssam2 that due to the nature of package management, many of the details of operation of these tools are encoded in ways that are not immediatly obvious in the configuration of these tools.14:38
* SotK wonders what tiagogomes_ meant earlier about the wiki looking bad14:38
straycatZara, are you currently editing devel-with.mdwn?14:39
Zarastraycat: no14:39
straycatcool14:39
Zara:)14:39
straycatcause plain text conflicts :(14:39
radiofreeaaah14:40
radiofreeignore that14:41
paulsherwoodpersia: so, i'm interested in causing our terminology to become consistent here14:42
persiapaulsherwood: I don't believe there *can* be consistency between Debian terminology and Baserock terminology.14:42
paulsherwoodin both ybd and morph, the code refers to 'build-systems', and those are autotools, cmake etc.14:42
jmacsSo ecjsrc-3.8.2 requires java 1.6 to build, but gcj only provides 1.5; this is a problem I had last week with openjdk.14:43
paulsherwoodpersia: fair enough. but i'd like consistency in baserock if f possible - then we could map to others' terminologies?14:43
persiaThe main issue being that the tools that one uses in Debian to deal with defining systems, etc. are often not generalised, because almost nobody uses them.  As rjek pointed out, the vast majority of folk use debootstrap to create a rootfs.14:43
persiaThe secondary issue being that the nature of packages and chunks is very intentionally semantically distinct, to the degree that by comparison to chunks, all package systems have broadly similar semantics (whereas when looking at details, there are heaps of arguments about the differences in semantics between .ebuilds, .debs, .rpms, etc.)14:44
paulsherwoodso, back to baserock - can/should we agree that 'build system' means autotools and co, or would there be a better term?14:45
persiajmacs: Aha.  Hrm.  doko worked through it once, years ago, and since has been working off that bootstrap.  I can do more archeology if we need.14:45
persiaIt may be that we need to build ancient versions of things, and then go through more bootstrap cycles than we like.14:45
jmacsI've been doing archeology for weeks now, hence feeling like going in circles. It doesn't seem like a problem anyone else really cares about.14:46
*** mdunford has joined #baserock14:46
persiajmacs: I think there are about three other people in the world who care, but yeah, they all already solved it, or have an environment where they can do a dirty bootstrap once, and then be clean thereafter.14:47
paulsherwoodjmacs: that's interesting in itself, i think. go decides to bootstrap on go.  why do we care, when no-one else seems to?14:47
robtaylorjmacs: boostrapping java?14:48
persiaI thought Debian also cared about bootstrapping go.14:48
jmacsrobtaylor: Yes14:48
robtaylorjmacs: http://layers.openembedded.org/layerindex/branch/master/layer/meta-java/14:48
jmacsI think I would need to know yocto for that to be any use14:50
* radiofree seems to spend his life rebuilding from stage1-gcc14:52
robtaylorjmacs: i can talk you through it if you'd like14:52
paulsherwoodpersia: ok, us and some folks in debian and a few other folks... let me re-phrase my question..14:52
jmacsrobtaylor: Yes please14:52
paulsherwoodif the few folks who care about bootstrapping stop caring about it... what will break?14:53
paulsherwoodradiofree: poor you!14:54
jmacsAlthough maybe just the fact that it uses ecjsrc-3.6.2 might be enough14:55
persiapaulsherwood: If we can't bootstrap, then we depend on there being a working system.  This is already true, in that we don't have a means to bootstrap from pure source today.14:55
paulsherwoodok, but it's unlikely there will ever not be, for example, a working debian system?14:56
persiaThere are projects going on to resolve that: I think the combination of 0pf and aboriginal is probably the best start: this ought give us enough that we can then bootstrap Baserock or Debian.14:56
ssam2it would mean we have to deal wit the fact that different Baserock systems have different requirements to build them from scratch. Mostly important for people who release reference systems and who cross-bootstrap to new architectures. Whether that adds up to more or less work than working out how to bootstrap Java from a C compiler, I have no idea14:56
richard_mawstraycat: pong?14:57
* paulsherwood googles 0pf14:57
persiaDepends.  The classic thought experiment is someone trapped on a desert island with a CD containing lots of source code, and a computer with no software loaded.14:57
straycatrichard_maw, hey yeah i wasn't sure exactly what change you wanted me to make to the yarn14:57
richard_mawoh, right14:57
richard_mawit was the "GIVEN    a new system to build called expanded-system in new definitions branch two" I preferred to be different14:58
richard_mawwe should have a different IMPLEMENTS for creating and changing branch, and adding the new system14:58
straycathrm, well checking out isn't required here since anchor doesn't actually need a definitions repo, we just keep it around to verify that all the shas got pushed15:00
straycatrichard_maw, nonetheless i'm happy to make that change15:13
richard_mawthanks straycat15:24
*** mdunford has quit IRC15:28
nowsterpersia: is the CD bootable?15:33
rjekIt uses http://bellard.org/tcc/tccboot.html15:34
SotKlast call for opinions on the release notes: http://paste.baserock.org/elovopemes15:35
KinnisonSotK: erm15:36
Kinnison* Updates to Morph, Baserock's build tool, including:15:36
Kinnison    - Support for definitions version 4 (allow symlinks to be15:36
Kinnison      created in install-files manifests)15:36
rjekHave we updated OpenSSL?15:36
rjek(Since the 12th of June)15:36
KinnisonSotK: aren't those in definition now, not morph?15:36
KinnisonSotK: or is this why you're doing the release?15:37
persianowster: Depends on who is telling the story.  Sometimes, sometimes not.  When not, there is usually some other reader device that can extract the text.15:38
rjek(Latest OpenSSL fixes a downgrading attack)15:38
rjekpersia: I would say that somebody stranded on a desert island probably has better things to do than build the copy of Tuxcart on the CD :)15:38
SotKthe manifests have always been in them, but when we added the ability to create symlinks in them by modifying install-files we had to bump the definitions version, as old morph would break if it tried to use a new manifest15:38
persiaNewer SSL also forces anyone who has been using SSL for a long time to upgrade their certificates, as it gets overexcited by false positives.15:38
SotKwe haven't done a release since we implemented that feature, so don't actually use it in definitions master at the moment afaik15:39
persiarjek: Presume they have already undertaken all activities in the Mysterious Island, with the plot twist that Nero's cavern does not exist.15:39
rjek:)15:39
SotKthe content of the brackets is just to explain what the "version 4" we support is for15:39
KinnisonSotK: my point was more -- aren't configuration extensions now in definitions and *not* in morph?15:40
persiaKinnison: Isn't that a version 5 thing, so that version 4 still has meaning for install-files in morph?15:40
Kinnisonaah, if that's v5 then fine15:40
SotKKinnison: yes, but I don't see how that affects the meaning of the "version 4" morph supports?15:40
* Kinnison finds it very hard to keep track of those things, even with reference to the wikipage15:41
* persia rereads the release notes15:41
SotKthe extensions in definitions thing didn't need a version bump as it wasn't an incompatible change, it just meant that future minor changes to them also wouldn't cause compatibility issues15:41
persiaAh, no.  Apparently deployment extensions may be in definitions *OR* morph from version 3, and I don't see any statement that they must be in definitions from some given version.15:41
persiaRelated to the topic of the version increases: I had an interesting discussion today with someone about the long-term viability of build systems.15:43
persiaDuring this discussion, a team was mentioned that had given up on their build system because they were unable to backport changes for new upstream versions to their old description format.15:44
persiaAnd they were unable to upgrade their tool, because so much of their work was in the old description format, making this onerous.15:44
persiaWhile it seems that morph is supporting multiple definitoins versions, we should keep this story in mind when we consider how fast to change the format.15:45
*** mdunford has joined #baserock15:47
SotKI'd like opinions on how to word the "Support for definitions version 4 (...)" section of the release notes, anyone have any advice?15:49
SotKKinnison pointed out that since the new behaviour (symlinks in manifests) isn't conditional on the version of the definitions file, its a feature change in morph rather than a version of the definitions format, the version bump happened because we've been using the version to protect against using old morph to build new definitions15:50
KinnisonWhich is not what the version in definitions was meant to represent15:51
SotKit would be fine to put symlinks in manifests and leave the definitions version as 3 as far as new morph is concerned15:51
* SotK has thought about having a list of "required features" or similar in definitions somewhere for changes like this15:52
SotKrjek: I don't think we've updated OpenSSL, certainly I didn't notice it in the git log between my release branch and 15.1915:55
*** jonathanmaw has quit IRC15:56
franreddefinitions says we are using OpenSSL_1_0_1m which is 3 months old15:57
straycathi, i've got a draft for an update of the devel-with page can anyone help me proofread it? https://bitbucket.org/richardipsum/baserock-wiki/raw/45e3bbcfd42fb4ec48a577bacdab038f6c47a543/devel-with.mdwn15:57
straycatssam2, Zara ^15:57
ssam2one minute, need to read release notes first15:58
* straycat will also look at those15:58
ssam2SotK: it's not 'support for symlinks' that is the feature, it's not really a feature addition at all15:58
* SotK gets even more confused15:58
ssam2the sole purpose of version 4 is to handle this situation: old Morph tries to build definitions that have a manifest file that overwrites a symlink15:59
ssam2if we merged a patch to definitions.git that caused install-files to install a symlink over the top of an existing file, old morph would crash16:00
ssam2that will make people trying to build our new definitions with old Morph very sad and confused16:00
ssam2but if we increment the version of definitions to version 4 before merging that change, they will get an error saying "Your Morph is too old", instead of a crash16:00
ssam2so hopefully less sad16:00
Zarastraycat: looking now. no workspaces anymore? (was aware it was on the cards but didn't realise it was already the case)16:00
ssam2SotK: I'm not sure how to express that in the release notes16:01
SotKZara: they can still be used at the moment, but are being phased out16:01
SotKssam2: neither am I :(16:01
straycatZara, it's already the case, but you can still use workspaces if you choose, though we're clearly looking to remove them altogether16:01
Zaraah, okay, if they're not neccessary then that's fine.16:02
ssam2how about "Support for version 4, which is used as a marker to prevent versions of Morph that do not contain the fix from http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/commit/?h=93b034f045a2c7443122d7b082ad4460339907d0 from crashing"16:02
SotKsounds fine to me16:03
ssam2or maybe "to cause version of Morph ..... to exit with a version error, instead of crashing"16:03
ssam2the way I worded it there makes it sound like it actually fixes the crash, but it just avoids it16:03
ssam2with a different error16:03
Zaraon 'deploy a raw disk image', I'd like it if 'create a cluster morphology' could be accompanied with a translation of: ('ie: make a file named foo.morph, containing the following, and put it in the clusters directory of the definitions repository)'16:04
ZaraI know it doesn't have to be there16:04
ssam2straycat: git clone git@git.baserock.org:baserock/baserock/definitions won't work I think16:04
ssam2unless the reader happens to have SSH access to g.b.o16:05
ssam2git://git.baserock.org/.. would be better16:05
straycatoops16:05
Zarabut as a newcomer a phrase like 'create a cluster morphology' would confuse me, because I'd assume it was way more complicated than it is16:05
straycatyes it was quite stupid of me to use the ssh url, thanks16:05
ssam2I agree it would be nice to avoid using the term 'cluster morphology', but that's kind of a separate task16:06
ssam2I prefer 'deployment .morph file', but opinions vary :)16:06
ZaraI'm just trying to spot things that have changed from the old page; I put a little translation up on the old one so I'd like an equivalent. :P16:08
ssam2the new 'add something to Baserock' raises the question of what to do if it's not something present in git.baserock.org16:08
* bashrc thinks of cluster computing16:08
ssam2mentioning how to build from file:/// URLs would be good as well, although maybe it doesn't need to be on that page16:09
SotKany more comments on the release notes? http://paste.baserock.org/ijulogufav16:09
ssam2beyond that, looks fine at a glance. thanks for doing it16:09
franredssam2, also you can build from any git repository16:09
straycatyeah i'm not sure, i mentioned aliases but didn't really explain it further, should probably link to somewhere that explains the repo aliases in more detail16:09
ssam2Sotk: 'Support for version 4' -> 'Support for definitions version 4'16:10
ssam2SotK: i haven't spotted anything else16:10
SotKoops, good spot16:10
SotKI'll send them in 5 minutes unless there are further objections from anyone then :)16:11
straycatthey look fine to me16:11
ssam2What are people's thoughts on having rough-n-ready versioned documentation on wiki.baserock.org ?16:11
ssam2by which I mean, move some subset of the pages into a 15.19/ folder, so people using 15.19 can continue to refer to them16:12
* straycat was thinking that might be a good idea16:12
* SotK would be fine with that16:12
ssam2cool. I'm not sure which pages matter yet, will have a look16:12
straycatat the very least I was thinking we could copy the existing devel-with page and link to it from the new one16:12
jmacsI'm mildly concerned that that will break search even more16:13
straycatbut doing it for a number of pages would probably be better, especially since i imagine i'll also need to update at least quick-start16:13
Zarahow come the informal definitions spec and the build-failures links have been removed?16:13
ssam2jmacs: hmm, very good point16:13
ssam2jmacs: another alternative is move them somewhere else16:14
SotKIt would be cool if branchable had a way to view different branches of the repo16:14
SotKlike MUSTARD had16:14
straycatyes it would16:14
ssam2what about setting up a separate branchable wiki for 'stable' docs?16:18
ssam2have spoke to our sponsors who are OK with doing this16:18
SotK+1 from me16:18
jmacsIf it were me making the changes I'd just write separate instructions for older versions on the same page, as we do now16:19
ssam2my thinking is that we have a customer using docs from the wiki16:19
Zarajmacs: I think the pages could get long fast.16:19
Zara+1 for stable docs16:19
SotKjmacs: I worry that would get very confusing after a while16:20
* SotK sends the release notes16:20
jmacsI can only think of OpenStack that does that; most other projects don't need such complicated documentation16:20
straycatI also think having 2 versions for every use case will be pretty cluttered16:20
jmacsstraycat: You will always have 2 version for every use case16:21
ssam2with a separate wiki, the old versions can be left alone pretty safely16:22
ssam2if it's in the same wiki there'll be an ongoing cognitive load to maintain the two separate versions, even though 1 version doesn't actually need to change16:22
franredor we can create a file with old documentation and attach to the wiki as a link?16:23
straycatI think I'm in favour of a separate wiki, I'm not sure I understand exactly the setup ssam2 has in mind for this though16:23
jmacsI might agree to a frozen copy of the wiki for old version16:23
jmacsversions16:23
jmacsAs long as it's very obvious in the instructions that the people may need to check archived versions16:23
straycatI guess, just a 15.25 wiki and a stable wiki?16:24
Zaracould make a wiki for each release, but that's prrrrrobably overkill.16:24
Zara(if you preserved them all, anyway :P)16:24
franreds/wiki/document/16:24
*** petefoth has joined #baserock16:25
Zarais there a way of seeing the whole wiki as it was in some point in time?16:27
*** a1exhughe5 has quit IRC16:28
ssam2I was imagining a separate wiki called baserock-stable.branchable.com with the whole contents of the current wiki copied to it, to begin with16:28
Zaraif so, then maybe we could just have different versions point to different times.16:28
jmacsOther than cloning it with git, checking out an old revision and reading the .mdwn, not that I can think of16:28
mwilliams_ctDjango do that nicely - https://docs.djangoproject.com/en/1.7/intro/tutorial01/ is an example, in the bottom right you can choose the version you're working with16:28
ssam2later (or hopefully, at the same time) we could get cleverer and (a) remove stuff that it doesn't make sense to archive (stuff that isn't documentation) and (b) add version directories16:28
SotKmwilliams_ct: that is similar to what I was thinking of when mentioning what MUSTARD had16:29
ssam2mwilliams_ct: I think Django do this by generating all their documentation using Sphinx from files in the django.git repo, then rendering it all as HTML at release time and uploading it to docs.djangoproject.com in a new folder16:29
ssam2and it'd be great if someone fancies doing that for the Baserock documentation, but I'm not going to volunteer16:30
mwilliams_ctssam2: sure, didn't mean to suggest it was easy or even appropriate, but thought a quick look at how another project do it might be handy16:30
ssam2the way they do it is good, indeed16:30
mwilliams_ctmost importantly, big difference between a wiki and static html documentation16:31
straycatIt's cool but I don't think there's an immediate need for it, this is arguably the largest ui change we've had, and I don't *think* we anticipate any similar large ui changes?16:31
jmacsmwilliams_ct: No 'edit' button? :)16:31
straycatat least not in the near future16:31
* SotK updates the wiki, and with that...16:31
SotKBaserock 15.25 is released!16:32
ssam2hooray!16:32
jmacs\o/16:32
perryl\o/16:32
ssam2i'll mail baserock-dev about this wiki thing, in fact16:32
straycatawesome :)16:32
straycat(both the release and the email about the wiki)16:32
franredcongrats SotK !!16:32
straycatso much awesome :)16:32
Zarawould it be possible to clone it, tag retrospectively and rebase, then use some mechanism to let a reader view all pages on a specific git tag?16:33
SotKI don't know if branchable can do that16:33
Zaraseems like it would be a simple (albeit probably a bit inaccurate) way to do it if it can.16:37
* SotK runs away for the night before anyone finds problems in the release :)16:37
Zaraheh, 'night16:37
straycato/16:37
jmacsgcj seems to be asking for ecj1 no matter how I invoke it16:38
*** rdale has joined #baserock16:43
*** dabukalam_ has joined #baserock16:44
*** rdale_ has quit IRC16:45
*** dabukalam has quit IRC16:45
ssam2Zara: I don't think branchable has a way to view anything other than 'master'16:48
ssam2otherwise, we probably could do something like that, instead of having a separate wiki16:48
Zarawah :( how hard and/or time-consuming would it be to give branchable that functionality?16:49
ssam2branchable is a commercial company, so it might not even be possible16:51
ssam2they are quite friendy, so maybe it would be :)16:51
ssam2and it is based around ikiwiki, which is open source16:52
ssam2but also written in Perl16:52
Zaraah, okay. I started looking at ikiwiki plugins, in case there was something useful there, but there are a lot... http://ikiwiki.info/plugins/16:53
Zaraso might look in more detail tomorrow16:53
ssam2it may be that Kinnison or cttpollard arrive tomorrow and point out there is a much easier way of doing this :)16:56
Zaraheh :)16:57
*** mariaderidder has quit IRC17:01
*** bashrc has quit IRC17:02
*** ssam2 has quit IRC17:10
CTtpollardZara: what exactly are you looking at trying to do17:13
robtaylorjmacs: http://archive.eclipse.org/eclipse/downloads/drops/R-3.6.2-201102101200/ecj-3.6.2.jar17:16
persiarobtaylor: That doesn't solve the problem: it's still a blob.17:22
persiaI was hoping that there was a way to compile gcj that didn't need ecj to build ecj-native: such a gcj would be *incapable* of producing Java bytecode.17:23
*** mdunford has quit IRC17:23
persiaAnd then using that to stumble forward.17:23
persiaOne of the things that is different in Baserock than package-based environments is that we want to be able to bootstrap from raw source from an arbitrary system, whereas in a package environment, once you upload the dirty object, you can use that to build successively cleaner ones until you are comfy.17:24
persiaIn Baserock, you have to presume the cache was wiped, so there are *no* blobs available.17:24
robtaylorpersia: not going to happen, i'm afraid17:24
robtaylorpersia: we'll have to mirror a bottom-of-the-stack jar or two into17:25
robtaylor- there is no javac you can build with c alone17:25
persiaIn that case, perhaps the right answer is to do something like I did for fpc (which is impossible to bootstrap with open code), where we have instructions to perform the bootstrap committed to git, so that the human doesn't need to do anything.17:25
robtaylorpersia: no human needed, we'll just mirror the jar(s)17:26
robtaylorand then rebuild those components from current source later down the build tree17:26
persiaTreat the blobs as source?17:27
robtaylorpersia: yep17:27
robtaylorotherwise you'd be trying to http get during build time..17:27
robtaylor- note it's all open, so not quite the same situation as fpc17:28
persiaThe way I did it for fpc was to have code that performed the mirror be committed to git, so that someone bootstrapping it could say "do the bootstrap" and work around things.17:28
persiaI suppose stuffing the blob into git isn't much worse than that, but somehow it seems wrong.17:28
robtaylorwell ideally we'd have lorry do it17:29
robtaylorbut its somewhat unecessary as it'd never change (e.g. itd always be ecj-3.6.2.jar with md5sum ....)17:30
persiafpc is now, and has always been, open source.  It self-hosts fine.  It just can't be built without either a blob or by building an ancient copy with an out-of-print ancient proprietary solution.17:30
persiatarball lorries don't change anyway.  Nothing like debian/watch exists (to the best of my knowledge)17:30
robtaylorgood point17:30
robtaylorcan lorry lorry just a file and not try to extract it?17:31
persiaI believe that would need another lorry type, but it ought be trivial to enable a "file" lorry type, based on tarball, but with 99% of the logic removed.17:31
*** edcragg has quit IRC17:39
*** bwh has quit IRC18:00
*** benbrown_ has joined #baserock18:22
straycathello benbrown_18:22
* benbrown_ has a tail.18:23
straycat:)18:24
*** paulsherwood has quit IRC18:31
*** perryl has quit IRC18:31
*** SotK has quit IRC18:31
*** Zara has quit IRC18:31
*** petefoth1ringham has quit IRC18:31
*** benbrown_ has quit IRC18:31
*** mwilliams_ct has quit IRC18:31
*** pdar has quit IRC18:31
*** jmacs has quit IRC18:31
*** kejiahu_ has quit IRC18:31
*** DavePage has quit IRC18:31
*** sebh has quit IRC18:31
*** bwh has joined #baserock18:32
*** petefotheringham has joined #baserock18:32
*** sebh has joined #baserock18:33
*** gary_perkins has quit IRC18:34
*** gary_perkins has joined #baserock18:34
*** lachlanmackenzie has quit IRC18:35
*** gary_perkins has quit IRC18:37
*** gary_perkins has joined #baserock18:38
*** lachlanmackenzie has joined #baserock18:41
*** lachlanmackenzie has quit IRC18:47
*** bwh has quit IRC18:47
*** zoli__ has quit IRC19:14
*** gary_perkins has quit IRC20:13
*** zoli__ has joined #baserock20:16
*** edcragg has joined #baserock20:19
*** zoli__ has quit IRC20:56
*** zoli__ has joined #baserock21:00
* richard_maw grumps21:36
richard_mawMy https://gerrit.baserock.org/#/q/status:open+project:baserock/baserock/morph+branch:master+topic:sourceresolver-objects rework looks great, except for the fact that the memoise is the wrong idea, since I need to cache by something derived from the input, rather than the input itself21:37
* richard_maw pre-emptively -1s the offending change21:38
*** mwilliams_ct has joined #baserock21:44
*** edcragg has quit IRC22:02
*** zoli__ has quit IRC22:20
*** zoli__ has joined #baserock22:33
*** edcragg has joined #baserock23:15
*** edcragg has quit IRC23:31

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!