IRC logs for #baserock for Wednesday, 2015-01-28

*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Remote host closed the connection]00:11
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock03:12
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Ping timeout: 256 seconds]03:16
*** petefoth_ [~petefoth@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock04:03
*** petefoth [~petefoth@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 265 seconds]04:04
petefoth_ is now known as petefoth04:04
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock04:36
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Remote host closed the connection]04:49
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock07:50
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Ping timeout: 265 seconds]07:54
*** sherm_ [~mike@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock07:58
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock08:08
perrylpersia: is there no 'hide joins/parts' function for quassel?08:21
*** grahamfinney [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has joined #baserock08:25
*** sambishop [~sambishop@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:26
*** tiagogomes_ [~tiagogome@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:42
*** mariaderidder [~maria@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock08:42
*** rdale_ [~quassel@81.Red-79-146-151.dynamicIP.rima-tde.net] has joined #baserock08:44
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Remote host closed the connection]09:02
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock09:02
*** bashrc_ [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:06
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Ping timeout: 246 seconds]09:06
*** CTtpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:11
rjekCan somebody confirm to me that we're using glibc 2.20?  The morphs are a twisty maze of many glibc references and I can't work it out.09:20
KinnisonLast time I checked, we were using 2.2009:21
tiagogomes_ldd (GNU libc) 2.2009:26
*** zoli__ [~zoli_@cpe.ge-0-2-0-366.abnqu1.dk.customer.tdc.net] has joined #baserock09:35
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:40
Mode #baserock +v ssam2 by ChanServ09:40
tiagogomes_anyone has ideas about the best way of updating config.guess and config.sub if they are not updated upstream? (a) download the files from http://git.savannah.gnu.org/gitweb/?p=config; (b) run `autoreconf -ivf` on the repositoy and commit; (c) add `autoreconf -ivf` to the pre-configure-commands of the morphology09:52
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock09:53
* tiagogomes_ favors (b)09:56
KinnisonIf (c) were possible, (c) is preferable.  Given you're clearly running from preconfigured tarball imports, I'd do (b)09:57
*** zoli__ [~zoli_@cpe.ge-0-2-0-366.abnqu1.dk.customer.tdc.net] has quit [Remote host closed the connection]09:57
ssam2agreed09:57
KinnisonI'd also be tempted to email the upstream project and ask them if they might do a new release with updated autoconfery09:58
tiagogomes_Kinnison, why (c) is not possible?09:59
tiagogomes_(c) is dangerous, as if you update automake, things could break when running the pre-configured-commands10:00
KinnisonPresumably this is too early in the bootstrapping phase10:00
Kinnisonfor autoconf, automake, libtool etc to be available10:00
tiagogomes_righto10:01
bashrc_why isn't there any manpage for the morph command?10:02
KinnisonMost of morph's help is built into it10:03
Kinnisonbecause it's dynamic10:04
*** Krin [~mikesmith@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:05
persiaperryl: The problem is that when I'm on both sides of a netsplit, I have to keep track of which others are in which segments.  In my ideal world, my client follows the splits so that I have a consistent datastream from all segments, and am posting to all segments.10:05
ssam2bashrc_: there is one10:06
straycatwhat's the deal with cliapp in baserock? we seem to have an unofficial fork?10:07
persiaIs the fork required for morph?10:08
bashrc_maybe morph isn't installed10:08
perryla number of morph plugins use cliapp functions10:08
persiaI know that we've forked some of the key support components for morph, lorry, etc.  In some cases, even when we've landed things upstream, we haven't gone to an upstream branch in order to preserve history.10:08
ssam2straycat: i believe richard_maw has talked to lars a bit about our fork of cliapp, and at the time, lars didn't have time to do any kind of maintenance work on cliapp10:09
ssam2there are quite a few differences, it seems, it would be good to resolve them10:09
straycatokay, just asking because richard_maw fixed a bug I found yesterday and the fix has been sent upstream, so we'll want that10:10
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 252 seconds]10:11
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:12
*** grahamfinney [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has quit [Quit: Ex-Chat]10:14
*** grahamfinney [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has joined #baserock10:14
*** lachlanmackenzie [~lachlan@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:15
*** zoli__ [~zoli_@cpe.ge-0-2-0-366.abnqu1.dk.customer.tdc.net] has joined #baserock10:19
straycatssam2, i it's just this commit by doffm really, since the rest is chunk morph stuff we can remove10:19
pedroalvarezthat commit was to improve the `help` in morph IIRC10:20
tiagogomes_`build-depends: []` means that the chunk doesn't depend on anything on the strata were it was defined right10:25
ssam2tiagogomes_: yes10:26
* tiagogomes_ considers moving sqlite3 down on core10:26
ssam2since it's already in core, changing the order is fine if something else depends on it10:29
tiagogomes_ssam2 I was thinking in moving it to do autoreconf as part of pre-configure-commands, not minimize the delta agains upstream. But I see now that's a tarball, so I don't think it matters10:30
tiagogomes_s/not minimize/to minimize10:30
ssam2none of our systems at baserock.org are vulnerable to http://www.openwall.com/lists/oss-security/2015/01/27/910:34
Kinnisonphew10:34
pedroalvarezup-to-date systems ftw!10:35
paulsherwoodw00t!!!!!10:36
paulsherwood'a working security system  is indistinguishable from blind luck' :)10:36
* paulsherwood wonders by what magic gcc in autotools gets built with /usr/bin in its path10:38
paulsherwoods/autotools/build-essential/10:38
*** locallycompact [~lc@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:39
ssam2paulsherwood: there is a little cleverness going on, there's a special 'prefix' field for each chunk in a stratum10:40
ssam2'prefix' is /usr/bin by default10:40
ssam2sorry, 'prefix' is /usr by default10:40
paulsherwoodssam2: i can see the special prefix set to '/tools' but nothing else.10:40
paulsherwood(in definitions)10:40
ssam2yeah, if 'prefix' is absent then it is assumed to be /usr10:40
ssam2to avoid having to write 'prefix=/usr' for every single chunk10:41
ssam2then, this: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/tree/morphlib/buildcommand.py#n34010:41
paulsherwoodso that's one of many magic assumptions hidden in morph, i think. 10:41
ssam2i'd call it a 'default value' rather than a 'magic assumption'10:42
ssam2but I agree it could be documented better10:42
ssam2or, at all. :(10:42
paulsherwooddefault values could be in definitions too, easily enough. hiding this in code is unhelpful10:43
paulsherwood(imo)10:43
ssam2yeah, it'd be better if that could be controlled from definitions, I think10:43
persiaI'd agree.  Some systems may not think /usr/ is a sensible path.10:43
* Kinnison thinks defaults should be in definitions, but so should the build systems10:43
paulsherwoodagreed. 10:44
KinnisonThis would be another move toward as much as possible being declarative10:44
paulsherwoodyup.10:44
KinnisonPerhaps a topic worth discussion next week?10:44
paulsherwoodyes, absolutely :)10:45
paulsherwoodincidentally, ybd now builds up to gcc, hence my question. i have not verified it builds the same things, exactly, but it's a start :)10:45
*** grahamfinney [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has quit [Quit: Ex-Chat]10:45
ssam2cool!10:46
*** grahamfinney [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has joined #baserock10:46
*** grahamfinney_ [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has joined #baserock10:46
paulsherwoodi hope so. on this journey i've discovered another reason to be on it - ie a possibility to provide an independent view on morph's assumptions10:47
*** petefoth [~petefoth@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: petefoth]10:48
ssam2good for ensuring the definitions format is solid and well-defined, certainly10:48
KinnisonIndeed10:49
paulsherwoodssam2: i'd already seen that in buildcommand.py - but at gcc, all prefix can come back as is '/tools', so /usr must be slipped in somewhere else10:49
ssam2it includes its own prefix, I think10:49
KinnisonIdeally we'd be able to have a multiplicity of tools all able to work from definitions and all producing the same stuff10:49
paulsherwood+110:49
* Kinnison has been tempted to write some haskell to represent morphologies in an attempt to see if he can come up with an algebra of chunks10:49
* Kinnison is odd though10:50
paulsherwoodssam2: possibly, but i'm trying to understand what sets PATH in this way10:50
paulsherwood-PATH=/tools/bin:/usr/bin:/usr/lib/ccache:/sbin:/usr/sbin:/bin:/usr/bin10:50
paulsherwood+PATH=/tools/bin:/usr/lib/ccache:/sbin:/usr/sbin:/bin:/usr/bin10:50
paulsherwoodnot to worry, i'll track it down :)10:51
straycatrichard_maw, Did we submit doffm's patch to liw?10:53
paulsherwoodKinnison: that would be quite exciting. we are exploring an R&D project which would be interested in that thinking :-)10:53
Kinnisonpaulsherwood: I'd love to chat with someone who understands FP more than I do about it10:54
* Kinnison is a dabbler in it10:54
paulsherwood:)10:54
* paulsherwood is a dabbler in general10:55
richard_mawstraycat: yes, liw couldn't accept it because it changed things in a not backwards compatible way. IIRC he said he'd look at a better way to do it, but that was back when he had time to look at it.10:56
*** petefoth [~petefoth@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock10:56
straycatright10:57
straycati guess we can wait till he merges the fix then merge master into baserock/morph?11:00
richard_mawthat's what I'd do11:12
bashrc_is there a way to determine if an image is devel or not?11:14
*** zoli__ [~zoli_@cpe.ge-0-2-0-366.abnqu1.dk.customer.tdc.net] has quit [Ping timeout: 245 seconds]11:15
ssam2bashrc_: ls /baserock/*-system-*.meta11:18
ssam2will give you the name (and much more info) of the system artifact11:18
bashrc_thanks11:18
nowsteris it a bug in me or morph that you get "file exists" when doing "morph branch baserock:foo my/branch ; morph branch upstream:bar my/branch" ?11:46
ssam2morph, really11:46
nowsterI get "ERROR: /src/workspace/my/branch: File exists" on the second command11:46
pedroalvarezyou are trying to create 2 branches with same name11:47
nowsterpedroalvarez: yes, that is true11:47
ssam2`morph branch`, `morph checkout` and `morph edit` are quite flawed11:47
nowsterand it's what I desire11:47
ssam2but we've not yet done much documentation or work on how to use Morph without needing those commands11:47
*** De|ta [~arc@195.242.156.171] has quit [Quit: reboot]11:51
Zara_incidentally, the 'man morph' synopsis is a bit terrifying when you see it for the first time. I'm not sure if all that stuff should be in the synopsis, or if we could move some of it elsewhere.11:54
*** De|ta [~arc@195.242.156.171] has joined #baserock11:54
persiaUnfortunately, morph really doesn't work without at least one execution of `morph checkout` or `morph branch`.  Working without `morph edit` is as well documented as working with `morph edit`, but mostly because of the lack of clear documentation for either choice.11:54
petefothZara_: sadly, for baserock generally there is no specification (that I am aware of) for what documentation is required, and what criteria can be used to to determine whwther what is implmeneted meets those requirements. So we could (we have) change it to meet something we believe is a requirment, then change it again later to meet a different requirment without checking that it still fulfils the earlier requirement.11:58
*** zoli__ [~zoli_@cpe.ge-0-2-0-366.abnqu1.dk.customer.tdc.net] has joined #baserock11:58
persiaSome of that is exposed in the tests, but only with Mason v3 are we reaching a point where we have credible system-level testing, and are able to write verification criteria that match documentation, etc.12:00
persiaSpecification is a bit of an illusion in a collaborative environment: unless there is absolute consensus, the implementation rarely matches, and when there is delay for absolute consensus, the feature is often dropped in favour of something similar but different or a workaround that someone developed while everyone was waiting for consensus.12:01
Zara_I see. Just for context, atm I'm trying to find/flag up any UI issues which might discourage people from using baserock, and work out how easy they would be to fix.12:04
petefothZara_: good luck! ;)12:05
mauricemoss_is there an easy way to force a morph doing a clean build? I fiddle around with deleting chunks and the ccache, but morph won't do a clean build.12:10
tiagogomes_ mauricemoss_ did you deleted the artifacts as well? /src/artifacts12:10
SotKmauricemoss_: rm -rf /src/cache/artifacts12:11
SotKassuming your cachedir is set to /src/cache12:11
mauricemoss_ok, I'll try12:11
mauricemoss_worked :) Can I document that as "Clean build" under build issues in the wiki?12:13
persiaDocumentation for that should probably include information on how *not* to use a cache server.12:20
persiaWhile some builds don't use the cache server, higher-level stuff often does, and if one wants a true clean build, one wants not to use cached artifacts.12:21
mauricemoss_ok12:21
*** jonathanmaw_ [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock12:40
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Read error: Connection reset by peer]12:40
nowstermorph branch *appears* happy if you move the branch-named directory somewhere else12:48
nowstereg. morph branch baserock:foo my/branch; mv my/branch my-branch12:49
nowsteror  morph branch baserock:foo my/branch; mv my/branch my-branch-foo12:49
persiaMorph keeps some hidden files at the workspace level that may be confused as a result of that sort of thing.12:53
nowsterit does? My /src/workspace/.morph/ is empty12:54
nowsterI haven't got rid of the .../.morph-system-branch/config stuff. There's one in each checkout.12:55
persiaI believe morph makes some assumptions about directory layout, so you may want to adjust the config and the branch names to suit.12:57
persiaOr this may not be an issue for you.12:57
nowsterlet's see what happens12:57
* SotK thinks morph can cope with the directory being renamed12:59
persiaSotK: When running `morph`, it doesn't use the branchname from config to determine the directory in which to find definitions?13:00
SotKpersia: I don't think so13:04
jonathanmaw_ is now known as jonathanmaw13:05
SotKIt determines that based on which directory you are in (it knows how to find the root directory of the "system branch", and from there can get the path to definitions)13:05
SotKwhere the root directory of the "system branch" is the one containing the ".morph-system-branch" directory13:06
SotKI thinK :)13:06
SotKq13:11
*** CTtpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Ex-Chat]13:20
ssam2yes, moving the directories created by 'morph branch/checkout' and 'morph edit' around should work fine13:20
tiagogomes_I am trying to build m4 after doing a `autoreconf -ivf`. It fails with: aclocal-1.14: not found . It shouldn't need aclocal with the files updated by autoreconf committed. Any idea13:20
*** CTtpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock13:22
pedroalvarezmaybe you ran autoreconf in a machine with different autoconf version?13:22
pedroalvarezjjardon: have you ever tried `gst-launch-1.0 audiotestsrc ! audioconvert ! pulsesink` in baserock?13:24
tiagogomes_the machine the I ran autoreconf is the same as I am trying to build the chunk and it is failing13:24
jjardonpedroalvarez: yep, if you can only use a VM, simply check the pipeline change to state PLAYING without errors13:41
pedroalvarezjjardon: I was asking because i tried to test Alsa but looks like we are missing some kernel drivers to get it running13:42
radiofreedid you add a soundcard in qemu?13:42
jjardonpedroalvarez: alsasink maybe doesnt work because the device is being used by pulseaudio13:44
jjardongenerally is not a good idea to access alsa devices directly when pulseadio is running13:45
persiaIf `aplay -l` returns "no devices", this isn't the problem.13:46
radiofreetry qemu with -soundhw all13:48
radiofreeyou might have the driver for at least one of them...13:48
radiofreepretty sure hda (intel hda) is on by default in the defconfig13:49
jjardonmmm, seems pulseadio is intelligent enough to not make the pipeline to fail even if the alsa device is not there13:50
*** CTtpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Read error: Connection reset by peer]13:57
*** CTtpollard [~tom@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock13:57
nowsterERROR: Push to remote "trove", push url ssh://git@ct-mcr-1.ducie.codethink.co.uk/delta/cpython failed with exit code 114:09
nowsterThat's helpful.14:09
nowster"ssh git@ct-mcr-1.ducie.codethink.14:09
nowsterco.uk whoami" is fine.14:09
SotKwhat did you run to get that error?14:10
KinnisonFor those following tiago's attempts to get armv8l64 up, I can now confirm I have a minimal patch series against 3.19-rc5 (which should therefore be good against 3.19 real) and a devicetree which can boot an HP Moonshot m400 cartridge14:10
nowstermorph push trove baserock/nowster14:10
nowstermorph push trove baserock/nowster/mips14:10
richard_mawKinnison: woo!14:11
SotKKinnison: nice!14:11
ssam2sweet!14:11
ssam2nowster: avoid `morph push`14:12
SotKnowster: try naming your branch ct-mcr-1/nowster/mips, and using git push :)14:12
ssam2`git push` is fine and does the same thing, unless you happen to have committed binaries using `morph add-binary`14:12
ssam2anyone able to do an in-IRC review of this patch to fix a stupid mistake I made with the lvm2 chunk morph? http://sprunge.us/giKV14:12
nowsterAha!14:13
nowster ! [remote rejected] baserock/nowster/mips -> baserock/nowster/mips (hook declined)14:13
nowsterThe server, she no like me.14:13
nowsterdoes the branch name need to start with ct-mcr-1?14:14
SotKnowster: I think so14:14
perrylnowster: try git push <branchname>:ct-mcr-1/<branchname>14:14
KinnisonIf you're pushing to a trove called XXX then the branch will typically need to start XXX/14:14
pedroalvarezssam2: +1 if you move "I'm dubm" out of the commit message and you put it on a sticker in your t-shirt14:15
pedroalvarezs/dubm/dumb/14:16
nowsterok... it's accepted it.14:16
nowsternot ideal, but it's there14:16
SotKnowster: why not ideal?14:17
persiassam2: I don't like the commit message, but +1 for the rest of it.  Please explain that it was because Fedora links them, rather than making inaccurate claims about your thinking processes.14:17
persiaThe idea being that future developers looking through the commit messages can learn something useful or avoid the same trap.14:17
ssam2it's not to do with Fedora, it's to do with 'prefix' meaning the path *containing* ./lib, not ./lib itself14:18
persiaAh.  I thought this was related to the /lib/udev vs. /usr/lib/udev issue discussed recently14:18
ssam2ok if I change it to: 'The rules were being put in /lib/lib/udev/rules.d because I specified the wrong value for --with-udev-prefix=' ?14:18
persiaBut anyway, it ought be explained :)14:19
persiaI'd prefer "The rules were being put in /lib/lib/udev/rules.d/ because --with-udev-prefix is the path *containing* ./lib, not ./lib itself."14:20
ssam2ok, i'll commit with that message. thanks14:20
persiaThanks for finding this and fixing it :)14:20
tiagogomes_how to I attach a second volume to my VM in openstack14:21
tiagogomes_s/how to/how do14:21
ssam2persia: it found me. :)14:21
persiatiagogomes_: http://docs.openstack.org/user-guide/content/dashboard_manage_volumes.html14:22
persiatiagogomes_: Also, if you don't get enough help here, the folk in #openstack are also helpful (and may know more than us).14:22
tiagogomes_persia thanks14:22
persiatiagogomes_: There are also some nova command-line calls that would do that, but I fear the nova command-line options.14:23
jmalkI'm currently trying to run `morph cross-bootstrap` to x86_32 and getting the following error: http://paste.baserock.org/onubazukoh . For comparison I tried running cross-bootstrap to ppc64 and it completed with no problems.14:28
persiajmalk: To confirm, your local system is not x86_32, right?14:30
jmalkpersia: it is not. trying this on a VM, which is using baserock-14.29-devel-system-x86_64-generic14:30
persia14.29?  You might want to pull latest definitions, and run cycle.sh to get something fresher.14:31
jmalkpersia: I shall take a look at that14:31
richard_mawjmalk: what's in your cross-bootstrap-sysem-x86_32-generic.morph?14:31
jmalkrichard_maw: a paste of "Add the following" from http://wiki.baserock.org/x86_64_to_x86_32/14:32
jmalkrichard_maw: the writers of that tutorial had not encountered the error14:32
richard_mawjmalk: If you can confirm that it still has the `- morph: strata/build-essential.morph` line in it, I'm out of ideas14:34
persiaThe reference definitions have cross-bootstrap morphologies for armv7lhf, ppc64, and x86_64, which differ only by arch name.  Should the documentation perhaps reference one of these, and indicate that one copies the file and changes the architecture string?14:34
richard_mawthat error's only supposed to happen if you ended up with strata that don't contain any bootstrap chunks, and build-essential is the stratum which should have bootstrap chunks14:35
persiaDoes the x86_64 compiler have some special magic that makes it think x86_32 isn't cross-compilation?14:35
richard_mawpersia: it's failing well before a compiler gets involved14:36
jmalkpersia: that documentation links to http://wiki.baserock.org/guides/how-to-cross-bootstrap/ where the original "ARCH_NAME" example is14:36
persiajmalk: Right.  I'm not a fan of having multiple different copies of a file under different version control systems, which is why I thought it might be best to just reference the files in definitions master :)14:37
SotKDoes anyone remember why when serialising artifacts in distbuild we do "json.dumps(yaml.dump(contents))"?14:41
Kinnisoncontent which cannot be encoded in json14:42
SotKwhich morphologies contain that? :(14:44
KinnisonI think it's more logs14:48
Kinnisone.g. filenames with byte sequences which are invalid utf8 i think make json barf14:48
SotKwe don't serialise the logs there though afaict14:48
Kinnisonmetadata comes through perhaps?14:49
* Kinnison isn't clear on it and perhaps running 'git annotate' and examining the commits which put it in place might help?14:49
persialogs can contain anything, in any encoding (or even several)14:49
SotKhm, the commit message suggests its to "allow distbuilding morphologies with binary data embedded"14:50
persiaPerhaps someone wanted to put some binary blobs in the foo-commands sections?  I'd think it would be more sensible to base64- or uuencode them.14:50
* SotK imagines this was discussed at the time and looks on the ML14:51
*** zoli__ [~zoli_@cpe.ge-0-2-0-366.abnqu1.dk.customer.tdc.net] has quit [Remote host closed the connection]14:54
ssam2the reason distbuild needs to handle binary data is for when we transfer stdout and stderr from chunk builds14:59
ssam2because we can't expect those to be valid UTF-815:00
ssam2i'm pretty sure any other communcation can be easily validated before sending15:00
ssam2we should probably reject any chunk names that aren't valid UTF-8 straight away in morphloader.py15:00
ssam2the 'encode as YAML then JSON' thing was always a bit of a hack though, I think it was done in quite a rush15:01
*** grahamfinney [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has quit [Quit: Ex-Chat]15:01
richard_mawssam2: yeah, it should just be yaml, but there's code that expects the output to be a one-line string that encodes a json object15:05
richard_mawit would be better to make the parser handle multi-line messages and send the whole thing in yaml, but we were in a rush15:05
SotKrichard_maw: it looks to me like we just use yaml.load(json.loads(encoded_stuff)) when deserialising artifacts though?15:06
richard_mawpossibly, there's two different communications streams using the same idea going on. There's controller ←→ worker and `morph serialize-artifact` → worker15:08
ssam2richard_maw: ah, I see, so the line-breaks in yaml break the code which expects messages to be newline-separated15:08
Kinnisonssam2: chunk metadata might also contain non utf-8 (e.g. filenames)15:09
ssam2Kinnison: true, but I don't think distbuild ever talks about that right now15:09
tiagogomes_do we have morphs on the repos because nobody delete them, or because they are needed for backwards compatibility15:11
ssam2i wonder if there's something other than JSON or YAML we could use that would be (a) debuggable (b) fast (c) not limited to UTF-8 strings and (d) terminated in a way that code listening to the sockets can handle easily15:11
ssam2tiagogomes_: chunk morphs in chunk repos are there only because nobody has deleted them15:11
tiagogomes_ssam2, good to hear, thanks15:11
persiadeleting things in git is hard: better to just drop the relevant patch when moving to a new commit.15:12
* SotK hopes so, the current method is horribly slow15:12
ssam2there's https://developers.google.com/protocol-buffers/, I've never used it15:12
* richard_maw dislikes protocol buffers, as it's a meta-language that generates C or Go code, rather than having its own compiler that produces object code that you can link in15:13
ssam2does it not generate Python?15:16
ssam2it's hard to tell from https://developers.google.com/protocol-buffers/docs/pythontutorial15:16
*** franred [~franred@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]15:17
rdale_yes, you can use python protobufs15:17
jmacsWhy do you dislike generating C?15:18
richard_mawjmacs: it usually ends up generating hard to debug C when something does go wrong15:21
jmacsHarder than debugging object code?15:21
rdale_protobuf is c++, not c and the python protobuf code is pure python afaik15:23
ssam2I have a feeling that to have a Gerrit running on Baserock at baserock.org that sends forgot-password emails and the like, I need to add sendmail to Baserock15:33
pedroalvarezi managed to configure exim4 on it15:34
ssam2ok, is that better or worse than sendmail ?15:34
ssam2(i know nothing here)15:34
richard_mawexim4 is a sendmail15:36
ssam2ok, so ftp://ftp.sendmail.org/pub/sendmail/ is not 'sendmail' but 'a sendmail' too?15:36
pedroalvarezit wasn't complicated to configure, and I remember that Kinnison and rjek helped me with it. And the fact that they didn't say "don't use it"  makes me think that is a good option :)15:36
ssam2exim4 seems to have git repos to15:36
ssam2o15:36
persiassam2: You want exim or postfix: not authentic sendmail15:37
ssam2ok, thanks15:37
ssam2I'll go with exim415:37
pedroalvarezssam2: if the old testgerrit is still alive, it will have the configuration15:37
ssam2great15:37
persiaActually, you may not even need something that complex, if you are only dealing with limited outbound.15:40
persiahttps://github.com/ajwans/sSMTP as an example15:40
* rjek oohs at exim415:44
rjekISTR exim4 is exciting* to build with morph, but it's been 2 years since I tried.15:44
persiaI also worry about the security of exim vs. postfix (single binary vs. privilege separation), but I know of no exploits that use that vector today.15:45
persiaBut ssmtp or nullmailer are more useful candidates for appliances, unless one is setting up a mail host.15:45
* persia doesn't like nullmailer because it doesn7t have TLS support, but that may not be important for a controlled network15:46
nowsterone wee problem with cross-bootstrap... it leaves stuff in /bin -> /tools/bin15:47
persianowster: At which point?  My thought is that the model is 1) run something to generate a cross-tarball, 2) boot the cross-tarball to get a very basic system, 3) use this to run morph to generate a real build system, 4) deploy the build system somewhere, and start from there (generating devel, possibly deployiing trove/distbuild, etc.)15:49
nowsterat what point should /tools be removed?15:50
ratmice__fwiw, the author of protobufs went on to write cap'n'proto, which i'm not sure if its capnp compile with various output plugins would cover any of richard's objections 15:50
Kinnisonnowster: once the native-bootstrap completes, the system is "enough" to build something with morph15:50
Kinnisonnowster: we suggest you build a build system15:50
Kinnisonnowster: then use that build system to build a devel system15:50
Kinnisonnowster: at that point you're fairly well clean and away15:51
richard_mawratmice__: ooh, I like the look of that15:51
persiaWith this model, /tools is never explicity removed: the entire rootfs that contained it is ignored instead.15:51
Kinnisoncorrect15:52
persiaThat said, does the use of binaries from /tools ever cause a reduction in bootstrap steps?15:52
ssam2i don't understand the question16:01
KinnisonThe presence of /tools after the completion of the native part of the bootstrap phase is sad but not actually relevant.16:06
KinnisonIt should be ignored and a clean system built out of the tottering doom16:07
persianowster reports that there is some /bin->/tools/bin stuff left over.  My question is whether this matters at all.16:07
pedroalvarezssam2: given that the patch already had a +2, I merged it :/16:07
KinnisonIt shouldn't matter, no16:07
ssam2pedroalvarez: thanks. I'm sure it won't hurt :)16:10
ssam2persia: the symlink from /bin to /tools/bin is an expected part of the stage 3 bootstrap16:11
ssam2I forget the details of why16:11
ssam2it should be documented in the comments of the build-essential stratum16:11
*** sherm_ [~mike@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]16:39
bashrc_getting an error: Couldn't find morphology: strata/build-essential/stage2-linux-api-headers.morph16:46
bashrc_the thing is that morph file exists16:46
pedroalvarezbashrc_: can you please paste a detailed log of the error? With `morph version` output too please16:49
bashrc_morph version is e8adedb8f3f27d9212caf277b8e8f7c6792a20c216:51
bashrc_and I'm running the command morph cross-bootstrap $TARGET_ARCH file:///src/workspace/crossboot/baserock/baserock/definitions HEAD systems/cross-bootstrap-system-$TARGET_ARCH-generic.morph16:52
pedroalvarezbashrc_: can you please paste a detailed log of the error?16:52
pedroalvarezbashrc_: also, this version of morph is pretty old, can you update it please?16:54
pedroalvarezbashrc_: this is the easiest way to do it: http://wiki.baserock.org/using-latest-morph/16:56
bashrc_tried to get verbose output, but morph doesn't like that16:57
pedroalvarezit liked it every time I used it16:58
bashrc_just using -v or --verbose16:58
pedroalvarezanyway, please upgrade morph, the version that you are using is from before we moved all the chunks to definitions16:59
pedroalvarezif it's still failing, then please send the logs16:59
pedroalvarezis --verbose16:59
*** jonathanmaw [~jonathanm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]17:01
ssam2if --verbose doesn't work, '--log=/dev/stdout --log-level=debug' should give you even more info on stdout17:02
locallycompactWhat does lp: mean in the url field of make.lorry?17:07
richard_mawlaunchpad17:08
richard_mawit's a bzr import17:08
* persia continues to believe the easiest way to move a devel system to latest morph is to have a definitions master checking, and to run `git pull; morph upgrade` therein.17:10
locallycompactWhy isn't it being updated?17:10
persias/checking/checkout/17:10
bashrc_ah the classic error: Nothing to cross-compile. Only chunks built in 'bootstrap' mode can be cross-compiled17:11
bashrc_verbose output http://0bin.net/paste/V4j7twvXVphvRZKz#NNHOhyBdOQzNE92VgTDj+mtkrvc0cgggeNmQ6p-1ESS17:12
ssam2locallycompact: is there a problem with the delta/make lorry not updating? I can look at why, but will probably have to wait til tomorrow17:13
locallycompactssam2: Latest master on g.b.o is 3 years old which is 3.8.something, I'm interested in bringing it up to latest.17:14
ssam2oh dear17:15
ssam2but the launchpad make repo is up to date? or are we lorrying from the wrong place?17:15
locallycompactI'm trying to find where that lives17:16
locallycompactThink it's up to date https://launchpad.net/make17:17
ssam2good spot, the lorry is failing17:19
ssam2log shows: bzr: ERROR: These branches have diverged. Use the missing command to see how.17:19
ssam2    Use the merge command to reconcile them.17:19
ssam2I don't have a clue how to fix that17:19
Kinnisonsounds like they've force-pushed their bzr branch and lorry didn't know what to do17:19
Kinnisonssam2: bzr pull --overwrite might be what you need17:20
ssam2cool, I'll try that17:20
pedroalvarezbashrc_: from your log I don't see what command are you running, nor what morphology trying to build17:21
ssam2also, it seems silly to me that we keep these lorry logs secret. If they were public it'd be easier to notice errors like this..17:21
bashrc_the full script is here: http://wiki.baserock.org/x86_64_to_armv8l6417:21
pedroalvarezssam2: I'd like to make public also if they are working or not.17:22
pedroalvarezssam2: one day I thought that pubilishing the results of the latest run would be ok17:22
*** tiagogomes_ [~tiagogome@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]17:23
ssam2Kinnison: `bzr pull --overwrite` did a lot of stuff, but it still shows the error17:23
Kinnisonwhat does 'bzr missing' tell you?17:23
ssam2'Branches are up to date. '17:24
ssam2and if I run 'bzr pull' in the /bzr directory, it succeeds17:24
Kinnisonodd17:24
ssam2ah, I might need to add 'lp:make' to the end of the 'bzr pull' command17:24
Kinnisonheh17:25
ssam2ah, and I need to run it in the bzr/trunk directory, not the bzr/ directory17:28
pedroalvarezbashrc_: sorry, I don't have time to understand your script right now. 17:28
ssam24th time lucky17:28
Kinnisonbzr is confusing when you're used to git17:31
Kinnison(and git is confusing when you're used to bzr)17:31
ssam2this time, it failed because git-fast-import returned non-zero17:32
ssam2but its stderr only shows warnings17:32
KinnisonI assume git-fast-import can do non-ff updates17:33
ssam2I see this:17:33
ssam2 git fast-import --export-marks=/home/l17:33
ssam2orry/working-area/delta_make/git/marks.git --import-marks=/home/lorry/working-ar17:33
ssam2ea/delta_make/git/marks.git17:33
ssam2  warning: Not updating refs/heads/trunk (new tip 0c1aa83e432e8ea521f70db6065daf5382eb1426 does not contain ec421e7dda7788ac19e912fcc4e07f9a5809f8d5)17:34
ssam2which suggests it's avoiding doing a non-ff update17:34
*** nowster [~pm@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Client exiting]17:34
Kinnison:(17:36
Kinnisonpop --force on the import cmdline maybe?17:36
ssam2i'll try running it manually with --force17:36
*** mariaderidder [~maria@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Ex-Chat]17:36
* ssam2 does so as root, and thus breaks more things17:39
Kinnisonoops17:39
ssam2locallycompact: fresh new commits of make! http://git.baserock.org/cgi-bin/cgit.cgi/delta/make.git17:40
ssam2please wipe the tears off them17:41
locallycompactssam2: Thanks very much17:41
* locallycompact notices no tags of 4.x, hmm17:41
*** gary_perkins [~gary_perk@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock17:42
radiofreewouldn't it be easier to use make from git?17:43
ssam2definitely. is there one?17:44
radiofreehttp://git.savannah.gnu.org/cgit/make.git ?17:45
ssam2cool, let's switch to that17:46
ssam2I wonder why we were using it from launchpad?17:46
ssam2for compatibility reasons we'll have to keep the existing delta/make repo and import the git one as delta/make-git or something...17:46
radiofreethere's a tag in there "moved-to-git" from 2 years ago17:47
radiofreemaybe when it was originally lorried there was no git repo17:47
ssam2ah, right17:49
ssam2one problem of this 'everything is reproducible for ever' idea we have is that we can't rename the existing delta/make repo without breaking lots of existing commits of definitions.git17:50
ssam2unless we implement some super clever 'redirect this repo to this one but only in commits older than this' system, but that sounds a bit risky17:50
*** nowster [~pm@82-70-136-246.dsl.in-addr.zen.co.uk] has joined #baserock17:51
ssam2so I guess we're stuck with delta/make-2 or delta/make-git or something...17:51
pedroalvarez+1 for make-git17:51
ssam2unless of course the sha1s for the savannah version of make.git happen to correspond to those in our bzr import... but i think that's unlikely17:52
*** Krin [~mikesmith@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Remote host closed the connection]17:54
richard_mawespecially since they've apparently fast-forwarded their own master17:54
gary_perkinsHi, can anyone help me? I'm trying to run morph deploy ct-trove.morph. It has /root/.ssh/authorized_keys in the manifest. But the deploy errors with the following: ERROR: install-files.configure failed with code 1: ERROR: [Errno 2] No such file or directory: '/src/tmp/deployments/tmpWZb3He/tmpf9X8Ng/./root/.ssh/authorized_keys'17:55
gary_perkinsI'm sure I had this same problem when deploying this trove the first time, but I can't remember how I fixed it17:55
pedroalvarezgary_perkins: can I see the manifest?17:56
gary_perkinspedroalvarez: sure... 17:56
pedroalvarezI assume that in the root folder of your definitions you have a root/.ssh/ folder17:56
pedroalvarezwith the file inside17:57
gary_perkins0100600 0 0 root/.ssh/authorized_keys17:57
gary_perkinsI have a ct-trove-files/root/.ssh/authorized_keys17:58
gary_perkinsshould I have added the ct-trove-files ?17:58
*** bashrc_ [~motters@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Lost terminal]17:59
pedroalvarezgary_perkins: no as long that your manifest is also in the ct-trove-files folder18:00
pedroalvarezsorry about the confusion18:00
gary_perkinspedroalvarez: it is18:00
pedroalvarezyo need to add:0040755 0 0 /root18:01
pedroalvarez0040755 0 0 /usr/lib18:01
gary_perkinsI had included root/.ssh directory too as an entry in the manifest, but that didn't work either18:01
gary_perkinsahh /root! 18:01
pedroalvarezgary_perkins: oops, my paste is maybe wrong18:01
gary_perkinsahh! so obvious! I'll try that. Thanks pedro18:01
pedroalvarezgary_perkins: just to be clear: http://paste.baserock.org/axeqisigap18:02
gary_perkinspedroalvarez: Ta :)18:02
pedroalvareznp! :)18:02
*** mdunford [~marcdunfo@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 255 seconds]18:05
*** jonathanmaw [~jonathanm@host5-81-53-4.range5-81.btcentralplus.com] has joined #baserock18:06
*** ssam2 [~ssam2@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Quit: Leaving]18:07
*** kejiahu [~kejiahu@access.ducie-dc1.codethink.co.uk] has quit [Remote host closed the connection]18:15
*** jmalk [~joshmalki@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** mwilliams_ct [~mikewilli@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** petefoth1 [~petefothe@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** Zara_ [~zarazaime@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** pdar [~patrickda@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** wdutch [~yourname@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** jmacs [~jimmacart@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** paulsherwood [~paulsherw@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** benbrown_ [~benbrown@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** SotK [~adamcoldr@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** bwh [~benhutchi@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** DavePage [~dpage@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:15
*** perryl [~laurenper@access.ducie-dc1.codethink.co.uk] has quit [Read error: Connection reset by peer]18:16
*** grahamfinney_ [~grahamfin@cpc14-know11-2-0-cust234.know.cable.virginm.net] has quit [Quit: Ex-Chat]18:18
straycato/18:22
*** zarazaimeche [~zarazaime@access.ducie-dc1.codethink.co.uk] has joined #baserock18:23
zarazaimeche is now known as Zara18:23
*** perryl [~laurenper@access.ducie-dc1.codethink.co.uk] has joined #baserock18:24
*** jmacs [~jimmacart@access.ducie-dc1.codethink.co.uk] has joined #baserock18:32
*** kejiahu [~kejiahu@access.ducie-dc1.codethink.co.uk] has joined #baserock18:35
*** locallycompact [~lc@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 246 seconds]18:46
*** petefoth [~petefoth@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 256 seconds]18:58
*** lachlanmackenzie [~lachlan@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Ping timeout: 240 seconds]19:26
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock20:03
*** jonathanmaw [~jonathanm@host5-81-53-4.range5-81.btcentralplus.com] has quit [Quit: Leaving]20:15
*** gary_perkins [~gary_perk@82-70-136-246.dsl.in-addr.zen.co.uk] has quit [Read error: Connection reset by peer]20:36
*** petefotheringham [~petefothe@access.ducie-dc1.codethink.co.uk] has joined #baserock20:53
*** rdale__ [~quassel@200.Red-83-47-16.dynamicIP.rima-tde.net] has joined #baserock20:57
*** rdale_ [~quassel@81.Red-79-146-151.dynamicIP.rima-tde.net] has quit [Ping timeout: 256 seconds]20:59
*** SotK [~adamcoldr@access.ducie-dc1.codethink.co.uk] has joined #baserock21:18
*** SotK [~adamcoldr@access.ducie-dc1.codethink.co.uk] has quit [Client Quit]21:20
*** SotK [~adamcoldr@access.ducie-dc1.codethink.co.uk] has joined #baserock21:26
*** paulsherwood [~paulsherw@access.ducie-dc1.codethink.co.uk] has joined #baserock21:39
*** paulsherwood [~paulsherw@access.ducie-dc1.codethink.co.uk] has quit [Client Quit]21:39
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has quit [Remote host closed the connection]22:55
*** zoli__ [~zoli_@0x5e91887a.adsl.cybercity.dk] has joined #baserock23:16

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!