IRC logs for #baserock for Friday, 2015-08-21

*** petefoth has quit IRC05:14
*** petefoth has joined #baserock06:25
*** paulw has joined #baserock06:26
*** petefoth has quit IRC06:39
*** petefoth has joined #baserock06:40
paulsherwoodnowster: yup - each sandbox is a tmp directory created (with symlinks) specifically for each command07:01
paulsherwoodnowster: there's a known problem with the linux-user-chroot, where BIND can fail because another instance's sandbox gets deleted between listing things to bind and doing it07:07
paulsherwood(so by default i'm ignoring linux-user-chroot on systems where i run ybd)07:07
paulsherwood(until i find a fix)07:14
paulsherwoodnowster: here's another fail log - stage1-gcc this time http://sprunge.us/BbcN07:21
paulsherwoodpedroalvarez: shouldn't those notes mention the genivi baseline?07:25
*** jonathanmaw has joined #baserock08:00
*** bashrc has joined #baserock08:09
pedroalvarezpaulsherwood: probably08:14
*** bashrc has quit IRC08:21
pedroalvarezdone08:24
*** bashrc has joined #baserock08:30
paulsherwood:)08:32
*** bashrc has quit IRC08:33
pedroalvarezIf it weren't for the GENIVI manifests, the release process would be really easy. Less than 10 minutes of my time (not including the release notes)08:36
paulsherwoodinteresting. what is so hard about the GENIVI manifests?08:38
*** bashrc has joined #baserock08:43
pedroalvarezactually you are right, I guess I'm just disappointed I still need to create a workspace to be able to generate one of them08:43
paulsherwoodpedroalvarez: presumably your time excludes the build time?08:43
pedroalvarezit does08:44
*** ssam2 has joined #baserock08:44
*** ChanServ sets mode: +v ssam208:44
pedroalvarezI hope `morph edit` still works, or that we have replace its use for something else when generating the manifests08:44
paulsherwoodpedroalvarez: which script generates manifests? easiest would be to fix that, i assume?08:45
pedroalvarezscirpts/licensecheck.sh08:45
pedroalvarezit might be easy to replace with the get-repo command08:47
paulsherwoodwhat is the get-repo command?08:47
SotKyou could use `morph get-repo` if `morph edit` doesn't work08:47
* paulsherwood would like to use ybd, and be done08:47
SotKheh, you beat me to it08:47
pedroalvarezI assume it still works, but it might be about to dead08:48
pedroalvarezto die*08:48
pedroalvarezactually, this is one of the things that remains morph specific in definitions08:49
pedroalvarezwell, I guess the entire release process08:50
* paulsherwood wonders why git blame shows no lines of scirpts/licensecheck.sh written by paulsherwood, yet he has strange memories of writing it or something extremely similar08:50
pedroalvarezI guess it will show you the commit when we moved it to the scripts/ folder08:51
paulsherwoodso i'm pretty sure i can make it not require anything of morph08:52
pedroalvarezhahah I knew you would feel challenged after my comments08:53
paulsherwoodlol08:53
*** wschaller has joined #baserock09:16
*** paulw has quit IRC09:18
*** paulw has joined #baserock09:20
*** lmackenzie_2015 has joined #baserock09:22
*** paulw has quit IRC09:45
nowsterpaulsherwood: that fail is a binutils one, I think09:45
nowsterpaulsherwood: try running without parallelism09:45
*** paulw has joined #baserock09:47
paulsherwoodnowster: i know it works without parallelism. some also work *with* paralellism. the question is why it occasionally fails. stage1-gcc explicitly depends on stage1-binutils, so that should be present in its staging area, parallel or no?09:53
nowstermaybe it's disappearing09:53
paulsherwoodnowster: /win 5309:54
paulsherwoodbah!09:55
nowsterI think it's not good that there are multiple parallel builds happening of binutils, all making the same asset.09:55
nowsterYou may be setting yourself up for a race condition.09:55
paulsherwoodthey are all making it in different places09:55
paulsherwoodit's *supposed* to race09:55
paulsherwoodbut clearly there's something going wrong ;)09:55
nowsterand are multiple gcc builds happening too?09:56
paulsherwoodyup09:56
paulsherwooduntil there's enough scope for each to find something different from the others09:56
nowsterso you don't know whether one was succeeding when one failed...09:57
paulsherwoodi do... they all get logged09:57
nowsterso did they all fail the same way?09:57
paulsherwoodno, only one failed, afaict09:57
nowsterso, why are you stopping?09:58
nowsterHave you worked out what's different in the environment of the failed gcc compared with a single task build?09:59
paulsherwoodyou mean why does the instance hardstop? because it's a build fail that should not happen09:59
paulsherwoodno - i was hoping for a clue from someone who actually understood make/gcc/autotools, hence this conversation :)09:59
nowster"configure: error: cannot compute suffix of object files: cannot compile" is a linker not found bug.10:00
paulsherwoodand the linker we are expecting to use should be in stage1-binutils?10:01
nowsteryes10:02
paulsherwoodok, i need to look harder on that basis10:03
nowsterit would normally be in /tools/bin/ld at that point10:04
nowsternamed according to the pattern /tools/bin/$ARCH-bootstrap-linux-$ABI-ld10:05
nowsterand even /tools/$ARCH-bootstrap-linux-$ABI/bin/ld10:06
paulsherwoodok10:06
nowsterIs there *any* possibility that two copies of the gcc build might be making the "unpacked" artifact at the same time?10:08
paulsherwoodhmmm... actually, yes. the artifact should be unpacked when it's created (before the os.rename(tmpdir, target) not when something realises it's not unpacked)10:11
paulsherwoodgood spot10:12
*** wschaller has quit IRC10:17
paulsherwoodnowster: are ybd system meta files expected to be only repo: null, ref: null10:29
*** wschaller has joined #baserock10:29
nowsteryes10:31
nowster...as that's what they've always been10:31
paulsherwoodok great :)10:31
* paulsherwood has done pep810:31
nowsterpaulsherwood: have you got a tool for that?10:33
* nowster is pleased how few tweaks were needed10:34
paulsherwoodnowster: i am the tool for that :)10:35
paulsherwoodnowster: by which i mean, i just run pep8 *py, edit stuff and run pep8 again until it goes silent10:51
nowsterah10:51
nowsteraha!10:51
paulsherwood?10:51
nowsterI hadn't realised there was a tool with that name.10:51
paulsherwoodthere isn't in baserock... i run it from my host on an sshfs10:52
nowsterI've also found flake810:53
*** petefoth has quit IRC10:53
ssam2you might like autopep8, which supposedly fixes things automatically10:54
ssam2never tried it though10:54
ssam2I use whatever one Vim python mode comes with10:54
nowsterpaulsherwood: http://paste.baserock.org/eliwovecib10:54
paulsherwoodooh.. .what's that from nowster ?10:59
nowsterflake8 (and a newer pep8 program)11:00
nowsterhttps://gitlab.com/pycqa/flake811:00
nowsterpaulsherwood: after the fixes I've just requested, it leaves http://paste.baserock.org/ezehefotag, which may need further investigation rather than a blind fix11:15
*** wschaller has quit IRC11:30
paulsherwoodnowster: great, thanks11:53
paulsherwoodnowster: looks like your unpacked spot idea is correct11:54
nowsteraha!11:55
ssam2it seems that it's possible to run BitBake builds outside of BitBake.11:57
ssam2ah! I knew it would fail once i talked about it.11:57
ssam2I think the build failure I just hit is one I saw when running *in* BitBake, too, though11:57
ssam2and it got thru about 20 packages before breaking, anyway11:58
paulsherwoodssam2: that *must* be a w00t11:58
* pedroalvarez is amazed11:58
ssam2there's about 2.5GB of generated shell + Python code. But most of that is just a dump of all the variables in the gigantic 'data' dict that BitBake keeps12:01
ssam2one dump per task12:01
ssam2i think it's certainly going to be possible to convert them to a format which is nice to browse thru and see what configure flags are used, etc.12:01
ssam2converting them to actually usable Baserock definitions would be /much/ more of a stretch, but we can see, anyway :)12:02
paulsherwoodso do they go for an over-arching dict, similarly to ybd?12:03
ssam2I wouldn't really compare it to YBD12:03
ssam2it's more like Automake12:03
paulsherwoodoh? pls elaborate...12:04
ssam2there's an 'environment' in which recipes run, basically12:04
ssam2which has all these variables set12:04
ssam2and the recipes can also change the environment12:04
ssam2and the variables can expand to other variables, or expand to Python code, which is great12:05
ssam2so it feels quite like 'make'. I think YBD is far less flexible than that12:06
ssam2but with flexibility comes incredible complexity :)12:06
paulsherwoodnowster: did you test your more pep8?12:08
nowsterI'll do so now.12:08
* paulsherwood merged it, but has seen odd behaviour since12:09
pedroalvarezrandom thought: you can add this check in a Travis CI job12:11
nowsterpaulsherwood: such as?12:11
paulsherwoodi've lost my paste, now sorry12:12
paulsherwoodit failed fast on something to do with gitdirs i think12:12
paulsherwoodnowster: http://paste.baserock.org/ihififupoy12:14
* nowster tests12:15
paulsherwoodthis is running on clusters/ci.morph12:16
SotKtransl(x) doesn't appear to return anything12:17
nowsterpaulsherwood: usually the fault is a good few lines before where you started that paste12:18
paulsherwoodnowster: there's nothing above that but normal log?12:19
paulsherwoodhttp://paste.baserock.org/eqididusif12:19
nowsterI can reproduce12:20
nowstertry this: http://paste.baserock.org/unewupikic12:22
* nowster blushes12:22
paulsherwoodlol12:22
nowsterthat fixed it12:24
* paulsherwood looks forward to a pr :)12:24
*** wschaller has joined #baserock12:26
nowsterpaulsherwood: sent12:28
paulsherwoodtvm12:28
paulsherwoodnowster: what is a brown paper bag bug? :()12:29
nowsterone that makes you want to sit in a corner with a brown paper bag on your head12:30
nowsterLinus uses the term a lot.12:30
paulsherwoodaha :)12:31
paulsherwoodthanks for the patches, anyway... you're rapidly becoming a key contributor12:31
pedroalvarezI've documented a bit my failure trying to convert nixos expressions to something usable by baserock: http://wiki.baserock.org/projects/importing-from-nixos/12:37
paulsherwoodpedroalvarez: thanks!12:39
nowsterpaulsherwood: http://www.catb.org/jargon/html/B/brown-paper-bag-bug.html12:43
pedroalvarezpaulsherwood: we haven't looked at debian/germinate yet12:44
pedroalvarezthat one would be interesting12:44
paulsherwoodack12:45
pedroalvarezI've always found similarities between baserock definitions and debian's equivalent12:45
pedroalvarezsame with fedora12:46
nowsterdebian's are a little more spread out12:47
paulsherwoodyup, but germinate seems to do magic12:47
nowsterInteresting. I wasn't aware of germinate before.12:48
nowsterOh dear! Written by Scott and Colin. :)12:49
*** paulw has quit IRC12:52
*** paulw has joined #baserock13:01
pedroalvarezGenivi release notes draft: http://paste.baserock.org/upuzumefis13:40
pedroalvarezBrief, but I don't have more to say13:40
paulsherwoodpedroalvarez: looks good to me13:41
pedroalvarezgood13:41
pedroalvarezI'm going to send the emails soon13:42
SotKpedroalvarez: looks fine to me too13:42
* pedroalvarez sends the emails13:51
pedroalvarezAny with powers to moderate baserock-announce?13:51
ssam2you don't have them? i'll see if I can give you them :)13:51
pedroalvarezplease, I should :)13:52
pedroalvarez**Baserock 15.34 is released!**13:56
SotK\o/13:56
ssam2nice work!13:57
pedroalvarezI'm tempted to remove any mention of workspaces in the wiki13:57
ssam2That's a really good idea13:58
SotK+113:58
ssam2I think documentation is the main thing blocking us removing the commands from Morph13:59
ssam2although I still like the idea of us having some sort of workflow helper tool'13:59
radiofreepedroalvarez: "Baserock 15.34 (Kronos K-0.2) is released"14:01
radiofreeBaserock 15.34 is our Kronos K-0.1-compliant release of the Baserock GENIVI baseline.14:01
radiofrees/K-0.1/K-0.2/14:01
radiofree?14:01
paulsherwoodprobably14:01
pedroalvarezI know :/14:02
* pedroalvarez updates the version of morph in Mason(s)14:15
jjardon\o/14:19
jjardonssam2: version 6 coming?14:19
jjardonmmm, morph is trying to build binutils instead using the mason cache, anyone more with the same problem?14:20
ssam2jjardon: we can use version 6 now in master of definitions.git, yeah14:20
ssam2jjardon: which means we can delete a bunch of code from Morph too14:21
* jjardon likes deleting code14:21
* jjardon notices v7 is waiting in gerrit too14:22
pedroalvarezjjardon: the cache-key of everything changed recently14:23
pedroalvarezthere is no cache for latest morph, that's why I just updated the version of Morph in Mason(s)14:23
paulsherwoodv7 has some bugs, if i remember rightly14:24
jjardonpedroalvarez: ah, ok, thanks14:28
ssam2v7 is still a proposal on the mailing list. I think only Paul has responded so far14:30
ssam2the implementation I proposed for YBD might well have some bugs, too14:30
paulsherwoodssam2: v7 being defaults? (sorry i'm in other conversations... can't google enough)14:31
pedroalvarezyes14:35
pedroalvarezmaybe something else14:35
paulsherwoodybd has implemented defaults14:35
ssam2oh, cool14:35
ssam2if we want the Baserock definitions format to make sense independently of the tooling, we need to think about it independently of the tooling. right now we are talking about specific implementations, again14:36
paulsherwoodssam2: it was implemented based on your proposal... but there were bugs, is all. it will use defaults in definitions, if/when they are there14:38
paulsherwood(i was only replying to your comment)14:38
ssam2nice! But of course there are no bugs in my proposal :-)14:38
jjardonpaulsherwood: what was the URL to that online service with a massive amount of CPU avaialble?14:38
paulsherwoodaws?14:39
pedroalvarezhahaha14:39
jjardonthat one, thanks14:40
paulsherwoodssam2: https://github.com/devcurmudgeon/ybd/blob/master/config/defaults.conf14:45
paulsherwoodi'll try to reply to your email with what additions/changes happened vs your original14:46
paulsherwood(i needed indicators, for example... for backwards compatibility)14:46
paulsherwoodand i think python install-commands needed to be - python setup.py install --prefix "$PREFIX" --root "$DESTDIR"14:47
pdarare the artifacts for the release not cached yet?14:48
* pdar wonders if that sentence means what he thinks...14:48
paulsherwoodssam2: note the split-rules are there too14:49
ssam2pdar: pedro would know, but he has just left!14:50
ssam2the artifacts are very likely to be in cache.baserock.org already14:50
ssam2they need to be manually uploaded to git.baserock.org for each release, I don't know if that's been done or not14:50
ssam2paulsherwood: oh yeah, I guess if you don't want to remove build-system autodetection then you need the indicators still14:51
pdarha, alas. I asked cos I just started bulding stuff quite early on.14:52
pdarthanks ssam214:53
ssam2why do you think that about python install-commands? have you seen the previous defaults (`python setup.py install`) failing to work correctly?14:53
paulsherwoodssam2: i believe nowster hit that14:53
paulsherwoodnowster: ^^?14:53
ssam2right. It makes sense, it just would surprise me if they were broken for 4 years and nobody realised14:53
nowster?14:54
paulsherwoodcan you recall anything about defauly python install commands?14:54
nowsterSomething didn't work.14:54
paulsherwood:)14:54
nowsterI have a script:14:55
nowster#!/bin/sh14:55
nowsterXDG_CACHE_HOME=/src/cache PYTHONPATH=/src/src/sandboxlib /src/ybd/ybd.py $@14:55
*** wschaller has quit IRC14:55
nowsterbecause I couldn't get sandboxlib to install14:55
*** wschaller has joined #baserock15:10
*** CTtpollard has quit IRC15:35
*** wschaller has quit IRC16:16
*** jonathanmaw has quit IRC16:17
paulsherwoodhmmm.... ybd has started reporting 'sh: update-ca-certificates: not found'16:31
paulsherwoodnowster: sorry to bug you with this, but did you do anything that could have affected system-integration commands?16:33
nowsterNo, I don't think so.16:34
nowsterThey weren't running.16:34
paulsherwoodwhere is updat-ca-certificates now??16:40
jjardonstrata/core/ca-certificates.morph says /sbin/update-ca-certificates16:49
jjardonpaulsherwood: ^16:49
paulsherwoodyup, thanks16:49
paulsherwoodmy error is 15-08-21 14:06:15 [156/568/662] [genivi-baseline-system-x86_64-generic] ERROR: command failed in directory /src/workspace/reference/baserock/baserock/definitions:16:50
paulsherwoodi wonder why it's there, rather than in a sandbox16:51
ssam2system-integration commands16:52
ssam2I think they are sandboxed differently16:52
ssam2(I can't remember what Morph does, I think it's different to how it runs chunk build commands, though)16:52
paulsherwoodnowster: pretty sure git bisect will point me to... *you* :)16:56
nowster:P16:57
*** bashrc has quit IRC16:57
paulsherwoodfa38c448388d45fd020b1f556983593a4da984f416:58
nowsterneither morph nor ybd were running the integration commands for minimal-system-*16:59
paulsherwoodok, but for other systems16:59
paulsherwoodi think somehow cwd is different17:00
nowsterodd. I can't find where this['install-commands'] gets called17:01
jjardonmmm, seems the cache is still not available, morph is still trying to build binutils here17:04
paulsherwoodnowster: it's a system-integration command... http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/strata/core/ca-certificates.morph#n2017:05
ssam2jjardon: what is your artifact-cache-server variable set to? git.baserock.org:8080 or cache.baserock.org:8080 ?17:05
nowsterpaulsherwood: what calls the install-commands?17:05
ssam2oh, https://mason-x86-64.baserock.org/current.log  shows that it's only just starting building binutils17:06
ssam2makes me wonder how pedroalvarez did the release though17:06
paulsherwoodnowster: l205 of assembly.py stuffs them into 'install-commnands' for the system17:06
nowsteryes, but what calls the install-commands?17:07
ssam2actually the release commit is 6cb639983 which is showing as built on https://mason-x86-64.baserock.org/17:07
paulsherwoodnowster: not sure. never mind, i'll figure it out17:08
nowsterpaulsherwood: it's most likely to have been broken by the shuffling of the default build-system stuff around17:09
jjardonssam2: http://cache.baserock.org:8080/17:10
paulsherwoodnowster: i doubt it. bisect shows it's definitely that patch17:10
paulsherwood(i think. it's technically past beer o'clock here, my brain may be underperforming :)17:12
ssam2jjardon: ok. and are you definitely building the release tag, or 'master' ?17:15
ssam2here is notes from my digging into BitBake: http://wiki.baserock.org/projects/importing-from-bitbake/17:15
ssam2*here are notes17:15
nowsterpaulsherwood: you're not using linux-user-chroot?17:16
jjardonssam2: I tried with both with the same result: [Build 1/278] [stage1-binutils]17:19
ssam2some artifacts in /home/cache/artifacts on cache.baserock.org were owned by 'root' instead of 'cache'17:24
ssam2that often breaks things17:24
ssam2i'm changing them now17:25
ssam2ok, try now. I think it will be fixed17:25
ssam201b04620275d829d187b6c7023903897100e7be248f45718355554919164fea3.chunk.binutils-bins definitely exists in the cache, anyway17:25
paulsherwoodnowster: not for multi-instance, no17:26
paulsherwoodwell, actually s/not for multi-instance, // (since i moved it out of my path)17:26
jjardonssam2: it worked, thanks!17:31
nowsterpaulsherwood: is ca-certificates actually used by the system you're building, or is it just part of the build-deps of something?17:35
nowsterI can see that if it is excluded from the final tarball it shouldn't be running its system-integration commands.17:36
nowster...and the chdir/chroot will fail, possibly silently17:36
paulsherwoodnowster: i'm building genivi-baseline-x86_64 - it includes core, so the si command gets run at system artifact creation time17:37
paulsherwoodhttp://paste.baserock.org/wukaxaquhu17:37
nowsterwhat's in the artifact metadata for ca-certificates*.meta?17:40
paulsherwoodhttp://paste.baserock.org/vumozizoxe17:41
nowsteridentical to mine17:42
nowstervery odd17:45
nowsterAnyway, I'm off for the next week. Have fun. :)17:45
paulsherwoodkey is in the error message...17:45
paulsherwoodERROR: command failed in directory /src/workspace/reference/baserock/baserock/definitions17:45
paulsherwoodwe shouldn't be in that directory when we run it17:45
paulsherwoodenjoy your weekend17:45
paulsherwood:-)17:45
*** ssam2 has quit IRC17:51
*** lmackenzie_2015 has quit IRC18:22
*** mdunford has quit IRC18:28
*** petefoth has joined #baserock18:49
*** petefoth has quit IRC18:55
*** kejiahu has quit IRC19:37

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!