IRC logs for #baserock for Monday, 2014-07-28

*** fay_ [] has joined #baserock06:44
*** fay_ [] has quit [Ping timeout: 245 seconds]08:03
*** fay_ [] has joined #baserock08:18
*** jonathanmaw [] has joined #baserock08:32
*** ripsum [] has joined #baserock09:05
*** fay_ [] has quit [Ping timeout: 255 seconds]09:06
*** fay_ [] has joined #baserock09:07
*** motters [] has joined #baserock09:07
*** inara [~inara@] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)]09:08
*** inara [~inara@] has joined #baserock09:12
*** ripsum [] has quit ["Leaving"]09:36
motters is now known as bmottram10:09
bmottramQuestion: is it possible to create licensing reports for baserock builds?10:10
persiaThere's a check-licenses script, which I heard might do something like that, but I don't know how to run it.10:12
* pedroalvarez wonders if bmottram is looking for an output like this:
persiaMorphologies don't capture anything specific about licensing, so it depends on the original sources having valid parseable data.10:12
bmottramsimilar, but the reverse10:12
bmottramfor each license type10:13
persiaHow do you mean "the reverse"?10:13
bmottramfor every license type what packages exist in the build under those10:13
persiapedroalvarez: That comes from the licensecheck script, right?10:14
bmottramIf I were a lawyer first I'd want to know what licenses exist within the build, then go from there. I might have predelections for some and aversions to others10:16
persiagiven my experience with practicing litigants, the format of the information is rarely interesting10:17
persiaLooking quickly at the implementation, it loops over the set of components in a system, and then extracts licenses, so the implrmentation drives the format (there does not appear to be an intermediate representation from which a report is generated).10:17
pedroalvarezpersia: yes10:18
persiaIt should be relatively easy to parse that output, and generate the alternate hierarchy.10:18
rjekWhat's involved in setting up a dist-build network?  Is there a page on the wiki I am yet to find?10:18
persiapedroalvarez: Thanks for the confirmation.  You've just made me confident what I just said was true, rather than hopeless speculation :)10:18
persiarjek: You need a bunch of machines.10:18
* persia finds the page10:18
bmottramFrom what I've read, Yokto can automatically extract licenses and highlight any known incompatabilities10:19
persiaThere's some stuff in there which is jetson-specific, but the equivalent for x86 is essentially the same.10:20
persiabmottram: Are they using SPDX?10:20
bmottrampersia: I don't know10:21
persiarjek: if the documentation doesn't work, please complain (ideally with a lsit), and someone (quite possibly me) will fix it.10:21
persiabmottram: It might be worth investigating what the described feature claim means: it may be that baserock does it (or can trivally do so with a simple script), or it may be that it doesn't, but it's hard to know without specifics.10:22
rjekpersia: It's going to be a long while before I can try it, I think...10:24
rjek2014-07-28 10:05:54 [Build 7/751] [stage1-gcc-bins] Running build-commands10:24
persiastage1 again :(10:42
rjekWell, this is morph build rather than the native-bootstrap script.10:44
rjekSo hopefully in a couple of days I'll have a real BR system :)10:44
persiaOoh!  That is progress.10:44
rjek(But the USB is unreliable, so I expect to have to restart this several times.)10:44
persiaYou're running native in the bootstrapped environment?10:44
persiaMy recommendation would be to do a trove next.10:45
persiaWhile I've been seen to argue against troves, there's not currently any other sensible way to share cache, and you'll want to get the cache somewhere safe soonest.10:45
persia(Ideally to, but I suspect someone will want to replicate your work all over again before it lands there)10:45
rjekWell, it'll need merging, and we have another problem before we can do that.10:46
persiaOnce you have a trove, a distbuild cluster should be relatively easy.10:46
persiaWhat's the other pre-merge problem?10:46
rjekI've had to use GCC 4.6 for both stage 1 and 2, as well as 3.  On other platforms, we use GCC 4.6 only for 1 and 2, and use 4.7 for 3.10:46
paulsherwoodbmottram: iiuc yocto extracts information which humans have added to recipes - ie the actual consideration of applicable license(s) per component is done manually10:46
rjekpersia: And currently there's no way to change which compilers are used dependant on architecture.10:46
persiaAh, yes, logic in stratum morphologies.10:47
persiaTechnically, it could be done by mangling all the chunk morphologies very aggressively, but that's an unpleasant prospect.10:47
persiapaulsherwood: So a model akin to Debian's DEP5?10:48
paulsherwoodpersia: for better or worse, i have no experience at all with debian10:49
persiaOh heh.10:49
persiapaulsherwood: e.g.
persiaAh, yes, yocto *is* using SPDX10:52
paulsherwoodjust because SPDX says a component is foo-licenced, doesn't make it true, afaik.10:53
persiaSame for DEP5 or the output of licensecheck.pl10:53
persiaI don't know if they ever implemented it, but I remember speaking to some SPDX  folk about a format to express warrants of licensing, with the warrantor identified, to ease indemnity and liability considerations for recipient organisations.10:55
persia(because, we can't actually *know* the license for a given piece of code with any specificity, regardless of the textual content made available for download)10:56 at least reports all license instances, based on checking each file. that's better, imo.10:56
* persia will now stop writing about issues with licensing and compliance until and unless someone else particularly wants to have a discussion now, and there is a sane plan to turn said discussion into something useful10:57
paulsherwoodi was only rplying to bmottram's earlier comment10:57
* paulsherwood goes back to work :)10:57
* persia fails to refrain from writing that assumes 1) the text of the source code has any relation to the user's license terms, and 2) there are no irregularities in the format of expressed licenses (the number of sources with spacing issues in copied GPL preambles is not small).10:59
* persia tries harder10:59
* richard_maw boggles at /bin/sh command not found13:24
KinnisonThat sometimes happens if the dynamic loader isn't found13:24
KinnisonI seem to recall that symptom when pedro was doing ppc64 support13:25
Kinnisonwhen the loaders got installed to a different path13:25
rjekAny suggestions for what I should do to debug?  Will it have left the build directory around for me to nose in?13:26
KinnisonIt will have left the staging area in $MORPH_TEMPDIR/failed/13:26
Kinnisonfor you to poke at13:26
* richard_maw still boggles, since morph runs `sh -c 'make mrproper'`13:26
rjek/src/tmp/failed # chroot tmpeViDuV/ /bin/sh13:27
rjekchroot: can't execute '/bin/sh': No such file or directory13:27
rjekThat chroot doesn't even have a bin directory13:27
richard_mawah, it must be /tools/bin/sh13:27
KinnisonWhile bootstrap is still underway, /bin is likely a symlink to /tools/bin13:28
rjekYeah, that exists.13:28
* richard_maw thought the bootstrap created the /bin/sh symlink for you13:28
rjekIn that, I mean that /src/tmp/failed/tmpeViDuV/ has no bin.13:28
Kinnisonrjek: Hmm13:28
rjekIt does have tools/bin/13:28
KinnisonThis is most confuzzling :-(13:29
KinnisonWe've recently done from-scratch builds on other architectures13:29
Kinnisonwhich worked13:29
Kinnisonso it's not a systemic fault13:29
rjek"recently" ?13:29
Kinnisonlast Friday13:30
KinnisonCould you pastebin the meta file for stage2-fhs-dirs misc chunk?13:33
Kinnisonit should be in the /baserock of that failed chroot13:33
Kinnisonand the -bins ?13:35
Kinnisonthe symlink is entirely missing13:37
rjekHow do I fix this? :)13:40
richard_mawrjek: once we know how it happened we can provide advice, but until then any suggestions might be entirely wrong13:42
KinnisonStage2 FHS-Dirs does:13:43
Kinnison- ln -s "$PREFIX/bin" "$DESTDIR/bin"13:43
Kinnisonwhich *SHOULD* have put a bin symlink in place13:43
Kinnisonthe presence of etc/passwd etc suggests that the chunk ran properly13:43
KinnisonIt'd be handy if you happen to have the morph log of running the build for stage2-fhs-dirs so we can confirm if it ran that command13:43
richard_mawso, 1. is this file in the hardlink cache 2. is this file in the chunk artifact13:43
Kinnisonrichard_maw: I think you'll need to help rjek identify the chunk file so he can find out13:44
richard_mawok, run `morph --dump-config | grep tempdir` and look in the chunks subdir of the directory that command lists13:45
richard_mawthere's a cb3184d7bbd3991492725de83743fd48b9479a70d7b0ab42a858b6fffaaba1ae* subdirectory for each of the artifacts produced by stage2-fhs-dirs13:46
richard_mawI don't recall the naming off the top of my head, but look in those to see if any has a bin symlink13:46
richard_mawok, that's a possible culprit, now we need to work out whether it's the cache that's wrong or the chunk artifact13:49
richard_maw`morph --dump-config | grep cachedir` for the directory your cache is in, and `tar -tf $cachedir/artifacts/cb3184d7bbd3991492725de83743fd48b9479a70d7b0ab42a858b6fffaaba1ae.chunk.stage2-fhs-dirs-misc` and `tar -tf $cachedir/artifacts/cb3184d7bbd3991492725de83743fd48b9479a70d7b0ab42a858b6fffaaba1ae.chunk.stage2-fhs-dirs-bins` please13:51
rjekThere is no artifacts directory in /src/tmp13:53
richard_mawyour cache dir, not your temp dir13:53
rjekOh, sorry13:54
Kinnisonso the bin symlink is indeed entirely missing from the artifacts13:57
KinnisonVery odd13:57
* richard_maw can feel this draining his sanity13:58
Kinnisonrjek: Could you please confirm the SHA1 of stage2-fhs-dirs in your build-essential morphology?13:59
* rjek spends a few moments parsing that request to work out what is being asked.13:59
rjekKinnison: Where is it?  There isn't a stage2-fhs-dirs in build-essential.14:00
Kinnisonerm, there really ought to be14:01
Kinnisonit's line 116 in master's build-essential.morph14:01
Kinnisonwith the SHA1 on line 11814:01
rjekAh.  Sorry.  I couldn't see the SHA1 for the hashes.14:02
rjek  ref: 41bbb474cd4647ee715bc94c21c161d12a20deb414:02
richard_mawthat's up to date, and makes the symlink14:03
* Kinnison concurs14:04
* Kinnison also boggles14:04
rjekThis isn't the only thing that isn't working for me today that should, btw.14:04
KinnisonWhat FS is /src ?14:05
rjekSteve Jobs had a reality distortion field.  I have a reality breaking field.14:05
rjekKinnison: ext3.  (same filesystem as /, which is actually /mnt but I chrooted in)14:05
KinnisonOkay, that should be fine14:06
KinnisonWe're currently thinking that we want to cause the fhs-dirs chunk to fail to build so that we can see if the symlink is present in the .inst directory14:10
KinnisonI think the easiest (sic) way to do that is to morph edit stage2-fhs-dirs14:10
Kinnisonand then add 'false' to the end of the install commands14:10
KinnisonAnd then examine the failed/ chroot14:10
richard_mawwell, in stage2 it's not a chroot14:10
Kinnisonrjek: Do you think you could do that?14:10
Kinnisonrichard_maw: true14:10
Kinnisonrichard_maw: failed/ space then14:11
rjekGive me a few moments to try to understand what it is you're asking.14:11
rjek/src/workspace/baserock/robkendrick/mips64/baserock/baserock/definitions # morph edit stage2-fhs-dirs14:14
rjekERROR: morph edit needs the names of a system, a stratum and optionally a chunk as parameters14:14
* rjek tries to untwist in his mind where he is14:15
paulsherwoodrjek: wrong version of morph14:15
KinnisonThat's a surprisingly old morph error message actually14:15
rjekWell, it's what it's built and installed.14:15
Kinnisonrjek: based on the branch you and I made the other day?14:15
rjekIt's whatever's mentiong in baserock/robkendrick/mips64/14:16
KinnisonHow odd14:16
Kinnisonthat's ancient14:16
Kinnisonshows your commit on top of one from march14:16
* Kinnison is sure when we made your branch we based it off the current morph at that point14:17
rjekI'm not sure what I'll do if it turns out I need to build all this again.  But it will involve tears, and possibly a tantrum.14:20
KinnisonWe might get away with replacing your morph with one in /src and then making sure what you build subsequently uses the right thing14:20
* Kinnison is trying to work out how you ended up with a morph this old14:20
* Kinnison remembers sitting with you to make the branch14:21
Kinnisonand I'm sure there were commits from Sam from last week in it14:21
persiaShould be able to quickly rebase it against master.14:22
persiaOr, for safety, the commit in definitions.git today14:22
KinnisonCurrently richard_maw and I are erring on the side of rjek having a haunted machine because this is super-duper-wierd14:23
SotKKinnison: the morph ref in cross-bootstrap.morph points to a commit from March14:25
KinnisonOkay, for reference, I've helped rjek to update his morph to be rebased from current morph master14:44
Kinnisonand on his branch we've updated both the cross-bootstrap and the tools references to morph14:44
KinnisonUnfortunately this means his build is going from step 1 again14:45
Kinnisonso it'll be a while before we know if this worked14:45
persiaI suspect the chances are much better.14:45
rjekAt least 6 hours.14:45
rjekWhich basically means: tomorrow morning.14:45
persiaOut of curiosity, is there a way to cause cross-bootstrap to use the morph ref from tools?14:45
KinnisonCurrently, no.14:45
KinnisonI'd be interested in ideas on that front14:45
persiaWhat blocks it?14:45
Kinnisonpersia: cross-bootstrap needs to be minimal in order to be buildable -- the cross built environment is very quirky due to bootstrap stuff still being around, so we need to keep minimal the stuff we build14:46
Kinnisonthere's no easy way to refer from one stratum to another, and no way to share that data (currently)14:46
Kinnisonthus the separation and confusion14:46
* Kinnison would welcome any ideas on fixing this14:46
persiaCould morph (and a few other bits) be split out, to be consumed by cross-bootstrap, and whatever else needs them?14:47
KinnisonIt's tough because cross-bootstrap is a cross (sic) between core, foundation and tools14:47
Kinnisonso there's different build-dep structures in place14:48
* Kinnison would love to be able to unify things, but until and unless we have conditional control over stratum content, I doubt we can. (note: I don't say we can't, just that I doubt we can)14:49
persiaI don't understand why more modularisation wouldn't help.14:49
persiaHave core-minimal, foundation-minimal, and tools-minimal, and then build atop them.14:49
KinnisonI seem to recall there being a reason that wasn't easy to do, but I don't recall what the reason was14:50
persiaI suspect it of being easy to do, but hard to manage.14:50
persiaLots more strata adds confusion.14:51
persiaBut in this case, less strata has wasted a week or so :(14:51
Kinnisonfewer strata has indeed caused confusion14:52
KinnisonTo be fair, it'd have caused less confusion if we hadn't done the "morph edit chunkonly" syntax change14:52
Kinnisonbecause I'd have been more aware of the possibility of it being elsewhere14:52
rjekIt would be nice if "morph edit foo", when faced with multiple options for foo, instead of checking out out at random instead give you a list of more precise descriptions that you could copy-and-paste the appropriate one from.14:52
Kinnisonbut I take on responsibility14:52
persiaIt would be nice if `morph edit` returned an error about the lack of such a command.14:52
rjekActually, I'm not sure morph edit does select at random.  IME, it always selects the least appropriate :)14:52
* Kinnison is going to have to bow out of this discussion now. rjek: please hilight me if you have any updates on how your build goes :_)14:53
rjek2014-07-28 14:52:08 [Build 1/752] [stage1-binutils-bins] Removing staging area15:29
rjek2014-07-28 14:52:11 [Build 7/752] [stage1-gcc-bins] Building chunk stage1-gcc-bins15:29
* rjek wonders what happened to builds 2, 3, 4, 5, and 6.15:29
richard_mawrjek: they were stage1-binutils-{libs,misc,doc,...}15:32
richard_mawrjek: I've been working on a patch to fix this, and your issues with needing to remove large amounts of the cross-bootstrap script15:32
richard_mawsince their root cause is the same15:32
rjekAh, right.15:35
*** jonathanmaw [] has quit [Quit: Leaving]16:35
*** tiagogomes [] has joined #baserock16:47
*** fay_ [] has quit [Remote host closed the connection]16:50
*** tiagogomes [] has quit [Ping timeout: 264 seconds]16:52
*** bmottram [] has quit [Quit: Lost terminal]16:59

Generated by 2.15.3 by Marius Gedminas - find it at!