| *** gfinney_ has joined #baserock | 00:06 | |
| *** gfinney__ has quit IRC | 00:06 | |
| *** zoli__ has joined #baserock | 00:49 | |
| *** gfinney_ has quit IRC | 01:06 | |
| *** gfinney_ has joined #baserock | 01:06 | |
| *** zoli__ has quit IRC | 01:33 | |
| *** gfinney_ has quit IRC | 02:10 | |
| *** zoli__ has joined #baserock | 04:35 | |
| *** zoli__ has quit IRC | 04:39 | |
| *** zoli__ has joined #baserock | 05:45 | |
| *** zoli__ has quit IRC | 07:47 | |
| *** a1exhughe5 has joined #baserock | 08:04 | |
| pedroalvarez | radiofree: it looks standard (if it's this one) http://source.icu-project.org/repos/icu/icu/ | 08:10 |
|---|---|---|
| pedroalvarez | from lorry source code: | 08:12 |
| pedroalvarez | if layout == "standard": \n layout = { "trunk": "trunk", "tags": "tags/*", "branches": "branches/*" } | 08:12 |
| *** gfinney_ has joined #baserock | 08:25 | |
| *** mdizzle has joined #baserock | 08:45 | |
| *** jonathanmaw has joined #baserock | 08:51 | |
| *** zoli__ has joined #baserock | 08:51 | |
| *** bashrc has joined #baserock | 08:58 | |
| *** rdale has joined #baserock | 09:10 | |
| *** gary_perkins has joined #baserock | 09:14 | |
| *** lachlanmackenzie has joined #baserock | 09:14 | |
| petefoth | Interesting LWN Article on reproducible builds (for Android apps) at http://lwn.net/Articles/633106/ which references the work being done by F-Droid https://f-droid.org/wiki/page/Deterministic,_Reproducible_Builds | 09:14 |
| *** tiagogomes_ has joined #baserock | 09:28 | |
| *** ssam2 has joined #baserock | 09:34 | |
| *** ChanServ sets mode: +v ssam2 | 09:34 | |
| *** edcragg has joined #baserock | 09:41 | |
| *** straycat is now known as ofcourseistilllo | 09:51 | |
| *** lachlanmackenzie has quit IRC | 10:03 | |
| simonh_ | Can someone have a quick look at this morph file: http://paste.baserock.org/kexivovodu For mips64 we need to change a h-file to set the ABI correct, it worked with the repo being checked out locally, but not with sed. Does the sed instruction (Line 59) need to be done before configuration? | 10:09 |
| *** Krin has joined #baserock | 10:09 | |
| *** simonh_ is now known as mauricemoss_ | 10:10 | |
| *** flatmush has joined #baserock | 10:10 | |
| *** lachlanmackenzie has joined #baserock | 10:10 | |
| ssam2 | simonh_: surely you need a ; after it ? | 10:10 |
| ssam2 | maybe not | 10:10 |
| ssam2 | actually, your \'s will be eaten by the shell | 10:11 |
| ssam2 | try using ' quotes instead of " | 10:11 |
| mauricemoss_ | ssam2, I checked the mips.h while building gcc and the file was changed correct. | 10:12 |
| ssam2 | oh, then I don't know | 10:13 |
| ssam2 | you could try moving the sed command to pre-configure-commands | 10:13 |
| mauricemoss_ | Ok, I'll do that. | 10:14 |
| radiofree | lorry request http://paste.fedoraproject.org/187428/34130914/ | 10:22 |
| ofcourseistilllo | radiofree, looks fine to me | 10:28 |
| *** rdale_ has joined #baserock | 10:29 | |
| pedroalvarez | radiofree: i'd like to know if it's going to be possible to build from git before we lorry it | 10:30 |
| radiofree | yes it is | 10:30 |
| radiofree | why should that matter though? | 10:31 |
| radiofree | (it doesn't currently build for me on arm from a tarball...) | 10:31 |
| pedroalvarez | we have lorried some repos and after that we have noticed that those repos could only be built from tarball | 10:31 |
| radiofree | ah i see | 10:32 |
| *** rdale__ has joined #baserock | 10:32 | |
| *** rdale has quit IRC | 10:32 | |
| radiofree | well, to be honest, i haven't actually built it from this repo | 10:32 |
| radiofree | i downloaded the latest tarball release and built that | 10:32 |
| radiofree | what's the process for checking it builds from svn then? | 10:33 |
| * tiagogomes_ thinks that we should have the VCS lorried even if a tarball is being used | 10:34 | |
| * pedroalvarez thinks is a waste of resources | 10:34 | |
| radiofree | which resources? | 10:35 |
| *** rdale_ has quit IRC | 10:35 | |
| *** rdale has joined #baserock | 10:35 | |
| pedroalvarez | cpu and space in g.b.o, (if we lorry things that are not needed) | 10:35 |
| radiofree | it takes more time to submit a lorry request for a new tarball than to use a tag in a repo | 10:35 |
| pedroalvarez | I see, icu is in qtx strata. There shouldn't be a problem to install it from source | 10:36 |
| *** rdale_ has joined #baserock | 10:37 | |
| radiofree | ok, i'll try building it from svn release tag then, what's the process for that | 10:37 |
| *** rdale__ has quit IRC | 10:37 | |
| radiofree | i can use an svn repo in a strata? | 10:37 |
| pedroalvarez | no, that's not possible. | 10:38 |
| franred | radiofree, you can git-svn the repo into your devel machine and point it in your stratum with file:/// | 10:38 |
| pedroalvarez | let me check | 10:38 |
| franred | I think | 10:38 |
| ssam2 | you can use 'lorry' to run git-svn too, it's in devel systems now | 10:38 |
| ssam2 | it also doubles as a test that your lorry won't cause git-svn to go mad and break g.b.o. which it seems to sometimes do | 10:39 |
| ofcourseistilllo | heh maybe we could make morph run lorry in these situations | 10:39 |
| * ofcourseistilllo hides | 10:39 | |
| *** rdale has quit IRC | 10:39 | |
| radiofree | /bin/sh: lorry: not found | 10:40 |
| ofcourseistilllo | might need a more recent devel system? | 10:41 |
| pedroalvarez | anyway, the lorry looks ok, and I don't think you will have problems to build it | 10:42 |
| pedroalvarez | +1 | 10:43 |
| *** rdale_ has quit IRC | 10:43 | |
| radiofree | was that a +1 from you ofcourseistilllo? | 10:43 |
| * ofcourseistilllo nods | 10:43 | |
| radiofree | ta | 10:44 |
| *** fay__ has quit IRC | 10:59 | |
| franred | I have a question about apr and libapr, I've submitted a patch about apache where I create an apr chunk but I've seen (and pedroalvarez has pointed me out) that we have already a chunk called libapr which is the same. Im about to resubmit the 3rd version of the patch, do the people think I should replace my chunk name from apr to libapr? | 10:59 |
| *** fay_ has joined #baserock | 11:00 | |
| ofcourseistilllo | if they're idential and we already have libapr, then yeah i guess it'd be simpler to just use libapr? | 11:01 |
| tiagogomes_ | apr is a better name as it is the name of the project, however the lorry strata uses the libapr name | 11:02 |
| tiagogomes_ | franred, I think I wouldn't rename it for apache, and hope that someone sends a patch to rename libapr to apr for the lorry strata to be consistent | 11:04 |
| *** a1exhughe5 has quit IRC | 11:07 | |
| franred | ok, then I will leave it to apr and we can rename libapr afterwards | 11:17 |
| franred | thanks for the comments | 11:17 |
| *** a1exhughe5 has joined #baserock | 11:22 | |
| *** richard_1aw is now known as richard_maw | 11:46 | |
| pedroalvarez | I think that icu-svn lorry is going to timeout. Has been lorrying from one hour, and is still in converting commits of 2001 | 11:48 |
| tiagogomes_ | you may want to lorry just "trunk" | 11:49 |
| pedroalvarez | just trunk? and tags, no? | 11:50 |
| tiagogomes_ | does subversion has tags? | 11:50 |
| pedroalvarez | tiagogomes_: http://source.icu-project.org/repos/icu/icu/ | 11:51 |
| radiofree | :\ | 11:51 |
| tiagogomes_ | it seems so, but are branches and tags in subversion cheap? I had the idea that they were copies of the repository, which makes the repo to be very large in terms of disk space | 11:53 |
| pedroalvarez | those were my concerns before | 11:57 |
| bashrc | so what is a BSP? | 12:06 |
| nowster | bug squashing party? board specific something | 12:07 |
| CTtpollard | british sea power | 12:07 |
| tiagogomes_ | a customized kernel to run in some board | 12:08 |
| tiagogomes_ | Board Support Package | 12:08 |
| radiofree | not necessarily just the kernel, e.g for an i.mx bsp you might want to include the gpu binaries | 12:12 |
| radiofree | qt5 demos 'work' with xwayland, however mouse events don't seem to get passed through | 12:20 |
| radiofree | keyboard is fine | 12:22 |
| *** wschaller has joined #baserock | 12:23 | |
| Zara | Okay, working on horizon.morph; not totally sure what counts as configure-commands, build-commands and install-commands (so I'm not sure how to divide stuff up). Also not sure if I need to add some kind of 'cd' before python manage.py, or if I should be in the right place to just run it. | 12:25 |
| Zara | (horizon.morph is basically lines 25 and lines 31-55 of what was in openstack/usr/share/openstack/openstack-horizon-setup in fran's openstack branch, with a few little extras. Just in case we have some morphology experts hanging around.) | 12:25 |
| Zara | oh, yeah, also 'post-install-commands' | 12:27 |
| radiofree | it might be easier if you pastebin horizon.morph | 12:27 |
| ofcourseistilllo | isn't horizon just standard pbr? | 12:27 |
| pedroalvarez | ofcourseistilllo: yeah, it works without a morphology, but I believe Zara wants to add more commands to the install process | 12:28 |
| ofcourseistilllo | oh right | 12:28 |
| *** gary_perkins has quit IRC | 12:28 | |
| *** gary_perkins_ has joined #baserock | 12:28 | |
| pedroalvarez | Zara: afaik, it doesn't matter where you put things, but they will be run in the configure - build - install order | 12:30 |
| Zara | radiofree: okay, not sure how to copy out of vim into pastebin (I'm running vim on baserock) | 12:31 |
| pedroalvarez | is possible to put all of them in configure, I think, but is better to split them | 12:31 |
| tlsa | `tar` in the webtools stratum does not build at the moement | 12:31 |
| tlsa | /tar.build/gnu/quotearg.c:968: multiple definition of `quote' | 12:31 |
| pedroalvarez | tlsa: I think I remember that problem in the past | 12:31 |
| tlsa | collect2: error: ld returned 1 exit status | 12:31 |
| tlsa | I think it's because acl has been added to the foundation stratum recently | 12:32 |
| ofcourseistilllo | Zara, sprunge is pretty useful in situations like that http://sprunge.us/ | 12:32 |
| radiofree | there was some script to send it to paste.baserock.org as well | 12:33 |
| Zara | pedroalvarez: cool, I'll bear that in mind. | 12:33 |
| jjardon | Zara: or you can push your branch somewhere so we can take a look | 12:34 |
| Zara | currently skeletal and messy: http://paste.baserock.org/befuwuqige | 12:37 |
| ofcourseistilllo | Are you intentionally overriding the setuptools stuff? | 12:38 |
| Zara | What is the setuptools stuff? | 12:43 |
| radiofree | what exactly gets installed? there's no install commands | 12:44 |
| ofcourseistilllo | morph will automatically run python setup.by build, python setup.py install, but I think (though I'm not sure) that if you set each *-commands: to empty as you do in that morph you'll be overriding that default behaviour | 12:44 |
| radiofree | i'm assuming you want stuff in /etc/horizon/ at least, but i can't see where that's created | 12:44 |
| ofcourseistilllo | Might want to ask someone else to check that | 12:44 |
| Zara | ofcourseistilllo: ah, right, they're only empty atm because I'm not sure how to divide things up | 12:45 |
| ssam2 | i'm thinking of doing a Baserock 15.08 release tomorrow, anyone strongly for or against that ? | 12:45 |
| ssam2 | mostly to save everyone from having to faff with manually installing PyLRU in order to get the Morph build-graph speedups | 12:45 |
| ofcourseistilllo | sounds cool | 12:46 |
| ofcourseistilllo | Right, well I'm guessing you just want all your stuff in post-install? probably | 12:47 |
| SotK | ssam2: sounds like a good idea | 12:51 |
| Zara | ofcourseistilllo: Thanks, I think I'll do that for now since I want to check that the commands themselves work! Does anyone know, off the top of their head, an example morphology with multiline instructions? pipes were mentioned, not sure where to place them. | 12:54 |
| ofcourseistilllo | gitano.morph installs manually | 12:55 |
| franred | Zara, you only need to add build-system: python-distutils, remove configure- build- install- commands line | 12:57 |
| franred | and then you need to add a - in every command | 12:57 |
| franred | remember that if you have multiline commands you will need "- |" | 12:57 |
| franred | and you will add the commands in the next line | 12:58 |
| Zara | hm, just got 'KeyError: 'Unknown build system: python-disutils' | 13:05 |
| ssam2 | should be python-distutils | 13:05 |
| Zara | whoops | 13:06 |
| * Zara hangs head in shame | 13:06 | |
| Zara | thanks :) | 13:06 |
| Zara | I have only just realised that in 'removing temp sudirectory: failed', 'failed' is the name of a directory, not a message telling me that the removal failed. flagging it up since I might not be the only one who got the wrong end of the stick. | 13:14 |
| SotK | it took me a while to get that too, perhaps that output should be made more clear | 13:16 |
| Zara | yeah. I found it pretty funny when I realised, though, so it has that going for it. | 13:17 |
| petefoth | Seems like an opportunityu for a patch :) | 13:18 |
| *** brlogger has joined #baserock | 13:24 | |
| Zara | :) btw, other than morph gc, are there any quick things people know of that can speed up baserock? I find that the more builds I do, the slower it gets, so I'm assuming that something gets left behind somewhere. | 13:25 |
| ofcourseistilllo | ssam2, added a new option that can speed things up, and i think SotK and ssam2's speedups have been merged, so using a newer version of morph may help you, but as ssam2 said you'll need pylru installed | 13:28 |
| *** wschaller has quit IRC | 13:44 | |
| robtaylor | Zara: is your local filesystem btrfs? | 13:47 |
| robtaylor | Zara: or your /src mount in your vm? | 13:48 |
| robtaylor | Zara: if so, try a btrfs filesystem defrag on the folder (sometime when you have time and power) | 13:49 |
| robtaylor | and that reminds me, we need a unit that defrags/balances user mounts, and we should defrag/blanace on upgrades | 13:49 |
| Zara | okay, I think that /src is, since the quickstart guide says to format it that way. (unless I've massively misinterpreted something!) | 13:50 |
| * ofcourseistilllo uses ext4 for /src | 13:51 | |
| ofcourseistilllo | robtaylor, there should be a unit for /var now at least, which should speedup journalctl | 13:51 |
| tiagogomes_ | the last systemd release will speed up journalctl as well | 13:52 |
| ofcourseistilllo | oh except i don't think it got merged yet | 13:52 |
| Zara | :( running install_scripts | 13:52 |
| Zara | # post-install | 13:52 |
| Zara | # # mkdir -p /var/lib/horizon | 13:52 |
| Zara | mkdir: can't create directory '/var/lib/horizon': Read-only file system | 13:52 |
| Zara | ERROR: In staging area /src/tmp/staging/tmpu13Twf: running command 'sh -c mkdir -p /var/lib/horizon' failed. | 13:52 |
| ofcourseistilllo | Zara, I think you want to use $DESTDIR in that morph | 13:52 |
| robtaylor | ofcourseistilllo: it could just defrag any non-root rw btrfs mounts | 13:52 |
| ofcourseistilllo | $DESTDIR/$PREFIX usually | 13:53 |
| ofcourseistilllo | robtaylor, it could, I think richard_maw said autodefrag is going to be enabled by default soon though | 13:54 |
| robtaylor | that would be good | 13:55 |
| franred | Zara, you don't need $PREFIX this time, just $DESTDIR | 13:55 |
| franred | mkdir -p "$DESTDIR"/var/lib/horizon | 13:56 |
| richard_maw | ofcourseistilllo: I have no data about when autodefrag will be the default, just that the btrfs guys wanted to make it so. If we update to systemd v219 though, the journal will also be more performant, since systemd turns off CoW for journal files | 13:56 |
| * ofcourseistilllo nods | 13:57 | |
| Zara | okay, for now I'm going to make a new devel vm, so it might take me a little while to get back to sorting out horizon.morph | 13:57 |
| robtaylor | yeah, i would think it'd be good to do a timer unit, and maybe even sumbit it to systemd if anyone feels up to that. glory and mailing list discussion awaits the intrepid! | 13:57 |
| *** wschaller has joined #baserock | 14:03 | |
| Zara | saw this on irc a while ago but can't remember how to get round it: ERROR: Unknown architecture armv8l64 in morphology systems/build-system-armv8l64.morph | 14:07 |
| Zara | (when trying to use cycle.sh to upgrade to devel system) | 14:08 |
| robtaylor | Zara: a defrag will probaly take less time, tbh | 14:08 |
| SotK | Zara: are you using latest morph? | 14:10 |
| SotK | Zara: if not, either use latest morph or remove systems/build-system-armv8l64.morph from your definitions checkout | 14:11 |
| Zara | thanks, everyone :) I went with the 'build anew' option; slower but I'll be busy shortly so it can run in the background, and it has solved this issue for me before. | 14:21 |
| tiagogomes_ | SotK why remove that file would help? Does morph reads *every* definition? | 14:24 |
| SotK | tiagogomes_: yes, sometimes :( | 14:25 |
| tiagogomes_ | SotK, that's odd, any reason why? I'd expect that it would recursively read the morphs that were on the system definition | 14:26 |
| SotK | I think its historical reasons, but I could be mistaken | 14:27 |
| robtaylor | no, it'll be even more so going forward, if the conculsions from the summit play out | 14:28 |
| tiagogomes_ | which conclusion in particular? | 14:29 |
| robtaylor | tiagogomes_: see the writeup on declaritive defintiions | 14:31 |
| robtaylor | tiagogomes_: specifically " Components may be nested and contain other components. The | 14:32 |
| robtaylor | information which specifies a Component may be split across multiple | 14:32 |
| robtaylor | Definition files." | 14:32 |
| robtaylor | there was a general consensus that we wanted to remove meaning from file locations in definitions | 14:33 |
| robtaylor | - at least, that was my understanding ;) | 14:33 |
| * robtaylor is just a lurker really | 14:33 | |
| SotK | So we want to go back to referencing things by name rather than filename? | 14:35 |
| tiagogomes_ | I don't like that. I'd prefer something explicit like "component A is corresponds to subcomponent A1 and subcomponent A2 together" | 14:37 |
| tiagogomes_ | reading all morphs doesn't scale | 14:37 |
| SotK | I agree, I want to be able to say "component A contains component foo (from foo.def) and component bar (from things/bar.def)" | 14:39 |
| SotK | then morph only needs to look at A.def, foo.def and things/bar.def | 14:39 |
| tiagogomes_ | SotK exactly | 14:40 |
| paulsherwood | SotK: i want to say component A contains foo and bar. period. bar and foo are forced unique | 14:47 |
| paulsherwood | i'm happy for morph to parse all the definitions, because that takes almost no time | 14:48 |
| paulsherwood | but i expect we disagree on this | 14:48 |
| tiagogomes_ | I do :) | 14:48 |
| SotK | it depends on how big we expect the definitions repo to get I guess | 14:49 |
| paulsherwood | so you end up having to have two fields to identify a definition, as opposed to one | 14:49 |
| tiagogomes_ | Why not remove the name field instead? | 14:50 |
| SotK | but I also dislike that loading and validating them all causes errors seemingly unrelated to what you did, like Zara's earlier. | 14:50 |
| paulsherwood | tiagogomes_: because 'name' means name. so if there is only one field, it shoudl be name, imo | 14:51 |
| *** zoli__ has quit IRC | 14:51 | |
| paulsherwood | SotK: fair point. but that's morph specific. there's no reason the code *has* to behave that way | 14:51 |
| paulsherwood | for example - morph spits if build-depends: is missing. it could trivially default to [] | 14:52 |
| tiagogomes_ | what about doing similar to C. You define componentA to be composed of componentB and componentC, and you have a includes section that has filepaths for the components that define component A and B | 14:56 |
| paulsherwood | tiagogomes_: i think that's unnecessarily complicated? | 14:56 |
| tiagogomes_ | I don't think that is that complicated tbh, has an analogy to C regarding functions and include files, and adds extra flexibility | 14:58 |
| tiagogomes_ | just throwing ideas... | 14:58 |
| robtaylor | tiagogomes_: cos includes are the worst thing about C | 15:01 |
| robtaylor | tiagogomes_: richard_maw tells me parsing all of definitions is actually pretty fast | 15:02 |
| ratmice__ | I like not referencing file paths, among other reasons it makes it possible to send them easily via a port | 15:02 |
| robtaylor | (i think it was richard_maw at least, my memory isn't the best...) | 15:02 |
| ratmice__ | not sure if they are parsed incrementally e.g. parse the name field and only the rest if its required? | 15:03 |
| ratmice__ | anyhow that is a potential optimization that could occurr | 15:04 |
| robtaylor | yeah, tricky to get right but potential | 15:05 |
| robtaylor | tbh there's probably low hanging fruit in just fixing the yaml parser | 15:05 |
| richard_maw | robtaylor: it was paulsherwood who asserted it was actually fast, and Kinnison who backed it up that for the amount of yaml we have, yaml's not too slow | 15:07 |
| wdutch | I tried to deploy the latest devel x86_64 to openstack but it sticks at syslinux when booting, does anybody know what might be causing this? | 15:08 |
| robtaylor | richard_maw: ah thanks :) | 15:09 |
| richard_maw | wdutch: we've encountered some strangeness related to btrfs recently. I'd guess syslinux is having trouble with a btrfs format change | 15:10 |
| richard_maw | wdutch: which would require us to update our version of syslinux to cope, unfortunately syslinux has split out functionality into more binaries since the version we currently have integrated | 15:10 |
| richard_maw | wdutch: so things like pxe booting would also require update | 15:11 |
| robtaylor | we can probably parallelise yaml parsing as well for a decent speedup if needed | 15:11 |
| Kinnison | Parsing the yaml of a definitions repository should not be in the least bit slow | 15:13 |
| paulsherwood | < 1 second for me on macbookpro | 15:16 |
| SotK | what if the definitions repository in question was much larger than our current definitions repository? | 15:21 |
| jjardon | tlsa: tar builded fine here with this morphology: http://paste.baserock.org/titavipeco | 15:23 |
| robtaylor | SotK: if it was 10x bigger, you' might hit 8 seconds? | 15:24 |
| robtaylor | with no optimisations | 15:24 |
| robtaylor | its not going to be the main time taker in a build... | 15:24 |
| jjardon | Does anyone know what FORCE_UNSAFE_CONFIGURE=1 is for? | 15:24 |
| jjardon | nevermind: http://www.linuxfromscratch.org/lfs/view/development/chapter06/tar.html | 15:25 |
| *** petefoth has quit IRC | 15:26 | |
| richard_maw | we don't run `make check` anyway, so it doesn't really matter | 15:26 |
| tlsa | jjardon: thanks, I'll try that | 15:27 |
| SotK | If I want to build a component with very few contents (say I need to look at 3 or 4 definitions), I don't want to spend 8 seconds loading all ~6000? files when they aren't useful to me. | 15:27 |
| SotK | Especially when I've just merged a patch series reducing the time taken for morph to decide what it needs to build | 15:29 |
| *** zoli__ has joined #baserock | 15:30 | |
| richard_maw | SotK: that's one trade-off, but to be able to do that you need every definition file to list file paths to what it needs to look at next, which means we either need one defined thing you can build per-file, or we need to specify both file path and name all the time | 15:30 |
| paulsherwood | SotK: in the use-case you describe, i think you would discover other issues, deeper than a few seconds | 15:30 |
| richard_maw | plus, it makes it more annoying to rearrange the definitions tree if you have to refer to everything by file path rather than name | 15:30 |
| * richard_maw has been on both sides of this argument before | 15:30 | |
| paulsherwood | heh :) | 15:30 |
| rjek | The faster the whole lot is parsed, the sooner morph can tell me to fix my stupid typo. | 15:34 |
| rjek | :) | 15:34 |
| robtaylor | richard_maw gives a good summary | 15:35 |
| paulsherwood | actually, rjek has a point. if the typo is in a later step, and you haven't parsed it, you won't know til it gets there | 15:35 |
| ofcourseistilllo | we install pyyaml without libyaml? | 15:35 |
| ofcourseistilllo | could we not parse faster if we installed with libyaml and used Cloader? | 15:37 |
| pedroalvarez | slow parsing is not an issue afaik | 15:37 |
| * SotK wonders how much faster that would make the huge yaml dump when distbuild calculates the build graph | 15:37 | |
| richard_maw | ofcourseistilllo: possibly, but you get better error messages out of the python loader, and the python loader is more extensible | 15:38 |
| ssam2 | SotK: I've realised that huge yaml dump isn't actually needed any more | 15:38 |
| ssam2 | the controller doesn't need to pass the build graph to the workers, the workers can just calculate it for themselves | 15:38 |
| ssam2 | since it will take 1-2 seconds each time | 15:39 |
| ssam2 | so the output of serialise-artifact can be greatly simplified too | 15:39 |
| rjek | paulsherwood: Yes, I've been bitten by that more times than I care to enumerate. | 15:39 |
| ssam2 | SotK: also, I think I've hit a bug in morph master which we have caused | 15:40 |
| SotK | ssam2: oops :) | 15:40 |
| ofcourseistilllo | richard_maw, *nod* | 15:40 |
| SotK | ssam2: what is it? | 15:40 |
| ssam2 | SotK: morph was ignoring a .morph file even though one was present. when I deleted /src/cache/detect-chunk-buildsystems.pickle it stopped ignoring it | 15:40 |
| ssam2 | i was using temporary build branches created by 'morph build', which might be part of what caused it | 15:41 |
| SotK | hmm | 15:42 |
| tlsa | jjardon: with your morphology, tar failed to build for me, with /tar.build/gnu/quotearg.c:968: multiple definition of `quote' | 15:42 |
| tlsa | jjardon: are you using the latest definitions? The change that broke it was just 1 week ago | 15:44 |
| jjardon | tlsa: mmm, strange, Im using current definitions master, yes | 15:44 |
| tlsa | oddness | 15:44 |
| SotK | ssam2: I guess we should try and get the morphology before we get the cached build system then? | 15:45 |
| jjardon | tlsa: are you using lastest morph? | 15:45 |
| tlsa | I did what the wiki said to get the latest morph on friday | 15:46 |
| tlsa | don't know beyond that | 15:46 |
| jjardon | uh, after upgrading morph I can not run it in my chroot anymore: http://paste.baserock.org/acacuvofug | 15:47 |
| pedroalvarez | jjardon: http://wiki.baserock.org/using-latest-morph/ | 15:47 |
| tlsa | tar builds when I add --without-posix-acls the configure command | 15:48 |
| ssam2 | SotK: would that not negate the speedup? | 15:48 |
| ssam2 | maybe if we try and get it from local definitions only it'd be fine | 15:49 |
| jjardon | pedroalvarez: are the pylru change announced somewhere? | 15:49 |
| ssam2 | yeah, that makes sense actually | 15:49 |
| ssam2 | jjardon: only as part of the patch reviews on baserock-dev | 15:49 |
| SotK | yeah, if we only look in local definitions it will be OK (which was something we were going to do anyway I think) | 15:49 |
| jjardon | pedroalvarez: thanks btw | 15:51 |
| radiofree | ssam2: 2015-02-19 15:54:04 [Build 271/277] [qtwebkit] Starting actual build: qtwebkit bc9843a | 15:56 |
| ssam2 | thanks :) | 15:57 |
| ssam2 | hopefully it'll complete before I go home | 15:57 |
| jjardon | richard_maw: FYI, the build actually fail if you dont use FORCE_UNSAFE_CONFIGURE=1 : "configure: error: you should not run configure as root (set FORCE_UNSAFE_CONFIGURE=1 in environment to bypass this check)" | 15:57 |
| richard_maw | sheesh | 15:58 |
| tiagogomes_ | I wonder what weird things configure is doing to not be recommended to run as root | 15:59 |
| tiagogomes_ | jjardon for which repo that happens? | 16:00 |
| jjardon | tiagogomes_: tar | 16:00 |
| pedroalvarez | tiagogomes_: good thing is that we don't care, given that the build environment is isolated and protected :) | 16:01 |
| tiagogomes_ | pedroalvarez true :) | 16:02 |
| *** CTtpollard has quit IRC | 16:03 | |
| jjardon | can I force morph to build a chunk? | 16:03 |
| radiofree | ssam2: it should take about 2 hours... | 16:04 |
| ssam2 | jjardon: what do you need to do? | 16:05 |
| ssam2 | if you change the .morph file it'll rebuild the chunk | 16:05 |
| jjardon | ssam2: I upgrade morph I and want to build tar again, but morph seems to think its not needed | 16:05 |
| tiagogomes_ | I thought the chunks depended on the morph version | 16:07 |
| jjardon | I removed the tar cache in the end | 16:07 |
| ssam2 | jjardon: morph is probably right in that case. upgrading morph shouldn't change how chunks are built usually | 16:08 |
| jjardon | ssam2: sure, thats why I ask if you can force a rebuild ;) | 16:09 |
| ssam2 | tiagogomes_: there's a separate 'definitions-version' field (or something) that goes into the cache key and that can be used to cause rebuilds | 16:09 |
| ssam2 | I think | 16:09 |
| ssam2 | it's called 'metadata-version' in fact. in morphlib/cachekey.py | 16:09 |
| ssam2 | morphlib/cachekeycomputer.py even | 16:10 |
| ssam2 | i wonder if that should be linked to the definitions.git/VERSION file or not ... | 16:15 |
| ssam2 | probably should be separate, because Morph could change how it builds stuff in ways that don't affect the actual functionality, and aren't necessarily tied to the format of definitions.. | 16:17 |
| *** CTtpollard has joined #baserock | 16:18 | |
| * richard_maw thinks we ought to define an intermediate format that contains everything needed for the build, so we can decouple the format in definitions.git from what's needed to build | 16:19 | |
| *** jonathanmaw has quit IRC | 16:22 | |
| *** ssam2 has quit IRC | 16:24 | |
| *** ssam2 has joined #baserock | 16:32 | |
| *** ChanServ sets mode: +v ssam2 | 16:32 | |
| Zara | has anyone seen this? ERROR: Failed to list tree in ref 2bc8555f0428af81c2d067aa76168ed7bc5e0179 of repo github:giampaolo/psutil | 16:34 |
| ssam2 | not seen that before. check if there's more detailed info at the bottom of /src/morph.log | 16:36 |
| ssam2 | might be a random network error | 16:36 |
| jjardon | tlsa: yeah, latest morph and latest definitions and I can compile tar without problems with the morph file I pasted before. weird | 16:50 |
| Zara | morph.log: File "/src/morph/morphlib/remoterepocache.py", line 79, in ls_tree | 16:52 |
| Zara | raise LsTreeError(repo_name, ref) | 16:52 |
| Zara | LsTreeError: Failed to list tree in ref 2bc8555f0428af81c2d067aa76168ed7bc5e0179 of repo github:giampaolo/psutil | 16:52 |
| Zara | I could pastebin some of the stuff before that, but that seemed like the most relevant bit | 16:52 |
| Zara | oh wait, actually I should pastebin, gimme a sec | 16:53 |
| tlsa | jjardon: that is strange :s | 16:54 |
| tlsa | jjardon: I did get it working for me anyway | 16:54 |
| Zara | http://paste.baserock.org/ezodegetef | 16:54 |
| pedroalvarez | Zara: oh, the problem might be that the sha1 doesn't exist anymore | 16:55 |
| ssam2 | zara: from the log file: 2015-02-19 16:51:30 ERROR Caught exception: HTTP Error 404: Not Found | 16:56 |
| pedroalvarez | that too | 16:57 |
| nowster | is there something that morph would do which would cause a kernel tree to become "-dirty" when compiled? | 16:58 |
| *** a1exhughe5 has quit IRC | 16:59 | |
| ssam2 | the configure or build commands in the reference systems might do something | 17:00 |
| ssam2 | e.g: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/strata/bsp-x86_64-generic/linux-x86-64-generic.morph does 'make defconfig' | 17:01 |
| ssam2 | would that make the tree 'dirty' ? | 17:01 |
| nowster | it's just 'make foo_defconfig; scripts/config -e BLAH; yes "" | make oldconfig' | 17:01 |
| ssam2 | right. | 17:02 |
| ssam2 | morph only does what it's told inside the build directory, it doesn't write any random files in there as far as I know | 17:03 |
| nowster | the only thing I should have done is "make $MAKEFLAGS" instead of "make -j3 all" | 17:03 |
| nowster | I wonder if it's timestamps. Cached chunks extract with very old dates, eg. "1991-08-24 23:00:00" | 17:07 |
| ofcourseistilllo | to be fair if that's the error you get now because you get a url not found then we should probably give a better message | 17:16 |
| *** CTtpollard has quit IRC | 17:17 | |
| ssam2 | zara, franred: try branch sam/fix-lstree-error | 17:18 |
| ofcourseistilllo | :) | 17:19 |
| jjardon | Can I have a quick review to update to latest libtool tarball? http://paste.baserock.org/enocihixod | 17:20 |
| pedroalvarez | jjardon: +1'5 | 17:22 |
| pedroalvarez | jjardon: why this minour upgrade? | 17:22 |
| jjardon | pedroalvarez: it fixes a regression that makes builds slower | 17:23 |
| pedroalvarez | jjardon: here you have the +0'5 left | 17:23 |
| jjardon | :) thanks! | 17:23 |
| jjardon | I tried to use libtool from git instead a tarball but that will require quite a lot of changes | 17:24 |
| nowster | gah! | 17:26 |
| nowster | HEAD detached at d1a9b75 | 17:26 |
| nowster | nothing to commit, working directory clean | 17:26 |
| nowster | # make kernelreleas | 17:27 |
| nowster | e | 17:27 |
| nowster | 3.19.0-gd1a9b75 | 17:27 |
| nowster | it's nice and clean... | 17:27 |
| nowster | I wonder what's dirtying it... | 17:27 |
| franred | ok, I always has a problem when I create link in baserock on install-commands or post-install-commands, if I want to link something from an installed file should I do: ln -sf "$DESTDIR"/path_to_foo "$DESTDIR"/path_to_bar or I don't have to put the latest "$DESTDIR"? | 17:30 |
| pedroalvarez | you don't have to put the former | 17:31 |
| richard_maw | you must not put DESTDIR in the first argument, as it's interpreted at link dereference time, where DESTDIR will no longer exist | 17:41 |
| *** mdizzle has quit IRC | 17:42 | |
| jjardon | to avoid busybox to put binaries in /bin, would it be enough to create a symlink from bin -> /usr/bin just before the busybox configure step? Or another stuff has to be taken in account? | 17:49 |
| radiofree | fedora uses symlinks for that, i thought pedroalvarez was looking into that? | 17:49 |
| ssam2 | jjardon: I believe /bin is already a symlink to /tools/bin in stage 3. I might be wrong though | 17:50 |
| ssam2 | try it and see :) | 17:51 |
| *** bashrc has quit IRC | 17:55 | |
| pedroalvarez | jjardon: are you looking at the "kill /bin /sbin /lib" story? | 17:56 |
| jjardon | pedroalvarez: yes | 17:57 |
| pedroalvarez | jjardon: I've been working on it (as the story says) | 17:57 |
| pedroalvarez | jjardon: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/log/?h=baserock/pedroalvarez/usr-merge | 17:58 |
| pedroalvarez | jjardon: still a WIP, though | 17:59 |
| jjardon | pedroalvarez: oh, brilliant | 17:59 |
| pedroalvarez | jjardon: you are welcome to continue the work, I haven't had spare time last weeks to play with this | 18:00 |
| jjardon | Is ther any way to get notifications from storyboard? Ive created that story and Id expect to receive an email or something | 18:00 |
| nowster | grr DEPMOD 3.19.0-gd1a9b75-dirty | 18:02 |
| nowster | so something between make and install has dirtied it... | 18:02 |
| ssam2 | jjardon: I think it doesn't work yet | 18:02 |
| ssam2 | as in, feature missing in upstream storyboard. I could be wrong | 18:02 |
| jjardon | mmm, building current definitions morph is trying to build systemd instead get it from the cache ... Any idea what can be the problem? | 18:15 |
| jjardon | seems cache.baserock.org is down? | 18:17 |
| ssam2 | nope, cache.baserock.org seems fine | 18:19 |
| ssam2 | systemd 219 didn't get merged yet afaik | 18:20 |
| jjardon | yep, I know. Nevermind, probably I did something weird here | 18:21 |
| richard_maw | ATTENTION: The current combination of Linux 3.19 and btrfs-progs 3.18.2 results in it creating disk images that are not bootable by syslinux. http://paste.baserock.org/ibuqiyucup will force btrfs-progs to something that our current version of syslinux can read. | 18:22 |
| richard_maw | I haven't proposed a proper patch series for it because I'm head-down in OpenStack stuff, and we might need to support building with latest morph and older btrfs-progs, so we'd need to test for available features first. | 18:23 |
| richard_maw | I just thought it would be useful for anyone who is encountering this bug to have a work-around until we get around to fixing it properly | 18:24 |
| ssam2 | thanks for the fix up | 18:24 |
| pedroalvarez | indeed thanks | 18:27 |
| ssam2 | i propose merging this to master of morph, so I can do a 15.08 release tomorrow that isn't broken | 18:28 |
| ssam2 | this might cause new morph to not work in old baserock (with old btrfs-progs), but we can cross that bridge if we come to it | 18:28 |
| ssam2 | i'll send a patch tomorrow anyway | 18:31 |
| *** wschaller has quit IRC | 18:34 | |
| *** tiagogomes_ has quit IRC | 18:34 | |
| *** lachlanmackenzie has quit IRC | 18:34 | |
| nowster | now I'm really puzzled! | 18:37 |
| nowster | 3.19.0-gd1a9b75-dirty | 18:37 |
| nowster | HEAD detached at d1a9b75 | 18:37 |
| nowster | nothing to commit, working directory clean | 18:37 |
| jmacs | What's the confusion? | 18:39 |
| nowster | just wondering why it was clean during the build but is now -dirty. | 18:40 |
| jmacs | Ah. No idea. | 18:41 |
| ofcourseistilllo | thanks richard_maw | 18:42 |
| *** edcragg has quit IRC | 18:47 | |
| *** edcragg has joined #baserock | 18:49 | |
| *** zoli__ has quit IRC | 18:50 | |
| *** zoli__ has joined #baserock | 18:50 | |
| *** ssam2 has quit IRC | 18:54 | |
| *** gfinney_ has quit IRC | 19:10 | |
| *** gfinney__ has joined #baserock | 19:10 | |
| *** gfinney__ has quit IRC | 19:15 | |
| *** gary_perkins_ has quit IRC | 19:44 | |
| *** zoli__ has quit IRC | 19:44 | |
| *** edcragg has quit IRC | 21:02 | |
| radiofree | lol | 22:11 |
| radiofree | jetson build times, qtwebkit, max-jobs:1 = [Build 271/277] [qtwebkit] Elapsed time 03:44:45 | 22:12 |
| radiofree | add swap (just a 10GB file on the SSD, overkill...) [Build 271/277] [qtwebkit] Elapsed time 01:12:48 | 22:12 |
| radiofree | (that's with max-jobs: 4) | 22:12 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!