| *** pacon has joined #baserock | 00:45 | |
| *** rdale_ has joined #baserock | 02:30 | |
| *** rdale has quit IRC | 02:34 | |
| *** jjardon has quit IRC | 03:07 | |
| *** zoli__ has joined #baserock | 05:08 | |
| *** zoli__ has quit IRC | 05:16 | |
| *** pacon has quit IRC | 05:17 | |
| *** sambishop has quit IRC | 05:26 | |
| *** jjardon has joined #baserock | 06:09 | |
| *** zoli__ has joined #baserock | 06:22 | |
| straycat | wow #perl is hard work | 07:23 |
|---|---|---|
| paulsherwood | indeed :) | 07:26 |
| paulsherwood | ah, you mean the channel, not the language? | 07:26 |
| Kinnison | https://twitter.com/fioroco/status/608051214027259906 | 07:30 |
| *** gary_perkins has joined #baserock | 07:34 | |
| *** Stanto has joined #baserock | 07:36 | |
| straycat | paulsherwood, yes the channel :) | 07:39 |
| *** mariaderidder has joined #baserock | 07:42 | |
| *** edcragg has joined #baserock | 07:46 | |
| paulsherwood | :-) | 07:51 |
| Kinnison | Large IRC channels are hard work | 07:51 |
| Kinnison | s'one reason I only tend to contribute to smaller projects | 07:51 |
| paulsherwood | Kinnison: interesting. i had not realized that. how dumb am i?! | 07:52 |
| Kinnison | No idea, but your shift-key seems to be slightly broken | 07:53 |
| paulsherwood | you mean, missing caps at the start of sentences, i assume :) | 07:56 |
| Kinnison | You, once again, assume incorrectly. I wonder how long it'll take you to work it out? :-D | 07:57 |
| paulsherwood | oh dear, that's a puzzler. is there a prize? | 07:58 |
| paulsherwood | s/YBD/ybd/ ? | 07:58 |
| Kinnison | A reduction in my ire, should you get it right; perhaps. | 07:58 |
| paulsherwood | i give up, already. | 07:59 |
| Kinnison | Your brother has a similar issue, but I managed to correct him on it. | 08:00 |
| *** mariaderidder has quit IRC | 08:05 | |
| *** sambishop has joined #baserock | 08:06 | |
| *** jonathanmaw has joined #baserock | 08:14 | |
| straycat | SotK, I don't exactly get what you're on about with https://gerrit.baserock.org/#/c/793/2/morphlib/plugins/deploy_plugin.py@515 I'm not sure why you think prefixing PYTHONPATH with a colon is a good idea | 08:20 |
| SotK | I guess I can just set PYTHONPATH to $extensions_dir rather than trying to append $extensions_dir to the existing PYTHONPATH? | 08:24 |
| straycat | no | 08:25 |
| Kinnison | I'd much rather you appended it | 08:25 |
| Kinnison | or pre-pended it if you're worried about needing to override stuff | 08:25 |
| SotK | straycat: then what should I do? | 08:27 |
| straycat | if 'PYTHONPATH' in final_env: final_env['PYTHONPATH'] += ':%s' % exstensions_dir else: final_env['PYTHONPATH'] = extensions_dir ... or something? | 08:27 |
| SotK | sure, that just seemed messier and as far as I know having PYTHONPATH start with ':' doesn't harm anything, though I could be wrong | 08:29 |
| *** edcragg has quit IRC | 08:30 | |
| pedroalvarez | heh, so max_connections=100 in postgresql is not enough for openstack | 08:31 |
| Kinnison | pedroalvarez: erk | 08:31 |
| Kinnison | pedroalvarez: Surely each API needs 1 connection and that's it | 08:32 |
| pedroalvarez | Kinnison: I hope that's true.. http://paste.baserock.org/nuweyosetu | 08:33 |
| Kinnison | Holy batshit insanity, Stackman! | 08:33 |
| straycat | you could always final_env['PYTHONPATH'] = ''.join((pythonpath, ((':%s' if 'PYTHONPATH' in final_env else '%s') % extensions_dir)) ? :3 | 08:34 |
| * Kinnison is never quite happy with python's use of if/else as ternary operator | 08:35 | |
| straycat | i don't know i quite like it | 08:37 |
| straycat | it's certainly more readable to someone unfamiliar with the language | 08:37 |
| *** zoli__ has quit IRC | 08:42 | |
| *** mariaderidder has joined #baserock | 08:47 | |
| paulsherwood | +1 | 08:48 |
| *** ssam2 has joined #baserock | 08:56 | |
| *** ChanServ sets mode: +v ssam2 | 08:56 | |
| straycat | Zara, I made a bit of a hash of the change history on your import change, sorry, next time I won't use the web ui to update commit messages >.> | 08:59 |
| Kinnison | Web UI :( | 08:59 |
| straycat | it needs more vim :( | 09:00 |
| Kinnison | and vigor | 09:00 |
| *** zoli__ has joined #baserock | 09:02 | |
| richard_maw | straycat: let's have multiple lines for readability :-) | 09:10 |
| richard_maw | if 'PYTHONPATH' not in os.environ:\n final_env['PYTHONPATH'] = extensions_dir\nelse:\n final_env['PYTHONPATH'] = ':'.join((os.environ['PYTHONPATH'], extensions_dir)) | 09:12 |
| *** edcragg has joined #baserock | 09:16 | |
| *** lachlanmackenzie has joined #baserock | 09:22 | |
| straycat | richard_maw, sure :) | 09:23 |
| radiofree | 2015-06-09 00:52:49 [build-system-armv8l64] Elapsed time 03:13:39 | 09:28 |
| radiofree | \o/ | 09:28 |
| paulsherwood | radiofree: on mooshot, or mustang? | 09:36 |
| radiofree | mustang | 09:36 |
| paulsherwood | (or something else? | 09:36 |
| paulsherwood | ) | 09:36 |
| paulsherwood | aha! w00t! | 09:36 |
| radiofree | apart from that gcc hitch it went smoothly | 09:36 |
| radiofree | (you have to do something similar with gcc as you do with fhs-dirs, i.e when it fails, start the build again, then it should work) | 09:36 |
| Kinnison | that's "not good"(tm) | 09:37 |
| paulsherwood | radiofree: i think i'll remove the fork, and settle for the slowdown, unless someone has a better idea? | 09:37 |
| paulsherwood | Kinnison: i fork to remove current sandbox, while starting work on next chunk... | 09:37 |
| Kinnison | That seems.... risky | 09:37 |
| paulsherwood | it clearly is | 09:38 |
| Kinnison | Removal of the sandbox shouldn't be that expensive | 09:38 |
| radiofree | +1 remove fork | 09:38 |
| paulsherwood | but it saves a significant amount of time on my machine | 09:38 |
| paulsherwood | (eg a few mins over a whole build) | 09:38 |
| paulsherwood | i'll remove the fork | 09:38 |
| Kinnison | Start by removing the /foo.build/.git then /foo.inst/ and then the rest | 09:38 |
| radiofree | i think a few mins is better than the build failing when you're asleep | 09:38 |
| paulsherwood | yup | 09:38 |
| Kinnison | Optimising things like sandbox removal time seems fairly daft IMO | 09:39 |
| rjek | Move it out of the way, unlink it in the background | 09:39 |
| Kinnison | the issue is that he's doing background unlinking and the result is failed builds due to disk space issues presumably | 09:40 |
| SotK | the builds fail trying to mount a non-existent directory if this is the issue I think it is | 09:41 |
| Kinnison | aah | 09:41 |
| Kinnison | whatever it is, it's madness to background the unlinking IMO | 09:41 |
| paulsherwood | it's not in the way. it's that the sanbox command tries to lock it iiuc | 09:41 |
| Kinnison | it's not expensive | 09:41 |
| paulsherwood | anyway, i've pushed the fix. radiofree would you mind re-running from clean on that mustang? i'd really like to know how much time the full process takes on that board | 09:42 |
| paulsherwood | Kinnison: ybd is a series of experiments. this one taught me something :-) | 09:42 |
| Kinnison | :) | 09:43 |
| straycat | ssam2, think the sandboxlib could be useful for the import tool, given we're executing package code in some cases? | 09:46 |
| radiofree | paulsherwood: ok | 09:47 |
| Zara | straycat: nw. I'm about to push some changes, so sorry in advance if the commit message goes weird for a bit and there's noise | 09:50 |
| ssam2 | straycat: definitely... good thinking | 09:51 |
| straycat | Zara, ahh okay, maybe download the new change set and squash your changes into that? | 09:51 |
| straycat | ssam2, cool | 09:52 |
| SotK | straycat: I don't understand, what needs changing in the commit message? | 09:52 |
| SotK | (of https://gerrit.baserock.org/#/c/793/) | 09:52 |
| Zara | straycat: if I do git rebase -i and copy the current commit message on gerrit with line breaks, will that work out the same? (given that that's what I just did but I haven't pushed it yet, so there's still time :P) | 09:53 |
| ssam2 | perryl: do you fancy putting together a public page somewhere for the bit-for-bit work so we can link to it from http://wiki.baserock.org/doing-stuff-with-baserock/ ? | 09:53 |
| straycat | "Version 6 allows deployment extensions to depend only on code which is also in definitions and the Python standard library." except I could set PYTHONPATH before running morph deploy? | 09:53 |
| ssam2 | perryl: nothing complex, can be a storyboard story, or a wiki page, or anything | 09:53 |
| richard_maw | SotK: perhaps rolling the PYTHONPATH change into version 5 would be more appropriate than adding a version 6 | 09:54 |
| perryl | ssam2: sure, i'll try and get something written down today if possible | 09:54 |
| straycat | Zara, sure that'd be fine, just thought it might be slightly more error prone, up to you :) | 09:55 |
| SotK | straycat: I discussed the "set PYTHONPATH when running morph deploy" change with ssam2 elsewhere and he pointed out that it is still a break in compatibility (even though a work aroundis possible) | 09:55 |
| SotK | richard_maw: I'd be fine with that | 09:55 |
| richard_maw | SotK: cool. Possibly the best way to do this would be to roll both into one series | 09:56 |
| paulsherwood | jonathanmaw, radiofree - please note jeremiah's email on GENIVI patch review - would either or both of you care to comment? | 09:56 |
| paulsherwood | (on the list) | 09:56 |
| straycat | SotK, okay, not sure I follow, i'm just saying the commit message doesn't match my understanding, there's nothing in that change that restricts extensions to using only code in the extensions dir and the python standard library | 09:56 |
| radiofree | well i already replied to him in the other thread | 09:57 |
| radiofree | will do it later i'm in the middle of something atm | 09:57 |
| richard_maw | SotK: shall I bring your patch into my series? | 09:57 |
| SotK | richard_maw: sure | 09:57 |
| paulsherwood | radiofree: ack | 09:59 |
| SotK | straycat: maybe it needs rewording then, I didn't mean to imply restriction, just that they could now use code which is in the extensions dir (I have a branch which moves writeexts.py into there in the same way as I moved the extensions, and further patches to make the extensions depend on just code in there rather than morphlib and cliapp) | 09:59 |
| richard_maw | SotK: don't you need to conditionalise your code on the version? | 10:00 |
| Zara | (huh, that's weird, I changed the error message but it seems it wasn't saved? gah, gimme a sec) | 10:00 |
| SotK | richard_maw: why would I need to? | 10:01 |
| paulsherwood | SotK: iiuc ybd manages to avoid having to think about definitions version so far... will that still be the case after this? | 10:01 |
| richard_maw | SotK: I needed to for my change, I'm just trying to work out for myself whether you need to do so as well, and hoped you'd already thought about it and had an answer. | 10:02 |
| richard_maw | paulsherwood: AIUI ybd manages that by only caring about needing to build the latest definitions anyway | 10:02 |
| ssam2 | paulsherwood: it can ignore the versioning information forever. but users may report bugs that turn out to be due to version mismatches, and you might waste time working out that's what they are | 10:02 |
| paulsherwood | richard_maw: correct :) | 10:02 |
| ssam2 | it might be better to keep track of which version YBD is tested against | 10:03 |
| paulsherwood | ssam2: that's the approach for the moment, but it may not scale, of course :) | 10:03 |
| SotK | I figured that the change shouldn't affect anything in older versions | 10:03 |
| paulsherwood | ssam2: yes, or have ybd and definitions in the same repo for example. i wonder what other approaches are possible. neither of these seems *ideal* | 10:04 |
| SotK | (except it would put the definitions repo in PYTHONPATH if being run on defintions from before the extensions were moved) | 10:04 |
| paulsherwood | good point, SotK | 10:05 |
| richard_maw | SotK: which theoretically could cause issues, but I'll accept that it's vanishingly unlikely | 10:05 |
| richard_maw | SotK: I'd be happier with it explicitly checking, but I'll accept it not doing so. I'll amend the commit message to mention that this is version 5 rather than 6. | 10:08 |
| SotK | richard_maw: thanks | 10:09 |
| paulsherwood | richard_maw: am i right in thinking that your patches addressed the binary-strip wishlist item at https://github.com/devcurmudgeon/ybd/issues/30 ? | 10:12 |
| richard_maw | paulsherwood: not entirely. The stripped symbols are currently included in the result system artifact by default, while you wanted them excluded by default. | 10:13 |
| paulsherwood | ah, ok. how would one cause them to be excluded? | 10:14 |
| richard_maw | For morph I was hoping to amend the artifact splitting rules, and extend the logic so that the split rules for an artifact can declare whether they are included by default. | 10:14 |
| kejiahu_ | build of syslinux fails on armv7 using YBD, seems the target platform parameter hasn't been passed to it correctly. http://paste.baserock.org/ufiwimodit | 10:14 |
| paulsherwood | radiofree: have you seen that before? ^^ | 10:15 |
| paulsherwood | richard_maw: and for ybd? | 10:15 |
| richard_maw | paulsherwood: you could just remove everything from /usr/lib/debug | 10:15 |
| SotK | paulsherwood: why do ybd system artifacts contain /foo-system.inst and /foo-system.build? | 10:15 |
| richard_maw | since you don't support artifact splitting in any meaningful way currently anyway | 10:15 |
| paulsherwood | SotK: i don't know - iirc Kinnison worked on the system-level processing. i can investigate, though... but not right now | 10:16 |
| richard_maw | SotK: because there's no special case for system artifacts, ybd does all the set up, but sets DESTDIR=/, so the whole thing gets tarred up | 10:17 |
| paulsherwood | there is some special-casing for system now iiuc - Kinnison tweaked it because system-integration-commands are different i think | 10:18 |
| * paulsherwood would prefer no special case, but this may be unreasonable | 10:18 | |
| paulsherwood | kejiahu_: what commandline did you use for ybd, and what are you running on? | 10:22 |
| paulsherwood | s/what are/what hardware are/ | 10:22 |
| radiofree | kejiahu_: don't build syslinux on arm | 10:22 |
| radiofree | (it's only needed for x860 | 10:23 |
| kejiahu_ | paulsherwood: I was building rdale's branch for openwrt on jetson. | 10:24 |
| kejiahu_ | radiofree: so syslinux shouldn't be included in the definition at all? | 10:24 |
| radiofree | kejiahu_: not for arm | 10:25 |
| paulsherwood | right, so this is the issue... rdale's branch has lots of stuff that isn't required for openWRt (eg systemd) | 10:25 |
| SotK | paulsherwood: why does ybd create /dev/null in its sandbox? | 10:25 |
| rdale_ | no it shouldn't have | 10:26 |
| paulsherwood | ybd parses all definitions... and finds multiple build-depeds for some things, and chooses the wrong set in some cases | 10:26 |
| kejiahu_ | radiofree: really? as far as I can remember, it seems pxelinux can be used on moonshot to boot it? | 10:26 |
| radiofree | that usesd u-boots implementation | 10:27 |
| richard_maw | kejiahu_: It uses something that quacks like pxelinux, but ducks don't run on ARM. | 10:27 |
| paulsherwood | heh | 10:27 |
| radiofree | kejiahu_: syslinux is only needed for x86, you can see this if you git grep "syslinux" | 10:27 |
| kejiahu_ | radiofree, richard_maw : oh, right. thanks | 10:27 |
| radiofree | by the looks of it you're trying to build the x86 bsp? | 10:27 |
| paulsherwood | thing is untangling why ybd chooses wrong sometimes has proved to be beyond me so far. the issue is https://github.com/devcurmudgeon/ybd/issues/21 if you want to add to it or suggest solutions | 10:28 |
| rjek | You probably need to sort everything | 10:29 |
| rjek | Or are you already doing that? | 10:29 |
| rjek | ie, enumerate whole directory, sort, start from first entry | 10:29 |
| paulsherwood | i'd prefer to remove all the warnings by forcing that a given checkout of definitions can't have multiple instances of foo chunk declared, but that might not work in practice | 10:29 |
| rdale_ | trying to prevent foo chunk being defined in more than one stratum? | 10:30 |
| paulsherwood | rjek: it's not that. it's that foo chunk is declared to depend on bar and bip strata, while alternatively to depend on bobble and bunx | 10:30 |
| paulsherwood | rjek: i'd prefer that, but i don't think it fits all use-cases | 10:31 |
| straycat | Zara, biff, | 10:31 |
| ssam2 | because build depends are defined per-strata | 10:31 |
| ssam2 | per-stratum | 10:31 |
| paulsherwood | ssam2: correct | 10:31 |
| straycat | Zara, (alternative suggestion) | 10:31 |
| kejiahu_ | radiofree: no. I was building from a definition modified from x86 system, and i didn't remove the syslinux from it. so I need to replace syslinux by u-boot in that case? | 10:31 |
| ssam2 | as with many proposed changes to the definitions format, the first step to changing that is probably to produce a migration script that can show us what the current definitions repo would look like if we made that change | 10:32 |
| radiofree | kejiahu_: see the jeston-bsp | 10:32 |
| kejiahu_ | radiofree: thanks | 10:32 |
| radiofree | you're just checking to see if this builds on arm right? | 10:32 |
| ssam2 | thanks to rdale's discovery of ruamel.yaml that's no longer an insurmountable task, I think | 10:32 |
| radiofree | you can probably just remove the bsp entirely then, if you're not actually going to use it for anything | 10:32 |
| paulsherwood | ssam2: yup. but no point in going too far down the road if no-one likes the expected destination | 10:32 |
| ssam2 | paulsherwood: exactly. I think a migration script is the shorted path to working out whether we like the destination or not | 10:33 |
| ssam2 | my concerns are pretty much entirely "will definitions.git be easier or harder to work with after this change" | 10:33 |
| ssam2 | *shortest | 10:34 |
| radiofree | kejiahu_: if you want to test it, then just use bsp-jetson http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/systems/devel-system-armv7lhf-jetson.morph#n33 | 10:34 |
| paulsherwood | the general problem is that we clearly need definition contains n clusters of m systems, and each system may have its own special requirements for foo chunk (including having a different version of foo) | 10:34 |
| radiofree | (instead of bsp-x86) | 10:34 |
| paulsherwood | ssam2: fair | 10:34 |
| paulsherwood | ssam2: i think rdale_'s discovery of the parse thing is worth looking at, i just haven't got around to it | 10:35 |
| kejiahu_ | radiofree: ideally, I'd want to deploy it on something to verify it works, yes, I'll use that one. thanks | 10:35 |
| ssam2 | paulsherwood: it doesn't *have* to be you :) but for me, I think automated testing and integration is higher on my list of priorities than improving the definitions format | 10:36 |
| paulsherwood | ssam2: i agree, in fact i'd prefer it wasn't :-) but if i don't understand the approach, i'll probably end up grumpy later :) | 10:37 |
| ssam2 | perryl: https://github.com/devcurmudgeon/ybd/issues/21 will definitely affect bit-for-bit reproducibility of systems ! | 10:40 |
| ssam2 | I wonder if the 'random.shuffle(dependencies)' in assembly.py is a problem.. | 10:41 |
| ssam2 | certainly removing it might work around bug #21 | 10:41 |
| perryl | is it building random extra components or is there some reason to it? i've also noticed erlang in /src/cache/ybd-artifacts | 10:42 |
| ssam2 | i'm not really sure | 10:42 |
| paulsherwood | nope, i'm entirely confident that the shuffling will not affect #21. i'm mostly confident it won't affect b4b either. if it does, something else is wrong | 10:44 |
| SotK | It parses things in the order it finds them in the filesystem, and the first version of a thing parsed becomes the canonical version, so if it parses something with multiple definitions with the "wrong" defintion first, then it will try to build with the wrong definition AIUI | 10:44 |
| radiofree | kejiahu_: when you come to deploying please shout up, there's some important things you'll have to make sure to set on the deployment cluster | 10:44 |
| ssam2 | paulsherwood, SotK: I see. I also see richard_maw has suggested a better workaround in bug #21 now | 10:44 |
| kejiahu_ | radiofree: will do | 10:45 |
| radiofree | paulsherwood: looks like that fixed the issue | 10:54 |
| radiofree | 2015-06-09 10:50:41 [gcc] Elapsed time 00:12:08 | 10:54 |
| paulsherwood | kewl! :) | 10:54 |
| radiofree | next arch.. ppc? :) | 10:55 |
| paulsherwood | absolutely :) | 10:55 |
| paulsherwood | we have some mips stuff to try, too | 10:55 |
| paulsherwood | nowster: ^^ ? | 10:55 |
| nowster | hmm? | 10:55 |
| paulsherwood | could you try building interesting definitions with latest ybd on mips at some point? | 10:56 |
| paulsherwood | also, i've never tried cross-bootstrap definitions.... i wonder what would happen with those? | 10:57 |
| nowster | At some point, yes. I doubt it'll be this week, though. | 10:57 |
| nowster | Does ybd do "cross bootstrap" builds? | 10:57 |
| SotK | `morph cross-bootstrap` does all kinds of magic to create the cross-bootstrap tarball I think | 11:02 |
| SotK | paulsherwood: why does ybd create /dev/null in its sandbox? | 11:02 |
| SotK | I tried to use the tarfile module rather than calling `tar` when extracting system artifacts, but it broke because /dev/null was already there | 11:04 |
| paulsherwood | SotK: you're asking the wrong person... | 11:05 |
| paulsherwood | 6a4aca35 (Sam Thursfield 2015-06-08 18:34:42 +0100 63) app.log(this, "Creating /dev/null inside sandbox") | 11:05 |
| SotK | paulsherwood: "0ccf0afe (Paul Sherwood 2015-01-03 15:22:04 +0000 67) call(['sudo', 'mknod', devnull, 'c', '1', '3'])" | 11:06 |
| pedroalvarez | git-blame war begins | 11:06 |
| SotK | it was added in the commit which added sandbox.py as far as I can tell | 11:06 |
| ssam2 | i don't remember doing that | 11:07 |
| ssam2 | I remember adding the log message, but not the /dev/null creation | 11:07 |
| Zara | straycat: I tried your alternative suggestion and it worked (with a little tweaking due to the npm view command being a bit fussy about its input) :) | 11:12 |
| * SotK successfully deploys a system with subsystems | 11:31 | |
| SotK | s/subsystems/a subsystem/ | 11:31 |
| * SotK notes also that the strip-gplv3.configure extension doesn't work with ybd, since it expects metadata files to be JSON | 11:32 | |
| pedroalvarez | ughh | 11:32 |
| *** a1exhughe5 has joined #baserock | 11:34 | |
| radiofree | i hope the solution isn't to json the ybd metadata files | 11:48 |
| * radiofree likes how easy they are to parse | 11:48 | |
| rdale_ | what format are they in? | 11:49 |
| SotK | rdale_: unformatted text I think | 11:49 |
| rdale_ | hmm | 11:49 |
| straycat | weirdly that's what cpan's done with its metadata | 11:52 |
| straycat | they started out with yaml and then moved to json | 11:53 |
| straycat | Zara, cool :) | 11:53 |
| rdale_ | unformatted text isn't the same as yaml or json though | 11:53 |
| radiofree | shouldn't it move to yaml eventually though | 11:54 |
| * SotK thinks both ybd and morph should use yaml in the metadata files | 11:54 | |
| rdale_ | it's easy to convert json <==> yaml, but some random invented text format isn't | 11:55 |
| pedroalvarez | isn't that extension coupled to the build tool and how they generate the metadata files? | 11:56 |
| SotK | nope | 11:56 |
| pedroalvarez | -*- SotK notes also that the strip-gplv3.configure extension doesn't work with ybd, since it expects metadata files to be JSON | 11:57 |
| SotK | http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/extensions/strip-gplv3.configure#n68 | 11:57 |
| pedroalvarez | well, if the fact that the metadata files are not in json makes it crash, then I guess it depends on how the build tool generates these files | 11:59 |
| pedroalvarez | I'm not saying it depends on morphlib | 11:59 |
| SotK | ah, I see what you are saying | 12:04 |
| SotK | I don't think thats a good thing, given the system definitions say it should be run on some systems (and that list can be used by any build tool) | 12:05 |
| straycat | "Sam Thursfield <sam.thursfield@codethink.co.uk> does not identify a registered user or group" | 12:06 |
| * straycat throws a wooden spoon at gerrit | 12:06 | |
| straycat | Zara, biff, assuming 'latest' does what i guess it does then +1 | 12:07 |
| * SotK sends a pull request | 12:08 | |
| Zara | straycat: should do (I tested it and it worked, anyway). :) | 12:09 |
| straycat | Zara, hrm, as long as packagName@latest is equivalent to just passing in packageName then it's fine, if we're not sure about that then we should just pass in packageName (assuming that providing a packageName alone defaults to the latest version) | 12:11 |
| paulsherwood | SotK: you say 'Building a cluster doesn't make sense' - what if it's not built? | 12:16 |
| paulsherwood | rdale_: i reckon it would be pretty easy to make the manifest into yaml... https://github.com/ColdrickSotK/ybd/blob/master/assembly.py#L192 | 12:18 |
| SotK | paulsherwood: building the actual cluster doesn't make sense though... what goes in a "cluster artifact"? | 12:18 |
| paulsherwood | SotK: oh, yes. agreed. but does ybd still build the systems and subsystems, with your patches? | 12:19 |
| rdale_ | paulsherwood: i don't mind if the manifest is either yaml or json, i only mind if it's neither | 12:19 |
| paulsherwood | s/still// | 12:19 |
| SotK | paulsherwood: it does | 12:19 |
| paulsherwood | SotK: merged! :) | 12:20 |
| SotK | \o/ thanks | 12:20 |
| paulsherwood | rdale_: ack! | 12:20 |
| * SotK likes how quick upstream is | 12:20 | |
| * paulsherwood does his best | 12:23 | |
| paulsherwood | :) | 12:23 |
| paulsherwood | downside is, upstream changes rapidly, which may be bad for some users/usecases | 12:23 |
| straycat | maybe you should have a separate development branch if you plan on breaking things a lot | 12:25 |
| paulsherwood | straycat: that seems sensible... but then what if i break master? | 12:25 |
| paulsherwood | (ie what if something gets through?) | 12:26 |
| paulsherwood | this way, i'm not makign any promises i can't keep :-) | 12:26 |
| paulsherwood | but i take your point. am thinkgin about it | 12:26 |
| paulsherwood | ybd really needs ci, and cd | 12:26 |
| Zara | straycat: yeah, it's the same. | 12:28 |
| straycat | okay | 12:29 |
| straycat | ssam2, i tried to add you to https://gerrit.baserock.org/#/c/814 but couldn't | 12:29 |
| straycat | paulsherwood, well it's never going to be perfect :) | 12:30 |
| * paulsherwood runs the python-free, arch-free commandline '/src/ybd/ybd.py release/clusters.morph' for the first time | 12:30 | |
| paulsherwood | straycat: i agree... but it's gonna be close :-) | 12:31 |
| straycat | Zara, btw, one of the comments on 814 was a question :) | 12:32 |
| straycat | paulsherwood, without a minimal test suite it might be difficult to know when master is broken | 12:32 |
| paulsherwood | straycat: yes, but in my worldview at the moment, definitions.git is my test-suite? | 12:33 |
| straycat | Okay | 12:33 |
| paulsherwood | i'm very interested to understand what the practical gaps are in my approach | 12:34 |
| paulsherwood | incidentally this could be a further reason for bringing the whole of definitions into ybd.git :) | 12:38 |
| SotK | that sounds like it would be annoying to keep up to date | 12:38 |
| paulsherwood | i'm wondering if i can somehow get upstream definitions to mirror into the ybd directory somehow | 12:39 |
| paulsherwood | (ie keep up-to-date automatically) | 12:39 |
| paulsherwood | everything would be in separate directories, so should be no clashes/conflicts | 12:40 |
| paulsherwood | anyway, i should get back to my day-job :-) | 12:40 |
| *** zoli__ has quit IRC | 12:45 | |
| *** zoli__ has joined #baserock | 12:48 | |
| kejiahu_ | YBD complains depmod not found, but it is exists under PATH, why does this happens? http://paste.baserock.org/ificequrob | 13:00 |
| paulsherwood | kejiahu_: nice find! i don't know, i'm afraid. maybe ssam2 could help? | 13:01 |
| paulsherwood | kejiahu_: which architecture/os are you running under? | 13:02 |
| richard_maw | kejiahu_: is that depmod in your host system, or in the staging chroot? | 13:02 |
| kejiahu_ | paulsherwood: it's on armv7 baserock | 13:02 |
| franred | does someone has a minute to review https://gerrit.baserock.org/#/c/821/? | 13:02 |
| kejiahu_ | richard_maw: is in host | 13:02 |
| richard_maw | kejiahu_: you need it to be available in the system you are producing then. | 13:03 |
| paulsherwood | richard_maw: you mean in build-essential? | 13:05 |
| * SotK sends another pull request :) | 13:06 | |
| richard_maw | paulsherwood: no, as part of the system artifact | 13:06 |
| richard_maw | you need to include the stratum that contains kmod if you want kernel modules | 13:07 |
| paulsherwood | SotK: will that pr break anything against definitions master? | 13:07 |
| ssam2 | edcragg: I notice there's a bunch of patches from you in gerrit now, what do these relate to? | 13:07 |
| kejiahu_ | richard_maw: I would expect that's already included, as the some branch built on x86. but I'll check | 13:07 |
| paulsherwood | (that isn't already broken, i mean) | 13:07 |
| ssam2 | edcragg: it's hard to get a complete picture looking at the individual commits in Gerrit, might be handy if you could send a summary to the mailing list | 13:08 |
| SotK | paulsherwood: I don't think so | 13:08 |
| * SotK can test | 13:08 | |
| radiofree | kejiahu_: that's for the nouveau modules | 13:08 |
| radiofree | kejiahu_: which you almost certainly don't need, however it's an interesting case for ybd | 13:08 |
| paulsherwood | SotK: if you don't mind :) i'll probably merge it anyway... :) | 13:09 |
| franred | richard_maw, ta! | 13:09 |
| radiofree | kejiahu_: do you have foundation in your system? | 13:10 |
| kejiahu_ | radiofree: I've found that, the kmod was included in foundation, but it's not included in definition. | 13:11 |
| paulsherwood | radiofree: this may be another example of #21 | 13:12 |
| SotK | paulsherwood: seems to work fine | 13:12 |
| paulsherwood | SotK: merged :) | 13:12 |
| paulsherwood | you're on *fire* today :-) | 13:12 |
| kejiahu_ | radiofree: I'll remove nouveau | 13:12 |
| SotK | paulsherwood: thanks! | 13:12 |
| radiofree | kejiahu_: seems sensible, nouveau is for hardware accelerated graphics, which you probably don't need on a router | 13:15 |
| kejiahu_ | radiofree: thanks | 13:15 |
| radiofree | kejiahu_: to clarify, the problem was you *didn't* have kmod installed? | 13:21 |
| *** fay_ has quit IRC | 13:21 | |
| kejiahu_ | radiofree: yes, the foundation is not included in system definition | 13:22 |
| radiofree | gotcha, NOTABUG | 13:23 |
| paulsherwood | :) | 13:25 |
| kejiahu_ | radiofree: yes, with nouveau removed, the build finished. although the resulting rootfs is surprising big.... anyway, should I try jetson-upgrade.morph to deploy it? what's the 'important things' you mentioned just now regarding the deployment | 13:29 |
| radiofree | kejiahu_: what branch were you using? | 13:30 |
| radiofree | rdale/openwrt? | 13:30 |
| kejiahu_ | radiofree: yes | 13:31 |
| radiofree | ok, jetson-upgrade should be fine then (as a template) | 13:31 |
| radiofree | make sure you change the system morph in there | 13:31 |
| kejiahu_ | radiofree: with bsp replaced by jetson-bsp, and nouveau removed | 13:31 |
| radiofree | http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/clusters/jetson-upgrade.morph?h=baserock/rdale/openwrt#n4 | 13:31 |
| radiofree | you'll need to change the system on line 4, however the rest of it should be fine | 13:32 |
| kejiahu_ | radiofree: thanks | 13:32 |
| *** mariaderidder has quit IRC | 13:37 | |
| paulsherwood | kejiahu_: make sure you have latest ybd, deployment has been tweaked today :) | 13:43 |
| kejiahu_ | paulsherwood: good reminder, just 1 second before I press enter | 13:44 |
| paulsherwood | :) | 13:44 |
| * paulsherwood is seriously considering making ybd check to update itself on every run | 13:45 | |
| pedroalvarez | ughh | 13:46 |
| paulsherwood | pedroalvarez: you don't like that idea? | 13:46 |
| pedroalvarez | that is not good for the user point of view I think | 13:46 |
| pedroalvarez | auto updates without notifiying | 13:46 |
| * Kinnison wouldn't like that | 13:47 | |
| Kinnison | esp. if I had explicitly checked out a particular ref to check something | 13:47 |
| Kinnison | I wouldn't want it updating | 13:47 |
| Kinnison | ybd --check-for-updates | 13:48 |
| paulsherwood | heh :) | 13:49 |
| paulsherwood | ok, thanks for the feedback. i'll think of something less invasive | 13:49 |
| Kinnison | Same way proper OSes tell you there's updates but don't auto-upgrade themselves | 13:49 |
| radiofree | paulsherwood: 2015-06-09 13:28:51 [TOTAL] Elapsed time 03:36:44 | 13:51 |
| paulsherwood | radiofree: w00t! thanks :) | 13:52 |
| radiofree | that's build-system-armv8l64 | 13:52 |
| paulsherwood | it's not super-fast, though, is it? | 13:52 |
| radiofree | i think that's pretty fast from start->finish | 13:52 |
| paulsherwood | how many cores/memory on that thing? | 13:52 |
| radiofree | 8 cores, 16G | 13:52 |
| paulsherwood | did that include pulling the gits? | 13:52 |
| radiofree | yes | 13:53 |
| radiofree | i think | 13:53 |
| radiofree | maybe not | 13:53 |
| radiofree | no | 13:53 |
| paulsherwood | i think on my MBP it's approx one hour... | 13:54 |
| paulsherwood | i wonder if anyone from ARM is listening | 13:54 |
| richard_maw | what, building Armv8l64 | 13:54 |
| paulsherwood | no, building the equiv x86 | 13:54 |
| persia | For tooling upgrades: consider the case of the user who wants to use the tool in a place without useful networking, based on local caches (or even local authoritative git repo). Lots of folk work in this mode, either for temporary reasons (e.g. certain flights), or administrative reasons (e.g. restrictive office firewalls). | 13:55 |
| radiofree | your MBP is some i7 monster though? | 13:55 |
| paulsherwood | this seems too slow on the Moonshot, tbh. maybe a bug somewhere? | 13:55 |
| radiofree | this isn't the moonshot, this is a mustang | 13:55 |
| paulsherwood | radiofree: yup. it's a cruel world | 13:55 |
| paulsherwood | radiofree: ah. even so. what's it clocked at? | 13:55 |
| rjek | Same CPU I thought. | 13:55 |
| rjek | Plus I'm told that AMCC's ARM implemention... leaves something to be desired. (It's not an ARM Cortex design.) | 13:56 |
| Kinnison | They fluffed the pipeline I believe | 13:56 |
| paulsherwood | i think it's the best there is, as of today? but i may be wrong | 13:56 |
| Kinnison | so it behaves more like an A53 than an A57 in terms of pipelined performance | 13:56 |
| radiofree | if ybd actually logged to files i could give you build times of key components | 13:56 |
| Kinnison | paulsherwood: It's certainly the most easily acquired | 13:56 |
| radiofree | but it's gone from my scrollback, i think it did gcc in 12 minutes | 13:56 |
| rjek | That sounds like a feature. | 13:56 |
| radiofree | which i didn't think was too bad | 13:57 |
| rjek | radiofree: ybd | tee juicylogfile.txt ? | 13:57 |
| DavePage | script ybd | 13:57 |
| radiofree | i'll remember to do that 3 1/2 hours ago | 13:57 |
| rjek | :) | 13:57 |
| paulsherwood | radiofree: build times should be in all the logs, now. cat ybd-artifatcs/*build-log | grep elapsed_time | 13:57 |
| radiofree | i don't have the build time in my build-logs | 13:58 |
| paulsherwood | oh? when did you last update ybd? | 13:59 |
| radiofree | aa71bf2ad5eabb955f058a97986a6d4308be5a11, Tue Jun 9 10:40:45 | 13:59 |
| radiofree | is this a new feature within the last 3 1/2 hours? :) | 13:59 |
| paulsherwood | nope | 13:59 |
| paulsherwood | a few days ago? look at the top of one of the log files please? | 14:00 |
| paulsherwood | weird | 14:00 |
| paulsherwood | ah, no. it's in the manifest | 14:00 |
| paulsherwood | silly me | 14:00 |
| radiofree | nope, no times | 14:00 |
| radiofree | ah | 14:00 |
| paulsherwood | i'll add it to the log, too | 14:00 |
| paulsherwood | sorry | 14:01 |
| radiofree | right, so gcc took 00:08:34 | 14:01 |
| paulsherwood | can you just cutnpaste your terminal log and grep that ? | 14:01 |
| paulsherwood | actually, this needs more thought, also. sticking the time in the manifest will break b4b | 14:02 |
| radiofree | terminal log is lost to the aether i'm afraid ( was running in screen ) | 14:03 |
| richard_maw | if you stick it in the artifact, yes, but I don't expect the accompanying metadata file to be b4b reproducible | 14:03 |
| Kinnison | Morph drops a metafile alongside the build with timings in | 14:03 |
| Kinnison | not part of the artifact, but spare data in the cache | 14:03 |
| paulsherwood | ah, ok. so i could create .meta as well as the artifact and .build-log per build | 14:04 |
| richard_maw | yes, though I'd recommend having a directory per artifact, rather than different suffixes per file | 14:05 |
| radiofree | zlib took 41 minutes | 14:05 |
| rjek | That seems a long time | 14:05 |
| * richard_maw has wanted to change the artifact cache format to either ostree, or a directory per source for a while | 14:06 | |
| Kinnison | that is an insane amount of time | 14:06 |
| Kinnison | zlib should take seconds to actually build | 14:06 |
| paulsherwood | maybe the python recursion is strangling the mustang? | 14:06 |
| paulsherwood | (i have no idea, really) | 14:07 |
| Kinnison | Without better timing data from the logs I couldn't say | 14:07 |
| paulsherwood | ack | 14:07 |
| * richard_maw likes to ponder how that sentence would be interpreted by people without computing context | 14:07 | |
| rjek | Does it spit out build output, prefixed with timing information? | 14:07 |
| DavePage | richard_maw: Euphemistically? "maybe the python recursion is strangling the mustang?" "That's what *she* said!" | 14:08 |
| radiofree | are you interested in the build times for components paulsherwood? | 14:09 |
| richard_maw | I was imaginging someone being worried about snakes attacking their horses | 14:09 |
| * radiofree grabs them anyway | 14:09 | |
| rjek | ybd 2>&1 | awk '{ print strftime() ": " $0 }' | tee myjuicylog.txt | 14:09 |
| radiofree | http://paste.fedoraproject.org/230349/38589401/ | 14:09 |
| *** mariaderidder has joined #baserock | 14:09 | |
| rjek | I assume there's a tool already that does what that awk does, but I don't know its name. | 14:09 |
| radiofree | why did zlib take so much time? | 14:10 |
| * Kinnison wonders if some of the components are including the cost to build what they depend on | 14:10 | |
| Kinnison | E.g. | 14:10 |
| Kinnison | strata-build-essential.morph.meta:elapsed_time: 00:58:33 | 14:10 |
| Kinnison | the stratum itself should take almost zero build time | 14:10 |
| Kinnison | so those numbers are wrong | 14:10 |
| rjek | Does it include unpacking/creating the sandboxes? | 14:10 |
| Kinnison | it should | 14:10 |
| * richard_maw wonders whether the zlib times are including all their dependencies | 14:10 | |
| Kinnison | richard_maw: welcome to the conversation of less than a minute ago :) | 14:11 |
| radiofree | so 2015-06-09 13:28:51 [TOTAL] Elapsed time 03:36:44 was from the now lost ybd terminal output | 14:11 |
| * rjek wonders if the times it is reporting also includes all the dependencies. | 14:11 | |
| richard_maw | Kinnison: I appreciate your hospitality towards time travellers. | 14:11 |
| radiofree | which is correct, since i started the build, from scratch, 3 1/2 hours ago | 14:11 |
| Kinnison | radiofree: Yeah, I think ybd's timing data is not useful | 14:12 |
| radiofree | individual components like gcc are really fast though gcc.meta:elapsed_time: 00:08:34 | 14:12 |
| * paulsherwood disagrees | 14:12 | |
| Kinnison | I imagine you already had all the rest of build-essential built then | 14:12 |
| paulsherwood | build-essential takes an hour. that's true :) | 14:12 |
| Kinnison | 'cause gcc taking 8 mins and zlib taking 40+ is mad | 14:12 |
| paulsherwood | that's an actual problem, is my guess | 14:13 |
| radiofree | strata-nfs.morph.meta:elapsed_time: 02:48:44 | 14:13 |
| paulsherwood | i think that's including the other strata that it needed to build first, so less useful | 14:13 |
| Kinnison | paulsherwood: It's one thing for aggregate components to include the build time of their contents, but IMO they shouldn't include the time for their build-dependencies | 14:13 |
| Kinnison | paulsherwood: How you make that distinction in ybd's timings is up to you :) | 14:14 |
| paulsherwood | Kinnison: agreed. i don't know if i can fix it | 14:14 |
| Kinnison | radiofree: Also, FYI, baserock has a pastebin, no need to use fedora's | 14:15 |
| radiofree | i can fpaste < foo to fpaste though | 14:15 |
| Kinnison | heh | 14:15 |
| radiofree | zlib taking 41 minutes seems wrong | 14:16 |
| Kinnison | It's including its build-deps | 14:16 |
| Kinnison | clearly | 14:16 |
| rjek | And baserock's pastebin randomises syntax colouring and doesn't word-wrap. | 14:16 |
| radiofree | Kinnison: in that case wouldn't gcc's time be much higher? | 14:16 |
| Kinnison | I'm confused a little about that | 14:16 |
| Kinnison | radiofree: My guess is that since binutils and gcc are early in stage 3, perhaps the build cost of the bootstrap components was somehow not included | 14:17 |
| Kinnison | radiofree: Diagnosing what went wrong with the timelogging with just a set a timings isn't easy :) | 14:17 |
| radiofree | lemme run the whole thing again and keep the ybd log | 14:17 |
| radiofree | i always forget it's not actually logged anywhere | 14:18 |
| paulsherwood | acually coming back to this, i think that something may be stuck accessing netwrok or something for ages, leading to lots of waiting. this build should have taken no more than a couple of hours, imo | 14:21 |
| radiofree | 3 hours is "a couple of hours" imo | 14:21 |
| paulsherwood | no, i mean it should build in less than 2 hours | 14:21 |
| paulsherwood | is muy guess | 14:22 |
| Kinnison | Treat an XGene as straight-line speed similar to an i3 of equivalent clock rate | 14:22 |
| pedroalvarez | <rjek> And baserock's pastebin randomises syntax colouring and doesn't word-wrap. | 14:24 |
| pedroalvarez | :( | 14:24 |
| pedroalvarez | why today? | 14:25 |
| rjek | pedroalvarez: It does it every day, AFAICT :) | 14:26 |
| pedroalvarez | rjek: your gratuitous complaint, I mean | 14:26 |
| Kinnison | We were talking about pastebins | 14:27 |
| pedroalvarez | :( | 14:27 |
| rjek | Kinnison asked radiofree why he wasn't using BR's pastebin. For radiofree it's convenience, for me it is because I don't want random highlighting and the last bit of the first line being obscured and lack of wordwrap :) | 14:27 |
| pedroalvarez | I sometimes find usefult tath there isn't auto-wraping | 14:29 |
| pedroalvarez | ew,, s/tath/that/ | 14:29 |
| pedroalvarez | and when I miss it, I just click in the button to see the raw content | 14:30 |
| persia | +1 to no autowrap | 14:35 |
| persia | But if there is a tool that can deal with hastebin in the manner of `${tool} < file` or `${command} | ${tool}`, it would be very good to put an appropriately configured binary into the development system. | 14:36 |
| *** petefoth has quit IRC | 14:36 | |
| pedroalvarez | `wget http://paste.baserock.org/hastebinit` | 14:36 |
| radiofree | if i redirect the output, ybd doesn't seem to do anything | 14:37 |
| radiofree | it gets up to 5-06-09 14:36:31 [python-ceilometerclient] Cache_key is python-ceilometerclient.4ecfbcb575e0252f5cdf8ebdc3ad50c6a48c631bcf | 14:37 |
| radiofree | then... nothing, at least according to the log | 14:37 |
| radiofree | oh never mind | 14:38 |
| pedroalvarez | I can add the hastebinit tool to the essential files. would that make sense? | 14:42 |
| pedroalvarez | 29 lines of python | 14:42 |
| Kinnison | radiofree: I imagine ybd isn't force-flushing after log lines | 14:42 |
| radiofree | pedroalvarez: go on then, call it "brpaste" | 14:43 |
| pedroalvarez | heh | 14:43 |
| persia | pedroalvarez: If you configure to default to paste.baserock.org, yes please. | 14:43 |
| radiofree | would be super useful actually | 14:43 |
| franred | pedroalvarez, +1 | 14:46 |
| paulsherwood | ybd runs (but may not do very much yet) on MacOS... yay! thanks ssam2 :-) | 14:57 |
| pedroalvarez | we will be sending soon the OpenStack on Baserock kilo upgrade soon, and following the tradition, it will include a mega-patch (-4453/+6181) | 14:58 |
| radiofree | :/ | 14:59 |
| Kinnison | pedroalvarez: cripes | 14:59 |
| radiofree | do you expect anyone who's not worked on openstack to actually review that? | 14:59 |
| pedroalvarez | I'm happy to discuss better ways to do this | 14:59 |
| pedroalvarez | radiofree: at least to give a opinion from his point of view, yes | 15:00 |
| pedroalvarez | that mega commit only sync's configuration files with the new version | 15:01 |
| paulsherwood | pedroalvarez: really? | 15:01 |
| pedroalvarez | this is the branch: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/log/?h=baserock/update-openstack-to-kilo | 15:02 |
| paulsherwood | i've been wondering are we missing something here? should each 'system' or 'cluster' be in its own branch, maybe? and then have major release brnaches for applicable systems (eg openstack) | 15:02 |
| * richard_maw doesn't like that model, but doesn't have time at the moment to articulate why | 15:03 | |
| pedroalvarez | I can articulate that | 15:03 |
| pedroalvarez | why | 15:03 |
| paulsherwood | richard_maw: ok. it won't happen today, anyway :) | 15:03 |
| * paulsherwood would like to understand the negatives | 15:04 | |
| pedroalvarez | paulsherwood: I would like to understand the model you are purposing first | 15:05 |
| radiofree | paulsherwood: wouldn't that require someone to make sure all the branches are in sync? | 15:05 |
| * persia remembers the branch debate | 15:05 | |
| paulsherwood | definitions contains many example systems/solutions. i'm suggestign each example could be in its own branch... | 15:05 |
| radiofree | i.e someone updates systemd in the "devel-system" branch, what's the process for merging that to the "build-system" branch? | 15:05 |
| paulsherwood | eg OpenStack/kilo, OpenStack/juno etc | 15:06 |
| radiofree | persia: there was some discussion about branching genivi | 15:06 |
| paulsherwood | radiofree: i don't know, fair question. i was hoping there'd be plenty of git magic for that? :) | 15:06 |
| persia | radiofree: I meant the definitions branching debate, but that makes sense. | 15:06 |
| persia | Why are juno and kilo being simultaneously maintained in master? | 15:07 |
| paulsherwood | that's the current situation. i'm suggesting we change it but maybe my idea is dumb | 15:07 |
| persia | For all that I'm generally opposed to releases, for that use case, I'd rather see release branches with bugfixes applied. | 15:07 |
| persia | paulsherwood: Please pick a different example: that particular one is obviously entirely wrong. | 15:08 |
| paulsherwood | is it? why? | 15:08 |
| paulsherwood | sorry, i don't understgand OpenStakc's release model | 15:09 |
| persia | Because maintaining two different release series of a project in a single git branch is inherently broken: one can't sync. | 15:09 |
| persia | But that is independent of the issues inherent in having multiple branches of definitions for different systems. | 15:10 |
| paulsherwood | i was saying those woudl be two branches? | 15:10 |
| jjardon | pedroalvarez: I guess that (the mega patch) is because we include generated files in the git tree, rigth? I remember discussion that before. Is there still not a way to generate them depending of configuration files? | 15:10 |
| paulsherwood | branch1: OpenStack/kilo, branch2: OpenStack/juno, branch3: OpenStack/master or whatever? | 15:11 |
| persia | paulsherwood: Yes, but the reasons it is bad to have multiple branches apply to circumstances other than the example you picked, which is why I asked for another example. | 15:11 |
| jjardon | and also, is the plan to support only kilo after the merge? or the previous version as well? | 15:11 |
| paulsherwood | GENIVI/foo-release-0.1 GENIVI/foo-release1.0 GENIVI/foo-release1.1 GENIVI/bar-release 0.1 etc? | 15:11 |
| franred | we will only support Kilo when it gets merged | 15:12 |
| persia | So, in the previous debate, we discussed multiple branches for multiple systems. | 15:12 |
| paulsherwood | i don't know whether we'd *support* any of these things... just suggesting separate branches as a means of keeping GENIVI apart from OpenStack apart from OpenWRT etc | 15:12 |
| persia | But all of current releases. | 15:12 |
| franred | jjardon, there are some configurations files which can not be autogenerated | 15:12 |
| persia | We concluded that it was too hard to synchronise base changes in a sensible way, and that it all became too painful. | 15:13 |
| pedroalvarez | jjardon: we actually have removed some of them, and included them at install time of some components | 15:13 |
| persia | In an entirely unrelated discussion, there was some talk about having Baserock releases, and a means to have bugfixes to prior releases indpedendent of changes to master. | 15:13 |
| persia | So that definitions would have a master branch, and several release branches. | 15:13 |
| jjardon | franred: pedroalvarez right, thanks for the info | 15:14 |
| paulsherwood | utltimately i think we'd be best off with OpenStack/latest and GENIVI/latest etc, where we push as close to master on all the things as we can, but those organisations may have imposed their own limits on how close specific components can get to mainline | 15:14 |
| ssam2 | persia: that's actually been done with 15.19.1 and 15.19.2 | 15:14 |
| persia | In the case where we want to freeze release systems and do bugfixes, the latter discussion seems relevant. When upstream follows this model, our alignment with usptream makes sense, and the merge issues with multiple branches go away, because we actively don't want to upgrade underlying components. | 15:14 |
| paulsherwood | persia: yup. but several might be many | 15:14 |
| persia | ssam2: Excellent. When the debate happened, I had the impression that folk thought it was a good idea, but nobody wanted to actually implement it. I'm delighted to hear definitions has useful release series now. | 15:15 |
| Zara | what happens if someone later wants to use baserock to build openstack juno? is there a quick way to go back to it? (and is that likely to happen?) | 15:16 |
| franred | Zara, why would you like to do that? | 15:16 |
| paulsherwood | in my dream scenario, there's an OpenStack/juno branch | 15:16 |
| paulsherwood | franred: reproducibility? | 15:16 |
| persia | paulsherwood: Many release branches is fine, if they are all maintained (and there is a point). Essentially, if one wants to freeze everything except for minor bugfixes, doing this within deinifions master is a recipe for pain. | 15:16 |
| franred | you may want to have some support but for how long? | 15:16 |
| franred | paulsherwood, ^^ | 15:17 |
| SotK | `git checkout $old_sha` would be fine, surely? | 15:17 |
| persia | SotK: Up to the point where you need a bugfix, yes. | 15:17 |
| Zara | franred: I wouldn't like to do it personally, but some groups that use openstack don't have kilo support yet, so I'm just wondering if that would put them off using baserock. | 15:17 |
| franred | Zara, you can build juno, it is in the latest release also it is in the definitions repo | 15:18 |
| franred | paulsherwood, reproducibility is guaranteed, it is in freeze in definitions.git, am I wrong? | 15:19 |
| SotK | Then `git checkout $old_sha -b juno-bugfix && git cherry-pick $bugfix_sha` or similar? I guess at that point you may as well have created the release branch though. | 15:20 |
| SotK | I was mainly commenting on the "how to get back to it" issue | 15:20 |
| persia | I agree on the getting back to it: for me, the main reason to create the branch is as a marker to indicate which ${old_sha}, and whether there has already been a bugfix. | 15:21 |
| paulsherwood | at the point that we decide not to 'maintain' a branch, maybe we just tag it and kill the branch? | 15:23 |
| persia | I'd just leave the branch alone. | 15:24 |
| persia | If we formalise lack of support as meaning something other than folk happening to not submit changes, then we can add a commit indicating the lack of support. | 15:24 |
| paulsherwood | ok | 15:24 |
| persia | But the process for formal declarations of support by some organisation is really complicated, and not something I think we need to do in baserock today. We don't have a strong enough meaning for "support" yet anyway. | 15:25 |
| paulsherwood | so then master becomes baserock-specific systems? build-system, trove etc... and other examples end up in branches? would that work? | 15:25 |
| * Kinnison worries that those other branches would atrophy | 15:25 | |
| * franred too | 15:26 | |
| paulsherwood | not if they're in ci/cd? | 15:26 |
| franred | paulsherwood, what happen when we move components around different strata? | 15:26 |
| persia | paulsherwood: No. For the special case of branching to support an upstream release branch, we branch. For other cases, why would we branch? | 15:26 |
| paulsherwood | (and folks who care are maintainign them... eg Codethink still cares about GENIVI, in spite of all the spats) | 15:26 |
| Kinnison | paulsherwood: In that context, it'd be okay | 15:27 |
| Kinnison | paulsherwood: But then I'd worry about splitting patches between master and branches for fixes | 15:27 |
| kejiahu_ | how does YBD handle extensions? I suppose this is where it gets extensions 'app.py: settings['extsdir'] = os.path.join(settings['defdir'], 'extensions')' but it seems there is no such extensions directory exists? | 15:27 |
| paulsherwood | persia: because OpenWRT may not be compatible with GENIVI may not be compatible with OpenStack etc | 15:27 |
| persia | How not? | 15:27 |
| paulsherwood | SotK: ^^ re kejiahu_'s question | 15:27 |
| paulsherwood | ? | 15:27 |
| paulsherwood | persia: may require different versions of things? | 15:28 |
| rdale_ | we can't support custom versions of build-essential for different systems currently, and openwrt and musl need that | 15:28 |
| persia | Then we offer a service by integrating newer components. | 15:28 |
| paulsherwood | who is 'we'? | 15:28 |
| persia | I seem to recall something about the OpenStack integration being one of the first times newer systemd was used with OpenStack, which led to useful patches to systemd to ease integration with other environments in the future. | 15:28 |
| Kinnison | Indeed | 15:29 |
| persia | We: The set of folk who care about the content of definitions (I'd claim the set of folk who hack on definitions, but that would exclude me, so make for a strange definition of "we") | 15:29 |
| Kinnison | The openstack work helped expose a bug in systemd which we (richard_maw) provided a fix for | 15:29 |
| SotK | kejiahu_: the extensions directory is in definitions.git | 15:29 |
| richard_maw | Yep, I was able to point someone on the Mailing List at the fix they would need to use OpenVSwitch with systemd | 15:30 |
| * SotK spots a bit he missed in one of his ybd patches | 15:30 | |
| paulsherwood | SotK: :) | 15:30 |
| radiofree | paulsherwood: bad news, your fix didn't fix the gcc issue :( | 15:30 |
| DavePage | persia: There's no I in we | 15:31 |
| paulsherwood | erk! | 15:32 |
| SotK | oh, I didn't, I was looking at an old branch | 15:32 |
| persia | DavePage: Yes, but if I define "we" to be people who regularly commute to the moon, I may not be the most qualified spokesperson. | 15:32 |
| paulsherwood | radiofree: the MS_BIND issue? | 15:32 |
| radiofree | noooo | 15:33 |
| kejiahu_ | SotK: I see, the particular branch I checked out doesn't have that directory in it. thanks | 15:33 |
| radiofree | something entirely different this, it'll occasionally fail with "checking for suffix of object files... configure: error: in `/gcc.build/o/aarch64-unknown-linux-gnu/libgcc':" | 15:33 |
| radiofree | if i run it again it'll be fine | 15:33 |
| radiofree | (run the build) | 15:33 |
| radiofree | i'll take a look at it anyway | 15:34 |
| paulsherwood | weird. maybe j1? | 15:34 |
| paulsherwood | or maybe Kinnison knows bettere? | 15:34 |
| Kinnison | You'll need to get the whole proper error for me to help | 15:34 |
| Kinnison | perhaps look at the config.log | 15:34 |
| radiofree | here's the offending config.log | 15:40 |
| radiofree | http://paste.fedoraproject.org/230395/14338644/ | 15:40 |
| radiofree | i'll try and grab one from a working build | 15:41 |
| Kinnison | /gcc.build/o/./gcc/cc1: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory | 15:41 |
| Kinnison | that happened to me last when the right zlib wasn't being build-depended upon | 15:41 |
| radiofree | hmm.. it does work occasionally though | 15:42 |
| Kinnison | Is it configuring gcc to use the internal libz at this point? | 15:42 |
| radiofree | i'll check the log from when it works | 15:43 |
| *** a1exhughe5 has quit IRC | 15:50 | |
| radiofree | so it is libz related | 15:51 |
| radiofree | gcc is build-depending on "libz! | 15:52 |
| radiofree | gcc is build-depending on "libz" | 15:52 |
| pedroalvarez | right, erlang broken because now tools doesn't depend on foundation | 15:53 |
| *** fay has joined #baserock | 15:53 | |
| pedroalvarez | I wonder what's mason doing these days | 15:53 |
| *** fay is now known as Guest86040 | 15:54 | |
| Kinnison | You mean changing those dependencies had further repercussions beyond that which the submitter could be bothered to test for? | 15:54 |
| * Kinnison is shocked | 15:54 | |
| pedroalvarez | :) | 15:55 |
| * rjek gets the valium | 15:55 | |
| * franred looks at erlang stratum and try to fix it and looks at mason asking for some kind of explanation (no new builds since sunday?) | 16:00 | |
| pedroalvarez | franred: is building things | 16:04 |
| pedroalvarez | I wonder how many times.. | 16:04 |
| *** mariaderidder has quit IRC | 16:11 | |
| franred | pedroalvarez, replacing tools.morph by foundation.morph in the erlang stratum, builds erlang | 16:11 |
| pedroalvarez | I wonder why it was depending on tools | 16:12 |
| rdale_ | shouldn't it just depend on core? | 16:13 |
| pedroalvarez | rdale_: it needs systemd | 16:14 |
| rdale_ | erlang? | 16:14 |
| richard_maw | rdale_: no, it needs systemd's headers | 16:14 |
| franred | rdale_, no erlang but erlang-sd_notify | 16:14 |
| rdale_ | i wanted to build minimal systems with erlang and elixir | 16:15 |
| pedroalvarez | franred: go ahead with that patch. Looks like there wasn't a reason to depend on tools | 16:15 |
| franred | pedroalvarez, yeah, it was long time ago... and tools.morph has been modified a lot to check what did have at that time | 16:16 |
| pedroalvarez | franred: 081ffc7d | 16:16 |
| franred | rdale_, we could move erlang-sd_notify to openstack because it is required by rabbitmq to be able to use notify in its systemd units | 16:17 |
| franred | I think rebar should remain in the erlang stratum as it is a erlang compiler | 16:17 |
| radiofree | paulsherwood: if you wanted to improve build times you could make compressing cache artefacts optional for people with lots of space | 16:18 |
| radiofree | for example, stage1-gcc] Elapsed time 00:08:59 - more than 2 minutes of that was creating the cache artefact | 16:19 |
| franred | pedroalvarez, should I move erlang-sd_notify to openstack-services and make erlang depends on core? | 16:19 |
| jjardon | franred: I would prefer that, and make the erlang stratum depend on core, if possible | 16:19 |
| paulsherwood | radiofree: iirc Kinnison added the compression? seemed like a good idea at the time | 16:19 |
| paulsherwood | radiofree: maybe add that as an option in ybd.def ? | 16:20 |
| radiofree | it is a good idea, but if you don't care about it then it just slows your build down | 16:20 |
| persia | rdale_: As is done with linux, use a marker to indicate the differences between the erlangs (one compiled with systemd support, the other not), and move ahead with a separate entry for your needs. | 16:20 |
| pedroalvarez | franred: will that stop you from sending the kilo update patches? | 16:20 |
| paulsherwood | right. i'll make default not compressed | 16:20 |
| franred | pedroalvarez, not at all, I would include this in them | 16:20 |
| persia | I thought the point of compressing cache artifacts was for folk with limited bandwidth, not limited space. | 16:21 |
| radiofree | when i'm building locally i'm not going to be pushing these anyway | 16:21 |
| paulsherwood | compressing to upload is a different use-case from build and deploy, interestingly | 16:21 |
| radiofree | does ybd support artifact cache servers yet? | 16:22 |
| pedroalvarez | franred: your call then | 16:22 |
| persia | I cared about *downloading* cache artifacts, not uploading, but I can see the parallel | 16:22 |
| paulsherwood | radiofree: you might be, there's code to do it... no server on the receiving end yet though :) | 16:22 |
| paulsherwood | radiofree: it needs hooking up | 16:22 |
| paulsherwood | https://github.com/devcurmudgeon/ybd/blob/master/cache.py#L88 | 16:23 |
| paulsherwood | needs some agreement with baserock upstream i think | 16:23 |
| paulsherwood | bear in mind, ybd isn't even lorried yet... i'd be surprised if folks would trust its artifacts :) | 16:23 |
| radiofree | you also don't want to enable that by default | 16:24 |
| paulsherwood | radiofree: yup | 16:24 |
| * paulsherwood tries all the things first by default, to see what breaks | 16:24 | |
| radiofree | maybe you could check to see if it's compressed there, if not, compress, then upload | 16:24 |
| radiofree | or maybe even on the server side | 16:24 |
| paulsherwood | that would be my plan... compress to upload it | 16:25 |
| radiofree | seems sensible to me | 16:25 |
| radiofree | and obviously if disk space is an issue you can always enable compression during build | 16:25 |
| paulsherwood | yup | 16:28 |
| *** zoli__ has quit IRC | 16:47 | |
| ssam2 | anyone noticed that the Morph test suite fails in baserock 15.19 ? | 17:00 |
| ssam2 | http://paste.baserock.org/dakakayewi | 17:01 |
| ssam2 | I used to see this in Fedora, seems to be present in BR now as well | 17:01 |
| jjardon | Does anyone heard of https://semaphoreci.com/ ? looks pretty cool project | 17:03 |
| jjardon | for systemd: https://semaphoreci.com/systemd/systemd | 17:04 |
| *** bashrc_ has quit IRC | 17:07 | |
| ssam2 | looks useful. is it open source? | 17:10 |
| *** gary_perkins has quit IRC | 17:13 | |
| jjardon | its what I was trying to know, but it doesn't look like | 17:13 |
| *** sambishop has quit IRC | 17:17 | |
| *** ssam2 has quit IRC | 17:20 | |
| *** lachlanmackenzie has quit IRC | 17:36 | |
| *** sambishop has joined #baserock | 17:45 | |
| *** edcragg has quit IRC | 17:51 | |
| *** edcragg has joined #baserock | 19:14 | |
| *** edcragg has quit IRC | 19:37 | |
| *** jjardon has quit IRC | 20:21 | |
| *** jjardon has joined #baserock | 20:24 | |
| *** edcragg has joined #baserock | 20:46 | |
| *** edcragg has quit IRC | 22:04 | |
| *** sambishop has quit IRC | 22:27 | |
| *** rdale_ has quit IRC | 22:35 | |
| *** edcragg has joined #baserock | 22:57 | |
| *** edcragg has quit IRC | 23:10 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!