IRC logs for #baserock for Tuesday, 2015-05-19

*** rdale has quit IRC02:34
*** zoli__ has joined #baserock04:13
*** zoli__ has quit IRC05:26
*** zoli__ has joined #baserock05:49
*** fay_ has quit IRC06:41
*** fay_ has joined #baserock06:42
*** a1exhughe5 has joined #baserock07:03
*** a1exhughe5 has quit IRC07:10
*** a1exhughe5 has joined #baserock07:12
*** Albert has joined #baserock07:44
*** rdale has joined #baserock07:50
*** mike has joined #baserock07:52
*** mike is now known as Guest2201707:52
*** mariaderidder has joined #baserock07:54
*** bashrc has joined #baserock08:03
*** Guest22017 has quit IRC08:05
*** 7YUAAB3S9 has joined #baserock08:09
*** Guest22017 has joined #baserock08:17
wdutchis there an example cluster morph for deploying for a chroot? I have a build system chroot and would like a devel chroot08:21
KinnisonI think in release.morph there may be sections for the chroot systems08:22
petefothwdutch: have you had a poke round on wiki.baserock.org?08:22
*** jonathanmaw has joined #baserock08:22
wdutchpetefoth: yes I couldn't find anything describing how to deploy a tarball for the chroot tool08:23
petefothwdutch: OK08:23
wdutchah yes I can see how it is done in release.morph, thanks Kinnison08:24
Kinnisonyou're welcome.08:24
*** gary_perkins has joined #baserock08:56
*** Krin has joined #baserock09:00
*** ssam2 has joined #baserock09:02
*** ChanServ sets mode: +v ssam209:02
straycatssam2, richard_maw, can i merge https://gerrit.baserock.org/#/c/622/ now?09:10
richard_mawstraycat: reading, allow me a few minutes to catch up on the responses09:12
straycatrichard_maw, thanks09:12
ssam2straycat: looks fine to me09:13
richard_mawstraycat: and to me as well, I'm happier with the "catch everything" than the just catching ValueError if the cause is to handle potentially invalid input09:15
straycatwell originally i thought i could remove the exception handler altogether, but ofcourse it's better to keep it. anyway thanks, i'll merge this09:19
SotKssam2: is there any chance you could make my account on testgerrit an Administrator please? :)09:24
ssam2sure09:25
SotKthanks!09:25
ssam2argh, i've broken it09:29
ssam2I disabled OpenID single sign-on again so that i could actually use my Administrator account09:29
ssam2now when I try add you as an administrator i get 'Account Not Found: Adam Coldrick <adam.coldrick@codethink.co.uk>'09:30
SotKo.009:30
richard_mawssam2: biff re https://gerrit.baserock.org/#/c/581/09:32
ssam2thanks09:33
ssam2SotK: seems if I add you by user ID number it works09:35
straycatworth mentioning that when I tried to add ssam2 to a review yesterday I got a similar error09:36
ssam2straycat: in the real gerrit.baserock.org? that's a pain09:36
* straycat nods09:36
ssam2oh well. Hopefully updating to 2.10 will fix, i want to do that at some point09:38
ssam2SotK: did you ever use 'tools/trigger-job.py' that comes in zuul.git?09:39
ssam2if so, do you think it might work as a way to trigger a build of 'master', rather than a build of a Gerrit change?09:39
SotKssam2: nope09:41
SotKssam2: I'm currently looking at using the ref-updated event as a way of doing that09:41
ssam2right09:41
SotKssam2: also, thanks :)09:42
ssam2SotK: ref-updated seems like a much more sensible approach09:43
*** pacon has joined #baserock09:43
wdutchis wiki.baserock.org down?09:43
*** Guest22017 has quit IRC09:43
ssam2wdutch: I can't reach branchable.com or wiki.baserock.org at the moment, so perhaps it is down09:43
ssam2actually, I can reach branchable.com, it's just a bit slow09:44
De|tawiki.baserock.org is responding too, very slowly.09:45
rjekI'd say something unsavory about cloud providers, but I like liw and joeyh.09:45
* Kinnison has raised the issue with liw and joeyh09:45
ssam2De|ta: oh yeah, i've managed to load wiki.baserock.org now too09:45
ssam2SotK: what do you think about the idea of having zuul-layout.yaml defined by the user who deploys the Mason, rather than being auto-generated?09:49
ssam2it seems like each user of a Mason might have different requirements. But at the same time, it'd be nice to have it work automatically without the user first having to write a zuul-layout.yaml09:49
ssam2s/user/administrator/ I guess09:49
SotKI think the best we can do is provide an example one (eg. one which we will use in our CI) and some documentation on how to set it up09:50
SotKI don't see how we can easily set up one which "works by default", given we won't know anything about the layout of other people's Gerrits09:51
ssam2that makes sense09:51
*** a1exhughe5 has quit IRC09:52
*** Guest22017 has joined #baserock09:56
*** a1exhughe5 has joined #baserock09:57
richard_mawhrm, we could do with patching morph gc so that it moves the extracted chunk tree in the hardlink cache to a tempdir, so that if you ^C a `morph gc` it can't leave you with a partially removed cached extraction10:10
richard_mawbut hopefully we'll be moving to ostree soon enough anyway, and we won't need this10:10
*** Guest22017 has quit IRC10:33
*** mike has joined #baserock10:33
*** mike is now known as Guest6837810:33
Zarahi, I tried to cross-bootstrap the armvl5 generic system, and this script was generated. Apparently it contains a bunch of things that it shouldn't contain (and it didn't build when I tried to run the script). Does anyone have any insight? http://sprunge.us/ihge10:38
pedroalvarezZara: the script looks ok, can you paste the error you got when running it?10:39
Zarathis is the full build.log (sorry about the length, it may freeze your computer for a second but it should load eventually): http://sprunge.us/OASH10:40
pedroalvarezZara: I assume that you have generated a tarball using `morph cross-bootstrap`, then moved the tarball to a armv5 (maybe armv7) board, chrooted, and ran the script10:41
pedroalvarezis that correct?10:41
nowsterIs it not making the link from /bin to /tools/bin ?10:42
Zarayes, I moved the tarball over (to an armv7 board), extracted it, chrooted in and ran the script10:42
pedroalvareznowster: hm.. why?10:46
nowster /usr/bin/libtoolize: line 2508: /bin/sed: not found10:46
nowsteretc10:46
pedroalvarezright10:46
pedroalvarezI think that /bin/sed is being used before10:47
nowsterI noticed that too10:47
pedroalvarezthis might be an issue with coreutils sed, then?10:47
nowsterIt might be a linker error.10:47
jjardonsed is not part of coreutils10:48
pedroalvarezjjardon: coreutils-common, sorry for the confusion10:49
pedroalvarezwhich is a stratum with coreutils, sed, diff..10:49
jjardonpedroalvarez: It looks a bit strange that you have to compile even systemd to build the cross-* system, isnt it?10:49
pedroalvarezdidn't notice that10:50
pedroalvarezok, let's take a deeper look10:50
jjardoneven coreutils stratum should not be build10:50
jjardonZara: to generate the tarball, you used the cross-bootstrap-system-armv5l-generic system, rigth?10:51
Zarayes10:51
pedroalvarezew10:52
pedroalvarezit should be built10:52
pedroalvarezostee-core depends on foundation10:53
pedroalvarezfoundation has systemd10:53
pedroalvarezand depends on coreutils-common10:53
pedroalvarezso the behavior is ok10:53
jjardonthats abug I think, I think ostree-core should not depend on foundation now that glib is in core10:54
pedroalvarezI hope so :)10:54
* SotK thinks jjardon is correct10:55
jjardonhaha10:56
* jjardon sent the path to move GLib to core but forgot to remove the 'foundation' dependency in ostree-core10:57
* jjardon will send a patch ASAP10:57
ratmice__Zara: soon before the error it switches from finding /bin/sed to /usr/bin/sed, maybe something to do with bash path caching? e.g. a missing hash -d sed?10:59
jjardonZara: remove the foundation stratum from the build-depends of the cross- system and generate the tarball again11:00
* SotK despairs at the length of the stratum list in this Mason system11:02
Zararatmice__: I don't know, it might be worth someone looking into it who needs to build that stratum (since the thing that fails to build is part of the foundation stratum that I'll be taking out, I'm not going to go digging just now)11:05
pedroalvarezjjardon: remember when I told you that you cross-bootstraping bigger systems will introduce errors?11:07
jjardonpedroalvarez : there is not another option for us11:08
jjardonpedroalvarez: as we have to build a build- system to deploy anyway11:09
jjardonZara: removing foundation stratum is not going to work, as ostree depends in more stuff that is currently in foundation (e2fsprogs)11:10
ZaraOh, I should cancel this build, then. :D11:12
jjardonZara: so, I will try to move e2fsprogs to -core, to try to keep foundation out of the ostree stratum deps.11:13
ssam2ouch11:14
* SotK notes that the x86_64 Mason's disk looks to be full11:14
ssam2jjardon: morph doesn't actually depend on ostree right now, so you could also just remove ostree from your system while bootstrapping11:14
perryli've been looking at adding a username field to distbuild-cancel requests to log whoever cancels a build, but i'm wondering if it would be better to only allow the user/machine that started a build to cancel it; would this be viable or too restrictive, and is it worth sending an RFC to baserock-dev?11:16
ssam2perryl: I think there's a bit of a wider discussion needed about the scope of distbuild, i.e. whether it should be something that only works within a private, trusted environment, or whether it should be secure in all cases11:17
jjardonAh! Zara what Sam said then :)11:17
ssam2perryl: so best so send something to the mailing list11:17
ssam2*to11:17
radiofreejjardon: i don't have merge powers https://gerrit.baserock.org/#/c/678/11:17
perrylssam2: thanks, i'll write something up and send :)11:17
radiofreegetting stuff lorried used to be so simple11:17
radiofreeshould i resubmit that patch again?11:18
ssam2SotK: weirdly, neither cache.baserock.org or mason-x86-64.baserock.org seem to be full11:18
rjekIf we just lorried all software in the universe, we wouldn't need to do this merge request thing.11:18
SotKssam2: very weird11:19
ssam2SotK: maybe pedroalvarez fixed it but didn't tell us11:19
straycatperryl, outside of a trusted environment distbuild needs an auth mechanism to do anything like that11:20
jjardonradiofree: done11:20
straycatperryl, and inside of a trusted environment using the hostname would probably be sufficient11:20
radiofreethanks jjardon11:21
perrylstraycat: indeed, i was considering a function more for inside a trusted environment (as currently anyone can cancel anyone else's build) but i've not put a lot of thought to cases outside that; might be worth looking into before i send an RFC!11:21
straycatwhy would anyone cancel anyone's build within a trusted environment though?11:22
pedroalvarezssam2: oh, just ran `morph gc` but I thought that wasn't going to be enough11:22
ratmice__straycat: because they fired off a build before leaving the office and it's presumably triggered some awful bug causing an infinite loop? :)11:23
pedroalvarezssam2: and is not enough, only 12G free11:23
perrylstraycat: mistaken build identity if anything, but it may be useful if a user suddenly wonders why they have no recorded history of cancelling their own build11:23
straycatwhat do you mean by "mistaken build identity"?11:23
pedroalvarezssam2, SotK: I'll clean them up11:24
SotKpedroalvarez: thanks!11:24
perrylstraycat: morph distbuild-cancel InitiatorConnection-11 instead of InitiatorConnection-111:24
straycatperryl, that's a ui problem though11:24
ssam2perryl: I think it'd be useful to make the job names less confusing11:24
straycatjob names are now uuids11:24
straycatat least https://gerrit.baserock.org/#/c/666/ makes them uuids11:25
straycatso you're unlikely to accidentally cancel someone elses build in that manner11:25
perrylstraycat: ahh, has that been merged?11:25
straycatperryl, no, it's an rfc i'm working on11:25
Zarassam2, jjardon: thanks, I'll take ostree out then!11:25
perrylah, not yet :) in that case i'll leave it for now11:25
straycati'd be up for having users, but only with a proper auth mechanism, otherwise i don't see the point11:26
jjardonstraycat: +111:27
ratmice__i think user identity is never an acceptable auth mechanism11:28
ssam2perryl: so, if you're up for it, maybe you could start a thread on baserock-dev about how best to add user authentication to distbuild11:29
ssam2I think there's still interest in having 'build as a service' in some form, so it's useful to think about that11:29
perrylssam2: i can look into it!11:29
ssam2there are some quite simple ways to get user auth, e.g. reusing SSH like 'gitano' does11:30
ratmice__it always leads to a case where it's easier to just share identity than give authorization in which case the whole metaphor is broken, e.g. sharing or not sharing build id's should be the discriminating factor of whether you can kill a job11:30
Zarajust to check, would this be removing ostree-core from the system, or removing ostree from ostree core? (ie: do I still need libgsystem?)11:33
SotKZara: removing ostree-core from the system11:33
Zaracool, glad I guessed that one :D11:34
Zarahad a moment of... not panic, but potential gloom.11:35
* SotK complains about the lack of runtime dependencies again11:35
SotKI wonder if morph should warn if you include something but not one of its build deps11:35
* straycat says he would like runtime dependencies again11:36
* pedroalvarez complains about the lack of build dependencies information11:37
paulsherwoodremind me, syntactically, how folks would like to do runtime dependencies?11:37
* paulsherwood agrees that build dependencies already *obscures* what is really depended on, in most cases11:37
paulsherwoodbut if we actually specified all the build-depends, definitiosn would be unworkable11:38
pedroalvarezI agree, that's why I'm not complaining louder11:38
paulsherwood(imo)11:38
straycathttp://listmaster.pepperfish.net/pipermail/baserock-dev-baserock.org/2014-November/009162.html was one suggestion11:39
pedroalvarezbut I also have a feeling that this might bite us in the future11:39
pedroalvarezor maybe is already bitting us every day11:39
ssam2paulsherwood: we could make use of strata to make things less unworkable. e.g. rather than saying 'this build-depends on make, gcc, libstdc++, glibc and busybox' you can say 'this build-depends on the build-essential stratum'11:39
paulsherwoodisn't that what we already do, ssam2 /11:40
paulsherwood?11:40
paulsherwoodbut the result of that is that the information is hidden11:40
pedroalvarezssam2: do you mean stop adding the strata build-dependencies recursively?11:42
paulsherwoodstraycat: i remember now. my problem was mainly that this would further complicate definitions in the same vein as stratum splitting. and in practice i fear that we don't use stratum-splitting because it's too hard to comprehend11:42
ssam2paulsherwood: well, right now we say 'this build-depends on foundation', then we say 'foundation build-depends on core', then we say 'core build-depends on build-essential'11:45
straycatfair enough i can't say i've ever used the split rules for anything, richard_maw suggested part of the problem there is that the split rules aren't very accessible, they're buried in morph11:46
paulsherwoodssam2: yes. because previously we had them explicit and *that* became unwieldy (the genivi/x saga)11:46
ssam2ah, ok11:46
ssam2i don't think the result of that information is 'hidden', it's all available in definitions.git11:46
radiofreewe've worked around runtime dependencies for now by just replacing the build-time dependency in the system with the runtime one11:47
ratmice__i'm not sure, graph clustering of the dependency graph seems alright and non-hiding iff things which depend on the cluster depend on everything inside the cluster, that is there is a way to go from absolute dependencies to clustering, but not so much depending on clusters directly11:47
radiofreee.g mesa-common because mesa-common-vm for x86 images11:47
paulsherwoodssam2: what i mean is, if foo depends on bar stratum, which depends on quux stratum, we are hiding that foo only depends on some unknown subset of bar and quux11:47
ssam2paulsherwood: ah, I see what you mean. It might be clearer to call that 'misrepresenting' the dependency info11:48
paulsherwoodok11:48
SotKmy problem was that I'm rebasing an old branch and a bunch of components have been moved into their own strata11:48
SotKthe new strata were in the build-depends for the strata they used to be in, but the changes obviously weren't reflected in the system definition from the old branch11:49
ratmice__not sure if that observation really helps though, it means we can bootstrap a clustered/simplified view of the dependency graph from the absolute dependencies, but we still have an awful large absolute dependency graph around we have to work with :/11:50
SotKso I keep rebuilding without trouble then noticing something else is missing from my deployed system11:50
paulsherwoodSotK: i don't think runtime depends would have made that easier? there would have been *more* stuff to fix i think? :)11:50
SotKs/missing/doesn't work/11:50
*** jmacs has joined #baserock11:50
* paulsherwood may be wrong, of course11:51
SotKpaulsherwood: in my head, runtime depends would have either caused the deployed system to contain the missing things, or caused an error to be raised because the system doesn't contain a runtime depend of some stratum foo11:51
ssam2ratmice_: I worry that the task of representing dependencies can always be done in finer detail11:51
paulsherwoodratmice__: yup, that's the problem. i've been aiming to keep definitions as simple as possible, for ease-of-use11:51
ssam2ratmice__: in Fedora, a lot of RPMs now depend on a specific feature rather than a package by name11:52
paulsherwoodowww11:52
ssam2e.g. they'll depend on /bin/sh, then a package will define that it provides /bin/sh11:52
SotKrather than the current situation of "build all of the things and their dependencies (even if the dependency is not listed in the system), then only deploy the listed things"11:52
ssam2or depend on 'pip package foo' rather than 'rpm python-foo-1.2.3'11:52
paulsherwoodSotK: so by default we just end up with most things having a runtime-depends list which is the same as its build-depends list?11:53
* SotK doesn't know11:54
SotKmy problem was the python-wsgi and python-cliapp strata didn't exist in my old branch, which are both strictly runtime depends but listed as "build depends" afaik11:55
paulsherwoodwe have a python-cliapp stratum?11:56
straycathaving re-read the stratum runtime deps patch i admit that i don't understand why stratum runtime dependencies can't be specified in a similar way to stratum build dependencies11:57
SotKpaulsherwood: indeed11:58
ratmice__SotK: is this something to do with BUILD_DEP'ing on a thing with runtime-dep's necessitates the inclusion of the runtime-dep at build time?11:58
SotKratmice__: I don't think so11:59
paulsherwoodso just call them 'dependencies' and be done? :)11:59
paulsherwoodstraycat: ^^11:59
* SotK notes that some of the things in python-cliapp look unrelated to cliapp, but may be wrong12:00
* paulsherwood notes that the only thing depending on cliapp is likely to be morph12:00
* paulsherwood notes that ybd can build all of release.morph without morph, or cliapp, but cannot deploy all the things because deployment is coupled to morph which is coupled to...12:01
paulsherwoods/deployment is/deployment extensions are/12:02
straycatpaulsherwood, that would lead to potentially quite large amounts of inefficiency though, e.g. i'd end up having swift build-depend on ansible for example, so a change in something unrelated like jinja2 would cause an unnecessary rebuild of swift12:03
* SotK was referring to python-coveragepy and python-coverage-test-runner, which are part of the python-cliapp stratum...12:03
paulsherwoodhow long does it take to build those things?12:03
paulsherwoodSotK: all only there because morph, i believe12:04
ratmice__longer than it takes not to :)12:04
paulsherwoodratmice__: actually, not necessarily.12:04
straycatnot much time at all, but i'm not really sure that's the point? if people have been complaining that morph takes 10 seconds to "decide on task order" then i hate to imagine how they're react when they work out morph is spending a total of let's say 3 minutes building stuff it didn't need to12:04
SotKprobably, but in that case the stratum probably shouldn't be called python-cliapp12:05
straycat*they will12:05
ssam2the reason for python-cliapp existing is that both lorry-controller and morph use it12:05
paulsherwoodif build is longer but normally works, that's better than shorter builds which normally break or lead to unexpected results and therefore need to be re-done12:05
ssam2and I wanted to have lorry-controller in the gerrit system, without Morph12:05
paulsherwoodah, i'm wrong, then. cliapp is required by lc12:06
paulsherwoodi don't think i'm wrong on the efficiency argument, though :)12:07
ssam2i'm sure we could find a pathological case12:07
paulsherwoodi daresay :)12:08
straycatpaulsherwood, so i can revert all the morph speedup commits then, and no one will mind? :)12:08
paulsherwoodi won't mind - i'm using ybd as you know :)12:09
straycatwith an awareness that i'm quite (very) crap, something i was told a few years ago which i always try to bear in mind is that, roughly, you haven't achieved simplicity by excluding a feature, you've just failed to solve a problem, simplicity isn't actually about just cutting things out, it's harder than that, and the fact that simplicity is hard is what makes it difficult to achieve, and why we have complex things.  so when you say "just treat eve12:21
* radiofree waits for that anecdote to finish12:22
SotK(it got cut off at "treat eve")12:22
straycatso when you say "just treat everything12:22
straycat                 as a dependency" or "use random.shuffle() on the chunk list to avoid redundant building in a12:22
straycat                 distbuild", you haven't actually made anything simpler, you've just failed to solve the12:23
straycat                 problem.12:23
DavePageI'm pretty sure Eve would appreciate being treated12:23
paulsherwoodstraycat: we'll have to see, i think. i've been wrong many times before. but here, i'm pretty confident :)12:25
rjekWe should keep a log; the statistics might be interesting.12:26
paulsherwoodfwiw i don't think i said the shuffle was to avoid redundant building. i'm *happy* with some redundancy, ad shuffling definitely causes some12:27
paulsherwoodanyway i've yet to prove my point, so i'll be quiet now :)12:28
*** zoli__ has quit IRC12:29
ratmice__I suppose this is where haskell would help in having some infinite list of dependencies, which we compute lazily :)12:29
*** zoli__ has joined #baserock12:29
KinnisonSadly haskell is a total pig to cross-bootstrap12:30
KinnisonOtherwise I'd have it in build-essential by now12:30
jmacsThe "Doing stuff on baserock" wiki page mentions Raspberry Pi - does that mean the ARMv6 version, or the ARMv7?12:30
Kinnisonjmacs: most likely the former12:30
rjekjmacs: I believe pedroalvarez did an ARMv6 port as a proof-of-concept12:31
ssam2jmacs: i wondered about separating the two12:32
ssam2it kind of means 'either' right now12:32
*** fay__ has joined #baserock12:33
*** fay_ has quit IRC12:33
*** zoli__ has quit IRC12:34
ssam2i'm having a nightmare trying to log into the openstack gerrit, I'm glad we decided not to use Ubuntu One ID's12:49
ssam2an attempt to reset my password produced an 'Invalid OpenID transaction' and then my password still isn't reset12:50
ssam22nd time round it seems to have worked12:50
*** pacon has quit IRC12:51
*** zoli__ has joined #baserock12:57
*** mariaderidder has quit IRC12:59
*** mariaderidder has joined #baserock13:11
ZaraI'm back! I built my beautiful tarball, extracted it, read the native-bootstrap script-- only to find it still contained ostree! oh no! closer inspection reveals that the devious morph-utils also depends on ostree-core, which in turn depends on foundation, etc etc etc...13:14
SotKyou can remove ostree-core from the build-depends list in morph-utils too13:16
SotKits not actually needed yet13:16
Zaraheh :) is it likely to be lurking anywhere else?13:16
SotKI don't think so13:16
SotK`git grep ostree-core` should show all the places it is, only some of which will matter to you I imagine13:17
Zaraoh good, just the two strata13:18
* SotK discovers that Zuul's documentation contains bugs13:19
pdaris anyone else having difficulty accessing w.b.o?13:37
rjekpdar: Branchable's having a bad day today.13:37
jmacsNow you mention it, yes13:37
Zarapdar: yeah, intermittently13:38
pdarphew, I'm glad im not alone13:40
petefothpdar: ti was v. slow for me earlier13:42
ssam2SotK: i hope you're going to submit fixes to review.openstack.org !13:44
ssam2i'm trying to fix the bug where the 'zuul' program hangs completely if it can't contact the server and has to be kiiled with -913:44
ssam2because that super annoying13:44
jmacsIn the absence of wiki.baserock.org, could someone point me to a recent build/dev image of baserock (x86)?13:52
jmacsOh, it's back up now13:52
jmacsWell... kind of back up13:59
*** mdunford has joined #baserock13:59
ssam2remember you can do `git clone git://baserock.branchable.com/` and get the whole wiki locally14:06
ssam2not ideal, but it's a workaround14:06
KinnisonApparently branchable is undergoing some crunchy rebuilds which are causing issues14:06
Kinnisonthe owners are working on it14:06
ssam2what would my Baserock system be missing in order that terminal colour codes wouldn't work?14:17
Kinnison1. is TERM set right14:17
ssam2TERM=xterm14:17
ssam2in my SSH session14:17
KinnisonDo you have /usr/share/terminfo/xterm ?14:17
ssam2yes14:17
Kinnisonmight be .../x/xterm14:17
ssam2yeah, it's that in fact14:18
Kinnisonnot sure, I'd expect that to be enough14:18
Kinnisonis ncurses built properly?14:18
ssam2 /lib/libncursesw.so.5.9 exists, at least14:19
KinnisonIs it that echoing colour codes isn't working, or simply that programs you expect to be colourised are not?14:26
DavePagessam2: Is the best workaroudn not to lorry baserock.brnachable.com to trove.baserock.org? ;)14:28
paulsherwood+114:30
jjardonwould it be ok to move e2fsprogs to -core so the -cross systems are not massive?14:31
paulsherwood+114:31
KinnisonDavePage: I wouldn't do that14:33
Kinnisonjjardon: are any of e2fsprogs' outputs used in the deployment mechanisms which might be used to create the proper build system out of a cross-bootstrapped environment?14:33
Kinnisonjjardon: if so, we cannot remove them from the cross systems14:33
SotKKinnison: I believe the idea is to move them into core so that they can be in the cross systems without the cross systems needing to include foundation+deps14:34
* SotK may be misunderstanding this though14:34
rdalei think core should contain build tools, it seems to be a general dumping ground14:34
jjardonKinnison: Im not suggesting that, but move e2fsprogs from foundation to core; morph depends on ostree, wich depends on the whole foundation because e2fsprogs only14:34
* Kinnison got the layer-up the wrong way around14:35
Kinnisonjjardon: So long as cross-bootstrapped systems can still deploy things effectively I'm okay with that14:35
Kinnisonthe sole purpose of the result of cross-bootstrapping is to have something capable of properly building and deploying a build system14:36
pedroalvarezindeed14:36
pedroalvarezalthough at the moment they have even systemd14:36
jjardonKinnison: well, until very recently they can not deploy as btrfs-progs is in foundation14:37
pedroalvarezjjardon: +1 to make the ostree depend on less things14:37
* Kinnison will gladly review anything which makes cross-bootstrapped systems smaller, but still fully functional14:37
jjardonI suggested to move btrfs-progs to core to avoid that but seems it was not a good idea14:38
* jjardon cooking a patch14:38
* pedroalvarez will wait for the chef14:38
rjekWe use ansible, pedroalvarez14:39
jjardonrdale: me too, but I can not find a better solution in this case. Any other idea?14:40
rdaleanother stratum between core and foundation?14:40
pedroalvarezrjek: :)14:41
* rjek fears we may end up with one chunk per stratum14:41
* SotK wonders if strata should be able to depend on single chunks14:42
* jjardon fears the same since a long time ago14:42
* richard_maw isn't worried about 1 chunk per stratum, provided we can reduce the overhead in the definition, and have runtime depends to ease the burden when defining systems14:45
ssam2Kinnison: the colour codes are showing up on screen14:46
ssam2it does seem that splitting 'foundation' is on the horizon14:48
ssam2that's even more of a dumping ground than 'core'14:48
pedroalvarezssam2: last time I had that issue was because my system didn't have `less`, and it was using busybox less14:48
ssam2ah14:48
ssam2pedroalvarez: that's probably the issue here too, thanks14:49
pedroalvarezssam2: sorry I didn't say that before, I thought you were trying to solve a different problem14:49
Kinnisonssam2: Hmm that's very odd14:50
* richard_maw kicks baserock-system-config-sync14:54
straycataww :(14:54
richard_mawbecause I have a systemd unit with escape sequences in its name (because it escapes the value in templated units) it's refusing to allow me to perform an upgrade14:55
richard_mawthe fault, is that the `read` builtin will unescape some things14:55
richard_mawhence when it goes to look for the file, it can't find it any more14:55
* tlsa would get rid of stratum, and just make chunks have explicit dependencies on chunks.14:57
jjardonpedroalvarez: dinner is ready: https://gerrit.baserock.org/#/c/681/14:58
rjekOne of the original design goals was to avoid that, IIRC14:58
Kinnisontlsa: that way "packaged" systems lies14:58
jjardonalso Kinnison ^^14:58
* Kinnison has a nibble14:58
* SotK gets hungry14:58
tlsaKinnison: to me the good thing about baserock is its well defined, and reproducable definitions14:58
tlsastratum just get in the way the whole time14:59
* rjek basically agrees.14:59
tlsait's an artificial indirection of dependencies14:59
rjekA warm fuzzy feeling, but not really pragmatic or workable14:59
jjardontlsa: so, for example, for every chunk you should have to specify all its dependencies ?15:00
tlsayes15:00
* Kinnison is worried that if we drop strata as a concept then we're simply back to the combinatorial explosion issues that plague normal "packaged" distros15:00
* richard_maw thinks there's still call for a physical format level aggregation of multiple definitions for well integrated components15:00
rjekmetapackages :)15:00
jjardontlsa: until binutils? thats a long list, and a lot of duplication15:00
* SotK thinks it would be nice to keep strata but allow them to depend on individual chunks15:00
richard_mawrjek: not for grouping, just to make it easy to define all the interrelations in one file15:00
* rjek nods15:01
tlsajjardon: I'd probably investigate scraping the base definitions from linux from scratch15:01
jjardonI like more the SotK idea. I didnt think in the implications though15:01
* Kinnison has no desire for Baserock to essentially become yet-another-source-based-distro15:01
Kinnisonwe may as well use Gentoo at that point15:01
pedroalvarezcan we now build a straum?15:02
Kinnisonssam2: Have you bugged anyone else for a review of https://gerrit.baserock.org/#/c/641/ ?15:02
tlsaKinnison: getting rid of stratum has no affect on it being about fully defined systems, with atomic upgrades15:02
pedroalvarezyes we can!15:03
ssam2Kinnison: no, jjardon maybe you could look at it again?15:04
Kinnisontlsa: My point is, if the tooling ends up defining something akin to that defined by portage, bitbake, whatever.  Then there's little benefit in maintaining our own tooling and we'd be better off augmenting existing stuff15:04
tlsaI don't know anything of these alternatives15:05
tlsaKinnison: my opinion is based on my experience of all baserock pain deriving from stratum15:05
rjekThey're all worse15:05
KinnisonSadly, software integration is not easy15:05
ssam2Kinnison: I agree, but I don't think that is a logical argument about whether to keep or drop strata15:06
ssam2gentoo supported metapackages since before Baserock existed, anyway15:06
Kinnisonssam2: If the defining characteristic difference is in our striated system definitions, removing the strata from the model leaves us with no defining characteristic not implementable in other systems15:06
Kinnisonssam2: metapackages are subtly different from a striated system15:07
ssam2but strata can be implemented in other systems anyway15:07
Kinnisonthough perhaps the subtlety is lost in our approach15:07
ssam2I really don't think they are the one differentiating factor of Baserock's tooling...15:07
jjardonssam2: done, next time assume the my +1 is still there if you are only improving things :)15:07
ssam2jjardon: ok15:07
ssam2Kinnison: I've no idea how strata differ from metapackages, it might be useful if you could explain what subtlety we have lost15:07
Kinnisonmetapackages are about runtime dependencies where strata are about both runtime and build-time behaviour15:08
ssam2ah, ok15:08
KinnisonAlso, strata are meant to be integration points (though we've never quite gotten to that)15:08
Kinnisonwhereas in a package/metapackage system, integration remains the domain of the package15:08
ssam2i'm pretty sure that could be implemented in Bitbake or Gentoo for less effort than it took to create the whole of Baserock ;)15:08
* richard_maw grumbles that https://gerrit.baserock.org/#/c/641/ was merged before he could point out that the conditional, while it will always work as expected, does so in a *nasty* way15:09
tlsastrata mean when you build something that's quite far up the tree of strata, there's masses of unrelated stuff in the build chroot15:09
Kinnisonssam2: perhaps so15:09
Kinnisontlsa: Yes, the striated model does mean build chroots are larger than they might be15:09
Kinnisontlsa: there's little we can safely do to find a halfway house between striation and packages though15:10
richard_mawssam2: the unquoted $sessions can cause parse errors in the `test` command, but since you invert the result, and can only get parse errors when you have multiple sessions, it works15:10
richard_mawbut… eww15:10
ssam2ah, I didn't realise. I hate shell!15:11
ssam2will it fix it to put "$sessions" in quotes there?15:11
richard_mawit isn't broken, it will behave as it's supposed to, but `test -n "$sessions"` is much nicer than `! test -z $sessions`15:11
ssam2ok, i'll try to remember that for the future15:12
* richard_maw prepares a patch to baserock-system-config-sync so he can submit a patch to morph to sort out --upgrade propagating to subsystem deployments so he can submit fixed versions of the shutdownramfs patches to he can get on with his experiments in fixing command-line parsing in systemd15:13
jjardonssam2: Would it I change your -1 for a +[0|1] if I promise to work in a filesystem-stratum as soon as it gets merged?15:14
tlsaKinnison: to me "packages" mean installing partial changes to the running system, and that's irrelevent to baserock.  I don't think "either you have stratum or you have packages" makes sense.15:17
jonathanmawrichard_maw: http://i.imgur.com/t0XHtgJ.gif15:17
Kinnisontlsa: without strata, you have no runtime groupings available15:18
Kinnisontlsa: which implies each software source needs to state its runtime dependencies15:18
Kinnisontlsa: thus packages15:18
richard_mawjonathanmaw: exactly15:18
Kinnisontlsa: whether your resultant system has binary packages or not is an implementation detail15:18
tlsaKinnison: no, I omitted the other part of my plan15:18
tlsa1. get rid of stratum15:18
tlsa2. add runtime deps to chunks, in addition to build time15:19
Kinnisonnow chunks are packages15:19
Kinnisonby any other name15:19
richard_mawhttps://gerrit.baserock.org/#/c/682/115:19
rjekKinnison: I never understood why that's a bad thing.15:19
KinnisonIt's to do with presenting well integrated collections, but frankly I'm fed up with going round and round this point15:20
tlsa3. systems just list the few things that make it stand out; e.g. linux, gnome shell, firefox (for a web browser system), and the system builder puts together a system that has all that incusing the deps it needs15:20
Kinnisonif enough people hate strata -- come up with a replacement set of tooling which does away with them and propose it15:20
rjekmorph doesn't really enforce their use, does it?15:21
Kinnisononly that currently systems contain strata15:22
tlsaI would have "groups" rather than strata.  e.g. "productivity" group would list chunks like "abiword, open office, inkscape, gimp", and a productivity system could have "linux, kde, group:productivity"15:22
ratmice__i'm a bit annoyed that the list of files installed by a build is only available as a effect of 'make install', so we can't even make some sort of nutso filesystem which builds packages on use15:22
tlsaand the system builder would flatten the groups out to chunk lists, pull in deps, etc15:23
Kinnisontlsa: So if I did away with strata and instead had "aggregate chunks" (essentially your 'group' concept) would you be happy?15:24
tlsaas long as "aggregate groups" couldn't be used as build deps15:25
ratmice__what annoys me is that all these dependencies are built into the blob, and then respecified outside of the blob, simply because it's all wrapped up in stupid effects, without a way to get the dependencies without producing the effect15:25
Kinnisontlsa: why prevent their use as build-deps?15:25
Kinnisonratmice__: yeah we did originally look into trying to auto-extract dependencies from the source trees but it turned out to be delightfully turing-complete15:25
tlsaKinnison: that inderection is where all my baserock pain comes from15:25
Kinnisontlsa: So in your model you would have to explicitly list every single one of your build-dependencies?15:26
Kinnisontlsa: from fhs-dirs up?15:26
tlsayes, probably.  Is that not how other systems do it?15:27
KinnisonNo15:27
tlsahow then?15:27
Kinnisonother systems allow you to build-depend on aggregations15:27
Kinnisone.g. meta-packages15:27
tlsawell, maybe the low level stuff you need can be grouped15:27
Kinnisonalso build-depending on foo, means you implicitly build-depend on all of foo's runtime dependencies15:27
Kinnisontlsa: But you said that the data model was not to permit build-depending on aggregations15:28
tlsabut atm, if you want to move a chunk from stratum A to stratum that A build depends on, B, the info about what in B it depended on is not stored anyware, so you have to include stuff by trial and error15:29
KinnisonCorrect15:29
* Kinnison doesn't see that as a problem15:30
KinnisonWe simply shifted work from when the chunk was in A to when you wanted to move the chunk to B15:30
Kinnisonit's not a case that more work has to be done, simply that when the work is done has moved15:30
KinnisonAnd that in the case the chunk is not moved, less work is done over-all15:30
tlsaanother is if you have a branch of defintions, and you add a stratum that depends on other stratum for some software, you get everything working, merge it to master, no conflicts happen, but your system now either fails to build or fails to run, because some chunk moved between 2 stratum on master while you were branched15:31
tlsawith explicit chunk dependencies, that couldn't happen15:31
ssam2jjardon: you can ignore my -1 if so15:32
ssam2jjardon: remember -1 doesn't block merging it15:33
Kinnisontlsa: Your best bet for getting your idea considered is to produce a definitions set which looks like you wish it did.  Then ask for commentary15:35
Kinnisontlsa: Right now I think your approach would be way more work in the general case, to solve a few niggles in rarer cases15:35
Kinnisontlsa: BICBW15:35
tlsaKinnison: there's much more work involved when you remove chunk from A and put it in B15:35
Kinnisonright, but is moving chunks a common action or not?15:35
KinnisonI think you need to do some analysis15:36
tlsait's not only finding out which things in B the moved chunk might dep on15:36
Kinnison"I have pain when I do foo, therefore we should redesign to eliminate foo" is only a good thing when the redesign doesn't make 'bar' more painful where 'bar' is far more common than 'foo'15:36
tlsadoes chunk A still need to dep on B, or does other stuff in there need it?15:36
tlsas/cunk/stratum/15:37
tlsaKinnison: moving chunks seems to be routine15:37
Kinnisontlsa: then you have a data point15:37
* Kinnison says again -- further discussion here without data to back things up, and a fully fleshed out proposed reshape will be less than useful15:38
ssam2agreed, one point in favour of the current system is that it more or less works, and we can use it15:39
ssam2not that your complaints aren't valid, just that i don't know of anyone personally motivated to take time to change things right now15:39
ratmice__i'd guess something can be done with atime to look at the files of a chunk and see if none of them were accessed?15:40
Kinnisonratmice__: I doubt atime is altered for a stat() and that might be all that's used in the build process15:41
* jjardon would like (and would work) to make everything explicit, if we agree on that (but keeping the stratum model, ie, build-deps explicit between strata, and explicit between the chunks within a stratum)15:42
Kinnisonratmice__: basically unless we had binary-reproducible builds (which I'd like us to work toward at some point), it's hard to determine if removing a b-dep causes a change in the resultant built chunk15:42
Kinnisonjjardon: We already had a discussion which ended with "explicit b-deps within a stratum is too hard / too much work / cannot be bothered"15:43
Kinnisonjjardon: No idea if that opinion has changed though15:43
tlsaearlier SotK mentioned the length of the mason system's stratum list.  It's long and full of many generic names that could mean anything, yet it would break without them.  With chunk runtime deps, you could have a system that has "linux-system, mason, morph".  i.e. just listing what it needs that differentiates it.15:43
tlsajust throwing out ideas :)15:43
* richard_maw *finally* updated his shutdownramfs patches15:45
ssam2i'm quite interested to see how the definitions would look with these different ideas, it'd make it much easier to evaluate the different approaches15:45
jjardonssam2: I will try to have something when I have time, can you point me to the part of the morph code where the dependencies are resolved, so I can set everything explicit?15:47
ssam2jjardon: it's around here: http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/morph.git/tree/morphlib/artifactresolver.py#n11815:52
jjardonssam2: cheers, thanks15:52
ssam2i've always found that code quite confusing15:53
paulsherwoodssam2: try ybd :) https://github.com/devcurmudgeon/ybd/blob/master/definitions.py15:53
ssam2looks simpler15:54
paulsherwoodthat's all of it :)15:55
* Kinnison cannot understand ybd15:56
paulsherwoodheh.15:56
Kinnisonit is too highly coupled and poorly documented15:56
* paulsherwood cannot understand morph, for the same reasons15:56
Kinnison(and given I wrote part of it, that's worrisome)15:56
* richard_maw rebuilds the world because of a change to tbdiff15:57
*** a1exhughe5 has quit IRC16:06
*** franred has quit IRC16:08
paulsherwoodgit.baserock.org downloads are stalling for me...16:15
*** Krin has quit IRC16:24
*** Krin has joined #baserock16:24
*** ssam2 has quit IRC16:26
*** flatmush has quit IRC16:26
*** fay__ has quit IRC16:26
*** mariaderidder has quit IRC16:26
*** nowster has quit IRC16:26
*** mdunford has quit IRC16:26
*** CTtpollard has quit IRC16:26
*** paulw has quit IRC16:26
*** gary_perkins has quit IRC16:26
*** bashrc has quit IRC16:26
*** petefoth has quit IRC16:26
*** Guest68378 has quit IRC16:26
*** sambishop has quit IRC16:26
*** jonathanmaw has quit IRC16:26
*** 7YUAAB3S9 has quit IRC16:26
*** Krin has quit IRC16:26
*** edcragg has quit IRC16:26
*** bashrc has joined #baserock16:26
*** edcragg has joined #baserock16:26
*** paulw has joined #baserock16:26
*** sambishop has joined #baserock16:26
*** Krin has joined #baserock16:26
*** tiagogomes_ has joined #baserock16:26
*** Guest68378 has joined #baserock16:26
*** mdunford has joined #baserock16:26
*** jonathanmaw has joined #baserock16:26
*** mariaderidder has joined #baserock16:26
*** petefoth has joined #baserock16:27
*** fay__ has joined #baserock16:27
*** nowster has joined #baserock16:27
*** CTtpollard has joined #baserock16:27
*** ssam2 has joined #baserock16:27
*** ChanServ sets mode: +v ssam216:27
*** gary_perkins has joined #baserock16:27
ssam2paulsherwood: if you're in the same place as me, perhaps it's our internet connection!16:27
*** flatmush has joined #baserock16:28
ssam2the machine has one CPU at 100% on a CVS import and another spending lots of effort to run lorry-controller-webapp (probably because the webapp.db file is huge again...), but it doesn't look broken16:28
KinnisonCVS import :(16:29
paulsherwoodi'm on  adifferent continent i think16:32
ssam2probably not that then16:33
pedroalvarezoh, cvs import processes again16:33
pedroalvarezkill 'em16:34
ssam2done16:34
pedroalvarezthanks16:34
pedroalvarezssam2: I normally notice that g.b.o is going wrong when the there are too many "now"s in the LC status page16:35
pedroalvarezand always related to these cvs processes16:35
*** mariaderidder has quit IRC16:39
*** mariaderidder has joined #baserock16:40
*** mariaderidder has quit IRC16:42
* pedroalvarez is not an emails guy16:51
*** Krin has quit IRC16:52
*** ssam2 has quit IRC16:54
* paulsherwood is failing completely to download dropbear. i wonder if my hotel has throttled me16:57
*** Guest68378 has quit IRC16:57
rdaleare you building the openwrt system?16:58
radiofreeany chance someone could +1 https://gerrit.baserock.org/#/c/656/ ?16:58
richard_mawradiofree: merged16:59
radiofreerichard_maw: thanks very much17:00
*** jonathanmaw has quit IRC17:02
*** bashrc has quit IRC17:28
*** mdunford has quit IRC17:31
*** gary_perkins has quit IRC17:38
*** edcragg has quit IRC17:40
paulsherwoodrdale: trying to, modulo throttling18:33
rdalepaulsherwood: ok i hope you can build it. we were thinking about a luci2 web based demo to show we had made some progress with the baserock openwrt port, but maybe demostrating an openwrt system being built under baserock is possible?19:30
paulsherwoodrdale: both :)19:30
paulsherwoodrdale: http://paste.baserock.org/sakotukore19:50
rdalei think uhttpd is lorried, and so it should be upstream:openwrt/uhttpd19:52
pedroalvarezjjardon: both lorries fixed (gnome-shell and perl)19:55
paulsherwoodrdale: http://paste.baserock.org/holozejoxu did you do something to it?19:56
jjardonpedroalvarez: thanks! :)19:56
pedroalvarezjjardon: keep reporting them, even if you don't mind they are not up-to-date. Is useful to know all of these problems19:57
paulsherwoodrdale: ?20:09
*** zoli__ has quit IRC20:32
*** Albert has quit IRC21:20
* paulsherwood gives up and tries master21:46
* radiofree is glad he checked on this build, nearly up to qtwebkit and forgot to do "swapon"21:47
paulsherwoodis that on jetson?21:48
radiofreeyes21:49
paulsherwoodhave you tried ybd on one yet?21:50
radiofreenot yet, i think that's more "something for the weekend", the one i have is currently very incrementally getting to version 0.1 of the genivi demo platform21:51
radiofreetweak, rebase against master... rebuild from core21:51
paulsherwoodah, right21:51
radiofreei really need to e-mail the gdp mailing list as well.... the genivi demo platform: a tale of two genivis21:52
paulsherwoodyou've reminded me, there was discussion about getting the gdp to be discussed on genivi-projects, as opposed to meta-ivi21:54
paulsherwoodi should follow up on that21:54
radiofreeyes, needs to be discussed properly21:55
paulsherwoodrdale: http://paste.baserock.org/figezolise another one21:56
radiofreeif it's supposed to be a demonstration of upstream genivi components it's failing miserably, based on the number of patches anyone integrating have to apply to these components to get this g-d-p to do anything21:56
paulsherwoodi feel your pain, rdale22:00
paulsherwoodradiofree, i mean22:00
rjekMy libvirt clone has finally finished.  Sadly my ansible role has failed.22:23
* rjek mutters; #ansible tells me to use Vagrent and he's not happy about that.22:23
* rjek does not want to have to learn another tool just to do something he already knew how to do.22:26
* rjek sadfaces and beds.22:26

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!