IRC logs for #baserock for Thursday, 2015-04-23

*** zoli__ has joined #baserock02:33
*** zoli__ has quit IRC02:38
*** zoli__ has joined #baserock03:11
*** zoli__ has quit IRC03:39
*** zoli__ has joined #baserock04:38
*** petefoth has joined #baserock05:08
*** mariaderidder has joined #baserock06:59
*** Albert_ has joined #baserock07:08
*** a1exhughe5 has joined #baserock07:13
*** franred has joined #baserock07:31
*** rdale has joined #baserock07:37
*** bashrc_ has joined #baserock08:02
*** sambishop has joined #baserock08:14
*** gary_perkins has joined #baserock08:26
*** edcragg has joined #baserock08:36
*** jonathanmaw has joined #baserock08:39
*** ssam2 has joined #baserock08:44
*** ChanServ sets mode: +v ssam208:44
ssam2jjardon: thanks for fixing up the ostree branch!08:51
tiagogomes_morning, how do I create an internal link to an section header for w.b.o using markdown09:06
Kinnisonw.b.o doesn't seem to have header-anchors turned on09:08
ssam2what is that? does that make it possible to have links to subsections of wiki pages that aren't useless ?09:09
* Kinnison is having to rebuild the wiki, please hold09:09
Kinnisonssam2: yes09:09
Kinnisonright that's done09:09
ssam2wow, we should enable that09:09
Kinnisontiagogomes_: if you look at the generated HTML for a page, you'll now see ids on <h1> etc09:09
Kinnisontiagogomes_: You can link to those with links along the lines of [something or other](#heading_anchor_id)09:10
* Kinnison hopes that helps09:10
tiagogomes_I tried the [something or other](#heading_anchor_id) thingy, but I may had set the wrong id09:10
* tiagogomes_ checks09:10
jmacs[[guides#GENIVI]] also works, although I'm not quite sure how09:12
Kinnisondouble-square-brackets are ikiwiki markup rather than markdown09:13
*** Krin has joined #baserock09:14
tiagogomes_Kinnison yep, using the wrong ID. I assumed that the ID would be the section header with the words separated by hyphens instead of spaces09:14
Kinnisonwikilinks are odd and magical so I'm not unsurprised that it works09:14
Kinnisontiagogomes_: I don't know the algorithm for generating ids :)09:14
*** lachlanmackenzie has joined #baserock09:15
jmacsI can't get [guides](#GENIVI) to work - I put that on the Sandbox page, and it creates a link to
tiagogomes_the generated link seems fine, the problem is that there isn't an anchor named 'GENIVI'09:17
jmacsAha, [Things](/guides#GENIVI) does though09:17
*** mauricemoss_ has joined #baserock09:23
jjardonssam2: no :)09:26
jonathanmawwell, that's odd. I deployed a fresh VM with my pam changes and I can't log in as root09:58
jonathanmaweverything's as expected for /etc/passwd, no password is set in /etc/shadow10:02
jonathanmawI'm going to add my ssh key and see if I can get in that way.10:02
KinnisonWoah, so close
KinnisonI have no idea how paste.b.o chooses its highlighting rules10:03
tiagogomes_richard_maw, does this makes sense: ? Or a different IP address was intentional?10:04
richard_mawtiagogomes_: what? I see no changes about IP addresses10:06
tiagogomes_richard_maw sorry, was this one
richard_mawI'm not sure how much of the HOSTS_SELF bit is needed, but that address was what dhcp was giving it10:08
richard_mawthe point was that it is the IP of the external interface10:08
tiagogomes_then pherpahs for the controller HOSTS_SELF should be the external IP addr as well10:13
jjardonjonathanmaw: in what order did you install the packages? for what I read about pam the order matter, and I think it should be: attr -> acl -> pam -> libcap -> shadow10:16
jonathanmawah, shadow isn't using libcap10:17
* jonathanmaw didn't know libcap had to be moved as well10:18
jonathanmawattr -> acl -> pam -> shadow10:18
jjardonjonathanmaw: I do not know neither for sure, only the little I read about it :). Also, did you configure /etc/login.defs ? You can see how in this guide to use pam with shadow:
edcraggsilly question... how is morph installed in built systems? i don't see any obvious strata10:24
jjardonjonathanmaw: mmm, seems we are missing several files in our shadow chunk. These are the ones in Arch:
tiagogomes_the reads obviously enough to me :)10:25
edcraggssam2: ah yes, thanks10:25
jonathanmawjjardon: I have a /etc/login.defs, nothing in it stands out as being wrong.10:35
richard_mawjonathanmaw: well, apart from it using DES for the password hashes10:41
jjardonjonathanmaw: rigth, check you have all the pam configuration files as well: you can take a look to the Arch ones here:
jjardonrichard_maw: yep, Arch overwrite the files and uses SHA51210:42
richard_mawI've got a patch that needs reviewing to make us do that too10:43
Krinrichard_maw, point mme at it and i'll give it a look over10:44
richard_mawKrin: enjoy:
Krini refuse to enjoy, i shal but do.10:45
*** ssam2 has quit IRC10:54
*** franred has quit IRC10:55
Krinugh, i forget, doing a review branch using gerrit is 'git review 'commit-id' '? or am i losing my marbles?11:00
*** Albert_ has quit IRC11:01
jjardonKrin: git review -d <gerrit change number>11:01
jjardongit review -d 284 for the Richard patch, for example11:02
*** franred has joined #baserock11:02
Krinok. yep my marbles are all over the floor XD11:02
Krinthanks jjardon11:03
*** ssam2 has joined #baserock11:03
*** ChanServ sets mode: +v ssam211:03
jjardonssam2: FYI: 22:45 <jjardon> uh, our u-boot version is 3 years old ... it can be the cause why highbank doesnt with new kernels:
jjardon22:47 <SotK> jjardon: looks very similar, repost that link tomorrow when ssssam is online I guess :)11:12
jjardon22:47 <jjardon> yep11:12
jjardon22:48 <jjardon> only so I do not forget: secure mode can be configured with the bootm_boot_mode u-boot option11:12
ssam2although i have no idea how to flash u-boot on calxeda11:17
ssam2or if that's even possible11:17
ssam2but thanks, that's really useful to know11:18
ssam2'That should save some other poor sucker many wasted hours.'  indeed.11:18
richard_mawIIRC the last time we tried updating the firmware we bricked half the nodes11:19
richard_mawand that was with a provided blob11:19
richard_mawand we no longer have the option of RMA11:19
ssam2new version of patches to add OSTree to definitions for use by Morph:
ssam2have built this up to ostree on x8611:20
ssam2jjardon: do you have time to check it over quickly?11:21
ssam2only change since the last version is to add morph to cross-bootstrap systems actually11:22
ssam2add ostree to cross-bootstrap systems even11:22
jjardonssam2: sure11:23
jjardonssam2: mmm, if you add python-pygobject to the cross-* systems, should not be foundation have to be added as well?11:26
jjardon(for the GLib dependency)11:26
ssam2oh, good point11:26
ssam2which is really defeating the point of the cross-bootstrap system...11:27
ssam2but the alternative is maintain code for 2 types of artifact caching, which is also far from ideal11:28
jjardonyeah :( maybe we should split GLib in its own stratum?11:28
jjardon(with gobject-introspection on it as well)11:28
ssam2doesn't really make sense having it in with foundation11:30
jjardonssam2: I already make a comment in my own patch but; are we sure we depend on python-core in python-pygobject? if yes it should be added in the corss-* systems as well11:31
ssam2v5 sent:
ssam2jjardon: hmm, ok11:31
jjardonsorry I didnt spot this yesterday :/11:32
ssam2sorry we don't have code to runtime dependencies11:33
ssam2it's not sensible to make humans have to work this stuff out when computers can do it11:33
ssam2ok, there's a v6:
ssam2it's shit that we have to make cross-bootstrap massive suddenly. but none of the extra python stuff should really cause problems11:34
ssam2thank you!11:39
jjardonyeah, lets fix dependencies when this got merged: Ive +1 all the patches in the series I didn't send myself (which have a implicit +1)11:39
ssam2cool. We now have OSTree in definitions!11:42
jjardon"2015-04-23 10:59:55 system minimal-system-armv5l-aspeed is cached at ..." :) <- Im interested myself on making the cross systems as small as possible :)11:42
jjardonssam2: \o/11:42
ssam2next stop: add unionfs-fuse, so the morph branch will work without needingt linux 3.1811:42
ssam2and because i have to rebuild everything above systemd to try it out, that may take the rest of the day.... :(11:46
*** Albert_ has joined #baserock12:22
*** ssam2_ has joined #baserock12:23
*** ssam2 has quit IRC12:25
*** franred_ has joined #baserock12:50
*** franred has quit IRC12:54
*** mariaderidder has quit IRC12:54
*** mariaderidder has joined #baserock12:55
richard_mawrecent commits to systemd imply that CONFIG_FHANDLE may return to being optional, but only with a recent kernel12:56
* Kinnison takes a deep breath and asks ybd to build and deploy a disk13:02
paulsherwoodwhoah! :)13:02
Kinnison2015-04-23 13:08:33 [] Deployment begins13:09
* paulsherwood imagines Kinnison, cackling with a maniac grin13:12
mwilliams_ctcan confirm13:12
* Kinnison actually fetched tea and quietly looked at snake and kitten pictures13:12
Kinnisonbut you can imagine whatever you want13:12
Kinnison2015-04-23 13:10:51 [] Running write method: ['/src/morph/morphlib/exts/rawdisk.write', '/src/tmp/deployments/tmpOVL3_v', 'build-system-x86_64-ybdtest.img']13:13
* Kinnison taps his fingers impatiently13:13
* paulsherwood too13:13
Kinnison2015-04-23 13:14:08 [TOTAL] Elapsed time 00:12:3513:14
Kinnison2015-04-23 13:14:08 [clusters/ybdtest.morph] Finished13:14
* Kinnison very gently pokes it13:14
paulsherwoodkaboom :)13:14
KinnisonHoly butts, it booted13:15
Kinnison*and* jonathanmaw just gave me money13:15
Kinnisonit must be my lucky day13:15
paulsherwoodwere the two incidents related?13:15
Kinnisonjonathanmaw just loves me that much13:16
paulsherwoodso does the booted system appear to be similar to a normal build system? does it appear functional?13:16
* Kinnison is poking it, gently, as stated13:16
Kinnisonso far, I can log in13:16
KinnisonIt does not seem to have DHCP'd13:17
Kinnisonthere are some errors in the boot log13:17
straycatdid you run the usual set of config extensions?13:17
KinnisonI did13:18
perrylin gerrit, do +1s work across patches? i'm looking at which has +1 each for the 4th and 5th revisions...looking at the changes between the revisions the only difference seems to be that one was rebased atop master, so they're essentially the same13:22
* straycat is looking at 408 as it happens13:22
straycatI can also test the change in theory13:22
perrylstraycat: ahh, thanks!13:23
*** zoli__ has quit IRC13:29
paulsherwoodKinnison: are your patches decent enough for public consumption?13:36
*** a1exhughe5 has quit IRC13:38
KinnisonNot until I work out what's going on13:38
Kinnisonand clean them up after13:38
KinnisonOkay, I may have found the hiccough there13:53
* Kinnison redeploys13:53
KinnisonWell, I believe that's plausible14:02
KinnisonI shall clean up these commits now14:02
Kinnisonpaulsherwood: how picky are you about whitespace14:03
*** a1exhughe5 has joined #baserock14:03
Kinnisonpaulsherwood: master of my ybd can now deploy in a rudimentary fashion14:06
Kinnisonpaulsherwood: I invoke it with: env PYTHONPATH=/src/morph python /src/ybd/ clusters/mytestcluster.morph x86_6414:07
Kinnisonpaulsherwood: it's all very *very* dodgy for now14:07
KinnisonMy brain is now frazzled, so for now it's not very nice14:08
*** zoli__ has joined #baserock14:28
* pedroalvarez tests linux 4 on ppc6414:30
pedroalvarezthat worked14:31
*** Albert_ has quit IRC14:36
*** Albert_ has joined #baserock14:37
persiaKinnison: WIth your ybd success today, are we closer to a workflow that gives a working development environment with `pip install ${FOO}; git clone ${BAR}`?14:55
KinnisonNo idea14:56
Kinnisonybd still depends on linux-user-chroot and is utterly unsuitable for anything other than experimentation by people very experienced in how Baserock works14:57
KinnisonAll I've done is hacked in some deployment capability so that paulsherwood or others can continue to experiment and learn14:57
jjardonssam2_: are you sure python-cliapp actually depends on python-core? I built a cross* systemd without having python-core on it  (python-core was not added to cross* systems when python-cliapp was introduced)15:00
persiaOh, hrm.15:00
ssam2_jjardon: ah, i'm not sure15:03
Kinnisonpersia: if someone actually tasked me with what you just asked about, I assume I'd spend some time first working out how to get there, and thence how to start15:03
ssam2_jjardon: remove it if it's not needed after all15:03
ssam2_me and richard_maw are wondering who pushed master of mime-types.git to git.git on git.baserock.org15:04
ssam2_that is definitely not the source code to Git!15:04
persiaKinnison: Fair.  When ybd started, I had the impression it would have different dependencies, and that these would end up documented, but things don't always work out the way one hopes.15:04
ssam2_I guess it must have been Lorry, i don't think anyone else can force-push to master of delta/15:05
Kinnisonpersia: right now, the critical thing missing is a formal specification for Baserock definitions -- who knows what ybd is "doing wrong" so far?15:05
persiaKinnison: BNR or some such?15:06
* persia can't spell15:07
persiaI apparently meant "BNF", although amusingly the wikipedia page for "Backus-Naur Form" is the first hit for "BNR" from my preferred search engine.15:07
KinnisonRight, erm, not necessarily15:08
persiaRight, so what level of formality do you imagine then.15:08
persiaTo put it another way: how formal is formal?15:08
KinnisonPerhaps I don't mean 'formal' so much as 'as complete as possible'15:08
KinnisonI want a document which I could use, without referring to morph's codebase at all, to implement something to process a definitions tree and build and deploy systems15:09
persiaIf we're going for complete, we may as well go for formal.  Hard to validate otherwise.15:09
jjardonIf the stuff in is not enough, please add what you think is missing15:10
persiaMy search engine fails me, but is there anything like a DTD for YAML that can be used to validate that a given document matches a given specification?15:10
Kinnisonpersia: there are some fairly simplistic structural schema things out there which I have used on other projects -- I think locallycompact might remember what it was called15:12
persiajjardon: The three issues are 1) the document claims not to be a specification (it also claims to be inaccurate), and 2) the document does not claim to be complete, and 3) we have no way to validate a given set of definitions against the content provided to determine if it complies with the non-specification.15:12
Kinnisonjjardon: I think that is entirely inadequate and have nowhere near enough time to "add what I think is missing" right now15:12
Krinhey all, trying to deploy a baserock system, and it's been sitting at this for a long time now (as can be seen on the timestamp)15:13
Krin2015-04-23 14:49:25 [systems/openstack-system-x86_64.morph][release]Fetching to local cache: artifact openstack-system-x86_64-rootfs15:13
Krinanyone have any ideas?15:13
KinnisonKrin: Is your network interface still transferring?15:13
KinnisonKrin: it could be a very slow internet connection15:13
Krinhow would i test that Kinnison?15:13
Krinth wifi light is still going like crazy, just realised i forgot the ethernet when i came back into the office today ¬.¬15:15
KinnisonIf your wifi isn't great, it could be that15:15
franred_Krin, you can use --verbose to check what is doing your morph command15:15
Krink. gonna plug the ethernet in, kill it and restart it with verbose, thanks all :)15:16
jjardonpersia: yeah, it has issues but its the best we have rigth now. The only way to improve it is to help adding the missing stuff15:16
persiajjardon: I'm much more tempted to toss it out completely and draft a formal document of what I want.  Trying to determine a specification from a working system is something I try to avoid.15:17
franred_Krin, also you are trying to download 10 GB of rootfs, I would suggest you to build your rootfs - I disabled the cached for this15:17
persiaThis isn't to say that someone else shouldn't add to it if they like, but rather that I think it worth considering what class of specification we want, and how we can validate it, before we try to populate it.15:18
persiafranred_: Why?15:18
franred_persia, build the rootfs with the cached artifacts was faster than download 10 GB15:19
franred_more if you are using the wifi15:19
persiaAh, so the optimisation to not bother downloading the artifacts if you can download the rootfs never landed?15:20
pedroalvarezfranred_: 10G??15:20
jjardonpersia: rigth, other projects (jhbuild) use a DTD, but as you Im not usre something similar exist for YAML. Do you know if Yocto has some kind of validation tool?15:20
KinnisonWhile I've not used it, might be a starting point15:21
franred_pedroalvarez, umm, 2.6 GB sorry15:21
Kinnisonthere's also
KinnisonI think rx is what I used before15:22
persiaNote that at high latencies from the DC, this may differ, as at high latencies, tcp windows open rather wide, so a large transfer is *much* faster than several small ones.15:22
persiajjardon: I know that bitbake cannot have any sort of validation tool because it uses a turing-complete language as the means of describing the builds.15:22
persiaI don't know if Yocto is considering other build tools.15:22
persiaNote that there do exist parsers to validate that bitbake recipies are syntactically correct: it's just the semantics which cannot be validated (as the developer has a very wide range of modification mechanisms from which to choose)15:22
persiaKinnison: pykwalify looks exciting to me: the idea of using YAML to write the specification for the definitions YAML has a nice smell.  Rx looks cool, but we run into the issue "How does one validate the format of the specification".15:25
KinnisonI think pykwalify may not have been as usable back when I looked first, but yes it looks quite good now15:27
*** ssam2_ has quit IRC15:41
*** edcragg has quit IRC15:54
*** ssam2 has joined #baserock15:55
*** ChanServ sets mode: +v ssam215:55
*** edcragg has joined #baserock15:56
*** a1exhughe5 has quit IRC15:58
*** mariaderidder has quit IRC16:01
jjardonDo we want GLib in its own stratum or can we move it to core? Its a "library required for the Foundation stratum", so maybe it fits there?16:06
persiaI'm a fan of putting things as low in the stack as possible, assuming there are no systems that do not need them.16:06
rdalei wouldn't have thought glib would be used in a server only system16:07
persiaIs there anything that uses core and/or foundation that does not need glib?16:07
jjardonsystemd currently depends on glib, so I do not think so16:08
jjardonminimal systems doesnt use systemd, but they do not have core neither16:09
persiaThen core sounds sensible to me.16:11
rdalei don't think a gui lib should be in core16:12
persiardale: The tricky bit is that to reuse a component, we have to provide everything it needs at build-time.  We could build two different components from the same source (with different build-deps and ./configure input) to reduce the total bindings size for some systems, but it's double the maintenance, so only worth it if the benefit is very large.16:12
ssam2rdale: I agree. Glib doesn't provide any kind of gui16:12
rjekglib isn't a gui lib16:12
rdaleit's only needed by a gui lib16:13
rjekIt's a data structure library and yet another typedef for char.16:13
persiardale: You'd rather maintain two different systemd artifacfts?16:13
jjardonrdale: sorry, thats not true16:13
*** Albert_ has quit IRC16:14
rjekrdale: ldd $(which pkg-config)16:23
rdalerjek: what does that tell me?16:24
rjekIt tells you that pkg-config depends on glib, and pkg-config has nothing to do with GUIs16:24
jjardonGLib is more similar to STL or boost for C16:25
rdaleok, fair enough. my pkg-config 0.28 in kunbutu 14.10 depends on glib, but pkg-config 0.28 in baserock doesn't16:27
rjekIt's just a type library, it has no GUI functionality whatsoever.16:27
persiaI suspect that pkg-config in Baserock is missing some functionality that it would gain did it have glib headers available at compile-time.16:27
jjardonthats because pkg-config embed a partial copy of GLib in its sources16:27
rdaleok, then i think you're right and glib/pkg-config should be in the core then16:28
rjekIt's probably that Debian ban embedded copies of libraries, and removing them and having them use the system copy is a requirement for the package to be uploaded, and then Ubuntu take Debian's package16:28
rjek(Were Baserock uses raw upstream)16:28
jjardongreat, that would save quite a lot of work :)16:29
persiaI suspect that we ought demonstrate how to avoid embedded source in one of our reference systems, for people who care about security maintenance, etc.16:29
rjekpersia: have lorry fetch Debian source patches and apply them? >:)16:30
* rjek wonders how many copies of zlib there are in a Baserock dev system.16:30
rjek(Debian were shipping hundreds at one point)16:31
rjekWhich is not fun when you get a zlib security update.16:31
persiaWe'd need a more advanced build environment construction mechanism, to get the patch to be applied into the build environment.16:31
rjekpersia: That sounds like something useful and nice to have anyway16:32
persiaBut other than that, lorrying the packaging from debian, and using that, seems like a low-effort way to do that sort of thing.16:32
persiarjek: Very much so, indeed.16:32
*** zoli__ has quit IRC16:33
persiaBut I think the current recommendation is to just create a new branch of the project with the patch applied.16:33
* persia isn't quite sure of the best workflow: having it in git is really nice, but avoiding manual rebases is also nice, plus rebasing branches in a curated git repo makes kittens cry.16:34
jjardonwe build zlib in build-essential, so if the autotools configuration of the rest of the chunks are correct, they should pick up the system installed zlib: if not lets fix it when we have time (or file a bug in our storyboard)16:37
persiaThere exists a vast quantity of software with embedded zlib sources that does not support using system zlib.  Lots of it even depends on versions of zlib with call semantics different than modern zlib.16:38
rjekjjardon: Nobody's autotools configuration is correct.16:39
rjekAlso, what persia said16:39
persiaThis is an unfortunate thing, but provides an excellent target for demonstrating how to reduce the security exposure of embedded sources using the Baserock tooling.16:39
tiagogomes_rjek +116:39
* Kinnison remembers doing the work to allow cherokee to use non-embedded zlib16:39
Kinnisonthat was "fun"16:39
*** zoli__ has joined #baserock16:46
jjardonssam2: FYI: I have a couple of patches here to make cross-* systems be of a sane size again, will submit when I finish building16:47
ssam2awesome, thanks16:47
*** tiagogomes_ has quit IRC16:49
*** zoli__ has quit IRC16:59
*** bashrc_ has quit IRC17:02
*** edcragg has quit IRC17:17
*** zoli__ has joined #baserock17:20
*** mauricemoss_ has left #baserock17:23
*** zoli__ has quit IRC17:24
*** jonathanmaw has quit IRC17:28
paulsherwoodKinnison: thanks for the patches. what do you mean 'no cleverness though' - do you have specific things in mind?17:38
paulsherwoodalso, what was the upshot on the erlang problem... is it a bug?17:42
ssam2could anyone do a quick review of ?17:56
ssam2i've tested it and it can't break anything for you unless you use a Calxeda server :)17:56
ssam2or even if you do17:56
Kinnisonpaulsherwood: it can't select which subsystems to deploy, it can't do sub-system deployments, it can't alter the deployment parameters from the cmdline, it can't deploy other-architecture stuff17:58
Kinnisonpaulsherwood: re:erlang -- ybd is sensitive to filesystem ordering and there are duplicate names causing issues17:58
*** zoli__ has joined #baserock17:59
*** gary_perkins has quit IRC18:03
*** ssam2 has quit IRC18:07
* paulsherwood +2s ssam2's patch and merges it18:08
*** ssam2 has joined #baserock18:09
*** ChanServ sets mode: +v ssam218:09
*** lachlanmackenzie has quit IRC18:10
*** ssam2 has quit IRC18:22
*** zoli__ has quit IRC19:11
*** zoli__ has joined #baserock19:34
*** zoli__ has joined #baserock19:54
*** zoli__ has joined #baserock19:56
*** edcragg has joined #baserock21:00
*** zoli__ has quit IRC22:14
*** zoli__ has joined #baserock22:14
*** edcragg has quit IRC22:30
*** zoli__ has joined #baserock23:15
*** zoli__ has quit IRC23:21

Generated by 2.15.3 by Marius Gedminas - find it at!