IRC logs for #baserock for Monday, 2016-03-14

*** gtristan has quit IRC05:48
*** gtristan has joined #baserock06:16
*** paulsherwood has quit IRC07:15
*** SotK has quit IRC07:15
*** persia has quit IRC07:15
*** pedroalvarez has quit IRC07:15
*** dabukalam has quit IRC07:15
*** paulsherwood has joined #baserock07:16
*** SotK has joined #baserock07:16
*** pedroalvarez has joined #baserock07:16
*** dabukalam has joined #baserock07:16
*** sets mode: +v pedroalvarez07:16
*** persia has joined #baserock07:18
*** franred has joined #baserock08:21
*** toscalix has joined #baserock08:55
*** N-a-N has joined #baserock08:56
*** bashrc has joined #baserock08:58
*** edcragg has joined #baserock09:15
gtristanI am hot on the trail of figuring out the build issue of building sed/diff/tar against freshly built coreutils09:24
*** rdale has joined #baserock09:25
gtristanand it's... quite interesting... I think pedroalvarez will understand when I call it "fun"09:25
* pedroalvarez is here waiting for some fun09:25
gtristanI have reduced the problem to this:
gtristanbasically, after compiling strace in the sandbox... I found that cat essentially fails to stat stdin09:26
gtristaninterestingly... busybox bash/sh feed stdin to programs in different ways09:27
gtristanso, I think that cat is "ok" actually, something is obviously wrong and it's hard to pinpoint09:28
pedroalvarezwow, this might be too much fun for me!09:28
gtristanI have built a static binary of that program above, both with glibc and with musl09:28
gtristanso in every case... echo "pony" | ./test... succeeds with stat09:29
gtristanthe problem occurs with bash, never busybox sh... when the redirection technique is used:09:29
gtristan./test <<_EOF\npony\n_EOF09:30
gtristannow, I also have several shells to test with... I have native musl busybox and native musl bash 2.15b09:30
gtristanthose are static compiled09:31
gtristanthen I have baserock staged busybox sh, and baserock staged bash 4.3+patches09:31
gtristanboth of those dynamically linked against glibc09:31
*** tiagogomes has joined #baserock09:32
gtristanboth static musl bash and baserock staged 4.3 bash fail this test when run inside the chroot09:32
gtristanoutside of the chroot, I only have must static bash... and it passes the test !09:33
gtristanwhen performed both on the virtfs share, or outside of the virtfs share09:33
gtristanso with all of these combinations handy, I think the interesting comparison is... static linked musl bash A.) Fails the test in the chroot... B.) Passes the test outside of the chroot09:34
pedroalvarezsame problem if you redirect from a file?09:35
pedroalvarezhm...same static linked musl bash? or recompiled inside the chroot?09:35
gtristanI tried with the same original static linked musl bash which is default in the aboriginal image, from both inside and outside of the chroot09:37
gtristanOk, I can reproduce completely without any baserock packages built09:38
gtristanI had been suspecting this but wasnt sure to believe it or not09:39
gtristanvirtfs related...09:39
gtristanSo, that test... with static musl bash... run in the chroot sandboxing script I have... without *any* baserock packages staged09:40
gtristanone sec...09:40
gtristanthis one:
gtristanthat sandbox script basically assumes that things might be staged... and stages toybox/busybox/make/bash in the absence of already staged tools09:41
gtristanso that sandbox... on a fresh directory... PASSES the test outside of the virtfs share09:42
gtristanFAILS the test inside the virtfs share09:42
gtristanthis has to do with mmap implementation of the filesystem at /09:43
* pedroalvarez feels like the best rubber duck in #baserock09:43
gtristanyou know... in montreal there was (is?) a company called discrete logic09:44
gtristanthey do stuff like image manipulation and such09:44
gtristananyway, I knew some people worked there, ... they had this teddy bear in the common lounge09:45
gtristanand people were supposed to go and tell that teddy bear about their problems09:45
gtristannot sure how well it worked :)09:45
*** jonathanmaw has joined #baserock09:49
*** ssam2 has joined #baserock10:03
*** ChanServ sets mode: +v ssam210:03
gtristanpedroalvarez, with help from teddy bears and rubber ducks in #bash, I was able to work around the issue10:07
gtristanseems that for that to work, bash does not require features from /... but it does require... probably the mmap implementation... from /tmp10:08
gtristanso bind mounting /tmp into the sandbox fixes this10:08
gtristanthis was quite a ride :)10:09
pedroalvarezwow! it was10:09
pedroalvarezto end up mounting /tmp10:09
* gtristan had started with a line-by-line in tar's configure10:09
gtristannarrowing it down to where it invokes cat with <<__EOF10:10
gtristanthen finding that all the stuff it set in the environment didnt matter, but #!/bin/bash vs #!/bin/sh made the difference10:10
gtristanthen to looking through cat.c10:10
gtristandownloading and building strace in the sandbox (which... we should totally have in some low level of baserock, btw)10:11
gtristanisolating the broken fstat6410:11
gtristanand... down to this yes... mounting /tmp :)10:11
robtaylorgtristan: you may want to bindmount a mktemp -d to reduce potential attack surface on build server10:16
*** ctbruce has joined #baserock10:21
gtristanrobtaylor, could be... could be just interesting to prevent accumulation of crap in /tmp in the aboriginal guest from build to build too10:26
gtristanI wonder if there is potential to fix the glibc build bug too10:27
gtristani.e. we suffer from
gtristanperhaps glibc folks could be convinced to build the archive in /tmp10:28
* gtristan deserves supper10:29
* gtristan back soon10:29
robtaylorgtristan: you do ;)10:31
robtaylorgtristan: seems the last entry was a request for patch, so maybe we could just do that10:31
*** gtristan has quit IRC10:33
*** gtristan has joined #baserock10:59
*** locallycompact has joined #baserock11:06
*** vgrade_w has joined #baserock11:08
pedroalvarezI think I missed this patch when upgrading fuse:
pedroalvarezedcragg: I guess fuse will fail to build in aarch6411:14
pedroalvarez(for whenever you hit the error..)11:16
tiagogomesis that still needed ¬_¬11:17
pedroalvareztiagogomes: well, I would assume so11:17
*** locallycompact has quit IRC11:18
*** franred has quit IRC11:21
*** locallycompact has joined #baserock11:25
* gtristan has never been so satisfied to see sed configuring11:34
*** franred has joined #baserock11:34
edcraggpedroalvarez: haven't seen fuse fail if it has indeed been built in any of the systems i've build11:37
pedroalvarezedcragg: that change was merged this weekend11:43
edcraggfuse built successfully as late as the 9th of march11:47
*** franred has quit IRC11:47
edcraggso, depending on when it was merged, i'm not sure if it is required...?11:50
pedroalvarezwhat we merged was an upgrade to fuse 2.9.4, and when upgrading I missed that patch for aarch6411:52
pedroalvarezthat merge was this weekend, so my guess is that whenever you update your definitions, your build of fuse will fail11:52
edcraggright, ok, i'll change the ref and try to rebuild that at some point11:54
edcraggit's the sort of thing you'd expect upstream to have fixed already though11:54
edcraggi'm guessing11:55
pedroalvarezI can see a few patches upstream in master12:00
pedroalvarezbut not in 2_9 branch12:01
*** franred has joined #baserock12:01
pedroalvarezso yes, fixed upstream, but not present in this release12:01
*** franred has quit IRC12:49
gtristaninteresting... building btrfs-tools complains with: No package 'ext2fs' found13:08
pedroalvarezbut fails?13:09
gtristanand fails13:10
gtristanoddly, I'm looking, and it really should be failing13:10
gtristanit's not configured with --disable-convert13:11
pedroalvarezis it build-logs time?13:11
gtristanand it's clearly asserts ext2fs in the case that --disable-convert is not passed13:11
* gtristan serves up build log13:13
pedroalvarezgtristan: dobule checking, btrfs-progs?13:16
gtristanthats the id which is in definitions for btrfs-progs13:17
gtristanright, you say: checking for EXT2FS... yes13:17
gtristanbut where could that possibly come from ?13:17
pedroalvarezmaybe.. EXT2FS was provided by something in build-essential?13:17
gtristanand I build busybox in build-essential in exactly the same way13:18
pedroalvareze2fsprogs in core13:18
pedroalvarez/baserock/e2fsprogs-devel.meta:        "usr/lib/pkgconfig/ext2fs.pc"13:19
gtristanI see13:19
gtristanok, that's my bad :)13:19
gtristanpedroalvarez, got it... it's on the way :)13:20
* gtristan had a bad e2fsprogs build13:21
gtristanfrom a day or two ago13:21
pedroalvarezgtristan: btw, I've sent this patch yesterday:
pedroalvarezadded you and jjardon as reviewers, for whenever you have time. Should be quick13:22
paulsherwoodfolks, jjardon has raised this but my immediate reaction is to wonder whether this could be related to the extension itself?13:27
pedroalvarezpaulsherwood: does ybd handle subsystems?13:31
gtristanI have _always_ had some issues with the extension... usually because my host tooling was not *exactly* the versions that the extensions wanted13:32
gtristannot sure if this can be related13:32
gtristanbut the extensions are finicky wrt to host tools13:33
pedroalvarezpaulsherwood: if it does, maybe subsystems are being deployed after the main system is deployed.. and that will trigger this error13:33
pedroalvarezthis check of the configuration extension is making sure that the path specified for the initramfs system actually exists13:33
paulsherwoodpedroalvarez: i believe it does handle subsystems13:33
paulsherwoodSotK implemented it, iirc :)13:34
paulsherwood(just in case he's listening :-)13:34
pedroalvarezheh, he won't be around today I think13:35
*** vgrade_w has quit IRC13:37
*** vgrade_w has joined #baserock13:37
gtristanlooks promising13:40
* gtristan is building systemd, btrfs-tools and fuse13:40
gtristantime to pack it in, I think I'll restart a clean build and see if I can get as far building overnight13:40
paulsherwoodjjardon: ah, i understand now, wrt
paulsherwoodall: wouldn't it make sense to have build-steps in DEFAULTS, rather than the schema?13:44
paulsherwoods/wouldn't it/surely it would/13:45
richard_mawhow do you define which keys are valid build steps?13:45
paulsherwoodby having them in defaults, rather than in the build tool?13:46
richard_mawno I mean when you say foo-commands, how do you know that it's a valid build step?13:46
paulsherwoodcurrently ybd falls back to
*** gtristan has quit IRC13:47
paulsherwooda valid build step is what we say it is, i think? previously morph had it hard-coded iirc13:47
* paulsherwood thinks schemas should define the syntax, not the content... but may be wrong13:48
paulsherwoodrichard_maw: have i answered your question?13:49
richard_mawwell, in the absence of a top-level "commands" key in the chunk definition file, the commands are syntactic elements, meaning you need to enumerate them in the schema as well as defaults13:49
richard_mawif you had the latter then you could say that every entry under "commands" is a command that your schema can validate the same way13:51
paulsherwoodi don't think that's necessary, but i'm probably missing your point13:51
richard_mawok, let's try it this way around13:51
paulsherwoodare you -1 on my suggestion to put the build-steps into DEFAULTS?13:51
richard_mawhow does the schema know which keys in a chunk definition are commands?13:51
paulsherwoodwhy should the schema care?13:52
richard_mawbecause the schema needs to be able to tell you "you've put a string there, rather than a list of strings"13:52
richard_mawwhich it can do currently, because we define *-commands at the syntactic level, so those keys are special and it knows that those need to be validated as commands13:53
*** gtristan has joined #baserock13:53
paulsherwoodok... so you *are* -1 on my suggested approach?13:53
richard_mawhow do you propose to validate commands after moving the definition of which keys are commands to the DEFAULTS file?13:55
paulsherwoodi'm struggling to get my head round this again, so please bear with me...13:57
paulsherwoodbut note that... if the build-steps change in the schema, which is now *outside definitions*, then it means that definitions won't tightly define build-steps any longer...13:58
paulsherwood(i think)13:58
paulsherwood... so we're introducing uncertainty (in the engineering sense) where none existed before, i fear13:58
richard_mawI'm all for defining the build steps in DEFAULTS (though I think they should be per-build-system rather than a global list).14:01
richard_mawI'm just failing to articulate14:01
paulsherwoodthis is a difficult topic14:02
paulsherwoodcan i get a pro-tem +1 on adding them to DEFAULTS while we reason about schemas separately?14:02
* richard_maw no longer votes, so merge it if you want14:03
paulsherwoodah, ok14:03
paulsherwoodssam2: ?14:03
richard_mawpaulsherwood: short term I think your easiest approach is to require that commands match ".*-commands$", and change the schema something like:
ssam2paulsherwood: are you envisaging something like this in DEFAULTs, then ?14:09
richard_mawsince if you can't syntactically tell which keys are meant to be commands, then you can't syntactically validate them14:09
* paulsherwood has brainfade... can not compute this :/14:11
richard_mawschema knows that build-commands is a command-sequence because it's in the schema that the build-commands key has a value that's a command-sequence14:12
richard_mawbecause that's in the schema, you can validate build-commands is a command-sequence14:12
ssam2basically the idea is allowing definitions.git to customise what command-sequences can be used, right?14:14
ssam2so you could invent blabble-commands or whatever14:14
ssam2the JSON-Schema schemas obviously can't validate such user-defined things, but that's not necessarily bad14:15
ssam2the question for me is whether it's a good idea to allow user-defined command sequences, or whether it's better to have a specific set that everyone has to use14:15
paulsherwoodssam2: yup. in which case, inventing them in DEFAULTS seems sensible to me, rather than having to futz with spec.git14:17
* richard_maw can't answer that, just that if you wanted to instead ignore user-defined commands then you lose the ability to catch that an optional field has been mis-spelled14:17
ssam2certainly, if they are to be user-defined then they should be defined in definitions.git, not spec.git14:18
ssam2richard_maw: yes, it does mean you could accidentally write 'buld-commands' and the json-schema wouldn't catch that. but the build tool could check that manually14:18
ssam2i guess allowing them to be user-defined is good, my gut feeling is to stick with the set that we have and mandate everyone use those, but i'm probably not thinking big enough here :-)14:19
* richard_maw previously considered the idea for the kernel build-system14:19
richard_mawsince you want to have the selection of the base config and the enabling of customisations to be different steps14:20
richard_mawsince you can then override them differently14:20
richard_mawso a per-device-config step and a baserock-userland-config step14:21
richard_mawsince then you can put the usual configs in DEFAULTS and only have to deal with replacing the per-device ones to fix quirks14:21
radiofreeso you'd just define the defconfig + any additional CONFIG options in your build instructions?14:22
radiofreerest is from a set of baserock defaults?14:22
ssam2that's an interesting idea14:23
richard_mawradiofree: that kind of thing, yeah14:23
radiofreethat would be rather nice14:23
richard_mawit'd certainly be handy to not have a whole bunch of duplicated config in every kernel14:23
ssam2on an unrelated note: :-)14:24
paulsherwoodjjardon: when you say "I've changed DEFAULTS to add a new build step" which commit did that, please?14:28
pedroalvarezooi, is anybody looking at gerritbot?14:42
paulsherwoodjjardon: tvm14:46
paulsherwoodjjardon: please try ybd master?14:54
jjardonpaulsherwood: seems to be working, thanks!15:06
*** cyndis has quit IRC15:24
jjardonhi, my system doesn't want to boot, I tihnk because baserock initramfs-scripts doesnt support a system without  /sbin/init; richard_maw am I rigth?15:25
pedroalvarezwithout /sbin/init? :/15:27
pedroalvarezwhich system is this ooi?15:28
jjardonone without sysv compatibilities (pure systemd)15:28
jjardonalso same problem if init is in /usr/bin/init, for example15:29
pedroalvarezjjardon: you could set a different init= in the kernel command line15:30
pedroalvarezinitramfs will handle that15:30
pedroalvarezso.. in a "pure systemd" system.. what do you have to run to boot the system if it doesn't have /sbin/init?15:31
jjardonpedroalvarez: /usr/lib/systemd15:32
pedroalvarezthen... init=/usr/lib/systemd ?15:32
pedroalvarezhehe, I've no idea about these things tbh15:33
jjardonin out systems, /sbin/init is simply symlink to /usr/lib/systemd15:33
pedroalvareztrue (to '/lib/systemd/systemd' in mine, but maybe that has changed)15:34
*** cyndis has joined #baserock15:34
pedroalvarezthen, what I've just said, should work15:35
pedroalvarezeven with initramfs15:35
jjardonIts still a bug in our initramfs scripts, I think15:37
pedroalvarezour initramfs scripts default to /sbin/init, but it has support to read the 'init' arg from kernel command line15:39
jjardonbecause changing the kernel command line every time I boot or locally patch initramfs is not the correct solution, I think; but I think I can keep going with this, thanks!15:41
pedroalvarezi don't get "changing the kernel command line every time I boot"15:42
pedroalvarezif you set "KERNEL_ARGS: init=/path/to/whatever" in your cluster morphology, should work15:43
jjardonoh, didn't think about that, cheers15:44
*** cyndis has quit IRC15:44
pedroalvarezif you want to try it quickly, just modify extlinux.conf15:46
*** paulw has joined #baserock16:12
*** cyndis has joined #baserock17:05
*** toscalix has quit IRC17:17
*** rjek_ has joined #baserock17:29
*** rjek has quit IRC17:37
*** rjek_ is now known as rjek17:38
benbrown_ssam2: I have no idea how to reply in gerrit in a way that will notify you, will you be CC'd as you have taken part?17:44
pedroalvarezbenbrown_: I think the email will go to anyone in the reviewers list17:46
benbrown_pedroalvarez: cool, ty17:46
pedroalvareznp :)17:46
*** vgrade_w has quit IRC17:53
*** ctbruce has quit IRC17:53
*** ssam2 has quit IRC17:55
*** bashrc has quit IRC18:01
*** jonathanmaw has quit IRC18:03
*** tiagogomes has quit IRC18:21
*** edcragg has quit IRC18:26
*** locallycompact has quit IRC18:29
*** rdale_ct has joined #baserock19:50
*** rdale_ct_ has joined #baserock19:52
*** rdale has quit IRC19:53
*** rdale_ct has quit IRC19:56
*** edcragg has joined #baserock20:48
*** edcragg has quit IRC21:27
*** edcragg has joined #baserock21:31
*** rjek has quit IRC23:39
*** rjek has joined #baserock23:39

Generated by 2.15.3 by Marius Gedminas - find it at!