IRC logs for #baserock for Tuesday, 2016-01-12

*** gtristan has quit IRC05:26
*** gtristan has joined #baserock05:56
*** paulw has joined #baserock08:23
*** edcragg has joined #baserock08:30
*** ssam2 has joined #baserock08:36
*** ChanServ sets mode: +v ssam208:36
*** toscalix has joined #baserock08:39
*** bruce_ has joined #baserock08:42
*** rdale has joined #baserock08:45
*** edcragg has quit IRC08:55
*** bashrc has joined #baserock09:09
gtristanpaulsherwood, I see that recently sandbox.py 'install' was patched to report an error if cache.get_cache() returns False, instead of trying to build a path with the boolean False return... any clues as to why this would happen, though ?09:17
* gtristan recalls working around this problem weeks ago, and now is hitting the error but having a hard time figuring out why it's happening09:18
paulsherwooderk :/09:18
paulsherwoodi was looking at this last night09:19
paulsherwoodfor example https://github.com/devcurmudgeon/ybd/issues/14909:19
paulsherwoodi don't know what's happening there tbh09:20
paulsherwoodgtristan: could you try with the previous release, please?09:21
paulsherwood(but i fear that it will do the can't concatenate bool + str fail)09:21
gtristanI was trying with 15.49 which blindly builds a path and causes an error, and then upgraded to the "nice" error :)09:22
paulsherwoodah, ok09:22
paulsherwoodcould you paste your log?09:24
paulsherwoodgtristan: i assume you're root?09:26
*** CTtpollard has quit IRC09:26
gtristanhttp://paste.baserock.org/somesofeci09:27
gtristanproblem is, this is pretty experimental territory :-/09:27
*** CTtpollard has joined #baserock09:28
paulsherwoodack.09:28
gtristanI built stage1-binutils alright and it created an artifact, then; I gutted build-essential to remove all the stage1/stage2 cross stuff, and am trying to hardwire the build env to use the installed distcc on aboriginal09:28
* gtristan runs off for feeding, will return shortly...09:32
* paulsherwood is breakfasting... this is not improving the taste of my food :-)09:32
*** faybrocklebank has quit IRC09:36
*** fay_ has joined #baserock09:36
*** fay_ is now known as faybrocklebank09:37
*** gtristan has quit IRC09:37
*** CTtpollard has quit IRC09:37
*** edcragg has joined #baserock09:39
*** CTtpollard has joined #baserock09:40
*** jonathanmaw has joined #baserock09:46
*** rdale has quit IRC09:53
*** rdale has joined #baserock09:54
pedroalvarezradiofree: well... it looks like gdp-hmi dididn't work before upgrading the components... sorry for the noise with this...09:59
pedroalvarez(in x86, btw)10:00
pedroalvarezit might work in arm, I dont know10:00
radiofreehmm10:00
radiofreeah!10:00
radiofreemaybe...10:00
radiofreehow is weston started?10:00
pedroalvarezExecStart=/usr/bin/weston-launch -u root -- --log=/tmp/weston.log --backend="fbdev-backend.so"10:01
radiofreeah, so still with the fbdev backend10:01
radiofreedoes stock desktop-shell work?10:01
radiofreelaunch weston with --no-config (i think, check man weston)10:01
radiofreethen try launch weston-simple-egl10:01
*** CTtpollard has quit IRC10:02
radiofreethere were some upgrades to mesa recently that allowed you to use the drm backend in qemu, potentially could have caused issues with the fbdev backend (though i'm not sure why...)10:02
*** CTtpollard has joined #baserock10:06
radiofreei could try it on a jetson if you want?10:06
*** franred has quit IRC10:08
*** CTtpollard has quit IRC10:09
*** gtristan has joined #baserock10:10
*** CTtpollard has joined #baserock10:13
*** franred has joined #baserock10:13
*** franred_ has joined #baserock10:14
radiofreehmm linux32 stopped working on this mustang after the kernel upgrade10:14
*** franred_ has quit IRC10:15
*** nowster has joined #baserock10:21
paulsherwoodgtristan: i'm a bit stumped on this get_cache returning false thing... i can't reproduce it10:29
*** Lachlan1975 has joined #baserock10:29
gtristanyeah I figured you cant reproduce :-/10:35
gtristanok so how do you get a shell in the build staging area instead of running the build ?10:49
tiagogomes__morph creates a script that allows you to enter a chroot with the build environment for some chunk. setup. ybd could provide the same functionality10:52
* gtristan wonders if it's as easy as invoking 'bash' somewhere... but suspects there is some redirection going on which will prevent that from working10:54
paulsherwoodgtristan: you could exit 1, then chroot into the staging area ?10:55
tiagogomes__that doesn't setup the build environment (e.g. environment variables)10:56
radiofreelinux32 issue was due to it prefering /bin/linux32 (busybox) over /usr/bin/linux32 (util-linux) for some reason10:56
tiagogomes__doesn't the one that gets chosen depends on $PATH?10:57
gtristanright, I'm mostly concerned with what the mounts are and what the env vars are10:57
gtristani.e. the real build env it would have ended up with10:57
paulsherwoodack. the build env is recorded in the log, maybe you could copy that and source it?10:58
gtristanpaulsherwood, iirc ybd builds stuff in /tmp/... is there a way to make it happen in all in ${base}/tmp ?10:58
paulsherwoodgtristan: it should all happen in ${base}/tmp10:59
gtristanhmm10:59
tiagogomes__chroot script example for morph: http://paste.baserock.org/uquyotinuy10:59
gtristanthat shouldnt be a problem then (I think my /tmp is not writable)10:59
paulsherwoodtiagogomes__: tvm10:59
paulsherwoodgtristan: your log  says '/home/ybd/tmp is directory for tmp'11:00
gtristanyeah, it is11:00
gtristanso as long as it really builds in /home/ybd/tmp, and not in /tmp, it shouldnt be a problem11:01
paulsherwood:)11:01
paulsherwoodgtristan: you could set log-verbose: True, maybe that will give a bit of extra info11:01
gtristangood hint11:05
*** tiagogomes__ has quit IRC11:05
paulsherwoodi'm clutching at straws, tbh :/11:05
*** edcragg has quit IRC11:07
gtristanhonestly I know the env vars which are are printed and which ybd says its setting up, and I see the code which is setting them up... but when you dont trust it... would be great to have a real shell that ybd setup for a build to know for sure whats up in there11:07
paulsherwoodi agree11:09
*** edcragg has joined #baserock11:09
gtristanI think what is happening, is that 'make' is not found in that shell11:09
gtristannot sure why that would result in the get_cache() failure though :-/11:10
* paulsherwood wonders how a python program can throw its user into a shell11:11
gtristanjhbuild does it11:11
radiofreewouldn't you just call the same command you use to do the build, but instead of "cd foo.build/ && make" (or whatever) you just run /bin/sh ?11:12
gtristanhttps://git.gnome.org/browse/jhbuild/tree/jhbuild/commands/base.py#n40911:12
radiofreethe morph log used to give you the linux-user-chroot command that failed, you could just copy that and run sh11:12
gtristanradiofree, right I think so11:12
radiofreebut yes "foo failed, would you like to enter this and prod about?" would be great11:13
gtristanradiofree, the only thing I wonder is with all the redirection and mounting and chrooting, if that shell will be accessible to the one who invoked ybd ;-)11:13
radiofree(and is great in jhbuild)11:13
gtristanok this is weird11:19
*** tiagogomes__ has joined #baserock11:20
radiofreepaulsherwood: todays ybd error11:21
gtristanah, I missed another possible False return... lets see11:21
gtristanin any case, the cache_key() is producing something11:21
radiofreehttp://fpaste.org/309737/52597729/11:22
radiofree"sh -c 'python setup.py build'11:22
radiofreeclone: Invalid argument"11:22
*** paulw has quit IRC11:24
paulsherwoodradiofree: that looks like something in the chunk?11:25
paulsherwood(but i may be wrong... this is not my finest hour)11:25
*** jonathanmaw has quit IRC11:26
radiofreehmm11:27
*** jonathanmaw has joined #baserock11:27
radiofreeisn't that "clone" error usually related to chroot/container issues?11:28
radiofreeit's actually happening on everything11:30
paulsherwoodi can't remember seeing 'clone' mentioned before11:31
radiofreethis is in a armv7lhf chroot on a aarch64 dev board11:31
radiofreeafter upgrading to 4.411:31
paulsherwoodwith latest ybd?11:31
radiofreeyes11:31
paulsherwoodradiofree: building what? i can try the same on on a pre 4.4 box11:33
radiofreebuilding anything11:34
radiofreethis time it's perl11:34
radiofreei'll switch back to 4.1 and see if it goes away11:34
radiofreei wonder if it has anything to do with me logging in via terminal11:37
radiofreei've always done it over ssh previously11:37
radiofreethe environment is different when i login via the serial11:37
radiofreemy chroot script would have *never* worked via serial... so i guess i've never done it like this11:37
radiofreehowever things seem to be working here11:38
paulsherwoodssam2: i wonder if it would be possible/sensible to have sandboxlib support opening up a shell?11:42
paulsherwood(for the usecase gtristan described above)11:42
ssam2paulsherwood: if the chroot contains a shell already, then that is super easy11:43
ssam2paulsherwood: just run it11:43
ssam2paulsherwood: if it doesn't, then it's quite hard because we'd have to somehow inject a shell into the sandbox that worked11:43
*** jonathanmaw has quit IRC11:44
ssam2paulsherwood: but ybd could simply try to run '/bin/sh' in the sandbox if something fails to build.. would be worth trying.11:44
*** jonathanmaw has joined #baserock11:44
radiofreepaulsherwood: https://irclogs.baserock.org/%23baserock.2014-10-30.log.html#t2014-10-30T14:15:1511:45
radiofreewould appear to be (of course) my fault11:46
gtristanok so where is the artifact directory *supposed* to be created ?11:54
gtristannot in assembly.py:setup() ?11:54
gtristanerr sandbox.py11:54
radiofreepaulsherwood: ok, fixed, CONFIG_NET_NS is absolutely essential11:57
radiofreeit wasn't a bug in ybd anyway11:57
paulsherwoodradiofree: yay! :)11:57
paulsherwoodgtristan: https://github.com/devcurmudgeon/ybd/blob/master/ybd/app.py#L132 creates the top-level directories, including the artifacts directory12:00
paulsherwoodbut are you asking about the sandboxes?12:00
gtristanpaulsherwood, yeah, I'm asking where does the $base/$cachekey directory get created12:01
paulsherwoodssam2: i tried that... it sits there silently12:01
gtristanor, I suspect it is moved from $base/tmp/$sandboxdir after a build ?12:01
gtristanerr, where $base/artifacts/$cachekey is created I mean, of course12:01
gtristanthat's the one which doesnt exist when get_cache() is called12:02
paulsherwoodhttps://github.com/devcurmudgeon/ybd/blob/f00d7c963c33eb63649d31ef5776ce0ee643e75e/ybd/cache.py#L13412:03
paulsherwoodthis is a bit convoluted, because it needs to be atomic and multi-instance12:03
paulsherwoodso the actual cache artifact is $base/artifacts/$cachekey/$cachekey, which ends up in place when we rename $base/tmp/$random to $base/artifacts/$cachekey12:05
gtristanso after a build, we rename $base/tmp/$random to $base/artifacts/$cachekey ?12:07
paulsherwood(and by this point we've also untarred the artifact, so if the tar was corrupted, no artifact arrives)12:07
gtristanbut that is not the referenced line right ? cause that is where we unpack a tarbal12:07
paulsherwoodthat referenced line is where the rename happens12:07
gtristanok, so we build, then we tar, then we untar and rename12:08
paulsherwoodso that's the point when the artifac springs into existence12:08
paulsherwoodcorrect12:08
gtristanso cache.py:cache() must be called at some point before cache.py:get_cache()12:09
paulsherwoodwell, n artifact can be created by cache.py:cache() or by fetching from artifact cache ... get_remote()12:10
paulsherwoodin the get_remote() case cache() would not be called, iirc12:10
gtristanright, assuming nothing exists yet of course :)12:11
paulsherwoodyup12:11
paulsherwoodmy current hunch is that an exception is happening, but it's being eaten12:12
gtristanyeah12:12
radiofreewhat exactly does the "[135/206/375]" in a ybd tell me?12:15
radiofreethere's 375 things to build, of which 206 have been built/there's a cache12:15
radiofreeso 169 things need building? and 135 of those 169 are complete?12:15
pedroalvarez375 components in total, 206 you have to actually build, and you are building the number 13512:15
radiofreeah12:15
pedroalvarez(IIRC)12:16
paulsherwooddoing job 135 of 206 jobs this run, of total 375 jobs required to build foo12:16
radiofreeyes that makes much more sense, thanks12:16
ssam2paulsherwood: try passing stdout=sys.stdout and stderr=sys.stderr  to sandboxlib.run_sandbox() when running the shell12:17
ssam2hmm.. probably needs a way to forward stdin too12:18
ssam2i guess it's not so simple12:18
paulsherwoodyup12:18
ssam2this is ugly in a way because some sandboxing methods might not make this possible at all12:18
ssam2well, not simple anyway12:18
paulsherwoodgtristan: pls could you try http://paste.baserock.org/senijayine12:18
ssam2e.g. if you were using a lightweight VM for sandboxing, or something12:18
paulsherwoodssam2: yup, but not as ugly as ybd reimplementing the call itself? :)12:19
gtristanpaulsherwood, ok... fwiw, in my case, it seems cache() is not getting called, but get_cache() is... and I expect the build failed, but have no indication that it did12:19
paulsherwoodgtristan: ack. i'm hoping that tweak to claim() may give a hint12:20
ssam2paulsherwood: true.. it would be better for sandboxlib to expose a 'run_sandboxed_interactive()' function that just might not work in all cases12:22
gtristanI think/suspect a build command is failing, i.e. ybd should get an error trying to execute 'make', which I suspect is not found12:23
gtristanhaha: NameError: global name 'true' is not defined12:24
gtristanI guess maybe you mean True12:24
paulsherwoodyup12:25
paulsherwoodbut ybd  normally will spot build errors and report them properly, i think?12:26
gtristanI *think* so too12:33
gtristanit's possible it doesnt even have a shell though12:33
gtristanso it might for instance... blindly launch the build commands in the inexistent shell and then expect to read the error back with something special12:34
gtristanno difference with that patch fwiw :-/12:34
gtristanoddly it removes the work directory it created12:35
* gtristan tries stopping the thing while its working12:35
gtristanpaulsherwood, where does executor get assigned in sandbox.py ?12:37
paulsherwoodhttps://github.com/devcurmudgeon/ybd/blob/master/ybd/sandbox.py#L9112:38
gtristanthat makes it assigned to something ?12:39
gtristanoh12:39
gtristanok better question; what is it ?12:39
paulsherwoodhttps://github.com/devcurmudgeon/ybd/blob/92727a5770592b433a9081af8d99e8883444fba5/ybd/__main__.py#L5312:40
gtristanah12:40
paulsherwoodsandboxlib decides wther it's linux-user-chroot or chroot12:40
paulsherwoodgtristan: i assume no l-u-c on your system?12:41
gtristanno12:42
gtristanno l-u-c12:42
* gtristan searches in https://github.com/collective/sandboxlib12:42
gtristanbut not sure that's the right place12:42
paulsherwoodhttps://github.com/codethinklabs/sandboxlib12:43
gtristanah yeah that's where I installed it from12:43
gtristanok so, I'll look deeper into this... looks like around here: https://github.com/CodethinkLabs/sandboxlib/blob/master/sandboxlib/chroot.py#L203 ... it's questionable whether the exception is reported correctly in the case that no shell is available to run the commands in the chroot12:48
gtristanwell, not quite sure, I'll have to debug around there anyway12:48
radiofreewould anyone mind me (on x86) i) moving extlinux.conf to /extlinux/ rather than / or ii) creating a symlink /extlinux.conf -> /extlinux/extlinux.conf if 1 doesn't work12:58
radiofreei'm trying to work about introducing even more configuration options (namely) EXTLINUX_CONFIG_PATH12:58
pedroalvarezI would be worried if upgrades stop working, but no, I don't mind13:01
radiofreei) doesn't work13:02
radiofreesyslinux obviously expects it to be in /13:02
radiofreeii works13:03
*** rdale has quit IRC13:15
*** gtristan has quit IRC13:26
*** rdale has joined #baserock13:28
*** gtristan has joined #baserock13:42
radiofreeoh no!13:48
radiofreessam2: "sam-jetson-mason" was that the one with known file system corruption?13:48
radiofree[87386.749943] systemd-journald[156]: Failed to write entry (12 items, 309 bytes), ignoring: Read-only file system13:49
rdaleis baserock-dev the 'official' mailing list for discussing the evolving ybd design - or is there another one?13:51
paulsherwoodrdale: please use baserock-dev, yes13:51
rdaleok. the reasone i ask is that i've just found out that it only uses a single sandbox temp dir for building everything, instead of a different one for every chunk. i don't know when that changed13:52
rdaleit means that artifact splitting can't possibly be done at build time and has to be done at system assembly time13:57
paulsherwoodrdale: that's not true?14:04
rdaleah ok14:05
paulsherwoodeach chunk gets its own sandboxdir14:05
paulsherwoodbut splitting *should* be done at system assembly time, imo14:05
rdaleright14:05
rdalesorry, your right about the sandbox paths - i'm logging the destpath in the _process_tree() function and initially it seemed to be always the same tmp sandbox, but now it isn't14:07
rdalei need to go back and analyse my debugging log to see whats happening. what i don't understand is how sandbox.install() is pulling in the build dependencies - it seems to only copy the files for the current chunk. in previous versions it iterated thru build-depends and content copying their files14:09
ssam2radiofree: yeah sam-jetson-mason goes read-only every few days14:20
ssam2perhaps because i resized the root btrfs partition too much. i got greedy!14:21
radiofreeyes i think that's the cause of that14:21
radiofreei'll have to do a guide on how to move / to the ssd so you have more space for upgrades14:22
radiofreeit's pretty easy14:22
radiofreei think it's pretty easy anyway....14:22
paulsherwoodrdale: the loop is in assembly... assembly.py:preinstall14:28
paulsherwoodeg https://github.com/devcurmudgeon/ybd/blob/master/ybd/assembly.py#L8414:28
ssam2radiofree: (3) on http://wiki.baserock.org/guides/baserock-jetson/ mentions this already14:30
rdaleok, thanks i've got it. i said in my stand up notes this morning that i was waiting for a 'lightbulb' moment, and i think it just happened14:30
paulsherwood:)14:33
*** bashrc has quit IRC14:41
*** bashrc has joined #baserock14:43
*** bashrc has quit IRC15:01
*** bashrc has joined #baserock15:06
radiofreewhat exactly is "menu.c32"?15:11
radiofreeoh ok, it is needed, for the menu that pops up in syslinux when there's > 1 systems to select15:12
* pedroalvarez nods15:19
radiofreeget_partition_by_mountpoint() isn't a very useful function if the mountpoint doesn't actually exist, since it just returns any old thing15:48
radiofreeactually it's probably me doing something wrong15:52
radiofreeno, it's the function that is wrong16:03
radiofreehttp://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.git/tree/extensions/pyfdisk.py#n46416:03
radiofreeonly will ever return "/", i've fixed this16:03
edcraggradiofree: iirc that is supposed to return a Partition class16:07
radiofreeedcragg: if i call it with "/boot" it will only ever return "/"16:12
radiofreewhich kind of defeats the purpose of having a function that returns by mountpoint?16:12
radiofreemy patch was http://fpaste.org/309862/45261518/16:13
*** gtristan has quit IRC16:18
*** gtristan has joined #baserock16:28
franredradiofree, gerrit?16:30
edcraggradiofree: yep, that works... thanks :)16:30
radiofreefranred: i'll post it with the jetson/potentially any board with u-boot sysboot support patches16:31
franredradiofree, cool, thanks :)16:32
*** bruce_ has quit IRC16:32
*** gtristan has quit IRC17:13
*** ssam2 has quit IRC17:24
*** edcragg has quit IRC17:31
*** gtristan has joined #baserock17:34
*** jonathanmaw has quit IRC18:04
*** bashrc has quit IRC18:05
pedroalvarezradiofree: so, yeah, GDP is broken in x86, so my changes weren't causing any error..18:28
*** Lachlan1975 has quit IRC18:29
pedroalvarezThe changes gdp-hmi changes are enough to make it compile, and show something, but something is wrong. The launcher doesn't take the whole screen, and then the apps don't take the whole launcher... Idk..18:29
*** rdale has quit IRC18:30
radiofreepedroalvarez: try launch weston with the drm backend18:43
radiofreeif you have javiers kernel patches + mesa updates that should work in qemu now18:43
*** toscalix has quit IRC20:45
*** edcragg has joined #baserock21:20

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!