IRC logs for #baserock for Wednesday, 2016-01-13

*** gtristan has quit IRC02:10
*** edcragg has quit IRC02:17
*** gtristan has joined #baserock02:45
*** radiofree has quit IRC04:45
*** gtristan has quit IRC07:06
*** gtristan has joined #baserock07:53
pedroalvarezI will try that08:02
pedroalvarezthough, they are not merged right?08:02
*** radiofree has joined #baserock08:03
gtristanAny idea why this line: https://github.com/CodethinkLabs/sandboxlib/blob/master/sandboxlib/chroot.py#L234 should cause:08:05
gtristanLookupError: unknown encoding: string-escape08:05
gtristan?08:05
gtristanit would seem someone is trying to decode the stderr/stdout coming from a subprocess through a pipe, instead of just printing it raw, but I'm not seeing where08:06
*** ssam2 has joined #baserock08:26
*** ChanServ sets mode: +v ssam208:26
*** toscalix has joined #baserock08:31
*** ssam2 has quit IRC08:33
*** ssam2 has joined #baserock08:33
*** ChanServ sets mode: +v ssam208:33
paulsherwoodssam2: ^^08:43
*** jonathanmaw has joined #baserock08:45
paulsherwoodgtristan: so i'm guessing your issue is an uncaught exception?08:47
*** bruce_ has joined #baserock08:48
gtristanpaulsherwood, not really... that helped btw, catching the exceptions help08:49
paulsherwoodcool08:49
paulsherwoodideally ybd shouldn't eat them anyway... but the retry logic is relying on exceptions. it needs a re-write08:50
gtristanmy issue is the chroot environment I'm trying to setup doesnt work, and I have no way to tell what went wrong if I reach it through ybd08:50
paulsherwoodack, so we still need the interactive option08:51
gtristanso, right now I'm trying to do the chroot entirely ybdless, I have a sort of hacky branch which tries to setup the environment 'just right', but I'll try to get it right first without that08:51
gtristanI'm not sure the interactive option is feasible for sandboxlib itself, it requires knowledge of the setup08:51
gtristani.e. you have to know where the shell is installed08:51
paulsherwoodnot if we could put the shell command in definitions?08:52
gtristanright now, chroot /sandbox /path/to/sh... reports "No such file or directory", even though I know I have the mounts setup, so that is odd08:52
gtristanputting the shell command in definitions would only technically let you run the shell once you get to the point where the shell is built, i.e. you are running the shell in the context of a package which already depends on the shell being there08:53
paulsherwoodtrue08:54
paulsherwoodiirc richard_maw calls this the 'crowbar inside the box problem'08:54
gtristanwhat I am trying on the other hand... is to set it up so that the base aboriginal system is fully mounted under '/base' in the given sandbox, and setting up environment variables to get the host tools to run08:55
paulsherwoodgood luck -)08:56
gtristanit should work08:56
gtristanthen you get a '/' populated only with artifacts, and pull some extra goodies out of '/base'...08:57
*** jonathanmaw has quit IRC08:59
*** jonathanmaw has joined #baserock08:59
tiagogomes__gtristan if there isn't yet a shell in the staging area, we are talking about chunks in build essential, and those don't use a chroot to be built as they need host tools09:00
*** jonathanmaw_ has joined #baserock09:00
paulsherwoodgood point, tiagogomes__ :)09:01
gtristantiagogomes__, yeah, but I'm trying to ditch that part at least for this exercise09:01
gtristanmaybe I should build those first, but it seems pointless, they have to be built anyway on the image, all the way up to python and git at least09:02
gtristanwell, maybe the image doesnt need git technically09:02
gtristandepending on how you slice it, you could slave the image to ybd on the outside, and have ybd prepare the checkouts and stage the artifacts outside of the build VM09:03
*** jonathanmaw has quit IRC09:04
*** rdale has joined #baserock09:04
*** bashrc_ has joined #baserock09:06
*** bashrc_ has quit IRC09:10
*** bashrc_ has joined #baserock09:10
*** CTtpollard has quit IRC09:10
tiagogomes__Ansible 2.0 was released yesterday.09:15
*** CTtpollard has joined #baserock09:16
*** bashrc__ has joined #baserock09:17
*** bashrc_ has quit IRC09:18
*** bashrc__ has quit IRC09:18
*** bashrc_ has joined #baserock09:18
ssam2hooray! much nicer tools for working with OpenStack in Ansible 2.009:18
*** jonathanmaw_ is now known as jonathanmaw09:32
gtristanah, progress... the shell is at least working :D09:46
paulsherwoodw00t! :)09:47
*** gtristan has quit IRC09:52
*** gtristan has joined #baserock10:14
*** edcragg has joined #baserock10:16
*** Lachlan1975 has joined #baserock10:23
pedroalvarezssam2: finally released? yay!10:24
*** jonathanmaw has quit IRC10:39
*** jonathanmaw has joined #baserock10:40
*** gtristan has quit IRC11:09
*** jonathanmaw has quit IRC11:14
*** jonathanmaw has joined #baserock11:14
*** jonathanmaw_ has joined #baserock11:14
*** jonathanmaw has quit IRC11:18
*** gtristan has joined #baserock11:20
*** jonathanmaw_ is now known as jonathanmaw11:25
paulsherwoodrdale: i see elsewhere you now understand how ybd assembly works :)11:28
rdalei like to think i do at least11:28
paulsherwoodunfortunately i think some stratum-specific and system-specific logic has crept in. ideally things should be generic11:29
paulsherwoodin my imaginary world, component could have component-specific processing, whether component is chunk, stratum, system, cluster11:30
rdaleyes, in my stand up notes i made a point about nowster's patch only special casing build-depends for a 'system' when i think it should be generalised11:30
rdaleyes, all of those things a part of the same recursive call tree to do a build afaik11:30
paulsherwoodi had a syntax for this, but it will require migrating definitions and i wanted to hold off on that until ybd is functionally sufficient, deterministic and bulletproof :)11:31
rdalei agree with 'contents' being a better name for a list of sub components11:31
rdalehmm11:31
paulsherwoodrdale: if you can fix the splitting stuff, i believe that would allow us to tick the functionally sufficient box11:32
rdaleyes, i think i can but you need to get the overall picture before it seems 'simple' which has taken me a few days and it still doesn't quite seem 'simple' yet11:33
paulsherwoodi agree... this is not simple11:34
rdalealso to verify the you've done the right thing is quite hard - it could be tests, or it could be a better visualation of what is going on11:34
paulsherwoodack11:36
*** CTtpollard has quit IRC11:39
*** CTtpollard has joined #baserock11:39
rdalei had to wait some time before the 'light bulb' moment and it happened yesterday, and now i wait for the 'hacking frenzy' moment11:39
paulsherwoodlol11:42
paulsherwoodrdale: to verify you've done the right thing, can't you just run against something in definitions? anything from stage1-binutils to ci.morph should work11:45
paulsherwoodi agree that some tests would be beneficial, though11:46
paulsherwood(and a bit more documentation)11:46
rdalei'm thinking of 'fake sandbox' tests where a set of files come in .build, and a set of files come out in .inst without really building anything11:50
paulsherwoodah yes... that would be useful11:51
*** jonathanmaw has quit IRC11:51
*** jonathanmaw has joined #baserock11:53
radiofreehmm.. any reason why ybd would stop on the deploy cluster stage after checking cache keys?13:21
radiofreei don't get any error13:22
radiofree(and it worked yesterday)13:22
radiofreenope never mind... the architecture is *not* armv7lhfr.....13:23
*** ssam2 has quit IRC13:53
*** ssam2 has joined #baserock13:59
*** ChanServ sets mode: +v ssam213:59
pedroalvarezerroring in that situation might be wise :)14:15
*** tiagogomes has joined #baserock15:06
*** tiagogomes__ has quit IRC15:06
* paulsherwood wonders if rdale is frenzied yet16:15
rdaleno, not quite - i'm still thinking it through. i written a couple of functions to dump the definitions after they've been loaded, which is useful16:17
paulsherwoodack16:18
rdalehttp://paste.baserock.org/udokehobit16:19
paulsherwoodcool16:20
rdalei don't like file paths in the definitions, and i agree with turning them into names16:22
paulsherwoodi went down that road initially, but it breaks because definitions has always had non-unique names16:23
rdaleso you don't use the file paths converted to names?16:24
paulsherwoodoh, yes, ybd does. it's keyed on that16:24
rdaleright, that was what i was agreeing with then16:24
paulsherwoodack16:25
paulsherwoodexplanation is in _fix_keys... "        Some definitions have a 'morph' field which is a relative path. Others16:25
paulsherwood        only have a 'name' field, which has no directory part. A few do not16:25
paulsherwood        have a 'name'16:25
paulsherwood        This sets our key to be 'path', and fixes any missed 'name' to be16:25
paulsherwood        the same as 'path' but replacing '/' by '-'16:25
paulsherwood"16:25
rdaleright, having file paths makes it hard to serialize into another format16:25
rdalethere seem to be two conditions to me: 1. artifact is not built yet, in which case use build-depends, 2. artifact is built, ignore build-depends and apply splitting rules16:27
rdaleas far as i can see if you apply those rules consistently at every level in heirarchy of components, it should be correct16:29
paulsherwoodrdale: but iiuc the only time we need to split is deploy time, ie this is processing on a built artifact?16:29
rdalei think the file system in the artifact at the top level should be correct before deploy time16:31
*** toscalix has quit IRC16:31
rdaleie the splitting rules and build-depends rules should have all be applied according to my 2 cases16:31
rdaleotherwise you need to recurse through the whole thing all over again at deploy time, which seems unecessary to me16:36
paulsherwoodi think it's not unnecessary, but i may be wrong16:37
paulsherwoodi'd prefer the spltting to be done at system creation time or artifact deploy time, not at build time16:38
rdaleyes, i'm saying it shouldn't be done at build time, only the generation of the yaml in the .meta files with split file lists as per nowster's patch16:39
paulsherwoodok, i think i should shut up and let you hack in a frenzy :)16:40
tiagogomesI really prefer using file paths16:40
tiagogomesBut that's an old argument16:41
rdaleyes16:41
paulsherwoodtiagogomes: in effect ybd is using filepaths... but converting '/' to '-' because it gets used in the cache artifact name16:42
tiagogomespaulsherwood, why to you need the converted filepath in the artifact name?16:43
pedroalvarezif they are names, then the tool has to load them all before starting16:45
paulsherwoodtiagogomes: because humans sometimes look at artifact caches16:45
paulsherwoodmorph folks and i disagreed about this - morph sticks the name at the end, ybd at the front. i personally find ybd's naming standard easier to grok/debug16:47
pedroalvarezbut even with the name at the front you still need to know the cache key16:48
tiagogomesProbably putting the name on the front would be better. But I don't know why still want the artifact file path there. It seems wrong to me that an artifact depends on the location of its source in definitions16:49
rdaleisn't that what is wrong with using file paths in the first place?16:50
tiagogomesHere we go again...16:52
paulsherwoodtiagogomes: because some definitions don't *have* names, for example.16:52
paulsherwoodbut yes, an old discussion16:52
*** CTtpollard has quit IRC17:10
*** jonathanmaw has quit IRC17:21
radiofreeoooh morphs progress bars are lovely17:23
radiofreepartitioning support worked for jetson (after a few attempts), will test it didn't break x86 and submit17:24
richard_mawradiofree: you can thank ripsum (whatever he calls himself now) for that17:24
rjekrichard_maw: I think he calls himself Richard.17:24
*** ssam2 has quit IRC17:41
*** bashrc_ has quit IRC18:03
*** jjardon has quit IRC18:08
*** lachlanmackenzie has quit IRC18:08
*** jjardon has joined #baserock18:23
*** lachlanmackenzie has joined #baserock18:27
*** bruce_ has quit IRC18:29
*** rdale has quit IRC18:33
*** edcragg has quit IRC18:40
*** edcragg has joined #baserock19:10
*** Lachlan1975 has quit IRC19:48
*** edcragg has quit IRC19:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!