IRC logs for #baserock for Thursday, 2016-10-06

*** gtristan has joined #baserock03:37
*** gtristan has quit IRC06:06
*** gtristan has joined #baserock06:34
*** fay has joined #baserock07:20
*** fay is now known as Guest591507:20
*** Guest5915 is now known as faybrocklebank07:20
*** ctbruce has joined #baserock07:31
*** rdale has joined #baserock07:38
*** locallycompact has joined #baserock09:26
locallycompactnow with drag drop and zoom
paulsherwoodlocallycompact: cool. while you're on, can you think back to how we handled git submodules before you improved definitions?09:43
paulsherwooddid we ever handle them recursively?09:43
paulsherwoodthere was some conversation here that claimed so, but i can't understand how it would have worked09:43
locallycompact*before* V8?09:44
paulsherwoodor after09:44
paulsherwoodit doesn't work now, that's for sure09:44
pedroalvarezI think I checked ybd code when we moved to v8, and it was recursive09:44
pedroalvarezor at least it looked recursive, and to do the right thing09:44
SotKyes, I'm pretty sure it used to be correctly recursive09:45
tiagogomesMorph used to do them recursively, by parsing .gitmodules09:46
paulsherwoodis there any example of this in our actual definitions? if ybd doesn't do the right thing, i wonder how come it builds ci.morph sueccessfully09:47
pedroalvarezpaulsherwood: it may build it, but fetching the code from a different place (not g.b.o)09:47
locallycompactIs there anything in definitions with two layers of submodules?09:47
SotKpedroalvarez: I think the issue was that the nested submodules weren't even being checked out09:47
pedroalvarezright, someone should try to reproduce it then]09:48
pedroalvarezlocallycompact: there was something.. introduced when integrating openstack09:49
pedroalvarezwill try to find out09:49
paulsherwood is the previous conversation on this09:50
* paulsherwood is curious how SotK could be so clear that was the breaking commit?09:50
locallycompactthis bit surely
SotKlocallycompact: yes09:52
SotKpaulsherwood: by keeping the call to checkout_submodules() in checkout() but making checkout_submodules() call _checkout() instead, checkout_submodules() is only called for the first level09:53
SotK(ie, it isn't called when checking out a submodule, and it should be)09:54
pedroalvarezwill the patch only remove the "_" :P09:55
* paulsherwood has been working on non-code things for so long, he is struggling to grok basic python09:56
SotKI'd move the checkout_submodules stuff into _checkout to replicate the old behaviour09:57
SotKwhere stuff == call09:57
paulsherwoodSotK: ack, thanks. i'm trying to find a 'volunteer' to do the actual work :)10:02
paulsherwoodlocallycompact: so looking at your visualisation, you've now got that base-system contains minimal-system + some other sets/strata?10:03
paulsherwood(shiny, btw)10:04
paulsherwoodso is your schema generic for nesting of these sets? eg foo could contain base-system + other sets10:05
paulsherwooddoes 'right' mean 'yes'?10:05
paulsherwoodgreat, then. and your new proposed name for these generic sets is 'assemblage'?10:06
paulsherwoodand an assemblage could be (say) a system (bootable set) or a stratum... but not a cluster?10:06
locallycompactyes, strata and system no longer have syntactic specifications, a strata is an assemblage that lives in another assemblage, a system is an assemblage that has certain constraints on its lower strata so that it is bootstrappable and also can boot, etc10:08
locallycompactthat's not in the ontology yet, but system and strata schemas are gone10:09
* paulsherwood likes this10:10
paulsherwooddid i hear someone mumble that 'defslib nearly builds' already?10:10
locallycompactDoes, nearly10:12
*** franred has joined #baserock10:12
tiagogomesIf ybd and and defslib are on gitlab, shouldn't sandboxlib be there was well10:13
locallycompactsanboxlib is on gitlab10:13
tiagogomesah ok, does it sync from github sandboxlib?10:14
locallycompactI think it is10:14
leemingit should do10:14
tiagogomesAnyway that is confusing10:14
paulsherwoodit's probably time for some cleaning up. i agree this and other things are confusing10:15
leemingi was only using gitlab for the ci10:15
tiagogomesI am planning to merge if no review happens by tomorrow.10:24
paulsherwoodleeming: ^^10:29
*** franred has quit IRC10:30
tiagogomesI would appreciate code review, and not only checking whether the tests pass or not.10:31
*** franred has joined #baserock10:44
leeminga few small issues, i am seeing if i can clean it up10:49
*** jjardon_matrix has quit IRC10:50
leemingtiagogomes, where do the params for test functions get sourced from?10:55
leemingah nevermind. I was just debugging some of your code... you seem to have the wrong indentation set10:57
*** gtristan has quit IRC10:58
jjardontiagogomes: are you one of the owners of the sandboxlib project? If yes, we can move it to gitlab instead mirroring it11:01
pedroalvarezis defslib going to be another builder? or will ybd sit on top of defslib to do the work?11:03
locallycompactdefslib has a reference implementation for V1011:03
pedroalvarezto parse definitions or to parse and build?11:04
locallycompactit wasn't going to but it needs to be able to clone in order to calculate cache keys11:04
*** jjardon_matrix has joined #baserock11:04
locallycompactand since it's all sorted and flattenable11:04
locallycompactthen building is not that much extra11:04
pedroalvareznot sure if that makes a lot of sense, but ok :)11:06
locallycompactdefslib can sort topologically by build dependency using this
locallycompactso it can do that for the whole assemblage11:07
locallycompactthen it can flatten the assemblage to a list of chunks, filling in the build-dependencies as it flattens11:08
locallycompactthen building is for x in chunks:11:08
locallycompactbecause they are sorted11:08
*** jjardon_matrix has quit IRC11:09
pedroalvarezI didn't expect another build tool, that's all11:10
pedroalvarezbut I welcome it11:10
locallycompactYou can just use the resolver validator and the cache key logic and use your own build logic11:11
locallycompactthat works already11:11
*** franred_ has joined #baserock11:13
*** jjardon_matrix has joined #baserock11:15
leemingso far it is easier to read too, although it is basic atm11:16
*** franred has quit IRC11:17
leemingneeds comments ;)11:17
pedroalvarezother build tools were easy to read when they only had basic functionality11:18
*** ctbruce has quit IRC11:19
*** ctbruce has joined #baserock11:20
tiagogomesjjardon I believe I am one of the owners yes, but I leave that work to someone that is more knowledgeable of what Baserock will use for its infrastructure.11:26
* tiagogomes is unsure if gerrit is still used or not11:26
jjardontiagogomes: sandboxlib is not in baserock infra at the moment ...11:26
jjardonbut sure your choice11:27
*** anahuelamo__ is now known as anahuelamo11:27
pedroalvarezcan any of the infrastructure creators join the infrastructure team?11:28
pedroalvarezand take some decissions and clean up this mess?11:29
jjardonwhat mess?11:29
* SotK wouldn't mind helping out but probably only has useful time at the weekends11:30
*** anahuelamo_ has joined #baserock11:31
pedroalvarezwhat is baserock infra and what not11:31
*** anahuelamo_ has quit IRC11:31
pedroalvarezwhat is baserock using and what not11:31
pedroalvarezin how many different places are we going to have the code, and accept patches11:32
pedroalvarezissue trackers11:32
pedroalvarezalso, what is baserock and what not11:32
jjardonAFAIK, nothing has changed (yet); I was about to send an email to change the canonical location of definition from to but didnt have the time. Maybe now is a good moment11:33
* SotK wouldn't like that11:34
pedroalvarezI wouldn't like that either, but I'm not going to vote11:34
jjardonSotK: why not? we can use the gitlab ci and get what we always wanted: pre-commin ci and a flexible ci system. Some people is already using the mirror in for this reason11:35
pedroalvarezI say it again, I'd appreciate if people wanting to make decisions about the infra, joinded the infra team11:35
SotKjjardon: what happens if ceases to exist for some currently-unforeseen reason?11:37
tiagogomesWho are in the infra team these days? Pedro and Gary?11:37
jjardonSotK: we can setup our own gitlab instance if that concerns you11:37
pedroalvarezyeah, sssam also, but he is not around these days11:37
SotKI thought we care about maintaining git.b.o to ensure that git repositories we care about don't disappear11:37
jjardongit.b.o doesnt have to disappear11:38
pedroalvarezmy question still stands11:38
jjardonpedroalvarez: what is your question?11:38
pedroalvarezwell, not question11:39
pedroalvarez<pedroalvarez> I say it again, I'd appreciate if people wanting to make decisions about the infra, joinded the infra team11:39
franred_jjardon, probably what pedroalvarez wants to say is given that someone has create the gitlab infra, and you want to make it canonical, one of you should be part or the operations team (it that replaces completely the whole infrastructure)11:40
jjardonI do not agree with that11:40
pedroalvarezI just want to get out of the way11:41
jjardonand setup that infra in a couple of minutes in my free time to test it; if people think is cool and useful (as I think it is), I propose to switch; I do not think is needed any bureaucracy joining other teams to do that11:42
jjardonpedroalvarez: nobody is on the way :) I was hopping the people to try it and have their own opinnion before propossing any change11:43
pedroalvarezso, you set it up, and then forget about it11:44
jjardonno at all; I'm using it all the time!11:45
* jjardon will write an email to the mailing list11:45
pedroalvarezusing != maintaining11:46
jjardonpedroalvarez: SotK any other concerns about moving the canonical location of definitions to ?11:46
leemingis this re: sandboxlib?11:46
leemingbecause it has existed on github for a few years now?11:47
pedroalvarezI don't have any concerns, just joing the infra team11:47
jjardonpedroalvarez: No need to mantain a project hosted in gitlab.com11:47
jjardongitlab do that for us11:48
pedroalvarezright, just join to maintain the bits you set up11:48
jjardonor Am I misunderstanding something?11:48
pedroalvarez(other things may happen)11:48
pedroalvarez(lorrying definitions may fail, forexample)11:49
jjardonlorrying defininitions?11:49
SotKif definitions' upstream is it'll need to be lorried into git.b.o is the meaning I think11:50
SotKsomeone should probably update the ybd lorry too incidentally11:51
jjardonSotK: do we actually need to lorry definitions back to g.b.o?11:53
* SotK thinks it is verging on hypocritical not to11:55
SotKsince we maintain a mirror of the various projects in g.b.o to make it more likely that we can reproduce things in the future11:56
SotKand by not mirroring the definitions themselves there (or having them live there) that is pretty pointless11:56
jjardonwell, if needed it will be lorried under /delta, so it should not conflict with the current repo location11:57
SotKbut if that lorry breaks for whatever reason, who investigates and fixes it?11:58
* SotK thinks pedroalvarez's point is that he doesn't have time to do so really11:58
* SotK may be misrepresenting there, and apologises if so11:59
locallycompactThings I still would like:12:51
locallycompactmorph: replaced by $include:12:52
pedroalvarezalso updaging the docstrings :P12:52
pedroalvarezoh no12:52
pedroalvarezit's ok, is v9, not v1012:52
pedroalvarezignore me12:53
locallycompactactually 10 has a typo12:53
pedroalvarezah, I see, 2 versions at the same time12:54
pedroalvarezany reason to not send that patch to upstream?12:54
* pedroalvarez runs away12:54
*** franred_ is now known as franred12:55
*** faybrocklebank has quit IRC13:04
*** fay has joined #baserock13:17
*** fay is now known as faybrocklebank13:18
locallycompacthow do I get actual information back from sandbox lib running a command14:30
locallycompactall I see is14:30
locallycompactcmd: ['sh', '-c', '-e', 'ls'], cwd: /, env: {'TERM': 'dumb', 'PROGRAM_REF': 'my-version', 'DESTDIR': 'stage1-binutils.install', 'DEFINITIONS_REF': 'def-version', 'LC_ALL': 'C', 'USER': 'tomjon', 'TARGET_STAGE1': 'cpu-bootstrap-linux-gnu', 'CCACHE_DIR': '/tmp/ccache', 'LOGNAME': 'tomjon', 'CCACHE_EXTRAFILES': '', 'USERNAME': 'tomjon', 'TZ': 'UTC', 'SHELL': '/bin/sh', 'MAKEFLAGS': '-j 1', 'MORPH_ARCH': 'x86_64'14:30
locallycompact, 'TARGET': 'cpu-baserock-linux-gnu', 'HOME': '/tmp'}, filesystem_root: tmp/tmpx3ohzmj5,filesystem_writable_paths: all, mounts: isolated, extra_mounts: [('tmp/tmpx3ohzmj5/', '', None), ('tmp/tmpx3ohzmj5/stage1-binutils.install', 'stage1-binutils.install', None), ('tmp/tmpx3ohzmj5/baserock', 'baserock', None), ('tmpfs', '/tmp', 'tmpfs'), ('proc', '/proc', 'proc'), ('dev', '/dev', 'dev')],14:30
locallycompactnetwork: isolated, stderr: -1, stdout: -114:30
locallycompacthm wat14:30
locallycompactleeming, what are those extra mounts?14:31
leemingextra as in thing.install,
locallycompactI'm simply perplexed14:39
locallycompactWhat is a mount14:39
leemingwhat are you asking? just how to get data out?14:39
locallycompactI checked out the thing to .build14:39
leemingthe mapping of host directory to sandbox directory14:40
locallycompactso it's there in the filesystem14:40
locallycompactthese are already in the sandbox directory14:40
locallycompactnow I get14:41
locallycompactRuntimeError: ['mount', '-t', 'dev', 'dev', 'tmp/tmpudjfj8lh/dev'] failed: mount: unknown filesystem type 'dev'14:41
leemingare we on the same page here? :\14:41
locallycompactno, clearly14:42
leemingohhh dev14:42
* leeming looks in the code + bwrap docs14:42
locallycompactthat's with chroot14:42
locallycompactwith bubblewrap it is ind of fine14:42
locallycompactbut still stdout: -1 with no information14:43
leemingyou downstairs ?14:43
*** gtristan has joined #baserock15:02
*** toscalix has joined #baserock15:29
locallycompactWhen I run the configure for stage1-binutils inside a sandbox with the host rootfs='/', I get a Makefile out15:50
locallycompactWhen I run it inside the sandbox I get gcc: error trying to exec 'cc1': execvp: No such file or directory15:50
locallycompactWhat's going on15:50
tiagogomesIf in the second time the root of the sandbox is not '/', they no compiler will be installed in the sandbox, hence that error16:13
locallycompactit is '/'16:13
locallycompactIt finds gcc fine16:14
locallycompactconfigure:4246: gcc --version >&516:14
locallycompactgcc (GCC) 6.1.1 2016070716:14
tiagogomesSo what is the difference between the two runs? Is not clear to me what you changed16:14
jmacslocallycompact: Does it find cc1?16:14
locallycompactthat's unclear16:15
locallycompacttiagogomes, sorry I misspoke16:15
locallycompactthe first one was on the host16:15
locallycompactthe second one was in rootfs='#/'16:15
*** ctbruce has quit IRC16:16
tiagogomesThat makes more sense. Are /usr or /lib{,64} in your host are in the same partition as '/'16:17
tiagogomesAnd as jmac said, your host can run cc1?16:17
locallycompactcc1 not on my path16:18
locallycompactI think16:18
locallycompactthe host completes the configure though16:18
jmacsThere needs to be a cc1 executable somewhere and your gcc executable needs to know where it is16:19
jmacsIt's possible you have a bad gcc executable16:19
jmacsBut without know how sandbox works I couldn't guess how you got there16:20
*** toscalix has quit IRC17:01
*** locallycompact has quit IRC17:23
*** ctbruce has joined #baserock17:28
JasonASotk: I see you were wondering my progress.  Sorry, its been busy around here.  I have not quite gotten the submodule fix working, and I have had to move on for now.  I will revisit once I get a chance in the next week or so17:53
*** gtristan has quit IRC19:19
SotKJasonA: no need to apologise, I was just wondering if my suggestion had worked :)21:36
SotKon an unrelated note: people who want yaml lorries may be interested in
*** ctbruce has quit IRC21:37
*** rdale has quit IRC22:03

Generated by 2.14.0 by Marius Gedminas - find it at!