IRC logs for #baserock for Friday, 2016-10-14

*** toscalix has joined #baserock05:24
*** toscalix has quit IRC05:39
jjardon_matrixYeah, " they eat their own food"05:53
*** gtristan has joined #baserock06:34
*** ctbruce has joined #baserock07:25
jjardonHi, any idea why deployment start failing? https://gitlab.com/baserock/definitions/builds/5083556 maybe some misconfiguration in the python PATHS? the strange thing is that the build step work with no problem08:06
gtristanjjardon, am I to understand that the extension is being run *inside* the sandbox ? is that a new behavior ?08:16
gtristanA wild guess, is that yaml is not installed in the sandbox where some python code is expected to run08:17
jjardonMmmm, no changes in ybd I think, but we are now running the install_dependencies script from ybd in the ci, instead install the steps manually08:19
*** CTtpollard has joined #baserock08:22
*** tiagogomes has joined #baserock08:23
gtristanodd indeed, clearly yaml was there to get up to that point08:24
gtristanjjardon, ok so it's running inside the sandbox, clearly08:28
gtristanjjardon, ybd deployment.py is running the install-files configure extension in a sandbox, that sandbox must then have a python environment with yaml installed for that to work08:28
gtristannot sure where whatever changed to cause that, but that seems to be happening08:29
gtristaninstall-files.configure imports writeexts which ends up indirectly importing yaml, and that all happens in the sandbox08:30
gtristanif that is really happening, which is quite seems to be, then that is quite wrong imo08:30
gtristanlooking at the ybd/deployment.py and extensions/install-files.configure, it looks like it was this way forever08:33
gtristanjjardon, so indeed some pathing thing seems to be the problem, and the requirement is that: If thou useth baserock to deploy any system using extensions, thou must deployeth thine python in there as well08:34
* gtristan believes the proper and expensive fix to end all of these problems is to never depend on a python environment in the sandbox (probably by using virtualenv) and to have extensions run outside of the sandbox as python scripts which are allowed to link to defslib and use all the same ybd tooling to do things inside the same sandbox08:37
tiagogomesI am not sure what is the problem where. Why can't we run deployments the same way the build essential chunks are built?08:47
gtristantiagogomes, you mean with build-mode: bootstrap ?08:58
gtristanthose parts of build essential ?08:59
gtristanoh wait08:59
gtristanas I return, I see that I can be reading that code completely wrong sorry09:00
*** edcragg_ has quit IRC09:00
gtristanit's not sandbox.run_in_sandbox(), rather it's sandbox.run_extension()09:00
gtristansandbox.run_extension() does not do any chroot09:02
tiagogomesI was talking about the fact that when build-mode == bootstrap, the sandbox root is '/' mounted as read-only.09:02
*** edcragg has joined #baserock09:02
gtristanwell, it should not make a huge difference, but indeed it seems strange to have a separate codepath for that09:03
gtristansandbox.run_extension just changes the CWD to the sandbox and runs the extension09:04
gtristanbut it also messes with PYTHONPATH to put the extensions in the path09:04
*** ctbruce has quit IRC09:04
gtristanactually runs it in defdir ... this codepath is very confusing09:06
*** ctbruce has joined #baserock09:08
SotKiirc it runs in the definitions repo because some extensions expect to be run from there09:16
* SotK may be totally making that up as its a long time since I worked on that code and its friday09:16
gtristanSotK, most probably the install-files want that09:17
gtristanthe extensions is quite limited and rigid in terms of what context can be passed from the builder to the extension09:18
SotKindeed09:18
* SotK assumes there was some reason for the limitedness, but doesn't know what it was (if there was one)09:18
pedroalvarezall my pipelines are stuck at fetching a docker container09:42
pedroalvarezfist attempt failed when cloning ansible10:02
tiagogomesI can clone it locally10:10
tiagogomesjjardon, is there a special reason why you are merging /usr/bin and /usr/sbin in your jjardon/usr-merge branch?10:11
pedroalvarezI asked about that, no answer yet ^10:11
tiagogomesostree seems to do that. But because ostree does it, it doesn't mean we should do it.10:12
tiagogomesFedora, for example, keeps them separate.10:12
tiagogomesI am looking at the artifact dir populated by ybd, but I can't find the stage2 artifacts10:52
paulsher1oodtiagogomes: if it has the rest of the build-essential artifacts, it doesn't download stage1* and stage2*10:53
pedroalvarezhow does that work? it ignores the "bootstrap" chunks?10:54
* gtristan wonders if it makes sense to cache the stage1/stage2 artifacts on the artifact server, do they work when shared ?10:54
tiagogomesah ok. morph used to download them all I believe10:54
tiagogomesAn Internals.md document for ybd would be mint10:55
gtristanpedroalvarez, I believe it's a question of dependency, the stage1/stage2 are not required to build a stratum above build-essential (as nobody asks for them to be staged -> built, nobody asks them to be downloaded)10:55
gtristanbootstrap happens to cause this dependency circumvention, but I dont like this; V10 will improve semantics around that10:55
pedroalvarezeverything depends on built essential, so I assume that it works as I said, by ignoring the bootstrap flagged chunks10:56
paulsher1oodpedroalvarez: correct10:56
tiagogomesThat looks a good idea to cut down download speeds, but I'd like that there was a way to download them all.10:57
* gtristan is very skeptical that stage1 or stage2 gcc would be useful to anyone who has to use them to build the rest of build essential10:58
gtristani.e. the built gcc outside of the sandbox used to build the real gcc would be sysrooted, so if I download tiagogomes's stage1, it will look for things in /home/tiago/baserock...10:59
paulsher1oodthe build-essential artifact contents  exclude bootstrap chunks. all other artifacts depend on the build-essential artifact10:59
pedroalvarezgtristan: should be easy to test. Trigger a build of a chunk that depends on gcc stage211:00
pedroalvarezor on gcc stage111:00
pedroalvarezthe lack of people reporting bugs about that makes me think that it works11:02
* paulsher1ood is confident ybd's approach for this works, with or without pre-cached kbas/local artifacts11:03
paulsher1oodso... folks are pushing to make ybd run with python3, and drop support for python211:04
paulsher1ooddoes anyone object to that?11:04
* gtristan has not ever run a build with a working connection to the artifact server... but I can try a simple test locally11:04
paulsher1oodi'm considering that if we do this, i'll tag one final release of ybd that supports python2, before the move11:05
tiagogomesgtristan unless the sysroot on morph was always a fixed path, but I don't think so11:06
gtristanI have a build-essential that works, now modify ybd.conf to put the tmp dir in a new location, and trigger a rebuild of gcc, actually it should break even without changing the config (because of tmp file staging dir path)11:06
pedroalvarezERROR: Build failed: execution took longer than 3600 seconds11:33
pedroalvarezis this new?11:33
pedroalvarez(only 1h of CI in gitlab)11:34
leemingwell that isn't helpful for some things right now :P11:36
* leeming wonders what the state of these private runners is11:36
pedroalvarezadd to that wondering if the private runners allow you to increase that limit11:37
locallycompactIs anyone else tackling gcc6? I can't make any progress without it11:37
leemingpedroalvarez, im sure they do, as other projects have run for longer than 1hr builds11:37
pedroalvarezleeming: seems to be configurable in the project (the timeout)11:38
pedroalvarezgiven that I've forked it, I need to configure mine11:38
leemingnever messed about directly with the runner configs, so wouldn't really know11:39
pedroalvarezI can't find where to configure the timeout though11:40
leeminglocallycompact, might know?11:40
locallycompactrepo?11:40
pedroalvarezfound it11:41
tiagogomeslocallycompact, what do you need gcc6 for?11:42
locallycompactI can't bootstrap on my laptop11:42
locallycompactgetting this https://gcc.gnu.org/bugzilla/show_bug.cgi?id=6995911:42
locallycompactthis seems to be the problem with gcc6 https://gitlab.com/baserock/definitions/builds/478901811:43
pedroalvarezlocallycompact: I looked into it a while ago, but hit a problem with libffi11:43
pedroalvarezhttp://git.baserock.org/cgit/baserock/baserock/definitions.git/log/?h=baserock/pedroalvarez/gcc611:43
pedroalvarezruntime problem, making python really broken11:44
tiagogomesmeh!11:44
tiagogomescan gcc-5 build gcc-6?11:44
locallycompactyes11:44
locallycompact6 can not build 5.311:45
* locallycompact overwhelms11:45
tiagogomesit would be worst if 5 couldn't build 6. I don't know if debian already uses 6 yet for example11:45
leemingwhy wouldn't you be able to build an older version?11:46
leemingassuming it defines the libs+versions correctly11:46
locallycompact<locallycompact> getting this https://gcc.gnu.org/bugzilla/show_bug.cgi?id=6995911:47
locallycompactdon't assume rhyme nor reason with gcc, it will break because you looked at it funny11:47
* leeming has broken bwrap11:48
* locallycompact looks for something to smash11:48
leemingmy brain, i dont think im using it right now11:48
tiagogomesleeming, I will have a look at the bbrwap PR today. Also, it would be nice that future PRs would go to gitlab11:49
leemingi wasn't sure what had been agreed on with lab vs hub11:50
leemingI updated the PR btw, just trying to get it working with ybd11:50
leeminggithub's interface for your suggested change confused me, as i couldn't see what was proposed11:51
pedroalvareznah, trove is still broken11:55
pedroalvarezhttp://paste.baserock.org/jofolifoya11:55
pedroalvarezI guess we need these deps: http://git.baserock.org/cgit/delta/python-packages/yoyo.git/tree/setup.py#n2211:55
* SotK is very confused as to how his trove was lorrying from g.b.o11:56
* leeming assumes when building in bootstrap mode host /dev sound be mounted12:00
leemings/sound/must/12:00
*** gtristan has quit IRC12:01
leemingah.. seems to get getting past my previous blocking point... ybd was missing a mount for dev (clearly bwrap is pickier than linux-user-chroot)12:05
* leeming lunches and hopes not to see an error message on his return12:05
paulsher1oodhttps://blogs.gnome.org/tvb/2016/10/13/software-build-topologies/12:17
*** tiagogomes has quit IRC12:35
pedroalvarezlorry-controller lorrying from yaml http://cache.baserock.org/lc-status.html12:38
pedroalvarezSotK: it works :)12:38
pedroalvarezalthough I wonder how did you manage to get it working with some of the yoyo dependencies needed12:38
pedroalvarezwe need to integrate these before I can upgrade git.baserock.org12:39
SotKyeah, I have no idea what I did that caused that not to happen12:40
pedroalvarez`pip install yoyo` :P12:42
leemingupdate: looks like ybd+bubblewrap (root) are working. still on stage2 though12:42
locallycompactnice12:42
leemingI guess what tiago said the other day to me was true then. that I had a cached stage1/2 artifact12:42
leemingthus why it seemed to work previously12:42
leemingonce this is done. I will nuke the cache and run non-root to see12:43
SotKpedroalvarez: oh, that could actually have been what I did12:43
* SotK admonishes himself12:43
leemingglibc is so huge :P12:44
pedroalvarezyou haven't met webkit12:44
leeming:)12:44
CTtpollardha!13:08
*** tiagogomes has joined #baserock13:27
tiagogomesleeming I reviewed your PR13:30
paulsher1oodleeming: are you testing the sandboxlib bubblewrap implementation with ybd?13:36
leemingtiagogomes, thanks. Reasonable suggestions. not sure why it isn't picking up the logger config tho13:37
leemingpaulsher1ood, yes, see above :)13:37
leemingstill running the root version13:38
leemingthen will check with non-root13:38
paulsher1oodah, perfect13:38
paulsher1oodso looks like you've found a bug in ybd also?13:38
leemingnot sure if it is a bug persay13:39
leemingjust probs differences between how bwrap and other sandboxes deal with /dev13:39
locallycompactper se13:39
leemingwhat he said, extertragh13:40
pedroalvarezsomeone should investigate what's going on with ybd+ansible13:51
pedroalvarezi haven't managed yet to test my changes using gitlab13:52
paulsher1oodpedroalvarez: what's the problem with ybd+ansible?14:13
*** CTtpollard has quit IRC14:13
pedroalvarezpaulsher1ood: this one: http://paste.baserock.org/utofupabiy14:14
pedroalvarezin gitlab ci pipelines, all the time :/14:14
paulsher1oodwarning: remote HEAD refers to nonexistent ref, unable to checkout.14:19
*** CTtpollard has joined #baserock14:21
pedroalvarezpaulsher1ood: is that regarding the same issue? I can't find that warning in my logs14:23
paulsher1oodthat's what happens when i run 'git clone --no-hardlinks git://git.baserock.org/delta/ansible.git'14:24
paulsher1oodwhich is what ybd does14:24
locallycompactthere's no master in cgit...14:26
locallycompactis that normal14:26
paulsher1oodit's not 'normal' but it's not impossible14:26
locallycompactfor ansible14:26
pedroalvarezpaulsher1ood: can reproduce the warning, but that is not a failure14:26
pedroalvarezhttp://paste.baserock.org/jowixaware14:27
paulsher1oodpedroalvarez: strange, then. https://gitlab.com/baserock/ybd/blob/master/ybd/repos.py#L20214:29
* paulsher1ood wonders how come 'call' returns something non-false if the git command returns 014:30
tiagogomesis the gitdir argument a git repo URL or a bare local directory?14:31
paulsher1oodgit repo url14:31
paulsher1oodoh, sorry14:31
pedroalvarezregarding 'master' not present, it's normal for ansible project14:32
paulsher1oodyup14:32
rjekThe docs for subprocess.call say it has the same fingerprints as subprocess.Popen, which returns a Popen object.14:32
paulsher1oodgitdir is the bare local directory14:32
jmacsNo they don't https://docs.python.org/2/library/subprocess.html14:33
jmacs(Same for Python3)14:34
jmacs"return the returncode attribute"14:34
* rjek was looking at https://docs.python.org/2/library/subprocess.html#subprocess.call14:34
rjek"The full function signature is the same as that of the Popen constructor"14:35
jmacsAh, signature14:35
jmacsYes, I suppose they don't include the return type in that14:35
rjekThe docs are rather ambiguous tbh14:36
jmacsAt least in Python 3 they say "mostly the same as"14:36
rjekhah14:37
jmacsAre you sure it's failing on the git clone, and not the git checkout?14:38
*** jude_ has joined #baserock14:39
paulsher1oodi believe so. the error reported from the app.log(name, 'Git clone failed for', ref, exit=True) line14:40
leeming subprocess.Popen seems to pass the CWD env though14:40
leemingeven when you do not include it as an arg14:41
*** gtristan has joined #baserock14:42
pedroalvareznitpick: the error string is confusing given that it's not cloning that ref14:43
paulsher1oodtrue14:44
paulsher1oodhttps://gitlab.com/baserock/ybd/commit/c24b2c96c3732fcd6d0677583feb9dab10f68d3214:47
tiagogomesI think ybd is innocent in this ansible problem15:08
pedroalvareztiagogomes: anything else to add?15:10
pedroalvarezor just a feeling15:10
tiagogomesHEAD in ansible from git.baserock.org points to refs/heads/master15:11
tiagogomesHEAD in ansible from git://github.com/ansible/ansible points to refs/heads/devel15:12
pedroalvarez<pedroalvarez> can reproduce the warning, but that is not a failure15:12
pedroalvarez<pedroalvarez> http://paste.baserock.org/jowixaware15:12
* SotK notes that morph built ansible fine for me last weekend :)15:14
tiagogomesthe warning can be an indication why it would fail later15:18
tiagogomesMy point is that the ansible repo in gbo is somewhat damaged15:19
*** toscalix has joined #baserock15:20
pedroalvarezI don't think that lorry also replicates the HEAD15:22
SotKthat'd be my guess as to the cause15:22
pedroalvarezso, I don't think that anything has changed in gbo to make it fail15:23
SotKalso the repo still works except for not being able to checkout HEAD when cloning15:23
*** ctbruce has quit IRC15:24
tiagogomesoh I can't reproduce the problem with ybd15:38
pedroalvarezthen the problem is gitlab? :/15:39
tiagogomesdon't think so, yesterday I reproduced the problem15:48
pedroalvareztiagogomes: maybe you need to clean your git cache?15:58
tiagogomesI did that :)15:58
tiagogomeswill try again15:59
pedroalvarez:/15:59
*** CTtpollard has quit IRC16:22
paulsher1ood:/16:53
*** locallycompact has quit IRC16:58
jjardonpedroalvarez; Is this definitions in gitlab? The timeout there is set to 1200 minutes17:00
jjardonGo to ci/CD settings if you are working on a personal repo and you want to change it17:01
*** locallycompact has joined #baserock18:11
*** inara` has joined #baserock18:43
*** tiagogomes has quit IRC18:45
*** paulsherwood has joined #baserock18:46
*** toscalix has quit IRC18:51
*** inara has quit IRC18:51
*** leeming has quit IRC18:51
*** paulsher1ood has quit IRC18:51
*** ChrisPolin has quit IRC18:51
*** leeming has joined #baserock18:57
*** ChrisPolin has joined #baserock18:57
*** locallycompact has quit IRC21:45
*** gtristan has quit IRC21:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!