IRC logs for #baserock for Tuesday, 2015-06-16

*** rdale_ has joined #baserock02:31
*** rdale has quit IRC02:34
*** zoli__ has joined #baserock03:16
*** zoli__ has quit IRC03:28
*** zoli__ has joined #baserock03:29
*** zoli__ has joined #baserock03:41
*** zoli___ has joined #baserock03:55
*** zoli__ has quit IRC03:59
*** zoli___ has quit IRC04:48
*** zoli__ has joined #baserock04:48
*** zoli__ has quit IRC06:38
*** zoli__ has joined #baserock07:01
*** CTtpollard has joined #baserock07:52
*** mariaderidder has joined #baserock07:54
* straycat finds https://gerrit-review.googlesource.com/#/c/58322/07:58
*** mdunford has joined #baserock08:01
*** bashrc has joined #baserock08:08
*** gary_perkins has joined #baserock08:10
jjardonpaulsherwood: not really, I have only compiled it a couple of times. But I know the main developer if you have any specific question08:13
*** ssam2 has joined #baserock08:18
*** ChanServ sets mode: +v ssam208:18
* SotK notices that both Masons are broken again :(08:18
SotKx86_64 is out of disk space08:18
SotKI can't tell whats up with x86_32, but its failing with "ERROR: Too many links" when it tries to build anything08:19
ssam2i'll add your key to both of them08:19
SotKssam2: thanks08:19
ssam2ok, you should be able to log in as root to them on their internal IPs08:22
SotKssam2: doesn't seem to have worked, I'm getting a password prompt08:26
ssam2are you logged into the devel VM with `ssh -A` ?08:26
SotKyep08:27
ssam2hmm, ok08:27
*** jonathanmaw has joined #baserock08:35
*** ssam2 has quit IRC08:36
*** ssam2 has joined #baserock08:44
*** ChanServ sets mode: +v ssam208:44
*** edcragg has joined #baserock08:52
*** mariaderidder has quit IRC09:06
*** franred has joined #baserock09:18
*** lachlanmackenzie has joined #baserock09:20
*** mariaderidder has joined #baserock09:21
ssam2jmacs: I've not forgotten about your cloud system, there's a network issue that seems to be stopping it working, so i'm waiting on a support ticket09:24
radiofreepaulsherwood: i can't reproduce the gcc issue on aarch64 anymore09:25
radiofreetried about 10 times now, so i'm going to say "ybd works on aarch64!"09:25
rjek:)09:26
*** ssam2_ has joined #baserock09:26
radiofreeof course, the next time someone uses ybd to build on aarch64 they'll come across it...09:26
jmacsssam2: It's not urgent.09:27
*** ssam2 has quit IRC09:28
nowsterpaulsherwood: It's not seeing "pbr"'s hooks, so /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'pbr'09:37
nowsterand then the package name is UNKNOWN09:37
straycatooi, what version of python?09:40
paulsherwoodradiofree: w00t!09:41
paulsherwoodssam2_: is nowster's error any help for diagnosing what's up?09:42
nowsterstraycat: Python 2.7.9 (default, Feb 10 2015, 04:30:22)09:43
ssam2_nowster, paulsherwood: I've no idea what's up, I'm afraid09:43
nowstermy guess is that the hooks that pbr is supposed to install aren't getting called09:44
ssam2_maybe the setup.py isn't using pbr correctly...09:44
ssam2_might be worth trying to run setup.py from a different package that also uses pbr09:44
ssam2_most of the openstack packages use it09:44
ssam2_https://git.openstack.org/cgit/openstack-infra/zuul/tree/ for example -- that's a small package, maybe you could try installing that from Git and see if it works09:45
ssam2_if it does, probably sandboxlib is doing something wrong09:45
ssam2_if not, probably pbr is broken on your system somehow09:45
* SotK has seen that error before, it was because the installed version of pbr was too old when I saw it IIRC09:46
ssam2_ah09:47
ssam2_maybe a requirements.txt would help09:47
nowsterssam2_: zuul gives same error.09:48
nowsterSotK: what is the ver of pbr that's needed?09:48
SotKnowster: I don't remember, what version do you have installed?09:48
nowsterIt claims to be 1.009:49
ssam2_I have 0.10.809:50
ssam2_but that version works09:50
SotKhm, 0.9.0 looks to be the version I had to update to when I was trying to build Zuul last year to fix that issue09:50
franred0.10.7 in Baserock In Openstack09:50
franreds/ Baserock In Openstack/ Openstack in Baserock/09:51
paulsherwoodssam2_: i guess this means that sandboxlib has introduced an implicity dependency on pbr?09:53
ssam2_paulsherwood: only at install-time09:53
paulsherwoodi take that as a 'yes' :)09:53
straycatnowster, i don't know what system you're running, but i just upgraded the pbr in this virtualenv to the latest and i can still create the egg-info for sandboxlib09:53
paulsherwoodwhy does sanboxlib need pbr?09:54
ssam2_paulsherwood: yeah. I'm just bristling slightly at calling it 'implicit' I think :)09:54
ssam2_paulsherwood: because setup.py files are a horrid mess09:54
paulsherwoodssam2_: it's implicit from ybd's point of view. ybd doesn't need it09:54
ssam2_paulsherwood: I can't really understand your use of the word 'implicit'09:55
ssam2_paulsherwood: if ybd did need pbr, it wouldn't make sandboxlib's dependency on pbr 'explicit'09:55
straycatybd isn't an installable python distribution anyway, if you wanted to make it one it would be a good idea to use pbr09:56
ssam2_pbr is nice because it allows you to write a declarative setup.cfg file to express the metadata and install instructions for the Python library09:56
paulsherwoodssam2_: i'm misuusing the word then. my point is ybd doesn't need pbr. sandboxlib does.09:56
nowstereg. baserock needs baserock to build, as morph assumes it's running on baserock.09:56
ssam2_indeed. without pbr, you have to write the metadata and install instructions as Python code09:56
ssam2_which I don't really like09:56
paulsherwoodi think we have a philosophical difference, here. what's the simplest way for ybd to have the functionality of sandboxlib without pbr?09:58
ssam2_'pip install pbr'09:58
ssam2_no, waiot09:58
ssam2_'pip install sandboxlib' I mean09:58
nowsterno pip09:58
paulsherwoodor, put another way is there a way for ybd to use sandboxlib without *installing* sandboxlib?09:59
ssam2_yeah, just set 'PYTHONPATH=path-to-sandboxlib' and run it from the source tree09:59
paulsherwoodnowster: ^^09:59
paulsherwooda little ugly, but may work :)09:59
ssam2_good thinking09:59
nowsterthat does seem to work.10:00
* nowster gets coffee and thinks a bit.10:00
paulsherwoodssam2_: how would you express my point about sndboxlib introducing pbr as a dependecnty to ybd, without ybd realising it?10:00
jmacsInteresting; some of our build process leaked into the gcc build results. /usr/lib/jvm/bin/java -> /gcc.build/oesult/gij10:01
paulsherwoodjmacs: morph, or ybd?10:01
jmacspaulsherwood: morph. I suppose this would be a good time to try ybd again10:01
* paulsherwood smiles :-)10:01
jmacsI think the problem will be in gcc, though10:02
paulsherwoodjmacs: however, i think ybd's sandboxing functionality is equivalent to morph's (albeit in the separated sandboxlib library) so the result may be the same10:02
ssam2_paulsherwood: I think i'd express it as you just did there10:03
ssam2_I can't think of a word that describes that situation more succinctly10:04
* paulsherwood still believes implicit conveys the point10:04
straycatI seriously don't know why you wouldn't have ybd use pbr, so you can install it with pip, I mean if you're trying to be accessible and all that10:04
paulsherwoodstraycat: because i have already one use-case where it doesn't work? ie nowster's situation above?10:05
straycatMkay10:05
ssam2_paulsherwood: ybd doesn't work on a system where I've broken python, either10:05
paulsherwoodlol10:05
straycatWell, pbr seems to be good enough for Openstack...10:05
paulsherwoodstraycat: my choices are my own. i don't think popularity is one of my criteria10:06
nowsterbeing able to bootstrap from a limited set of components does have its advantages10:07
paulsherwood:)10:07
ssam2_it does, but refusing to use any technology unless it's old enough to be widely available has its downsides too10:07
robtaylorpaulsherwood: you could submodule sandboxlib in10:08
paulsherwoodnowster: please could you raise a github issue documenting what you hit, either on sandboxlib or ybd as you think appropriate?10:08
ssam2_if you'd experienced the pain of trying to use Python setuptools / distutils / distribute, you'd probably appreciate how nice pbr is10:09
paulsherwoodrobtaylor: oh? how does that work, please? i'm still a python n00b10:09
paulsherwoodssam2_: i'm not saying it's not nice... i'm just not ok that it's a possible point of failure/friction for a new user of ybd10:11
robtaylorpaulsherwood: googling gives me http://stackoverflow.com/questions/29746958/how-to-import-python-file-from-git-submodule10:11
paulsherwoodrobtaylor: i see what you did there :)10:11
robtaylorheh10:11
ssam2_paulsherwood: fair enough. probably adding it as a submodule is a good plan, then10:12
paulsherwoodi think i'm against using git-submodule just for this, though10:12
ssam2_heh10:12
DavePageHas the problem with git submodules and troves been addressed?10:13
straycatpaulsherwood, packagers like yourself are the reason I have such a difficult time with the import tool :)10:13
paulsherwoodi don't understand git-submodule properly either, but implementing it for ybd it seemed like a kludge10:13
ssam2_I've also had issues with setup.py files that use setuptools, so I don't think removing pbr and using the older 'setuptools' or 'distutils' approaches would be an improvement10:13
straycatpaulsherwood, the proper way to package ybd is as ssam2_ has done for sandboxlib10:13
paulsherwoodstraycat: i don't doubt it. and if the pbr issue gets nailed, i'll be happy to use it packaged10:14
franredpaulsherwood, so you don't want to install sandbox (avoid to install pbr) and you don't want to make sandbox as a submodule, so are you going to add instructions to clone sandbox and set PYTHONPATH, does that no cause more friction than anything else?10:15
paulsherwoodincidentally, note that all i'm doing is trying to solve the *user* issue in a bulletproof way here. i don't know why there's so much pushback10:15
paulsherwoodfranred: it's a fair question. i might settle for a ybd.sh script that sets pythonpath or something10:16
paulsherwoodi just want something simple that works all the time10:16
paulsherwoodfranred: the friction in this current example is that nowster tried ybd yesterday, and it took til today to get to a solution10:18
franredpaulsherwood, in requirements.txt in sandbox I imagine ssam2_ can add which version of pbr is required10:19
*** ssam2_ has quit IRC10:19
franredthat would be the cleanest/more elegant solution, IMHO10:19
nowsterI think its more that my pbr isn't correctly installed.10:20
nowsterIt's supposed to add hooks, and hasn't.10:20
nowster2015-06-16 10:21:50 [systems/minimal-system-mips64b-openwrt.morph] ERROR: not a git repo /src/workspace/openwrt/github/nowster/definitions10:22
straycatnowster, you could also just remove pbr from your system, btw10:22
paulsherwoodnowster: is that true?10:23
nowsterit's not10:23
straycat?10:23
paulsherwoodare you running ybd in that directory?10:23
nowsterno10:23
nowsterIt is its current directory10:23
paulsherwoodthat's where you need to be10:23
nowster /src/workspace/openwrt/github/nowster/definitions # /src/ybd/ybd.py systems/mini10:23
nowstermal-system-mips64b-openwrt.morph10:23
* paulsherwood is confused. is /src/workspace/openwrt/github/nowster/definitions a git checkout, and is that your cwd?10:24
nowsteryesa10:24
paulsherwoodhmmm.10:24
paulsherwoodi assume you have git on your system?10:25
nowsterwould it matter if it's not clean10:25
paulsherwoodno, it shoudl be fine10:25
nowster# git --version10:25
nowstergit version 2.1.310:25
straycatnowster, if you remove pbr from your system, setuptools will install it in the distribution's directory before attempting to install sandboxlib10:25
straycatyou could also probably just upgrade it10:25
straycator you could do all this in a virtenv10:25
straycator in Baserock10:25
nowsterstraycat: will need to rebuild that baserock image to do that...10:26
straycatthis is way too hard, i'm trying to be helpful10:26
*** straycat has left #baserock10:26
nowstera full rebuild is not quick... we're talking about 36 hours10:26
rjekTrove question: if an upstream git repo remove, say, a tag, and lorry repulls or has to reclone, do we lose that tag?  Will any information ever get GCed?10:27
paulsherwoodnowster: can you try 'git describe' there please?10:27
rjekMy concern stems from GPL requirements of maintaining an offer of source code for two years from shipping a binary, and the risk of the exact code being shipped eventually vanishing.10:27
nowster# git describe10:28
nowsterfatal: No names found, cannot describe anything.10:28
paulsherwoodnowster: that's what ybd does to test for git repo-ness10:28
nowsterthere are no tags in that repo10:28
paulsherwoodi guess that's a bug10:28
nowster# git describe --all10:28
nowsterheads/openwrt10:28
paulsherwoodwell spotted, sir10:29
*** franred has quit IRC10:29
paulsherwooddo you want to patch it and offer a pr?10:29
nowsterOK.10:29
paulsherwoodrepos.py line 9010:29
*** zoli__ has quit IRC10:31
nowsternot line 68?10:32
nowsterit's in app.py10:33
paulsherwoodoops, sorry10:37
paulsherwoodyou're better placed to work it out i think :)10:38
nowsterpaulsherwood: you have a pull request :)10:38
paulsherwooddoes it work, nowster ?10:38
nowsteryes10:38
nowsterLots of warnings10:38
nowsterAnd it says  "finished" probably because all the bits are in the cache.10:39
paulsherwoodnowster: really? how could they be?10:39
nowsterI'd already built it with morph.10:40
paulsherwoodmorph and ybd do noy share cache-keys10:40
paulsherwoodwhat arch are you building for?10:40
nowstermips6410:40
Kinnisonmipsel64 or mipseb64 ?10:40
nowster2015-06-16 10:39:01 [systems/minimal-system-mips64b-openwrt.morph] Skipping assembly for mips64b10:40
Kinnisonb10:41
Kinnison:)10:41
SotKnowster: sounds like the arch auto-detection doesn't work there either10:41
nowsteryep10:41
SotKtry `ybd $system mips64b`10:41
nowsterI suspected it might need that changing too10:41
nowsterit's now building10:42
nowsterand it's failed10:42
nowsterDoing things in wrong order.10:43
nowsterIt's gone: minimal-system-mips64b-openwrt -> build-essential -> glibc -> stage2-fake-bash10:44
nowster...and ignored all the stage1 stuff.10:44
Kinnisonvery odd10:44
SotKI guess that stage2-fake-bash doesn't have any build-dependencies?10:44
nowsternope, it doesn't10:45
Kinnisonaah10:45
Kinnisonbecause it's essentially a shell script only10:45
SotKcan you paste the log of the failure?10:45
*** ssam2 has joined #baserock10:46
*** ChanServ sets mode: +v ssam210:46
nowsterIt probably needs to depend on stage2-busybox.10:46
paulsherwoodnowster: ybd does randomised build-order where it can10:46
SotKnowster: thats the "right" order then, since ybd just recursively tries building things until it gets to something for which the dependencies are satisfied10:46
SotKnowster: a bug in definitions then I guess :)10:46
paulsherwood:)10:46
*** franred has joined #baserock10:46
* nowster tries adding a build-depends10:47
paulsherwoodso in an exciting game, the score is currently nowster 2, ybd 1 :-)10:47
nowsternow it's building stage1-binutils, which is what i'd expect10:48
nowsterbut it's fauled10:48
nowsterclone: Invalid argument10:48
paulsherwoodouch!10:48
Kinnisonnowster: sounds like your kernel is missing stuff, perhaps namespaces?10:49
nowsterNot sure. Might be. Is that essential for ybd?10:49
ssam2nowster: yes, it could be that you have linux-user-chroot installed but some of the namespacing features in the kernel are missing10:49
paulsherwoodnowster: please raise that as an issue before we forget?10:49
ssam2sandboxlib does some autodetection of the backend, but it doesn't look at kernel config right now, it just looks at whether the linux-user-chroot binary is in the PATH10:50
nowsterpaulsherwood: which one? the namespace problem or the build-depends one?10:50
paulsherwoodssam2: does the sandboxlib chroot option work?10:50
ssam2paulsherwood: I've verified that it works for me. No idea if it will work for nowster :)10:50
paulsherwoodnowster: namespace, unless you believe the build-depends is a bug in ybd?10:50
paulsherwoodssam2: is there a way to force the chroot otion, even if linux-user-chroot is present?10:51
ssam2quickest way to try out the chroot backend is 'hide' the linux-user-chroot binary.10:51
ssam2probably sandboxlib should honour a SANDBOXLIB_BACKEND env var in its autodetection code to allow overriding it10:51
jmacsAre we still keen on using Storyboard? The wiki is inconsistent about it.10:52
ssam2I think we are inconsistent about it10:52
ssam2I would like to use it but it's not good enouigh10:52
ssam2but neither is anything else :)10:52
richard_mawssam2: perhaps sandboxlib should, on first use, check whether linux-user-chroot works, and flag that it has to use chroot otherwise10:52
paulsherwoodi think we are considering commiting engineering to improve storyboard10:52
paulsherwoodrichard_maw: that would be lovely :)10:53
nowsterlinux-user-chroot works here, AFAICT10:53
Kinnisoncould it be the unshare call for mount namespaces failing?10:53
ssam2nowster: define 'works'10:53
nowstermorph uses it10:53
nowsterhowever, # CONFIG_NAMESPACES is not set10:54
jmacsOK, I'll update the wiki to say you can use either storyboard or the mailing list, if that's OK.10:54
ssam2jmacs: that's fine10:54
ssam2nowster: interesting. could you try `linux-user-chroot --unshare-net / true` and see if you get the same error?10:54
nowsterI do get same error10:55
richard_mawlinux-user-chroot is not secure for containerising without mount namespaces, which are enabled with CONFIG_NAMESPACES10:55
* nowster goes to recompile kernel10:56
ssam2nowster: you could also hide linux-user-chroot so the chroot() backend of sandboxlib gets used :)10:56
nowsterssam2: but it'll build linux-user-chroot, won't it?10:57
ssam2nowster: building linux-user-chroot isn't the problem. the problem is that linux-user-chroot can't run10:57
richard_mawnowster: it will, but morph uses the linux-user-chroot provided by the host environment10:57
nowstermorph builds on this machine without namespaces10:58
nowster...and it uses linux-user-chroot10:58
ssam2nowster: that's really bizarre, as Morph unconditionally runs builds through linux-user-chroot10:58
paulsherwoodssam2: not for build-essential?10:58
ssam2paulsherwood: for build-essential it does too10:58
paulsherwoodhmmm. i don't know, then10:59
nowsterI've been building stuff with morph on that kernel config for about 4 months.10:59
ssam2is there any way I could have access to the machine we're talking about  so I could satisfy my curiousity about what is broken?11:00
nowsterroot@bono11:00
paulsherwoodnowster: could you paste the bottom of the broken log in that pr please?11:01
nowsterpaulsherwood: done11:02
ssam2ah! it's the --unshare-net flag that triggers the error11:02
paulsherwoodtvm11:02
ssam2morph doesn't pass that flag (at least, not the version you are using), while sandboxlib does11:02
* nowster remembers to wrap the log in markdown quotes11:03
ssam2I always thought Morph built in a chroot which was network isolated, but seems not11:05
paulsherwoodnowster: have you tried hiding linux-user-chroot?11:07
nowsterpaulsherwood: not yet. Currently recompiling kernel.11:07
nowstermv /usr/bin/linux-user-chroo11:08
nowstert /usr/bin/linux-user-chroot.gone11:08
nowsterhmm...11:10
nowsterchecking target system type... Invalid configuration `mips64b-bootstrap-linux-gnu': machine `mips64b-bootstrap' not recognized11:10
nowsterconfigure: error: /bin/bash ./config.sub mips64b-bootstrap-linux-gnu failed11:10
nowsterit needs some arch mangling.11:10
nowsterit needs to be told that TARGET_STAGE1 is mips64 in this situation11:11
nowsterarch_dict needs tweaking11:12
nowstertries again11:13
*** zoli__ has joined #baserock11:14
nowsternow appears to be running make11:16
ssam2I've filed https://github.com/CodethinkLabs/sandboxlib/issues/3 and https://github.com/CodethinkLabs/sandboxlib/issues/411:17
ssam2have created https://github.com/CodethinkLabs/sandboxlib/issues/5 for the pbr issue as well11:24
nowster2015-06-16 11:41:04 [stage1-binutils] Elapsed time 00:26:4511:44
nowster:(11:44
nowstersorry... :)11:44
nowster(would have been faster if I weren't making a kernel at same time)11:44
paulsherwoodthat's 26 times slower than my laptop!11:48
*** mdunford has quit IRC11:57
paulsherwoodnowster: do you have a pr for arch_dict?12:02
nowsterwill do12:04
*** mdunford has joined #baserock12:11
nowsterpaulsherwood: there's another tweak so that it recognises the host machine arch correctly12:22
nowster...which I'll need to test in a minute12:22
*** fay_ has quit IRC12:32
nowsterpaulsherwood: pull request sent12:34
*** tiagogomes has quit IRC12:34
*** fay_ has joined #baserock12:35
*** tiagogomes has joined #baserock12:47
nowster2015-06-16 13:01:47 [stage1-gcc] Elapsed time 01:47:29    .... so it looks like ybd is working.13:03
paulsherwood:)13:06
radiofreeis git.baserock.org ever going to get a valid certificate?13:06
*** straycat has joined #baserock13:08
paulsherwoodrjek: could you help with that?13:15
ssam2we have a cert, but i'm not sure what the best way is to make git.baserock.org use it13:15
rjekI can recommend somewhere to obtain the certificate from, but can't help much with its installation13:15
ssam2radiofree: is it causing you problems? there is a long list of things which need to be fixed in the infra13:16
radiofreessam2: well i was recommending someone update their repos to use GBO over https, then remembered that will fail13:18
ssam2mm, true13:18
radiofreeof course there's http, but they really shouldn't have to use that13:18
rjek+113:18
ssam2i've created https://storyboard.baserock.org/#!/story/49 to track this task13:19
ssam2if anyone knows what would be the "correct" way to do this for a Trove, please let me know. I've not got bandwidth to think about it right now, might have a think about it on friday13:20
rjekWhich http server do we use on Trove?13:21
Kinnisonlighttpd I believe13:21
ssam2yes13:22
ssam2presumably its config is generated by trove.configure13:23
rjekOK, I was just checking it wasn't busybox httpd :)13:23
KinnisonIIRC that doesn't do SSL at all13:23
* SotK thinks its config is in trove-setup.git13:23
ssam2oh yeah, actually it's the ansible script that does it13:23
ssam2ok, so shouldn't be too horrible a task13:23
*** mdunford has quit IRC14:29
*** mdunford has joined #baserock14:31
jjardonhi ssam2! about https://gerrit.baserock.org/#/c/853/ , not sure I'm adding more comments about the use of tarball-version?14:33
*** mdunford has quit IRC14:37
*** mdunford has joined #baserock14:44
radiofreehow would one go about contributing an ARM cache to the artefact cache server?14:44
radiofreeactually i probably should rebase this14:45
paulsherwoodradiofree: do you mean morph cache, or ybd cache?14:46
radiofreeerm.. morph cache, morph is still the official tool of baserock right?14:46
ssam2jjardon: oh, sorry14:47
ssam2i'm really confused by the diff then14:47
radiofreeit's for this genivi thing, i'm writing a guide (all over again because firefox crashed) and step one is "deploy a developement image to your jetson"14:47
paulsherwoodradiofree: i believe it is... but i'm hoping to propose ybd for 'official' recognition soon14:47
radiofreeso i do that, that's fine, but then i don't want them to have to sit through 5 hours of building Qt to get to the GDP14:47
paulsherwoodaha, understood14:47
radiofreewhich would mean having the cache in the upstream cache server14:47
ssam2jjardon: oh, did those comments appear in the diff between 1 and 2 due to rebasing perhaps?14:47
jjardonssam2: I think so, yes14:48
ssam2oh, sorry14:48
paulsherwoodradiofree: this was previously achieved by an arm mason... i wonder if that  can be re-established14:48
ssam2jjardon: i've undone by edit14:48
radiofreepaulsherwood: i seem to remember having this conversation.. weeks... months agao?14:49
radiofrees/agao/ago14:49
radiofreeanyway, that wouldn't solve the Qt problem, since that's not on any system in the CI14:49
radiofreeand i wouldn't like to see how long it would take the x86 mason to build an image with Qt in14:49
radiofreei'm happy to manually provide cache for this stuff (GDP)14:50
paulsherwoodmakes sense14:50
ssam2radiofree: there's no process for that because nobody ever does it, but i trust you not to hose cache.baserock.org and can give you access to upload stuff to it14:51
ssam2or git.baserock.org, that's probably a better place14:51
jjardonssam2: thanks!14:51
radiofreessam2: what's the "official" cache server? cache or git?14:52
radiofree(the one we document)14:52
ssam2morph defaults to git.baserock.org14:52
ssam2we document cache.baserock.org14:52
ssam2it's a mess, as usual14:52
radiofreeah, nice, it automatically checks for cache on gbo?14:52
radiofreebut the CI puts the cache on cache?14:52
*** mdunford has quit IRC14:53
radiofreei shall rebuild it with a clean cache folder, since this jetson has 70G of artifacts14:54
SotKyou could use `morph list-artifacts` to just get the relevant artifacts14:55
franredcan someone spot what is going on in https://mason-x86-32.baserock.org/log/284aec89f5de6f111702065bacd72c31e5fd0c9b--2015-06-16%2009:16:50.log ?14:55
franredI can't see an useful error14:56
radiofreeSotK: i had no idea it did that14:56
richard_mawfranred: could be failing to transfer the artifact to the cache then14:56
radiofreefranred: out of space?14:56
radiofree2015-06-16 09:30:02 Progress: Transferring gf-complete-misc to shared artifact cache14:56
radiofree2015-06-16 09:30:13 Build of gf-complete failed.14:56
SotKradiofree: it doesn't, but it gives you a list of the filenames14:56
SotKfranred: cache.b.o was full earlier IIRC14:57
franredradiofree, richard_maw, SotK, ta14:57
ssam2radiofree: a good thing about putting them on git.baserock.org is they are unlikely to get deleted any time soon14:59
ssam2whereas, because cache.baserock.org gets every build of everything, we have to delete stuff quite often14:59
ssam2and there's currently no way of saying 'wait, those artifacts are important, keep them'14:59
radiofreeah14:59
radiofreeok14:59
radiofreewhat happens if the cache is on gbo, but you set your cache server to cache.baserock.org14:59
radiofreewill it still check on gbo first?15:00
ssam2no15:00
ssam2sadly15:00
radiofreewell there's nothing populating arm cache on cache.baserock.org anyway, right?15:00
*** mdunford has joined #baserock15:00
ssam2no#15:01
radiofreeok, thanks15:02
persiaI seem to recall having an ARM Mason in the past: what is the current blocker for deployment?15:03
*** radiofree has quit IRC15:03
richard_mawpedroalvarez is probably the one to ask about that15:04
*** radiofree has joined #baserock15:04
radiofreepersia: we did15:04
radiofreeand we should still have one, i thought we agreed it could go back to datacentred?15:05
ssam2persia: nothing is really blocking having a single Jetson that acts as a Mason, other than time to do it15:05
ssam2I have one on my desk, in fact, it's just not sharing what it builds publically (and it breaks sometimes)15:05
persiaAh, indeed.  Tuits are ever in short supply.15:05
*** mdunford has quit IRC15:06
ssam2to have an ARM distbuild network, we are blocked by the fact that distbuild requires that the artifact cache needs to be able to contact each worker15:06
ssam2which conflicts with how the colo provider wants to set up the network for the Jetsons, if we put them back there15:06
ssam2which is solvable, but requires time to think it through15:07
*** mdunford has joined #baserock15:08
persiaWould virtual distbuild workers solve that, or is it just a fundamental issue with deployment?15:09
jjardonpaulsherwood: hi, just trying ybd for the first time in a long time, but I get this error: http://paste.baserock.org/fapabazeqi15:12
radiofreedid i break that?15:12
ssam2persia: It depends on the network topology of the virtual distbuild workers15:13
radiofreeno, i don't think i did15:13
ssam2if they were in the same network as cache.baserock.org, it would make it easier to deploy them15:13
jjardonpaulsherwood: nevermind, I think I fixed it15:14
ssam2we are also a bit stuck that someone left all the power supplies for the jetsons in the colo center, so we can't test anything here15:15
radiofree:|15:16
DavePageThere are a while bunch of Jetsons turned off in the server roo15:17
* rjek imagines a kangaroo waiter15:17
De|tassam2, we have bench PSUs. Shouldn't take much to connect a few Jetsons to one. From a quick look, we'd just need some 2.1mm barrel plugs and some wire.15:23
paulsherwoodjjardon: could you send a pull request if you've fixed that please?15:35
jjardonpaulsherwood: sure. BTW, is ybd meant to be running in a specific environment? seems it tries to write in /src15:37
jjardonah, just found ybd.def15:37
paulsherwoodit's defaulting to /src - you can change that in ybd.def i think15:37
jjardonpaulsherwood: I think it would be better to default to $XDG_CACHE_DIR/src . Would you be interested in a patch to implement that?15:39
* jjardon just remembers to sent a patch to do that a while ago15:40
ssam2De|ta: that's an idea15:46
paulsherwoodjjardon: how would that work if $XDG_CACHE_DIR is not honoured?15:52
rdale_that's certainly linux specific16:00
ssam2rdale_: XDG_CACHE_DIR? it's a freedesktop standard I think, so not specific to Linux16:09
ssam2http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html16:10
rdale_yes, but i would be surprised if it was set in a mac os x environment16:10
ssam2yes, probably OS X, Windows and commercial UNIXes don't have it set16:10
ssam2so would need to fall back to ~ or somewhere if it's not set16:11
persiaAt least AIX and Solaris should have $XDG_CACHE_DIR set.  I don't know about other commercial UNIXes (e.g. OS X).16:13
persiaBut the spec itself calls for having a fallback when unset.16:14
De|ta'echo $XDG_CACHE_DIR' returns nothing on my OS X machine16:15
persiaThat is a strong indicator that OS X does not support it by default.16:15
paulsherwoodjjardon: if your patch leaves behaviour unchanged in the absence of XDG_CACHE_DIR i'd be happy to take it?16:16
rdale_some suggestions about mac os x equivalents of $XDG* here: http://stackoverflow.com/questions/3373948/equivalents-of-xdg-config-home-and-xdg-data-home-on-mac-os-x16:16
jjardonpaulsherwood:ok, I will try16:17
persiaNote that the spec itself seems to define the variable as $XDG_CACHE_HOME rather than $XDG_CACHE_DIR16:17
nowsterybd went splat: http://paste.baserock.org/jiferuvuni16:17
paulsherwoodnowster: was this forked from definitions a while ago? i think there may have been a fix to fhs-dirs in the meantime16:18
nowsterah, right.16:19
nowstercould be16:20
rdale_the fhs-dirs chunk is the same in the master of definitions as it is in our openwrt build-essential as far as i can see16:24
straycathuzzah, ssam2's series is merged, so you can forget about workspaces now, if you like16:25
persiaHurrah!16:25
nowsterThe only difference I can see is the removal of $DESTDIR/var in the older one16:26
rdale_yes, nothing that would affect whether fhs-dirs.build is created16:27
nowsterI see no recent push to master for fhs-dirs16:28
nowstercould someone have made a fix to fhs-dirs recently and not pushed it back?16:32
rdale_it looks to me as though one thread created fhs-dirs.build and correctly ran the chunk, and then later another build thread came along expecting it to still be there16:32
ssam2straycat: which begs the question... when can we delete that code :)16:32
ssam2(`morph checkout`, `morph branch`, etc.(16:32
* Kinnison votes ASAP, in order to reduce confusion, but perhaps not until a document is written which explains "If you used to do X, you now do Y"16:33
straycatYes, I guess we need to go through the wiki and update before removing that code16:33
ssam2indeed. in fact, we can start doing that now16:34
ssam2oh, we should wait for a release I guess16:34
ssam2the release SotK is doing won't include this change, so will be a bit of time16:34
straycatother than that, if no one's actually using workspaces then yeah I guess we can remove all that code16:34
persiardale_: That sounds like my memory of the old problem.  There was some recursion issue, which I thought was resolved by some other change to definitions.16:37
straycatHrm, it's a pity we can't do this sooner, or get this change into the release16:39
persiaIndeed.16:41
*** mariaderidder has quit IRC16:47
nowsterIt's happening in linux-api-headers now.16:58
nowster(due to randomization of build order)16:58
rdale_perhaps it's a timing thing as the mips machine is slower than everything else16:59
*** jonathanmaw has quit IRC17:00
nowsterI *suspect* it's the attempted containerization.17:00
* paulsherwood believes he's seen this before, but can't remember what the outcome was.... ssam2 ?17:05
rdale_should we be using python3 for baserock in general?17:10
persiaProbably, but it's complicated, because some folk only have python 2.x on their systems.17:12
persiahttps://pypi.python.org/pypi/nine might be a reasonable solution for that though.17:13
jjardonpersia: why, we use a chroot/baserock system to develop exactly for that reason, not depend on any specific version of anything17:14
persiajjardon: Because the software should be resilient to multiple systems.  That I can't use lorry without having a baserock system, which I can't usefully maintain as a snowflake, makes me not want to use lorry.17:15
persiaThe same applies to any of the other tools.17:15
rdale_i'm just checking out ybd and the sandboxlib, and wondering which python to use - it sounds like i shouldn't use python3. but my version of tox is looking for that17:16
straycatrdale_, yeah it's python 217:16
rdale_ok17:16
persiardale_: Write your code so that you can run in either environment.  If you have issues, use six or (better) nine.17:16
jjardonrdale_: can you not make the code compatible?17:16
rdale_i'm only trying to run other people's python17:17
persiaWhen that fails, report bugs :)17:17
* persia vaguely suspects https://github.com/devcurmudgeon/ybd/issues/50 is due to an assumption that python 2 is always available17:18
rdale_this is the error i'm getting: http://paste.baserock.org/raqupiridi17:18
paulsherwoodpersia: i'll need to find a system with only py3 on it to test that assumption.17:19
persiapaulsherwood: You are in luck: rdale is currently working with such a system :)17:19
rdale_not intentionally17:19
paulsherwoodthat looks like sandboxlib specifically?17:20
persiardale_: Perhaps, but `python` appears to run `/usr/bin/python3`, which lets you test17:20
rdale_on my system python is 2.7.8. i installed a .deb called 'python-tox', not 'python3-tox' but it still seems to be python317:23
jjardonpaulsherwood: about the sandbox problem, you need to add python2 (or python3) to ybd.py: #!/usr/bin/env python217:23
jjardonwill send a patch if you are not faster :)17:23
persiardale_: Most modern deb-based systems default to python317:24
persiajjardon: Can we not just use six or nine instead?17:24
ssam2rdale_: 'tox' is a test tool, it tries to test sandboxlib under both python2 and python3 to ensure it works under both17:25
*** mdunford has quit IRC17:26
ssam2that error looks like you don't have 'setuptools' for python3 installed17:26
paulsherwoodjjardon: i can't patch here17:26
paulsherwoodi can merge, though17:26
rdale_ok, i'm really trying to use pbr - i have installed a .deb called python-pbr , but that doesn't seem to have a /usr/bin/pbr executable in it17:26
jjardonpersia: probably, but https://www.python.org/dev/peps/pep-0394/ recommends specify the version if the code is python2-only17:27
persiaYes, but I believe it to be poor practice for python2-only code to exist17:28
persiaEspecially given that for most code, it is relatively easy to make it work in either environment.17:28
ssam2rdale_: no idea about the Debian packaging, but I don't know if you need a 'pbr' executable -- it's a library that is imported by sandboxlib's ./setup.py17:30
straycatrdale_, setuptools should install pbr for you if you don't have it on your system, since it's listed in setup_requires in the setup.py17:30
paulsherwoodjjardon: my hope is that ybd can be made to work with both 2 and 317:31
*** lachlanmackenzie has quit IRC17:32
rdale_ah ok - i installed python3-setuptools and tox runs now, but i needed linux-user-chroot too. the tests aren't quite working17:32
rdale_i'm getting: AttributeError: 'NoneType' object has no attribute 'decode'17:32
jjardonpaulsherwood: nevermind, what is there seems to be correct17:33
rdale_here are my test errors: http://paste.baserock.org/abobedugot17:33
jjardonpaulsherwood: nice, because Im going to be using python3 here :)17:34
* paulsherwood crosses his fingers17:34
rdale_i need the python equivalent of rbenv to keep my versions separate i think17:36
straycatrdale_, so it's got a non-zero exit code but there's nothing on stderr, but sandboxlib is assuming there is17:36
straycattox looks awesome17:37
ssam2paulsherwood: I think that it did last  week :)17:42
ssam2rdale_: the python equivalent of rbenv is virtualenv, which tox uses to create a clean test environment for running the tests in17:43
*** edcragg has quit IRC17:43
ssam2you've clearly some mistakes of mine in the test suite, though ()17:43
ssam2actually, seems like it's all the same error17:43
ssam2if you change 'stdout=None, stderr=None' to 'stdout=sandboxlib.CAPTURE, stderr=sandboxlib.CAPTURE' I think it'll show you the actual error, instead of crashing when it tries to show you what went wrong17:44
rdale_ah ok - i need to find out more about virtualenv. in ruby the .deb ruby versioning is a mess, and i don't use it at all. it looks as though .deb python versioning is too difficult to use17:44
ssam2in the bit of code that the error log is pointing to, I mean17:44
ssam2the test suite may be failing if your environment can't do everything that it expects to work. I didn't do anything about making it skip tests that can't succeed on a given machine17:46
ssam2well, I did a bit of work on that, but it didn't seem like that useful a thing to do17:46
rdale_ok thanks17:47
*** ssam2 has quit IRC17:50
jjardonpaulsherwood: ybd is asking me to be root to start the assembly, is that expected?17:56
paulsherwoodjjardon: afraid so. sandboxing requires root so far17:57
paulsherwoodi was hoping we'd find a solution to that17:57
jjardonthats ok, maybe we shold fix the "quick start" section in https://github.com/devcurmudgeon/ybd ?17:58
paulsherwoodjjardon: absolutely17:59
*** gary_perkins has quit IRC18:28
juergbii expect this requirement could be avoided when creating a user namespace. i don't know whether that's already enabled on typical distro kernels these days18:53
juergbiit could use userns if available and otherwise require root, i suppose18:54
*** zoli__ has quit IRC19:20
*** tiagogomes has quit IRC19:59
*** zoli__ has joined #baserock20:15
*** zoli__ has quit IRC20:16
*** zoli__ has joined #baserock20:18
*** zoli__ has quit IRC20:20
*** zoli___ has joined #baserock20:20
*** zoli__ has joined #baserock20:21
straycatBit late but finally got around to discussing our changes in pip with the maintainer20:58
straycatHe's of the opinion that the functionality I added doesn't belong in pip, and whilst acknowledging that this will make our tool harder to implement he suggests that we should either write the internals we need or pull pip's internals into the tool with the awareness that they are internal and may break over time. He also added that pip should probably try and pull more stuff out into separate libraries.21:00
straycatSo it goes21:00
paulsherwoodstraycat: pls could you remind me, what functionality was this specifically?21:10
rjekneither approach sounds delightful21:10
paulsherwoodagreed21:10
straycatpaulsherwood, https://github.com/pypa/pip/pull/237121:11
straycatit adds a --list-dependencies option to pip21:11
paulsherwoodok, thanks. i don't understand rbtcollins' last comment?21:13
straycatyes i'm not sure about his point yet either, i'll check it out properly tomorrow21:14
paulsherwoodyup. it's late, there21:14
* persia believes lifeless to currently be in UTC+1221:16
persiaAs a result, soon might be the best time to catch him, if one wants realtime discussion21:16
*** zoli__ has quit IRC21:17
straycatI think it can wait till I have a better understanding of his point, also dstufft didn't acknowledge his point21:17
*** zoli__ has joined #baserock21:18
*** zoli__ has quit IRC21:23
*** persia_ has quit IRC21:23
*** persia has quit IRC21:23
*** persia has joined #baserock21:24
*** persia_ has joined #baserock21:24
*** persia has quit IRC21:24
*** persia has joined #baserock21:25
*** persia_ has joined #baserock21:25
*** persia_ has joined #baserock21:25
*** tiagogomes has joined #baserock21:28
*** zoli__ has joined #baserock21:30
*** edcragg has joined #baserock21:35
*** zoli__ has quit IRC21:51
*** persia has quit IRC22:23
*** persia_ has quit IRC22:23
*** persia_ has joined #baserock22:25
*** 21WACGOA9 has joined #baserock22:25
*** persia_ is now known as Guest1991422:25
*** Guest19914 has quit IRC22:30
*** Guest19914 has joined #baserock22:30
*** Guest19914 is now known as persia_22:30
*** 21WACGOA9 is now known as persia22:30
*** persia has joined #baserock22:31
*** edcragg has quit IRC22:53
*** edcragg has joined #baserock22:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!