*** rdale_ has joined #baserock | 02:31 | |
*** rdale has quit IRC | 02:34 | |
*** zoli__ has joined #baserock | 03:16 | |
*** zoli__ has quit IRC | 03:28 | |
*** zoli__ has joined #baserock | 03:29 | |
*** zoli__ has joined #baserock | 03:41 | |
*** zoli___ has joined #baserock | 03:55 | |
*** zoli__ has quit IRC | 03:59 | |
*** zoli___ has quit IRC | 04:48 | |
*** zoli__ has joined #baserock | 04:48 | |
*** zoli__ has quit IRC | 06:38 | |
*** zoli__ has joined #baserock | 07:01 | |
*** CTtpollard has joined #baserock | 07:52 | |
*** mariaderidder has joined #baserock | 07:54 | |
* straycat finds https://gerrit-review.googlesource.com/#/c/58322/ | 07:58 | |
*** mdunford has joined #baserock | 08:01 | |
*** bashrc has joined #baserock | 08:08 | |
*** gary_perkins has joined #baserock | 08:10 | |
jjardon | paulsherwood: not really, I have only compiled it a couple of times. But I know the main developer if you have any specific question | 08:13 |
---|---|---|
*** ssam2 has joined #baserock | 08:18 | |
*** ChanServ sets mode: +v ssam2 | 08:18 | |
* SotK notices that both Masons are broken again :( | 08:18 | |
SotK | x86_64 is out of disk space | 08:18 |
SotK | I can't tell whats up with x86_32, but its failing with "ERROR: Too many links" when it tries to build anything | 08:19 |
ssam2 | i'll add your key to both of them | 08:19 |
SotK | ssam2: thanks | 08:19 |
ssam2 | ok, you should be able to log in as root to them on their internal IPs | 08:22 |
SotK | ssam2: doesn't seem to have worked, I'm getting a password prompt | 08:26 |
ssam2 | are you logged into the devel VM with `ssh -A` ? | 08:26 |
SotK | yep | 08:27 |
ssam2 | hmm, ok | 08:27 |
*** jonathanmaw has joined #baserock | 08:35 | |
*** ssam2 has quit IRC | 08:36 | |
*** ssam2 has joined #baserock | 08:44 | |
*** ChanServ sets mode: +v ssam2 | 08:44 | |
*** edcragg has joined #baserock | 08:52 | |
*** mariaderidder has quit IRC | 09:06 | |
*** franred has joined #baserock | 09:18 | |
*** lachlanmackenzie has joined #baserock | 09:20 | |
*** mariaderidder has joined #baserock | 09:21 | |
ssam2 | jmacs: I've not forgotten about your cloud system, there's a network issue that seems to be stopping it working, so i'm waiting on a support ticket | 09:24 |
radiofree | paulsherwood: i can't reproduce the gcc issue on aarch64 anymore | 09:25 |
radiofree | tried about 10 times now, so i'm going to say "ybd works on aarch64!" | 09:25 |
rjek | :) | 09:26 |
*** ssam2_ has joined #baserock | 09:26 | |
radiofree | of course, the next time someone uses ybd to build on aarch64 they'll come across it... | 09:26 |
jmacs | ssam2: It's not urgent. | 09:27 |
*** ssam2 has quit IRC | 09:28 | |
nowster | paulsherwood: It's not seeing "pbr"'s hooks, so /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'pbr' | 09:37 |
nowster | and then the package name is UNKNOWN | 09:37 |
straycat | ooi, what version of python? | 09:40 |
paulsherwood | radiofree: w00t! | 09:41 |
paulsherwood | ssam2_: is nowster's error any help for diagnosing what's up? | 09:42 |
nowster | straycat: Python 2.7.9 (default, Feb 10 2015, 04:30:22) | 09:43 |
ssam2_ | nowster, paulsherwood: I've no idea what's up, I'm afraid | 09:43 |
nowster | my guess is that the hooks that pbr is supposed to install aren't getting called | 09:44 |
ssam2_ | maybe the setup.py isn't using pbr correctly... | 09:44 |
ssam2_ | might be worth trying to run setup.py from a different package that also uses pbr | 09:44 |
ssam2_ | most of the openstack packages use it | 09:44 |
ssam2_ | https://git.openstack.org/cgit/openstack-infra/zuul/tree/ for example -- that's a small package, maybe you could try installing that from Git and see if it works | 09:45 |
ssam2_ | if it does, probably sandboxlib is doing something wrong | 09:45 |
ssam2_ | if not, probably pbr is broken on your system somehow | 09:45 |
* SotK has seen that error before, it was because the installed version of pbr was too old when I saw it IIRC | 09:46 | |
ssam2_ | ah | 09:47 |
ssam2_ | maybe a requirements.txt would help | 09:47 |
nowster | ssam2_: zuul gives same error. | 09:48 |
nowster | SotK: what is the ver of pbr that's needed? | 09:48 |
SotK | nowster: I don't remember, what version do you have installed? | 09:48 |
nowster | It claims to be 1.0 | 09:49 |
ssam2_ | I have 0.10.8 | 09:50 |
ssam2_ | but that version works | 09:50 |
SotK | hm, 0.9.0 looks to be the version I had to update to when I was trying to build Zuul last year to fix that issue | 09:50 |
franred | 0.10.7 in Baserock In Openstack | 09:50 |
franred | s/ Baserock In Openstack/ Openstack in Baserock/ | 09:51 |
paulsherwood | ssam2_: i guess this means that sandboxlib has introduced an implicity dependency on pbr? | 09:53 |
ssam2_ | paulsherwood: only at install-time | 09:53 |
paulsherwood | i take that as a 'yes' :) | 09:53 |
straycat | nowster, i don't know what system you're running, but i just upgraded the pbr in this virtualenv to the latest and i can still create the egg-info for sandboxlib | 09:53 |
paulsherwood | why does sanboxlib need pbr? | 09:54 |
ssam2_ | paulsherwood: yeah. I'm just bristling slightly at calling it 'implicit' I think :) | 09:54 |
ssam2_ | paulsherwood: because setup.py files are a horrid mess | 09:54 |
paulsherwood | ssam2_: it's implicit from ybd's point of view. ybd doesn't need it | 09:54 |
ssam2_ | paulsherwood: I can't really understand your use of the word 'implicit' | 09:55 |
ssam2_ | paulsherwood: if ybd did need pbr, it wouldn't make sandboxlib's dependency on pbr 'explicit' | 09:55 |
straycat | ybd isn't an installable python distribution anyway, if you wanted to make it one it would be a good idea to use pbr | 09:56 |
ssam2_ | pbr is nice because it allows you to write a declarative setup.cfg file to express the metadata and install instructions for the Python library | 09:56 |
paulsherwood | ssam2_: i'm misuusing the word then. my point is ybd doesn't need pbr. sandboxlib does. | 09:56 |
nowster | eg. baserock needs baserock to build, as morph assumes it's running on baserock. | 09:56 |
ssam2_ | indeed. without pbr, you have to write the metadata and install instructions as Python code | 09:56 |
ssam2_ | which I don't really like | 09:56 |
paulsherwood | i think we have a philosophical difference, here. what's the simplest way for ybd to have the functionality of sandboxlib without pbr? | 09:58 |
ssam2_ | 'pip install pbr' | 09:58 |
ssam2_ | no, waiot | 09:58 |
ssam2_ | 'pip install sandboxlib' I mean | 09:58 |
nowster | no pip | 09:58 |
paulsherwood | or, put another way is there a way for ybd to use sandboxlib without *installing* sandboxlib? | 09:59 |
ssam2_ | yeah, just set 'PYTHONPATH=path-to-sandboxlib' and run it from the source tree | 09:59 |
paulsherwood | nowster: ^^ | 09:59 |
paulsherwood | a little ugly, but may work :) | 09:59 |
ssam2_ | good thinking | 09:59 |
nowster | that does seem to work. | 10:00 |
* nowster gets coffee and thinks a bit. | 10:00 | |
paulsherwood | ssam2_: how would you express my point about sndboxlib introducing pbr as a dependecnty to ybd, without ybd realising it? | 10:00 |
jmacs | Interesting; some of our build process leaked into the gcc build results. /usr/lib/jvm/bin/java -> /gcc.build/oesult/gij | 10:01 |
paulsherwood | jmacs: morph, or ybd? | 10:01 |
jmacs | paulsherwood: morph. I suppose this would be a good time to try ybd again | 10:01 |
* paulsherwood smiles :-) | 10:01 | |
jmacs | I think the problem will be in gcc, though | 10:02 |
paulsherwood | jmacs: however, i think ybd's sandboxing functionality is equivalent to morph's (albeit in the separated sandboxlib library) so the result may be the same | 10:02 |
ssam2_ | paulsherwood: I think i'd express it as you just did there | 10:03 |
ssam2_ | I can't think of a word that describes that situation more succinctly | 10:04 |
* paulsherwood still believes implicit conveys the point | 10:04 | |
straycat | I seriously don't know why you wouldn't have ybd use pbr, so you can install it with pip, I mean if you're trying to be accessible and all that | 10:04 |
paulsherwood | straycat: because i have already one use-case where it doesn't work? ie nowster's situation above? | 10:05 |
straycat | Mkay | 10:05 |
ssam2_ | paulsherwood: ybd doesn't work on a system where I've broken python, either | 10:05 |
paulsherwood | lol | 10:05 |
straycat | Well, pbr seems to be good enough for Openstack... | 10:05 |
paulsherwood | straycat: my choices are my own. i don't think popularity is one of my criteria | 10:06 |
nowster | being able to bootstrap from a limited set of components does have its advantages | 10:07 |
paulsherwood | :) | 10:07 |
ssam2_ | it does, but refusing to use any technology unless it's old enough to be widely available has its downsides too | 10:07 |
robtaylor | paulsherwood: you could submodule sandboxlib in | 10:08 |
paulsherwood | nowster: please could you raise a github issue documenting what you hit, either on sandboxlib or ybd as you think appropriate? | 10:08 |
ssam2_ | if you'd experienced the pain of trying to use Python setuptools / distutils / distribute, you'd probably appreciate how nice pbr is | 10:09 |
paulsherwood | robtaylor: oh? how does that work, please? i'm still a python n00b | 10:09 |
paulsherwood | ssam2_: i'm not saying it's not nice... i'm just not ok that it's a possible point of failure/friction for a new user of ybd | 10:11 |
robtaylor | paulsherwood: googling gives me http://stackoverflow.com/questions/29746958/how-to-import-python-file-from-git-submodule | 10:11 |
paulsherwood | robtaylor: i see what you did there :) | 10:11 |
robtaylor | heh | 10:11 |
ssam2_ | paulsherwood: fair enough. probably adding it as a submodule is a good plan, then | 10:12 |
paulsherwood | i think i'm against using git-submodule just for this, though | 10:12 |
ssam2_ | heh | 10:12 |
DavePage | Has the problem with git submodules and troves been addressed? | 10:13 |
straycat | paulsherwood, packagers like yourself are the reason I have such a difficult time with the import tool :) | 10:13 |
paulsherwood | i don't understand git-submodule properly either, but implementing it for ybd it seemed like a kludge | 10:13 |
ssam2_ | I've also had issues with setup.py files that use setuptools, so I don't think removing pbr and using the older 'setuptools' or 'distutils' approaches would be an improvement | 10:13 |
straycat | paulsherwood, the proper way to package ybd is as ssam2_ has done for sandboxlib | 10:13 |
paulsherwood | straycat: i don't doubt it. and if the pbr issue gets nailed, i'll be happy to use it packaged | 10:14 |
franred | paulsherwood, so you don't want to install sandbox (avoid to install pbr) and you don't want to make sandbox as a submodule, so are you going to add instructions to clone sandbox and set PYTHONPATH, does that no cause more friction than anything else? | 10:15 |
paulsherwood | incidentally, note that all i'm doing is trying to solve the *user* issue in a bulletproof way here. i don't know why there's so much pushback | 10:15 |
paulsherwood | franred: it's a fair question. i might settle for a ybd.sh script that sets pythonpath or something | 10:16 |
paulsherwood | i just want something simple that works all the time | 10:16 |
paulsherwood | franred: the friction in this current example is that nowster tried ybd yesterday, and it took til today to get to a solution | 10:18 |
franred | paulsherwood, in requirements.txt in sandbox I imagine ssam2_ can add which version of pbr is required | 10:19 |
*** ssam2_ has quit IRC | 10:19 | |
franred | that would be the cleanest/more elegant solution, IMHO | 10:19 |
nowster | I think its more that my pbr isn't correctly installed. | 10:20 |
nowster | It's supposed to add hooks, and hasn't. | 10:20 |
nowster | 2015-06-16 10:21:50 [systems/minimal-system-mips64b-openwrt.morph] ERROR: not a git repo /src/workspace/openwrt/github/nowster/definitions | 10:22 |
straycat | nowster, you could also just remove pbr from your system, btw | 10:22 |
paulsherwood | nowster: is that true? | 10:23 |
nowster | it's not | 10:23 |
straycat | ? | 10:23 |
paulsherwood | are you running ybd in that directory? | 10:23 |
nowster | no | 10:23 |
nowster | It is its current directory | 10:23 |
paulsherwood | that's where you need to be | 10:23 |
nowster | /src/workspace/openwrt/github/nowster/definitions # /src/ybd/ybd.py systems/mini | 10:23 |
nowster | mal-system-mips64b-openwrt.morph | 10:23 |
* paulsherwood is confused. is /src/workspace/openwrt/github/nowster/definitions a git checkout, and is that your cwd? | 10:24 | |
nowster | yesa | 10:24 |
paulsherwood | hmmm. | 10:24 |
paulsherwood | i assume you have git on your system? | 10:25 |
nowster | would it matter if it's not clean | 10:25 |
paulsherwood | no, it shoudl be fine | 10:25 |
nowster | # git --version | 10:25 |
nowster | git version 2.1.3 | 10:25 |
straycat | nowster, if you remove pbr from your system, setuptools will install it in the distribution's directory before attempting to install sandboxlib | 10:25 |
straycat | you could also probably just upgrade it | 10:25 |
straycat | or you could do all this in a virtenv | 10:25 |
straycat | or in Baserock | 10:25 |
nowster | straycat: will need to rebuild that baserock image to do that... | 10:26 |
straycat | this is way too hard, i'm trying to be helpful | 10:26 |
*** straycat has left #baserock | 10:26 | |
nowster | a full rebuild is not quick... we're talking about 36 hours | 10:26 |
rjek | Trove question: if an upstream git repo remove, say, a tag, and lorry repulls or has to reclone, do we lose that tag? Will any information ever get GCed? | 10:27 |
paulsherwood | nowster: can you try 'git describe' there please? | 10:27 |
rjek | My concern stems from GPL requirements of maintaining an offer of source code for two years from shipping a binary, and the risk of the exact code being shipped eventually vanishing. | 10:27 |
nowster | # git describe | 10:28 |
nowster | fatal: No names found, cannot describe anything. | 10:28 |
paulsherwood | nowster: that's what ybd does to test for git repo-ness | 10:28 |
nowster | there are no tags in that repo | 10:28 |
paulsherwood | i guess that's a bug | 10:28 |
nowster | # git describe --all | 10:28 |
nowster | heads/openwrt | 10:28 |
paulsherwood | well spotted, sir | 10:29 |
*** franred has quit IRC | 10:29 | |
paulsherwood | do you want to patch it and offer a pr? | 10:29 |
nowster | OK. | 10:29 |
paulsherwood | repos.py line 90 | 10:29 |
*** zoli__ has quit IRC | 10:31 | |
nowster | not line 68? | 10:32 |
nowster | it's in app.py | 10:33 |
paulsherwood | oops, sorry | 10:37 |
paulsherwood | you're better placed to work it out i think :) | 10:38 |
nowster | paulsherwood: you have a pull request :) | 10:38 |
paulsherwood | does it work, nowster ? | 10:38 |
nowster | yes | 10:38 |
nowster | Lots of warnings | 10:38 |
nowster | And it says "finished" probably because all the bits are in the cache. | 10:39 |
paulsherwood | nowster: really? how could they be? | 10:39 |
nowster | I'd already built it with morph. | 10:40 |
paulsherwood | morph and ybd do noy share cache-keys | 10:40 |
paulsherwood | what arch are you building for? | 10:40 |
nowster | mips64 | 10:40 |
Kinnison | mipsel64 or mipseb64 ? | 10:40 |
nowster | 2015-06-16 10:39:01 [systems/minimal-system-mips64b-openwrt.morph] Skipping assembly for mips64b | 10:40 |
Kinnison | b | 10:41 |
Kinnison | :) | 10:41 |
SotK | nowster: sounds like the arch auto-detection doesn't work there either | 10:41 |
nowster | yep | 10:41 |
SotK | try `ybd $system mips64b` | 10:41 |
nowster | I suspected it might need that changing too | 10:41 |
nowster | it's now building | 10:42 |
nowster | and it's failed | 10:42 |
nowster | Doing things in wrong order. | 10:43 |
nowster | It's gone: minimal-system-mips64b-openwrt -> build-essential -> glibc -> stage2-fake-bash | 10:44 |
nowster | ...and ignored all the stage1 stuff. | 10:44 |
Kinnison | very odd | 10:44 |
SotK | I guess that stage2-fake-bash doesn't have any build-dependencies? | 10:44 |
nowster | nope, it doesn't | 10:45 |
Kinnison | aah | 10:45 |
Kinnison | because it's essentially a shell script only | 10:45 |
SotK | can you paste the log of the failure? | 10:45 |
*** ssam2 has joined #baserock | 10:46 | |
*** ChanServ sets mode: +v ssam2 | 10:46 | |
nowster | It probably needs to depend on stage2-busybox. | 10:46 |
paulsherwood | nowster: ybd does randomised build-order where it can | 10:46 |
SotK | nowster: thats the "right" order then, since ybd just recursively tries building things until it gets to something for which the dependencies are satisfied | 10:46 |
SotK | nowster: a bug in definitions then I guess :) | 10:46 |
paulsherwood | :) | 10:46 |
*** franred has joined #baserock | 10:46 | |
* nowster tries adding a build-depends | 10:47 | |
paulsherwood | so in an exciting game, the score is currently nowster 2, ybd 1 :-) | 10:47 |
nowster | now it's building stage1-binutils, which is what i'd expect | 10:48 |
nowster | but it's fauled | 10:48 |
nowster | clone: Invalid argument | 10:48 |
paulsherwood | ouch! | 10:48 |
Kinnison | nowster: sounds like your kernel is missing stuff, perhaps namespaces? | 10:49 |
nowster | Not sure. Might be. Is that essential for ybd? | 10:49 |
ssam2 | nowster: yes, it could be that you have linux-user-chroot installed but some of the namespacing features in the kernel are missing | 10:49 |
paulsherwood | nowster: please raise that as an issue before we forget? | 10:49 |
ssam2 | sandboxlib does some autodetection of the backend, but it doesn't look at kernel config right now, it just looks at whether the linux-user-chroot binary is in the PATH | 10:50 |
nowster | paulsherwood: which one? the namespace problem or the build-depends one? | 10:50 |
paulsherwood | ssam2: does the sandboxlib chroot option work? | 10:50 |
ssam2 | paulsherwood: I've verified that it works for me. No idea if it will work for nowster :) | 10:50 |
paulsherwood | nowster: namespace, unless you believe the build-depends is a bug in ybd? | 10:50 |
paulsherwood | ssam2: is there a way to force the chroot otion, even if linux-user-chroot is present? | 10:51 |
ssam2 | quickest way to try out the chroot backend is 'hide' the linux-user-chroot binary. | 10:51 |
ssam2 | probably sandboxlib should honour a SANDBOXLIB_BACKEND env var in its autodetection code to allow overriding it | 10:51 |
jmacs | Are we still keen on using Storyboard? The wiki is inconsistent about it. | 10:52 |
ssam2 | I think we are inconsistent about it | 10:52 |
ssam2 | I would like to use it but it's not good enouigh | 10:52 |
ssam2 | but neither is anything else :) | 10:52 |
richard_maw | ssam2: perhaps sandboxlib should, on first use, check whether linux-user-chroot works, and flag that it has to use chroot otherwise | 10:52 |
paulsherwood | i think we are considering commiting engineering to improve storyboard | 10:52 |
paulsherwood | richard_maw: that would be lovely :) | 10:53 |
nowster | linux-user-chroot works here, AFAICT | 10:53 |
Kinnison | could it be the unshare call for mount namespaces failing? | 10:53 |
ssam2 | nowster: define 'works' | 10:53 |
nowster | morph uses it | 10:53 |
nowster | however, # CONFIG_NAMESPACES is not set | 10:54 |
jmacs | OK, I'll update the wiki to say you can use either storyboard or the mailing list, if that's OK. | 10:54 |
ssam2 | jmacs: that's fine | 10:54 |
ssam2 | nowster: interesting. could you try `linux-user-chroot --unshare-net / true` and see if you get the same error? | 10:54 |
nowster | I do get same error | 10:55 |
richard_maw | linux-user-chroot is not secure for containerising without mount namespaces, which are enabled with CONFIG_NAMESPACES | 10:55 |
* nowster goes to recompile kernel | 10:56 | |
ssam2 | nowster: you could also hide linux-user-chroot so the chroot() backend of sandboxlib gets used :) | 10:56 |
nowster | ssam2: but it'll build linux-user-chroot, won't it? | 10:57 |
ssam2 | nowster: building linux-user-chroot isn't the problem. the problem is that linux-user-chroot can't run | 10:57 |
richard_maw | nowster: it will, but morph uses the linux-user-chroot provided by the host environment | 10:57 |
nowster | morph builds on this machine without namespaces | 10:58 |
nowster | ...and it uses linux-user-chroot | 10:58 |
ssam2 | nowster: that's really bizarre, as Morph unconditionally runs builds through linux-user-chroot | 10:58 |
paulsherwood | ssam2: not for build-essential? | 10:58 |
ssam2 | paulsherwood: for build-essential it does too | 10:58 |
paulsherwood | hmmm. i don't know, then | 10:59 |
nowster | I've been building stuff with morph on that kernel config for about 4 months. | 10:59 |
ssam2 | is there any way I could have access to the machine we're talking about so I could satisfy my curiousity about what is broken? | 11:00 |
nowster | root@bono | 11:00 |
paulsherwood | nowster: could you paste the bottom of the broken log in that pr please? | 11:01 |
nowster | paulsherwood: done | 11:02 |
ssam2 | ah! it's the --unshare-net flag that triggers the error | 11:02 |
paulsherwood | tvm | 11:02 |
ssam2 | morph doesn't pass that flag (at least, not the version you are using), while sandboxlib does | 11:02 |
* nowster remembers to wrap the log in markdown quotes | 11:03 | |
ssam2 | I always thought Morph built in a chroot which was network isolated, but seems not | 11:05 |
paulsherwood | nowster: have you tried hiding linux-user-chroot? | 11:07 |
nowster | paulsherwood: not yet. Currently recompiling kernel. | 11:07 |
nowster | mv /usr/bin/linux-user-chroo | 11:08 |
nowster | t /usr/bin/linux-user-chroot.gone | 11:08 |
nowster | hmm... | 11:10 |
nowster | checking target system type... Invalid configuration `mips64b-bootstrap-linux-gnu': machine `mips64b-bootstrap' not recognized | 11:10 |
nowster | configure: error: /bin/bash ./config.sub mips64b-bootstrap-linux-gnu failed | 11:10 |
nowster | it needs some arch mangling. | 11:10 |
nowster | it needs to be told that TARGET_STAGE1 is mips64 in this situation | 11:11 |
nowster | arch_dict needs tweaking | 11:12 |
nowster | tries again | 11:13 |
*** zoli__ has joined #baserock | 11:14 | |
nowster | now appears to be running make | 11:16 |
ssam2 | I've filed https://github.com/CodethinkLabs/sandboxlib/issues/3 and https://github.com/CodethinkLabs/sandboxlib/issues/4 | 11:17 |
ssam2 | have created https://github.com/CodethinkLabs/sandboxlib/issues/5 for the pbr issue as well | 11:24 |
nowster | 2015-06-16 11:41:04 [stage1-binutils] Elapsed time 00:26:45 | 11:44 |
nowster | :( | 11:44 |
nowster | sorry... :) | 11:44 |
nowster | (would have been faster if I weren't making a kernel at same time) | 11:44 |
paulsherwood | that's 26 times slower than my laptop! | 11:48 |
*** mdunford has quit IRC | 11:57 | |
paulsherwood | nowster: do you have a pr for arch_dict? | 12:02 |
nowster | will do | 12:04 |
*** mdunford has joined #baserock | 12:11 | |
nowster | paulsherwood: there's another tweak so that it recognises the host machine arch correctly | 12:22 |
nowster | ...which I'll need to test in a minute | 12:22 |
*** fay_ has quit IRC | 12:32 | |
nowster | paulsherwood: pull request sent | 12:34 |
*** tiagogomes has quit IRC | 12:34 | |
*** fay_ has joined #baserock | 12:35 | |
*** tiagogomes has joined #baserock | 12:47 | |
nowster | 2015-06-16 13:01:47 [stage1-gcc] Elapsed time 01:47:29 .... so it looks like ybd is working. | 13:03 |
paulsherwood | :) | 13:06 |
radiofree | is git.baserock.org ever going to get a valid certificate? | 13:06 |
*** straycat has joined #baserock | 13:08 | |
paulsherwood | rjek: could you help with that? | 13:15 |
ssam2 | we have a cert, but i'm not sure what the best way is to make git.baserock.org use it | 13:15 |
rjek | I can recommend somewhere to obtain the certificate from, but can't help much with its installation | 13:15 |
ssam2 | radiofree: is it causing you problems? there is a long list of things which need to be fixed in the infra | 13:16 |
radiofree | ssam2: well i was recommending someone update their repos to use GBO over https, then remembered that will fail | 13:18 |
ssam2 | mm, true | 13:18 |
radiofree | of course there's http, but they really shouldn't have to use that | 13:18 |
rjek | +1 | 13:18 |
ssam2 | i've created https://storyboard.baserock.org/#!/story/49 to track this task | 13:19 |
ssam2 | if anyone knows what would be the "correct" way to do this for a Trove, please let me know. I've not got bandwidth to think about it right now, might have a think about it on friday | 13:20 |
rjek | Which http server do we use on Trove? | 13:21 |
Kinnison | lighttpd I believe | 13:21 |
ssam2 | yes | 13:22 |
ssam2 | presumably its config is generated by trove.configure | 13:23 |
rjek | OK, I was just checking it wasn't busybox httpd :) | 13:23 |
Kinnison | IIRC that doesn't do SSL at all | 13:23 |
* SotK thinks its config is in trove-setup.git | 13:23 | |
ssam2 | oh yeah, actually it's the ansible script that does it | 13:23 |
ssam2 | ok, so shouldn't be too horrible a task | 13:23 |
*** mdunford has quit IRC | 14:29 | |
*** mdunford has joined #baserock | 14:31 | |
jjardon | hi ssam2! about https://gerrit.baserock.org/#/c/853/ , not sure I'm adding more comments about the use of tarball-version? | 14:33 |
*** mdunford has quit IRC | 14:37 | |
*** mdunford has joined #baserock | 14:44 | |
radiofree | how would one go about contributing an ARM cache to the artefact cache server? | 14:44 |
radiofree | actually i probably should rebase this | 14:45 |
paulsherwood | radiofree: do you mean morph cache, or ybd cache? | 14:46 |
radiofree | erm.. morph cache, morph is still the official tool of baserock right? | 14:46 |
ssam2 | jjardon: oh, sorry | 14:47 |
ssam2 | i'm really confused by the diff then | 14:47 |
radiofree | it's for this genivi thing, i'm writing a guide (all over again because firefox crashed) and step one is "deploy a developement image to your jetson" | 14:47 |
paulsherwood | radiofree: i believe it is... but i'm hoping to propose ybd for 'official' recognition soon | 14:47 |
radiofree | so i do that, that's fine, but then i don't want them to have to sit through 5 hours of building Qt to get to the GDP | 14:47 |
paulsherwood | aha, understood | 14:47 |
radiofree | which would mean having the cache in the upstream cache server | 14:47 |
ssam2 | jjardon: oh, did those comments appear in the diff between 1 and 2 due to rebasing perhaps? | 14:47 |
jjardon | ssam2: I think so, yes | 14:48 |
ssam2 | oh, sorry | 14:48 |
paulsherwood | radiofree: this was previously achieved by an arm mason... i wonder if that can be re-established | 14:48 |
ssam2 | jjardon: i've undone by edit | 14:48 |
radiofree | paulsherwood: i seem to remember having this conversation.. weeks... months agao? | 14:49 |
radiofree | s/agao/ago | 14:49 |
radiofree | anyway, that wouldn't solve the Qt problem, since that's not on any system in the CI | 14:49 |
radiofree | and i wouldn't like to see how long it would take the x86 mason to build an image with Qt in | 14:49 |
radiofree | i'm happy to manually provide cache for this stuff (GDP) | 14:50 |
paulsherwood | makes sense | 14:50 |
ssam2 | radiofree: there's no process for that because nobody ever does it, but i trust you not to hose cache.baserock.org and can give you access to upload stuff to it | 14:51 |
ssam2 | or git.baserock.org, that's probably a better place | 14:51 |
jjardon | ssam2: thanks! | 14:51 |
radiofree | ssam2: what's the "official" cache server? cache or git? | 14:52 |
radiofree | (the one we document) | 14:52 |
ssam2 | morph defaults to git.baserock.org | 14:52 |
ssam2 | we document cache.baserock.org | 14:52 |
ssam2 | it's a mess, as usual | 14:52 |
radiofree | ah, nice, it automatically checks for cache on gbo? | 14:52 |
radiofree | but the CI puts the cache on cache? | 14:52 |
*** mdunford has quit IRC | 14:53 | |
radiofree | i shall rebuild it with a clean cache folder, since this jetson has 70G of artifacts | 14:54 |
SotK | you could use `morph list-artifacts` to just get the relevant artifacts | 14:55 |
franred | can someone spot what is going on in https://mason-x86-32.baserock.org/log/284aec89f5de6f111702065bacd72c31e5fd0c9b--2015-06-16%2009:16:50.log ? | 14:55 |
franred | I can't see an useful error | 14:56 |
radiofree | SotK: i had no idea it did that | 14:56 |
richard_maw | franred: could be failing to transfer the artifact to the cache then | 14:56 |
radiofree | franred: out of space? | 14:56 |
radiofree | 2015-06-16 09:30:02 Progress: Transferring gf-complete-misc to shared artifact cache | 14:56 |
radiofree | 2015-06-16 09:30:13 Build of gf-complete failed. | 14:56 |
SotK | radiofree: it doesn't, but it gives you a list of the filenames | 14:56 |
SotK | franred: cache.b.o was full earlier IIRC | 14:57 |
franred | radiofree, richard_maw, SotK, ta | 14:57 |
ssam2 | radiofree: a good thing about putting them on git.baserock.org is they are unlikely to get deleted any time soon | 14:59 |
ssam2 | whereas, because cache.baserock.org gets every build of everything, we have to delete stuff quite often | 14:59 |
ssam2 | and there's currently no way of saying 'wait, those artifacts are important, keep them' | 14:59 |
radiofree | ah | 14:59 |
radiofree | ok | 14:59 |
radiofree | what happens if the cache is on gbo, but you set your cache server to cache.baserock.org | 14:59 |
radiofree | will it still check on gbo first? | 15:00 |
ssam2 | no | 15:00 |
ssam2 | sadly | 15:00 |
radiofree | well there's nothing populating arm cache on cache.baserock.org anyway, right? | 15:00 |
*** mdunford has joined #baserock | 15:00 | |
ssam2 | no# | 15:01 |
radiofree | ok, thanks | 15:02 |
persia | I seem to recall having an ARM Mason in the past: what is the current blocker for deployment? | 15:03 |
*** radiofree has quit IRC | 15:03 | |
richard_maw | pedroalvarez is probably the one to ask about that | 15:04 |
*** radiofree has joined #baserock | 15:04 | |
radiofree | persia: we did | 15:04 |
radiofree | and we should still have one, i thought we agreed it could go back to datacentred? | 15:05 |
ssam2 | persia: nothing is really blocking having a single Jetson that acts as a Mason, other than time to do it | 15:05 |
ssam2 | I have one on my desk, in fact, it's just not sharing what it builds publically (and it breaks sometimes) | 15:05 |
persia | Ah, indeed. Tuits are ever in short supply. | 15:05 |
*** mdunford has quit IRC | 15:06 | |
ssam2 | to have an ARM distbuild network, we are blocked by the fact that distbuild requires that the artifact cache needs to be able to contact each worker | 15:06 |
ssam2 | which conflicts with how the colo provider wants to set up the network for the Jetsons, if we put them back there | 15:06 |
ssam2 | which is solvable, but requires time to think it through | 15:07 |
*** mdunford has joined #baserock | 15:08 | |
persia | Would virtual distbuild workers solve that, or is it just a fundamental issue with deployment? | 15:09 |
jjardon | paulsherwood: hi, just trying ybd for the first time in a long time, but I get this error: http://paste.baserock.org/fapabazeqi | 15:12 |
radiofree | did i break that? | 15:12 |
ssam2 | persia: It depends on the network topology of the virtual distbuild workers | 15:13 |
radiofree | no, i don't think i did | 15:13 |
ssam2 | if they were in the same network as cache.baserock.org, it would make it easier to deploy them | 15:13 |
jjardon | paulsherwood: nevermind, I think I fixed it | 15:14 |
ssam2 | we are also a bit stuck that someone left all the power supplies for the jetsons in the colo center, so we can't test anything here | 15:15 |
radiofree | :| | 15:16 |
DavePage | There are a while bunch of Jetsons turned off in the server roo | 15:17 |
* rjek imagines a kangaroo waiter | 15:17 | |
De|ta | ssam2, we have bench PSUs. Shouldn't take much to connect a few Jetsons to one. From a quick look, we'd just need some 2.1mm barrel plugs and some wire. | 15:23 |
paulsherwood | jjardon: could you send a pull request if you've fixed that please? | 15:35 |
jjardon | paulsherwood: sure. BTW, is ybd meant to be running in a specific environment? seems it tries to write in /src | 15:37 |
jjardon | ah, just found ybd.def | 15:37 |
paulsherwood | it's defaulting to /src - you can change that in ybd.def i think | 15:37 |
jjardon | paulsherwood: I think it would be better to default to $XDG_CACHE_DIR/src . Would you be interested in a patch to implement that? | 15:39 |
* jjardon just remembers to sent a patch to do that a while ago | 15:40 | |
ssam2 | De|ta: that's an idea | 15:46 |
paulsherwood | jjardon: how would that work if $XDG_CACHE_DIR is not honoured? | 15:52 |
rdale_ | that's certainly linux specific | 16:00 |
ssam2 | rdale_: XDG_CACHE_DIR? it's a freedesktop standard I think, so not specific to Linux | 16:09 |
ssam2 | http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html | 16:10 |
rdale_ | yes, but i would be surprised if it was set in a mac os x environment | 16:10 |
ssam2 | yes, probably OS X, Windows and commercial UNIXes don't have it set | 16:10 |
ssam2 | so would need to fall back to ~ or somewhere if it's not set | 16:11 |
persia | At least AIX and Solaris should have $XDG_CACHE_DIR set. I don't know about other commercial UNIXes (e.g. OS X). | 16:13 |
persia | But the spec itself calls for having a fallback when unset. | 16:14 |
De|ta | 'echo $XDG_CACHE_DIR' returns nothing on my OS X machine | 16:15 |
persia | That is a strong indicator that OS X does not support it by default. | 16:15 |
paulsherwood | jjardon: if your patch leaves behaviour unchanged in the absence of XDG_CACHE_DIR i'd be happy to take it? | 16:16 |
rdale_ | some suggestions about mac os x equivalents of $XDG* here: http://stackoverflow.com/questions/3373948/equivalents-of-xdg-config-home-and-xdg-data-home-on-mac-os-x | 16:16 |
jjardon | paulsherwood:ok, I will try | 16:17 |
persia | Note that the spec itself seems to define the variable as $XDG_CACHE_HOME rather than $XDG_CACHE_DIR | 16:17 |
nowster | ybd went splat: http://paste.baserock.org/jiferuvuni | 16:17 |
paulsherwood | nowster: was this forked from definitions a while ago? i think there may have been a fix to fhs-dirs in the meantime | 16:18 |
nowster | ah, right. | 16:19 |
nowster | could be | 16:20 |
rdale_ | the fhs-dirs chunk is the same in the master of definitions as it is in our openwrt build-essential as far as i can see | 16:24 |
straycat | huzzah, ssam2's series is merged, so you can forget about workspaces now, if you like | 16:25 |
persia | Hurrah! | 16:25 |
nowster | The only difference I can see is the removal of $DESTDIR/var in the older one | 16:26 |
rdale_ | yes, nothing that would affect whether fhs-dirs.build is created | 16:27 |
nowster | I see no recent push to master for fhs-dirs | 16:28 |
nowster | could someone have made a fix to fhs-dirs recently and not pushed it back? | 16:32 |
rdale_ | it looks to me as though one thread created fhs-dirs.build and correctly ran the chunk, and then later another build thread came along expecting it to still be there | 16:32 |
ssam2 | straycat: which begs the question... when can we delete that code :) | 16:32 |
ssam2 | (`morph checkout`, `morph branch`, etc.( | 16:32 |
* Kinnison votes ASAP, in order to reduce confusion, but perhaps not until a document is written which explains "If you used to do X, you now do Y" | 16:33 | |
straycat | Yes, I guess we need to go through the wiki and update before removing that code | 16:33 |
ssam2 | indeed. in fact, we can start doing that now | 16:34 |
ssam2 | oh, we should wait for a release I guess | 16:34 |
ssam2 | the release SotK is doing won't include this change, so will be a bit of time | 16:34 |
straycat | other than that, if no one's actually using workspaces then yeah I guess we can remove all that code | 16:34 |
persia | rdale_: That sounds like my memory of the old problem. There was some recursion issue, which I thought was resolved by some other change to definitions. | 16:37 |
straycat | Hrm, it's a pity we can't do this sooner, or get this change into the release | 16:39 |
persia | Indeed. | 16:41 |
*** mariaderidder has quit IRC | 16:47 | |
nowster | It's happening in linux-api-headers now. | 16:58 |
nowster | (due to randomization of build order) | 16:58 |
rdale_ | perhaps it's a timing thing as the mips machine is slower than everything else | 16:59 |
*** jonathanmaw has quit IRC | 17:00 | |
nowster | I *suspect* it's the attempted containerization. | 17:00 |
* paulsherwood believes he's seen this before, but can't remember what the outcome was.... ssam2 ? | 17:05 | |
rdale_ | should we be using python3 for baserock in general? | 17:10 |
persia | Probably, but it's complicated, because some folk only have python 2.x on their systems. | 17:12 |
persia | https://pypi.python.org/pypi/nine might be a reasonable solution for that though. | 17:13 |
jjardon | persia: why, we use a chroot/baserock system to develop exactly for that reason, not depend on any specific version of anything | 17:14 |
persia | jjardon: Because the software should be resilient to multiple systems. That I can't use lorry without having a baserock system, which I can't usefully maintain as a snowflake, makes me not want to use lorry. | 17:15 |
persia | The same applies to any of the other tools. | 17:15 |
rdale_ | i'm just checking out ybd and the sandboxlib, and wondering which python to use - it sounds like i shouldn't use python3. but my version of tox is looking for that | 17:16 |
straycat | rdale_, yeah it's python 2 | 17:16 |
rdale_ | ok | 17:16 |
persia | rdale_: Write your code so that you can run in either environment. If you have issues, use six or (better) nine. | 17:16 |
jjardon | rdale_: can you not make the code compatible? | 17:16 |
rdale_ | i'm only trying to run other people's python | 17:17 |
persia | When that fails, report bugs :) | 17:17 |
* persia vaguely suspects https://github.com/devcurmudgeon/ybd/issues/50 is due to an assumption that python 2 is always available | 17:18 | |
rdale_ | this is the error i'm getting: http://paste.baserock.org/raqupiridi | 17:18 |
paulsherwood | persia: i'll need to find a system with only py3 on it to test that assumption. | 17:19 |
persia | paulsherwood: You are in luck: rdale is currently working with such a system :) | 17:19 |
rdale_ | not intentionally | 17:19 |
paulsherwood | that looks like sandboxlib specifically? | 17:20 |
persia | rdale_: Perhaps, but `python` appears to run `/usr/bin/python3`, which lets you test | 17:20 |
rdale_ | on my system python is 2.7.8. i installed a .deb called 'python-tox', not 'python3-tox' but it still seems to be python3 | 17:23 |
jjardon | paulsherwood: about the sandbox problem, you need to add python2 (or python3) to ybd.py: #!/usr/bin/env python2 | 17:23 |
jjardon | will send a patch if you are not faster :) | 17:23 |
persia | rdale_: Most modern deb-based systems default to python3 | 17:24 |
persia | jjardon: Can we not just use six or nine instead? | 17:24 |
ssam2 | rdale_: 'tox' is a test tool, it tries to test sandboxlib under both python2 and python3 to ensure it works under both | 17:25 |
*** mdunford has quit IRC | 17:26 | |
ssam2 | that error looks like you don't have 'setuptools' for python3 installed | 17:26 |
paulsherwood | jjardon: i can't patch here | 17:26 |
paulsherwood | i can merge, though | 17:26 |
rdale_ | ok, i'm really trying to use pbr - i have installed a .deb called python-pbr , but that doesn't seem to have a /usr/bin/pbr executable in it | 17:26 |
jjardon | persia: probably, but https://www.python.org/dev/peps/pep-0394/ recommends specify the version if the code is python2-only | 17:27 |
persia | Yes, but I believe it to be poor practice for python2-only code to exist | 17:28 |
persia | Especially given that for most code, it is relatively easy to make it work in either environment. | 17:28 |
ssam2 | rdale_: no idea about the Debian packaging, but I don't know if you need a 'pbr' executable -- it's a library that is imported by sandboxlib's ./setup.py | 17:30 |
straycat | rdale_, setuptools should install pbr for you if you don't have it on your system, since it's listed in setup_requires in the setup.py | 17:30 |
paulsherwood | jjardon: my hope is that ybd can be made to work with both 2 and 3 | 17:31 |
*** lachlanmackenzie has quit IRC | 17:32 | |
rdale_ | ah ok - i installed python3-setuptools and tox runs now, but i needed linux-user-chroot too. the tests aren't quite working | 17:32 |
rdale_ | i'm getting: AttributeError: 'NoneType' object has no attribute 'decode' | 17:32 |
jjardon | paulsherwood: nevermind, what is there seems to be correct | 17:33 |
rdale_ | here are my test errors: http://paste.baserock.org/abobedugot | 17:33 |
jjardon | paulsherwood: nice, because Im going to be using python3 here :) | 17:34 |
* paulsherwood crosses his fingers | 17:34 | |
rdale_ | i need the python equivalent of rbenv to keep my versions separate i think | 17:36 |
straycat | rdale_, so it's got a non-zero exit code but there's nothing on stderr, but sandboxlib is assuming there is | 17:36 |
straycat | tox looks awesome | 17:37 |
ssam2 | paulsherwood: I think that it did last week :) | 17:42 |
ssam2 | rdale_: the python equivalent of rbenv is virtualenv, which tox uses to create a clean test environment for running the tests in | 17:43 |
*** edcragg has quit IRC | 17:43 | |
ssam2 | you've clearly some mistakes of mine in the test suite, though () | 17:43 |
ssam2 | actually, seems like it's all the same error | 17:43 |
ssam2 | if you change 'stdout=None, stderr=None' to 'stdout=sandboxlib.CAPTURE, stderr=sandboxlib.CAPTURE' I think it'll show you the actual error, instead of crashing when it tries to show you what went wrong | 17:44 |
rdale_ | ah ok - i need to find out more about virtualenv. in ruby the .deb ruby versioning is a mess, and i don't use it at all. it looks as though .deb python versioning is too difficult to use | 17:44 |
ssam2 | in the bit of code that the error log is pointing to, I mean | 17:44 |
ssam2 | the test suite may be failing if your environment can't do everything that it expects to work. I didn't do anything about making it skip tests that can't succeed on a given machine | 17:46 |
ssam2 | well, I did a bit of work on that, but it didn't seem like that useful a thing to do | 17:46 |
rdale_ | ok thanks | 17:47 |
*** ssam2 has quit IRC | 17:50 | |
jjardon | paulsherwood: ybd is asking me to be root to start the assembly, is that expected? | 17:56 |
paulsherwood | jjardon: afraid so. sandboxing requires root so far | 17:57 |
paulsherwood | i was hoping we'd find a solution to that | 17:57 |
jjardon | thats ok, maybe we shold fix the "quick start" section in https://github.com/devcurmudgeon/ybd ? | 17:58 |
paulsherwood | jjardon: absolutely | 17:59 |
*** gary_perkins has quit IRC | 18:28 | |
juergbi | i expect this requirement could be avoided when creating a user namespace. i don't know whether that's already enabled on typical distro kernels these days | 18:53 |
juergbi | it could use userns if available and otherwise require root, i suppose | 18:54 |
*** zoli__ has quit IRC | 19:20 | |
*** tiagogomes has quit IRC | 19:59 | |
*** zoli__ has joined #baserock | 20:15 | |
*** zoli__ has quit IRC | 20:16 | |
*** zoli__ has joined #baserock | 20:18 | |
*** zoli__ has quit IRC | 20:20 | |
*** zoli___ has joined #baserock | 20:20 | |
*** zoli__ has joined #baserock | 20:21 | |
straycat | Bit late but finally got around to discussing our changes in pip with the maintainer | 20:58 |
straycat | He's of the opinion that the functionality I added doesn't belong in pip, and whilst acknowledging that this will make our tool harder to implement he suggests that we should either write the internals we need or pull pip's internals into the tool with the awareness that they are internal and may break over time. He also added that pip should probably try and pull more stuff out into separate libraries. | 21:00 |
straycat | So it goes | 21:00 |
paulsherwood | straycat: pls could you remind me, what functionality was this specifically? | 21:10 |
rjek | neither approach sounds delightful | 21:10 |
paulsherwood | agreed | 21:10 |
straycat | paulsherwood, https://github.com/pypa/pip/pull/2371 | 21:11 |
straycat | it adds a --list-dependencies option to pip | 21:11 |
paulsherwood | ok, thanks. i don't understand rbtcollins' last comment? | 21:13 |
straycat | yes i'm not sure about his point yet either, i'll check it out properly tomorrow | 21:14 |
paulsherwood | yup. it's late, there | 21:14 |
* persia believes lifeless to currently be in UTC+12 | 21:16 | |
persia | As a result, soon might be the best time to catch him, if one wants realtime discussion | 21:16 |
*** zoli__ has quit IRC | 21:17 | |
straycat | I think it can wait till I have a better understanding of his point, also dstufft didn't acknowledge his point | 21:17 |
*** zoli__ has joined #baserock | 21:18 | |
*** zoli__ has quit IRC | 21:23 | |
*** persia_ has quit IRC | 21:23 | |
*** persia has quit IRC | 21:23 | |
*** persia has joined #baserock | 21:24 | |
*** persia_ has joined #baserock | 21:24 | |
*** persia has quit IRC | 21:24 | |
*** persia has joined #baserock | 21:25 | |
*** persia_ has joined #baserock | 21:25 | |
*** persia_ has joined #baserock | 21:25 | |
*** tiagogomes has joined #baserock | 21:28 | |
*** zoli__ has joined #baserock | 21:30 | |
*** edcragg has joined #baserock | 21:35 | |
*** zoli__ has quit IRC | 21:51 | |
*** persia has quit IRC | 22:23 | |
*** persia_ has quit IRC | 22:23 | |
*** persia_ has joined #baserock | 22:25 | |
*** 21WACGOA9 has joined #baserock | 22:25 | |
*** persia_ is now known as Guest19914 | 22:25 | |
*** Guest19914 has quit IRC | 22:30 | |
*** Guest19914 has joined #baserock | 22:30 | |
*** Guest19914 is now known as persia_ | 22:30 | |
*** 21WACGOA9 is now known as persia | 22:30 | |
*** persia has joined #baserock | 22:31 | |
*** edcragg has quit IRC | 22:53 | |
*** edcragg has joined #baserock | 22:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!