| *** ctbruce has quit IRC | 02:32 | |
| *** paulwaters_ has joined #baserock | 08:25 | |
| paulsherwood | now i find that 'pip install -U pip' on AWS AMI actually breaks pip :/ | 11:19 |
|---|---|---|
| paulsherwood | also http://paste.baserock.org/vaxohobupu shows master ybd on master definitions ci.morph appears to be broken | 11:24 |
| paulsherwood | build also broken for master definitions with last ybd tag (16.42) | 11:32 |
| paulsherwood | build *not* broken for last definitions tag (GENIVI-M1.0) with last ybd tag... | 11:55 |
| paulsherwood | (but deploy is broken... AttributeError: 'MultiFS' object has no attribute 'addfs') | 11:58 |
| jjardon | paulsherwood: master of definitions build fine in the CI; any idea how can we reproduce those problems? | 12:56 |
| jjardon | the MultiFS error in the latest ybf tag; thats probably because you are pulling current fs python package in your requirements.txt, wich have different api now: thats why we are using requirement.freeze.txt now, so we can reproduce build with the exac dependencies | 12:58 |
| * jjardon fires a GTK+ build | 12:59 | |
| jjardon | about deployment: this would help to detect that earlier ... https://gitlab.com/baserock/ybd/merge_requests/262 | 13:04 |
| jjardon | paulsherwood: mmm, strange, gtk+2 built fine in the CI: https://gitlab.com/baserock/definitions/builds/9461927 | 13:39 |
| paulsherwood | jjardon: does the gitlab ci build everything in ci.morph? | 18:14 |
| paulsherwood | jjardon: scarily, the gtk+ you are building has a different cache-key | 18:28 |
| paulsherwood | which suggests we have some overlapping dependencies in ci.morph | 18:28 |
| paulsherwood | or some other weirdness | 18:28 |
| jjardon | paulsherwood: it builds the same gtk+2; | 18:29 |
| jjardon | and it builds more systems, I think | 18:29 |
| paulsherwood | jjardon: different. you have gtk+.c660e2a005273921bc9044f5b4337931591d8c4988caf7eae10158dc9d4fe234 | 18:30 |
| paulsherwood | jjardon: my build has gtk+.fcfcb178b0ee81aca5b0ce9962e896dc7ca7b81b13d3a6bf5c219484ef04f449 | 18:30 |
| jjardon | I meant the same if you ook to definitions | 18:30 |
| jjardon | look* | 18:30 |
| jjardon | Its exactly the same sha and build instructions to build gtk+-2 | 18:31 |
| paulsherwood | yes. i'm saying that there must be two different possible paths to dependencies for it | 18:31 |
| paulsherwood | (or a bug in ybd, of course) | 18:31 |
| jjardon | ok, let me build cluster/ci.morph in the CI and see if we can reproduce | 18:34 |
| paulsherwood | well, i can reproduce your cache-key by building just the gnome system | 18:35 |
| paulsherwood | let's go back to the beginnign | 18:40 |
| paulsherwood | for some reason, an upgrade to pip was added to install_dependencies.sh ... | 18:41 |
| paulsherwood | when i run that on AWS, it breaks pip http://paste.fedoraproject.org/540015/15237148/ | 18:41 |
| jjardon | a modern version of pip is needed, or fs python package will not build | 18:43 |
| jjardon | (that's the problem you were having the other day) | 18:44 |
| paulsherwood | right | 18:44 |
| jjardon | one solution is to not install python-pip at all, and use get-pip.py as before; but then we can break systems that already have python-pip installed | 18:45 |
| paulsherwood | jjardon: were there actual examples of that breakage? | 18:52 |
| jjardon | so, we can revert bfb264cf872d7962a99db0db6fcd0501c139532c, but then we will not support system with a very old version of python-pip already installed with the system package manager (like debian stable) | 18:53 |
| paulsherwood | my problem is that what's been done since the 16.42 tag has definitely broken the install for me | 18:53 |
| jjardon | yeah, because what I said before: we install a very old version of python-pip and then it doesnt get upgrade, so the python package fs fails to install | 18:53 |
| jjardon | paulsherwood: can you check if this branch makes things work again in aws? https://gitlab.com/baserock/ybd/merge_requests/292 | 19:00 |
| paulsherwood | well, i guess the problem is we're running on different environments. the changes introduced since 16.42 work for the ones you use, but break both of mine. | 19:00 |
| paulsherwood | jjardon: that would work, because it effectively puts install_dependencies.sh back to 16.42 - but presumably that breaks things for other setups? | 19:02 |
| jjardon | it would break for setups that already have pip but its not new enough, yes | 19:03 |
| jjardon | it will not break the ci or anything we are currently using, so I think lets go ahead with the revert so at least we do not break yours :) (please do a quick test first) | 19:05 |
| jjardon | paulsherwood: ^ | 19:05 |
| * paulsherwood is thinking | 19:05 | |
| jjardon | paulsherwood: https://gitlab.com/baserock/definitions/builds/9469106 no problems building gtk+.fcfcb178b0ee81aca5b0ce9962e896dc7ca7b81b13d3a6bf5c219484ef04f449 | 19:07 |
| paulsherwood | great. so it looks like all my problems are ybd setup issues | 19:10 |
| paulsherwood | jjardon: so is python:2.7-slim ubuntu, or debian? | 19:10 |
| jjardon | paulsherwood: debian: https://github.com/docker-library/python/blob/master/2.7/slim/Dockerfile#L1 | 19:12 |
| jjardon | but It installs latest pip https://github.com/docker-library/python/blob/master/2.7/slim/Dockerfile#L22 | 19:13 |
| jjardon | paulsherwood: we can also make the ci test debian / fedora / ubuntu, so we will detect this next time | 19:17 |
| jjardon | I will submit a MR | 19:17 |
| paulsherwood | jjardon: i'm reluctant to revert if you're confident this works for the other distros | 19:19 |
| paulsherwood | the aws ami is clearly 'unusual' | 19:19 |
| jjardon | As its now, debian stable is currently broken (because it install an old python-pip but then it doesnt upgrade it), so I'm ok with the revert | 19:21 |
| jjardon | then we will add checks for several distros and we can be confident about future changes | 19:22 |
| paulsherwood | jjardon: you mean that the pip -U doesn't work for debian stable? | 19:22 |
| jjardon | correct | 19:22 |
| paulsherwood | are you confident that the revert won't break the ci? | 19:24 |
| paulsherwood | jjardon: the revert *doesn't* fix my situation, sadly | 19:30 |
| paulsherwood | http://paste.baserock.org/tuhavaheya | 19:31 |
| jjardon | yes: https://gitlab.com/baserock/ybd/merge_requests/292 can not be merged if the CI doesn't succeed | 19:31 |
| jjardon | paulsherwood: oh! thats a new one! :) | 19:32 |
| paulsherwood | indeed :) | 19:32 |
| * paulsherwood is expert at tripping over new ways to break software | 19:33 | |
| paulsherwood | jjardon: i suggest cancel that MR for now | 19:34 |
| paulsherwood | weirder still - the upgraded pip seems to be available for non-root, missing for root | 19:39 |
| jjardon | paulsherwood: debian stable failing to upgrade pip: https://gitlab.com/baserock/ybd/builds/9470584 | 19:40 |
| paulsherwood | ack | 19:42 |
| paulsherwood | i've cancelled that mr. given the odd behaviour of aws ami i'm going to switch to ubuntu from now | 20:09 |
| *** lachlan has joined #baserock | 20:56 | |
| jjardon | paulsherwood: any idea why this is happening : https://gitlab.com/baserock/ybd/builds/9481013 (only happen when I try to build in fedora:25 image !) | 22:01 |
| jjardon | https://gitlab.com/baserock/ybd/pipelines/6092446 | 22:02 |
| *** lachlan has quit IRC | 22:04 | |
| paulsherwood | no idea, but https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/1394 | 22:20 |
| paulsherwood | ok i have a better idea | 22:28 |
| paulsherwood | jjardon: please retry with https://gitlab.com/baserock/ybd/tree/ps-fresh-tmpdir | 22:32 |
| jjardon | paulsherwood: thanks, lets see: https://gitlab.com/baserock/ybd/pipelines/6092956 | 22:36 |
| jjardon | paulsherwood: can you review https://gitlab.com/baserock/ybd/merge_requests/293 and https://gitlab.com/baserock/ybd/merge_requests/294 when you have a second? | 22:37 |
| paulsherwood | i already approved them :) | 22:38 |
| jjardon | thanks! :) also https://gitlab.com/baserock/ybd/merge_requests/262 ? | 22:38 |
| paulsherwood | i can't rebase that, for some reason | 22:40 |
| jjardon | I've done it manually | 22:42 |
| jjardon | I think it's a problem with the new gitlab version; rebase through the UI used to work fine | 22:43 |
| paulsherwood | jjardon: https://gitlab.com/baserock/ybd/builds/9482675 looks happier :) | 22:44 |
| paulsherwood | until it fell over :/ | 22:45 |
| jjardon | paulsherwood: nice! seems puting a lot of distros in the ci already found a bug! | 22:45 |
| jjardon | paulsherwood: seems fedora doesnt comes with the find utility by default; I will send a fix for that | 22:46 |
| paulsherwood | cool :) | 22:47 |
| * paulsherwood hits the sack | 23:04 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!