| *** paulwaters_ has joined #baserock | 06:02 | |
| jjardon | paulsherwood: fyi, ybd master still fails: https://gitlab.com/baserock/definitions/builds/2537611#down-build-trace | 06:12 |
|---|---|---|
| *** toscalix has joined #baserock | 07:44 | |
| *** jonathanmaw has joined #baserock | 07:46 | |
| paulsherwood | :/ | 08:10 |
| *** edcragg has quit IRC | 08:12 | |
| *** edcragg has joined #baserock | 08:12 | |
| *** bwh__ has quit IRC | 08:13 | |
| *** bwh_ has joined #baserock | 08:13 | |
| richard_maw | that error is because pyfilesystem converted the path to unicode, and because the system calls take byte strings rather than unicode strings, decided it has to convert it back to a byte string, but for some reason decided it has to convert it to ascii | 08:34 |
| richard_maw | this is a different, but related problem, to there being file paths that aren't convertable to unicode | 08:34 |
| * richard_maw doesn't know how python decides what encoding to use, but if pyfilesystem had left the strings alone it would have been fine | 08:36 | |
| richard_maw | I *think* it may be related to what's set in the LANG and LANGUAGE environment variables, which would be unset in CI builds | 08:37 |
| richard_maw | so it might decide to fall back to ascii, since whoever specified python's behaviour here didn't understand file paths | 08:37 |
| richard_maw | http://stackoverflow.com/questions/2076708/python-os-stat-and-unicode-file-names | 08:39 |
| richard_maw | the recommendation is to re-encode it as a byte string before calling the syscall, but pyfilesystem doesn't do that | 08:39 |
| richard_maw | so… setting LANG=en_US.UTF-8 might be worth a try as a pragmatic solution, | 08:44 |
| *** rdale has joined #baserock | 08:46 | |
| richard_maw | even if it is caused by a fundamental misunderstanding of how file paths work, which ought to be fixed upstream | 08:47 |
| *** gtristan has joined #baserock | 09:01 | |
| *** locallycompact has joined #baserock | 09:15 | |
| * paulsherwood can't remember who has the superpowers to generate the pipelines at c.b.o | 09:45 | |
| paulsherwood | locallycompact: is that you? | 09:45 |
| paulsherwood | looks like we've dropped audio-bluetooth, but the pipeline has not been updated http://concourse.baserock.org/pipelines/genivi-demo-platform-x86_64-generic/jobs/audio-bluetooth/builds/34 | 09:46 |
| locallycompact | I don't know the aws details, mwilliams will | 09:48 |
| paulsherwood | does he still hang aroudn here? | 09:49 |
| CTtpollard | he'll be here in a sec | 09:50 |
| *** mwilliams_ct has joined #baserock | 09:50 | |
| mwilliams_ct | You rang? | 09:50 |
| * rjek wonders what the mwilliams summoning incantation is | 09:52 | |
| mwilliams_ct | ask tpollard to message me on another network. | 09:52 |
| mwilliams_ct | though he didnt give me much context and irclogs.br hasnt updated yet. Am I needed for something? | 09:53 |
| rjek | 10:45:25 -*- paulsherwood can't remember who has the superpowers to generate the pipelines at c.b.o | 09:53 |
| franred | audio-bluetooth is not a thing anymore --> https://gerrit.baserock.org/#/c/2197/ - so you will need to create 2 new pipelines one for bluetooth and another for sound-server-pulseaudio | 10:05 |
| mwilliams_ct | franred: pipelines apply to system rather than strata. I've updated genivi-demo-platform, is there any other needing doing? | 10:06 |
| franred | mwilliams_ct, oh! | 10:07 |
| mwilliams_ct | looks like baseline is broken too, will repair | 10:07 |
| mwilliams_ct | Also, who wants to take this mantle on as a deputy? | 10:08 |
| paulsherwood | richard_maw: so i'm thinking of just adding env['LANG'] = 'en_US.UTF-8' in the sandboxing process, to try your pragmatic fix | 10:40 |
| paulsherwood | question is (assuming it works), should it require a new artifact-version? | 10:41 |
| paulsherwood | ie all previous artifacts have been created without LANG being specified | 10:42 |
| persia | I believe it should. Several build systems behave differently in different locales. | 10:43 |
| paulsherwood | ok, i'll need to test without, first... bumping artifact-version and then discovering it doesn't work would be a shame | 10:44 |
| * richard_maw doesn't think it should | 10:44 | |
| paulsherwood | oh? | 10:44 |
| richard_maw | we're only doing this hack to make the build tool itself behave, not the build systems direcly | 10:44 |
| richard_maw | if we're resetting LANG to something else before running the build commands then it should be fine to have LANG set for the build tool itself | 10:45 |
| persia | Note that some software cannot be built in en_US.utf8, but presumably that could be adjusted in specific chunk definitions. | 10:45 |
| * richard_maw recalls that the sandboxing process in morph already did environment variable filtering | 10:46 | |
| persia | `make` behaves differently for different locales, for some sources, as a common example. | 10:46 |
| paulsherwood | richard_maw: the build tool isn't setting LANG. i'd be suprised if morph does, since i nicked the env variable filtering from morph | 10:46 |
| paulsherwood | LANG not present in morph source | 10:47 |
| * richard_maw remembered stuff like LOGNAME | 10:47 | |
| persia | morph does not: my initial interest in locales in Baserock was from systems that did not build in jp_JP.utf8 | 10:47 |
| paulsherwood | yes... that and the rest of the variables, i brought into ybd. it even has liw's tomjon user | 10:47 |
| paulsherwood | i think the reason morph users haven't hit this is that they only run in baserock systems | 10:48 |
| richard_maw | ok, morph's code doesn't carry the LANG environment variables into the sandbox, so assuming YBD does the same you don't need to change the cache keys | 10:48 |
| * richard_maw isn't sure setting LANG in the sandboxing process is early enough, python could compute the result on library load | 10:49 | |
| paulsherwood | ack. but if persia is correct, introducing LANG will break something else | 10:49 |
| persia | Unset vs. set can cause different result artifacts though. | 10:49 |
| richard_maw | paulsherwood: it won't, you're still leaving it unset in your actual builds | 10:49 |
| paulsherwood | persia: ack, which is unacceptable i think | 10:50 |
| persia | LC_COLLATION is the usual suspect. | 10:50 |
| richard_maw | paulsherwood: this is about making the python interpreter behave | 10:50 |
| persia | (which inherits if unset) | 10:50 |
| richard_maw | the builds themselves don't see your LANG | 10:51 |
| paulsherwood | richard_maw: so the fix can't go in sandbox.py https://github.com/devcurmudgeon/ybd/blob/master/ybd/sandbox.py#L256 | 10:51 |
| *** rjek has quit IRC | 10:51 | |
| *** vgrade has quit IRC | 10:51 | |
| persia | richard_maw: the python3 interpreter behaves differently in different host locales. | 10:51 |
| * persia is unsure about python2 | 10:52 | |
| richard_maw | paulsherwood: correct, that's for setting LANG in the sandbox, when the bug isn't in the sandbox, it's in the code that copies files out of the sandbox | 10:52 |
| paulsherwood | ack | 10:52 |
| * persia becomes less worried | 10:52 | |
| *** vgrade has joined #baserock | 10:53 | |
| *** rjek has joined #baserock | 10:53 | |
| paulsherwood | while we're on, though... if the locked-down env for sandbox doesn't include LANG, is it still possible for some host-specific stuff to be leaiking here? | 10:53 |
| persia | Note that some filenames will still be locale-dependent, but only those that use one of a very small number of rare characters (it was 3 last I checked, but that was a couple years ago, and I suspect it is only two now) | 10:54 |
| richard_maw | paulsherwood: the first thing that function does is build a new environment, it then selectively copies in things it assumes to be safe | 10:55 |
| richard_maw | paulsherwood: so unless it's explicitly copying stuff in from the environment, it's not going to leak in | 10:56 |
| richard_maw | umask might though | 10:56 |
| paulsherwood | jjardon: pls cna you grant me the powers so i can retry things like https://gitlab.com/baserock/definitions/builds/2537611#down-build-trace ? | 11:01 |
| paulsherwood | richard_maw: ack, i'm trying it | 11:01 |
| pedroalvarez | https://gerrit.baserock.org/2231 strata/bsp*: Use linux-stable when possible | 11:05 |
| * pedroalvarez is just another gerritbot nowadays | 11:06 | |
| paulsherwood | pedroalvarez: why use linux-stable? | 11:06 |
| pedroalvarez | for consitency, all of them pointing to linux-stable | 11:06 |
| paulsherwood | why not consistently point all of them at linux? | 11:06 |
| pedroalvarez | I can do that too | 11:07 |
| * paulsherwood has been fighting this 'stable' thing for years now | 11:07 | |
| pedroalvarez | oh, happy to fight in your side | 11:07 |
| paulsherwood | :) | 11:07 |
| pedroalvarez | i just though people wanted the other | 11:07 |
| pedroalvarez | thought* | 11:07 |
| paulsherwood | who are those people? | 11:07 |
| pedroalvarez | well, now I'm not sure those people exsist | 11:08 |
| paulsherwood | oh, they do, but in other channels i think | 11:08 |
| pedroalvarez | gah, typos | 11:08 |
| paulsherwood | ok, so i'd +1 a move to linux, but not away from it | 11:09 |
| * edcragg thought linux stable was maintained backports | 11:09 | |
| edcragg | (for particular old versions of linux) | 11:10 |
| franred | pedroalvarez, oh, different upstreams with same sha1? | 11:10 |
| richard_maw | edcragg: correct | 11:10 |
| pedroalvarez | franred: yes | 11:10 |
| paulsherwood | edcragg: yes, it is. no, we should not encourage that :) | 11:10 |
| pedroalvarez | I see now paulsherwood's point | 11:11 |
| pedroalvarez | better to move on, than use backported fixes | 11:11 |
| paulsherwood | exactly | 11:11 |
| richard_maw | if you're between mainline releases and there's a point release in stable for bugfixes for your current release it's preferable | 11:11 |
| paulsherwood | richard_maw: momentarily, maybe... but not for any production system, i think? | 11:12 |
| edcragg | i thought the idea was that it's *for* production systems | 11:14 |
| * richard_maw shrugs | 11:14 | |
| richard_maw | either way you want a stream of bugfixes | 11:14 |
| edcragg | if you can't/don't want to move to mainline, it's easier to let someone else maintain your bug fixes | 11:14 |
| paulsherwood | edcragg: for production systems with a lifetime of less than 2 years... which is not 'long term' in my vocabulary. and a new release every couple of weeks, with a couple of hundred patches, is not 'stable' for me either | 11:15 |
| paulsherwood | edcragg: i've been arguing for some years that mainline releases are the only sensible approach. this puts me at odds with a whole swathe of people and organisations who advocate LTS and LTSI, but it does not mean I'm wrong. | 11:16 |
| paulsherwood | approximately annually i end up in another project where the LTS/LTSI approach has led a project into a dead end | 11:17 |
| pedroalvarez | https://gerrit.baserock.org/2231 strata/bsp*: Use linux when possible | 11:18 |
| edcragg | yep, mainline is clearly better, but we all knew that already :) | 11:18 |
| paulsherwood | edcragg: surprisingly, even the people who i end up arguing with agree 'mainline is clearly better', while then recommending LTS/LTSI | 11:19 |
| paulsherwood | in fact, more generally, 'yes, paul is correct about the general principle' but then 'however we can't do that here because...' | 11:22 |
| paulsherwood | i'll stop whining, now | 11:22 |
| paulsherwood | feck. https://gitlab.com/devcurmudgeon/definitions/builds/2551154 | 11:25 |
| paulsherwood | which includes https://github.com/devcurmudgeon/ybd/commit/72ac4620f71f9a510714aade82f2c9c4d6ceea1e | 11:25 |
| pedroalvarez | gah | 11:26 |
| richard_maw | hmm, you could try setting it before python is invoked | 11:26 |
| paulsherwood | you mean outside ybd? | 11:26 |
| richard_maw | yeah | 11:27 |
| richard_maw | that's what the stackoverflow thing did | 11:27 |
| * richard_maw wondered if it might compute the locale stuff too early | 11:28 | |
| richard_maw | if that doesn't work, then it's worth checking whether the en_US.UTF-8 locale is installed in the container | 11:29 |
| *** jonathanmaw has quit IRC | 11:32 | |
| paulsherwood | ack | 11:33 |
| paulsherwood | no cigar. https://gitlab.com/devcurmudgeon/definitions/builds/2552430 | 11:38 |
| *** jonathanmaw has joined #baserock | 11:49 | |
| richard_maw | those perl warnings at the beginning sound like locales aren't installed in the container, which could be causing it | 12:09 |
| paulsherwood | oh! | 12:10 |
| *** lachlanmackenzie has quit IRC | 12:25 | |
| *** paulwaters_ has joined #baserock | 12:37 | |
| *** paulwaters_ has quit IRC | 12:39 | |
| paulsherwood | so i'm fumbling around in the dark, here https://gitlab.com/devcurmudgeon/definitions/builds/2554264 | 12:42 |
| paulsherwood | can someone guide me on how to script locale installation on this debian/ubuntu runner? | 12:43 |
| paulsherwood | rjek: ^^ ? | 12:43 |
| rjek | A debian system should already have locales installed | 12:44 |
| paulsherwood | ack. let's assume it's a ubuntu vm | 12:44 |
| paulsherwood | or worse, a docker container | 12:44 |
| * rjek steps away | 12:44 | |
| paulsherwood | hah | 12:44 |
| paulsherwood | come back! | 12:44 |
| rjek | But, perhaps apt-get install locales-all | 12:44 |
| rjek | If I never have to hear of docker ever again, my life will be better for it. | 12:45 |
| paulsherwood | rjek: https://gitlab.com/devcurmudgeon/definitions/builds/2554377 ? | 12:56 |
| rjek | paulsherwood: locale-gen is in /usr/sbin/locale-gen suggesting it must be run as root | 12:59 |
| paulsherwood | i think this is root | 13:00 |
| rjek | The prompt is $ ? | 13:00 |
| paulsherwood | ok, maybe not then... | 13:01 |
| rjek | Anyway, in Debian, /usr/sbin/locale-gen is in the "locales" package | 13:01 |
| paulsherwood | it is (approximately) root, apparently https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/263 | 13:05 |
| *** edcragg has quit IRC | 13:06 | |
| mwilliams_ct | so just to check, you're running this build using the "container" executor under gitlab-ci? (might be called "docker" executor, it's been a while) | 13:07 |
| *** edcragg has joined #baserock | 13:08 | |
| *** franred has quit IRC | 13:08 | |
| paulsherwood | yup | 13:08 |
| mwilliams_ct | ack. I'd suggest calling /usr/sbin/locale-gen rather than `locale-gen` then, in case docker image path is a bit borked. then some debugging ls's after the apt-get probably | 13:09 |
| paulsherwood | i'm managing to call that, now... but https://gitlab.com/devcurmudgeon/definitions/builds/2555008 | 13:10 |
| rjek | Have you read it? | 13:11 |
| rjek | It looks like it's asking you what locales you want to generate and then you feed it invalid input | 13:11 |
| mwilliams_ct | --yes will fix the apt-get problem as well | 13:11 |
| paulsherwood | rjek: yes. i don't know what to do about it | 13:13 |
| rjek | Why are you calling dpkg-reconfigure? | 13:14 |
| *** jonathanmaw has quit IRC | 13:15 | |
| *** jonathanmaw has joined #baserock | 13:29 | |
| locallycompact | could someone nudge this in for me https://gerrit.baserock.org/#/c/2227/ | 13:32 |
| paulsherwood | rjek: i have no idea. internet told me to | 13:35 |
| rjek | What did you ask the internet how to do? | 13:37 |
| rjek | Anyway, dpkg-reconfigure locales is an /interactive/ tool for editing the contents of /etc/locales.gen which then runs locale-gen | 13:38 |
| *** anahuelamo_ has quit IRC | 13:41 | |
| *** anahuelamo has joined #baserock | 13:42 | |
| *** jonathanmaw has quit IRC | 14:04 | |
| *** jonathanmaw has joined #baserock | 14:18 | |
| *** franred has joined #baserock | 14:36 | |
| *** franred has quit IRC | 15:07 | |
| *** mwilliams_ct has left #baserock | 15:07 | |
| *** gtristan has quit IRC | 15:16 | |
| *** edcragg has quit IRC | 15:16 | |
| *** gtristan has joined #baserock | 15:28 | |
| *** bjdooks_ has joined #baserock | 15:31 | |
| *** bjdooks has quit IRC | 15:35 | |
| *** cyndis has quit IRC | 15:39 | |
| *** gary_perkins has quit IRC | 15:39 | |
| *** vgrade12 has quit IRC | 15:39 | |
| *** gary_perkins has joined #baserock | 15:40 | |
| *** cyndis has joined #baserock | 15:47 | |
| *** jonathanmaw has quit IRC | 15:49 | |
| *** CTtpollard has quit IRC | 15:50 | |
| *** rdale has quit IRC | 15:54 | |
| *** toscalix has quit IRC | 15:56 | |
| *** gtristan has quit IRC | 16:34 | |
| jjardon | paulsherwood: I made you "master" of the definitions project; try again? | 16:36 |
| jjardon | (it should not be needed though) | 16:36 |
| *** locallycompact has quit IRC | 16:46 | |
| *** tiagogomes has quit IRC | 18:03 | |
| jjardon | paulsherwood: and probably difficult to agree in the versioning; I'm sure several people here actually use 'LTS' distros | 18:52 |
| paulsherwood | i exect so :) | 19:17 |
| *** radiofree has quit IRC | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!