IRC logs for #baserock for Friday, 2016-07-22

*** paulwaters_ has joined #baserock06:02
jjardonpaulsherwood: fyi, ybd master still fails: https://gitlab.com/baserock/definitions/builds/2537611#down-build-trace06:12
*** toscalix has joined #baserock07:44
*** jonathanmaw has joined #baserock07:46
paulsherwood:/08:10
*** edcragg has quit IRC08:12
*** edcragg has joined #baserock08:12
*** bwh__ has quit IRC08:13
*** bwh_ has joined #baserock08:13
richard_mawthat error is because pyfilesystem converted the path to unicode, and because the system calls take byte strings rather than unicode strings, decided it has to convert it back to a byte string, but for some reason decided it has to convert it to ascii08:34
richard_mawthis is a different, but related problem, to there being file paths that aren't convertable to unicode08:34
* richard_maw doesn't know how python decides what encoding to use, but if pyfilesystem had left the strings alone it would have been fine08:36
richard_mawI *think* it may be related to what's set in the LANG and LANGUAGE environment variables, which would be unset in CI builds08:37
richard_mawso it might decide to fall back to ascii, since whoever specified python's behaviour here didn't understand file paths08:37
richard_mawhttp://stackoverflow.com/questions/2076708/python-os-stat-and-unicode-file-names08:39
richard_mawthe recommendation is to re-encode it as a byte string before calling the syscall, but pyfilesystem doesn't do that08:39
richard_mawso… setting LANG=en_US.UTF-8 might be worth a try as a pragmatic solution,08:44
*** rdale has joined #baserock08:46
richard_maweven if it is caused by a fundamental misunderstanding of how file paths work, which ought to be fixed upstream08:47
*** gtristan has joined #baserock09:01
*** locallycompact has joined #baserock09:15
* paulsherwood can't remember who has the superpowers to generate the pipelines at c.b.o09:45
paulsherwoodlocallycompact: is that you?09:45
paulsherwoodlooks like we've dropped audio-bluetooth, but the pipeline has not been updated http://concourse.baserock.org/pipelines/genivi-demo-platform-x86_64-generic/jobs/audio-bluetooth/builds/3409:46
locallycompactI don't know the aws details, mwilliams will09:48
paulsherwooddoes he still hang aroudn here?09:49
CTtpollardhe'll be here in a sec09:50
*** mwilliams_ct has joined #baserock09:50
mwilliams_ctYou rang?09:50
* rjek wonders what the mwilliams summoning incantation is09:52
mwilliams_ctask tpollard to message me on another network.09:52
mwilliams_ctthough he didnt give me much context and irclogs.br hasnt updated yet. Am I needed for something?09:53
rjek10:45:25 -*- paulsherwood can't remember who has the superpowers to generate the pipelines at c.b.o09:53
franredaudio-bluetooth is not a thing anymore --> https://gerrit.baserock.org/#/c/2197/ - so you will need to create 2 new pipelines one for bluetooth and another for sound-server-pulseaudio10:05
mwilliams_ctfranred: pipelines apply to system rather than strata. I've updated genivi-demo-platform, is there any other needing doing?10:06
franredmwilliams_ct, oh!10:07
mwilliams_ctlooks like baseline is broken too, will repair10:07
mwilliams_ctAlso, who wants to take this mantle on as a deputy?10:08
paulsherwoodrichard_maw: so i'm thinking of just adding env['LANG'] = 'en_US.UTF-8' in the sandboxing process, to try your pragmatic fix10:40
paulsherwoodquestion is (assuming it works), should it require a new artifact-version?10:41
paulsherwoodie all previous artifacts have been created without LANG being specified10:42
persiaI believe it should.  Several build systems behave differently in different locales.10:43
paulsherwoodok, i'll need to test without, first... bumping artifact-version and then discovering it doesn't work would be a shame10:44
* richard_maw doesn't think it should10:44
paulsherwoodoh?10:44
richard_mawwe're only doing this hack to make the build tool itself behave, not the build systems direcly10:44
richard_mawif we're resetting LANG to something else before running the build commands then it should be fine to have LANG set for the build tool itself10:45
persiaNote that some software cannot be built in en_US.utf8, but presumably that could be adjusted in specific chunk definitions.10:45
* richard_maw recalls that the sandboxing process in morph already did environment variable filtering10:46
persia`make` behaves differently for different locales, for some sources, as a common example.10:46
paulsherwoodrichard_maw: the build tool isn't setting LANG. i'd be suprised if morph does, since i nicked the env variable filtering from morph10:46
paulsherwoodLANG not present in morph source10:47
* richard_maw remembered stuff like LOGNAME10:47
persiamorph does not: my initial interest in locales in Baserock was from systems that did not build in jp_JP.utf810:47
paulsherwoodyes... that and the rest of the variables, i brought into ybd. it even has liw's tomjon user10:47
paulsherwoodi think the reason morph users haven't hit this is that they only run in baserock systems10:48
richard_mawok, morph's code doesn't carry the LANG environment variables into the sandbox, so assuming YBD does the same you don't need to change the cache keys10:48
* richard_maw isn't sure setting LANG in the sandboxing process is early enough, python could compute the result on library load10:49
paulsherwoodack. but if persia is correct, introducing LANG will break something else10:49
persiaUnset vs. set can cause different result artifacts though.10:49
richard_mawpaulsherwood: it won't, you're still leaving it unset in your actual builds10:49
paulsherwoodpersia: ack, which is unacceptable i think10:50
persiaLC_COLLATION is the usual suspect.10:50
richard_mawpaulsherwood: this is about making the python interpreter behave10:50
persia(which inherits if unset)10:50
richard_mawthe builds themselves don't see your LANG10:51
paulsherwoodrichard_maw: so the fix can't go in sandbox.py https://github.com/devcurmudgeon/ybd/blob/master/ybd/sandbox.py#L25610:51
*** rjek has quit IRC10:51
*** vgrade has quit IRC10:51
persiarichard_maw: the python3 interpreter behaves differently in different host locales.10:51
* persia is unsure about python210:52
richard_mawpaulsherwood: correct, that's for setting LANG in the sandbox, when the bug isn't in the sandbox, it's in the code that copies files out of the sandbox10:52
paulsherwoodack10:52
* persia becomes less worried10:52
*** vgrade has joined #baserock10:53
*** rjek has joined #baserock10:53
paulsherwoodwhile we're on, though... if the locked-down env for sandbox doesn't include LANG, is it still possible for some host-specific stuff to be leaiking here?10:53
persiaNote that some filenames will still be locale-dependent, but only those that use one of a very small number of rare characters (it was 3 last I checked, but that was a couple years ago, and I suspect it is only two now)10:54
richard_mawpaulsherwood: the first thing that function does is build a new environment, it then selectively copies in things it assumes to be safe10:55
richard_mawpaulsherwood: so unless it's explicitly copying stuff in from the environment, it's not going to leak in10:56
richard_mawumask might though10:56
paulsherwoodjjardon: pls cna you grant me the powers so i can retry things like https://gitlab.com/baserock/definitions/builds/2537611#down-build-trace ?11:01
paulsherwoodrichard_maw: ack, i'm trying it11:01
pedroalvarez https://gerrit.baserock.org/2231 strata/bsp*: Use linux-stable when possible11:05
* pedroalvarez is just another gerritbot nowadays11:06
paulsherwoodpedroalvarez: why use linux-stable?11:06
pedroalvarezfor consitency, all of them pointing to linux-stable11:06
paulsherwoodwhy not consistently point all of them at linux?11:06
pedroalvarezI can do that too11:07
* paulsherwood has been fighting this 'stable' thing for years now11:07
pedroalvarezoh, happy to fight in your side11:07
paulsherwood:)11:07
pedroalvarezi just though people wanted the other11:07
pedroalvarezthought*11:07
paulsherwoodwho are those people?11:07
pedroalvarezwell, now I'm not sure those people exsist11:08
paulsherwoodoh, they do, but in other channels i think11:08
pedroalvarezgah, typos11:08
paulsherwoodok, so i'd +1 a move to linux, but not away from it11:09
* edcragg thought linux stable was maintained backports11:09
edcragg(for particular old versions of linux)11:10
franredpedroalvarez, oh, different upstreams with same sha1?11:10
richard_mawedcragg: correct11:10
pedroalvarezfranred: yes11:10
paulsherwoodedcragg: yes, it is. no, we should not encourage that :)11:10
pedroalvarezI see now paulsherwood's point11:11
pedroalvarezbetter to move on, than use backported fixes11:11
paulsherwoodexactly11:11
richard_mawif you're between mainline releases and there's a point release in stable for bugfixes for your current release it's preferable11:11
paulsherwoodrichard_maw: momentarily, maybe... but not for any production system, i think?11:12
edcraggi thought the idea was that it's *for* production systems11:14
* richard_maw shrugs11:14
richard_maweither way you want a stream of bugfixes11:14
edcraggif you can't/don't want to move to mainline, it's easier to let someone else maintain your bug fixes11:14
paulsherwoodedcragg: for production systems with a lifetime of less than 2 years... which is not 'long term' in my vocabulary. and a new release every couple of weeks, with a couple of hundred patches, is not 'stable' for me either11:15
paulsherwoodedcragg: i've been arguing for some years that mainline releases are the only sensible approach. this puts me at odds with a whole swathe of people and organisations who advocate LTS and LTSI, but it does not mean I'm wrong.11:16
paulsherwoodapproximately annually i end up in another project where the LTS/LTSI approach has led a project into a dead end11:17
pedroalvarezhttps://gerrit.baserock.org/2231 strata/bsp*: Use linux when possible11:18
edcraggyep, mainline is clearly better, but we all knew that already :)11:18
paulsherwoodedcragg: surprisingly, even the people who i end up arguing with agree 'mainline is clearly better', while then recommending LTS/LTSI11:19
paulsherwoodin fact, more generally,  'yes, paul is correct about the general principle' but then 'however we can't do that here because...'11:22
paulsherwoodi'll stop whining, now11:22
paulsherwoodfeck. https://gitlab.com/devcurmudgeon/definitions/builds/255115411:25
paulsherwoodwhich includes https://github.com/devcurmudgeon/ybd/commit/72ac4620f71f9a510714aade82f2c9c4d6ceea1e11:25
pedroalvarezgah11:26
richard_mawhmm, you could try setting it before python is invoked11:26
paulsherwoodyou mean outside ybd?11:26
richard_mawyeah11:27
richard_mawthat's what the stackoverflow thing did11:27
* richard_maw wondered if it might compute the locale stuff too early11:28
richard_mawif that doesn't work, then it's worth checking whether the en_US.UTF-8 locale is installed in the container11:29
*** jonathanmaw has quit IRC11:32
paulsherwoodack11:33
paulsherwoodno cigar. https://gitlab.com/devcurmudgeon/definitions/builds/255243011:38
*** jonathanmaw has joined #baserock11:49
richard_mawthose perl warnings at the beginning sound like locales aren't installed in the container, which could be causing it12:09
paulsherwoodoh!12:10
*** lachlanmackenzie has quit IRC12:25
*** paulwaters_ has joined #baserock12:37
*** paulwaters_ has quit IRC12:39
paulsherwoodso i'm fumbling around in the dark, here https://gitlab.com/devcurmudgeon/definitions/builds/255426412:42
paulsherwoodcan someone guide me on how to script locale installation on this debian/ubuntu runner?12:43
paulsherwoodrjek: ^^ ?12:43
rjekA debian system should already have locales installed12:44
paulsherwoodack. let's assume it's a ubuntu vm12:44
paulsherwoodor worse, a docker container12:44
* rjek steps away12:44
paulsherwoodhah12:44
paulsherwoodcome back!12:44
rjekBut, perhaps apt-get install locales-all12:44
rjekIf I never have to hear of docker ever again, my life will be better for it.12:45
paulsherwoodrjek: https://gitlab.com/devcurmudgeon/definitions/builds/2554377 ?12:56
rjekpaulsherwood: locale-gen is in /usr/sbin/locale-gen suggesting it must be run as root12:59
paulsherwoodi think this is root13:00
rjekThe prompt is $ ?13:00
paulsherwoodok, maybe not then...13:01
rjekAnyway, in Debian, /usr/sbin/locale-gen is in the "locales" package13:01
paulsherwoodit is (approximately) root, apparently https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/26313:05
*** edcragg has quit IRC13:06
mwilliams_ctso just to check, you're running this build using the "container" executor under gitlab-ci? (might be called "docker" executor, it's been a while)13:07
*** edcragg has joined #baserock13:08
*** franred has quit IRC13:08
paulsherwoodyup13:08
mwilliams_ctack. I'd suggest calling /usr/sbin/locale-gen rather than `locale-gen` then, in case docker image path is a bit borked. then some debugging ls's after the apt-get probably13:09
paulsherwoodi'm managing to call that, now... but https://gitlab.com/devcurmudgeon/definitions/builds/255500813:10
rjekHave you read it?13:11
rjekIt looks like it's asking you what locales you want to generate and then you feed it invalid input13:11
mwilliams_ct--yes will fix the apt-get problem as well13:11
paulsherwoodrjek: yes. i don't know what to do about it13:13
rjekWhy are you calling dpkg-reconfigure?13:14
*** jonathanmaw has quit IRC13:15
*** jonathanmaw has joined #baserock13:29
locallycompactcould someone nudge this in for me https://gerrit.baserock.org/#/c/2227/13:32
paulsherwoodrjek: i have no idea. internet told me to13:35
rjekWhat did you ask the internet how to do?13:37
rjekAnyway, dpkg-reconfigure locales is an /interactive/ tool for editing the contents of /etc/locales.gen which then runs locale-gen13:38
*** anahuelamo_ has quit IRC13:41
*** anahuelamo has joined #baserock13:42
*** jonathanmaw has quit IRC14:04
*** jonathanmaw has joined #baserock14:18
*** franred has joined #baserock14:36
*** franred has quit IRC15:07
*** mwilliams_ct has left #baserock15:07
*** gtristan has quit IRC15:16
*** edcragg has quit IRC15:16
*** gtristan has joined #baserock15:28
*** bjdooks_ has joined #baserock15:31
*** bjdooks has quit IRC15:35
*** cyndis has quit IRC15:39
*** gary_perkins has quit IRC15:39
*** vgrade12 has quit IRC15:39
*** gary_perkins has joined #baserock15:40
*** cyndis has joined #baserock15:47
*** jonathanmaw has quit IRC15:49
*** CTtpollard has quit IRC15:50
*** rdale has quit IRC15:54
*** toscalix has quit IRC15:56
*** gtristan has quit IRC16:34
jjardonpaulsherwood: I made you "master" of the definitions project; try again?16:36
jjardon(it should not be needed though)16:36
*** locallycompact has quit IRC16:46
*** tiagogomes has quit IRC18:03
jjardonpaulsherwood: and probably difficult to agree in the versioning; I'm sure several people here actually use 'LTS' distros18:52
paulsherwoodi exect so :)19:17
*** radiofree has quit IRC23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!