IRC logs for #buildstream for Wednesday, 2019-01-02

*** nimish has joined #buildstream01:08
*** max has quit IRC01:19
*** tristan has quit IRC01:43
*** nimish has quit IRC02:45
*** kapil___ has joined #buildstream03:00
*** tristan has joined #buildstream03:50
*** nimish has joined #buildstream05:51
*** mimaz has joined #buildstream06:15
*** kapil___ has quit IRC06:40
*** alatiera has joined #buildstream06:53
*** mimaz has quit IRC07:39
*** kapil___ has joined #buildstream07:53
*** rdale has joined #buildstream09:28
*** phildawson_ has joined #buildstream09:33
*** jonathanmaw has joined #buildstream10:02
*** tpollard has joined #buildstream10:03
*** kapil___ has quit IRC10:10
*** Nexus has left #buildstream10:36
gitlab-br-botLaurenceUrhegyi approved MR !1026 (chandan/update-doc-makefile-note->master: doc/Makefile: Update comment about sphinx entrypoint) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/102610:50
gitlab-br-botphildawson approved MR !1026 (chandan/update-doc-makefile-note->master: doc/Makefile: Update comment about sphinx entrypoint) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/102610:54
*** raoul has joined #buildstream10:59
*** nimish has quit IRC11:01
*** nimish has joined #buildstream11:01
phildawson_hmm, if i go to docs.buildstream.build, firefox is warning me about an expired certificate11:28
phildawson_buildstream.build is fine though11:28
tpollardwho controls those certs, jjardon?11:30
gitlab-br-botvalentindavid opened MR !1030 (valentindavid/remote_execution_configuration->master: Valentindavid/remote execution configuration) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/103011:41
laurencenice one, valentind :)11:51
valentindlaurence, I did that before holidays, but I did not want to submit it in a rush.11:52
laurencevalentind, cool, I think that use of the 'self-assign' feature on gitlab issues is very useful for this11:52
laurenceallows others to know you're planning to take it on11:53
laurenceI am a fan of it, and will incorporate it into the new guidelines11:53
laurences/will/think we should/11:54
valentindHappy new year by the way.11:54
*** cs-shadow has joined #buildstream11:58
*** raoul has quit IRC12:04
*** nimish has quit IRC12:06
*** nimish has joined #buildstream12:07
tlater[m]Happy new year :)12:15
*** rdale has quit IRC12:26
*** rdale has joined #buildstream12:28
gitlab-br-bottlater approved MR !1020 (coldtom/collections->master: Use collections.abc for Mapping, Iterable) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/102012:31
*** raoul has joined #buildstream12:40
gitlab-br-botcs-shadow opened issue #838 (Add tests for python 3.7) on buildstream https://gitlab.com/BuildStream/buildstream/issues/83813:01
*** nimish has quit IRC13:02
*** nimish has joined #buildstream13:02
cs-shadowHappy new year :)13:11
cs-shadowOne of my pipelines (https://gitlab.com/BuildStream/buildstream/-/jobs/140569928) recently hit a seg fault, which seems unrelated to the change. Has anyone seen seg faults in our CI before?13:11
cs-shadowMaybe I just got unlucky on first pipeline of the year13:11
tpollardI've definitely seen the debian pipeline seg before13:12
tpollardhappy new year also :)13:12
cs-shadowah, so I'm not alone. I assume we didn't manage to get any insights about it in the past?13:13
*** kapil___ has joined #buildstream13:13
cs-shadowand we keep hitting the job with the rebuild hammer until it passes13:13
tpollardI can't see an issue for it, we should probably at least raise it13:14
* cs-shadow is creating one now13:14
tpollard:)13:14
tlater[m]Ooi, do the aarch64 runners on buildstream-docker-images only work at a certain time or something? The CI for that was stuck waiting for those runners for a pretty long time yesterday.13:15
valentindtlater[m], they were down.13:15
valentindI put them back on.13:16
tlater[m]Ah, fair enough13:16
valentindNetwork goes does down from time to time. I still have to figure out why.13:16
cs-shadowtlater[m]: we were considering making the aarch64 builds optional in https://gitlab.com/BuildStream/buildstream-docker-images/issues/2413:17
gitlab-br-botcs-shadow opened issue #840 (CI Piepline sometimes seg faults on debian:9) on buildstream https://gitlab.com/BuildStream/buildstream/issues/84013:19
*** nimish has quit IRC13:27
*** nimish has joined #buildstream13:38
gitlab-br-botcs-shadow merged MR !1026 (chandan/update-doc-makefile-note->master: doc/Makefile: Update comment about sphinx entrypoint) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/102613:42
*** nimish has quit IRC13:48
*** nimish has joined #buildstream13:48
*** juergbi has quit IRC13:52
*** juergbi has joined #buildstream13:52
*** alatiera_ has joined #buildstream13:55
*** raoul_ has joined #buildstream13:55
*** nimish has quit IRC13:56
*** raoul has quit IRC13:56
*** rdale has quit IRC13:56
*** tpollard has quit IRC13:56
*** alatiera has quit IRC13:56
*** johnward has quit IRC13:56
*** lantw44 has quit IRC13:56
*** phildawson has quit IRC13:56
*** Trevinho has quit IRC13:56
*** jennis has quit IRC13:56
*** ikerperez has quit IRC13:56
*** adds68 has quit IRC13:56
*** coldtom has quit IRC13:56
*** persia has quit IRC13:56
*** ctgriffiths has quit IRC13:56
*** chill has quit IRC13:56
*** SotK has quit IRC13:56
*** tpollard has joined #buildstream13:57
*** persia has joined #buildstream13:58
*** ctgriffiths has joined #buildstream13:59
*** rdale has joined #buildstream13:59
*** johnward has joined #buildstream14:03
*** SotK has joined #buildstream14:06
*** chill has joined #buildstream14:11
*** raoul_ is now known as raoul14:19
laurencefor those interested in attending the Gathering at the end of January, in London, please either add your name here or ping me and I'll add it15:16
laurencehttps://wiki.gnome.org/Projects/BuildStream/Events/Gathering-201901#preview15:16
jmacDone15:31
tpollardyou can add me laurence :)15:33
raoulme too laurence15:35
laurencedone and done16:00
tlater[m]Would someone mind taking a look at https://gitlab.com/BuildStream/buildstream/merge_requests/638? It's a pretty small one.16:08
jmacI will16:12
* tlater[m] hopes rebasing didn't add new linting errors, but that should be pretty minor16:13
*** max has joined #buildstream16:14
jmacjjardon: Why did all the image names change in .gitlab-ci.yml in https://gitlab.com/BuildStream/buildstream/merge_requests/638 ?16:15
tlater[m]jmac: That's my commit16:16
tlater[m]We needed a new version of pycodestyle to be installed16:16
tlater[m]So the image versions need to be bumped (the new pycodestyle version was released two weeks ago)16:16
*** jennis has joined #buildstream16:17
*** coldtom has joined #buildstream16:18
*** ikerperez has joined #buildstream16:18
*** phildawson has joined #buildstream16:18
*** adds68 has joined #buildstream16:18
jmactlater[m]: Bumping the versions makes sense, I was just curious as to why it's changed from `master-123` to `06bab030`16:19
cs-shadowjmac: that's because we changed the versioning scheme of the testsuite images to have the commit sha16:20
tlater[m]I think that's a quirk from the recent updates to the docker images16:20
tlater[m]Yeah :)16:20
jmacSounds good. No other comments from me, looks good.16:25
*** lantw44 has joined #buildstream16:25
tlater[m]Thanks @cs16:27
tlater[m]cs-shadow:16:27
tlater[m]jmac:16:27
tlater[m]x)16:27
cs-shadow:)16:27
tlater[m]I'll check if we can move the configuration16:27
tlater[m]But I'm not sure it's possible, pytest-pycodestyle is how we currently run the linter.16:27
*** max has quit IRC16:28
cs-shadowFor example, if we can put it under something like `pycodestyle` section then I would imagine that both pycodestyle and pytest-codestyle are able to read it. But I haven't verified this just yet. I can also look into this later as this is somewhat orthogonal to your MR16:32
cs-shadowdefinitely not a blocker for me16:32
*** Trevinho has joined #buildstream16:33
tlater[m]cs-shadow: Eh, I'm already checking just that - might as well get it perfect ;)16:36
*** nimish has joined #buildstream16:38
tlater[m]Judging by 123 max-line-length failures, looks like that's not possible :| We could duplicate the configuration, but that seems off.16:39
*** mimaz has joined #buildstream16:41
gitlab-br-bottlater opened (was WIP) MR !638 (jjardon/pycodestyle->master: Use pycodestyle instead pep8 python module) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/63816:44
gitlab-br-bottlater opened issue #841 (There is no way to run pycodestyle without `setup.py` while maintaining our configuration) on buildstream https://gitlab.com/BuildStream/buildstream/issues/84116:49
raoulIs there a way of getting the gitlab CI to run a specific test? I'm having issues with 2 tests on the debian image, but it works locally and all other images16:50
*** nimish has quit IRC16:50
tlater[m]raoul: By "locally" do you mean you've tried running the test in a local docker container?16:51
tlater[m]If so, you'll need to modify .gitlab-ci.yml16:52
tlater[m]Remove all but the image you want to test, and modify the TEST_COMMAND variable16:52
tlater[m]But well, it's a lot simpler to spin up a testsuite image locally16:53
raoulah yeah, guess I should be trying it with docker images16:53
raoultime to remind myself how to use docker then16:54
tlater[m]raoul: If you can figure out how to use it, this may be handy: https://gitlab.com/BuildStream/buildstream/snippets/168493016:54
tlater[m]But it's not well maintained atm. I need to sit down and update it at some point.16:54
raoulI shall see if it works, should give an idea of what to do at least16:56
raoulcheers :)16:56
*** tristan has quit IRC17:08
*** max has joined #buildstream17:15
*** benschubert has joined #buildstream17:15
benschubertHey, I have a question about warnings, are they always project specific or can we have them globally at the user conf?17:16
*** raoul has quit IRC17:16
tpollardI think they're only parsed from project config17:18
*** raoul has joined #buildstream17:20
benschubertAnd thus, they are per project only and no way of overriding them, correct?17:20
*** tristan has joined #buildstream17:25
gitlab-br-bottlater opened MR !1031 (tlater/message-lines->master: Avoid "showing 0 lines" messages when there are no lines) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/103117:28
gitlab-br-bottlater merged MR !638 (jjardon/pycodestyle->master: Use pycodestyle instead pep8 python module) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/63817:32
tlater[m]Hmm... What's our policy on tests for buildstream output? I just fixed #799, but to write a test I'd need to write something that asserts a specific bit of text turns up in buildstream's output.17:32
gitlab-br-botIssue #799: Remote execution: Make TLS and client certificate optional for storage service https://gitlab.com/BuildStream/buildstream/issues/79917:32
tlater[m]We have a few tests that do that, but I've heard that we don't want to do that too much on occasion. Mostly citing possible text changes and future localization.17:32
jmacI don't see how we can avoid it if we want to do true end-to-end testing; we'd just have to localise the tests as well17:34
raoulafter faffing with docker for a bit to get it working your script seems to work tlater[m]17:39
tlater[m]\o/ It's a little quirky though, so don't expect launching it again to give you the same cache ;)17:41
raoulhaha, it's working for the one thing I want it for, so I'm happy :P thanks again :)17:45
cs-shadowcan someone please verify that https://docs.buildstream.build doesn't have a valid certificate and it's not just me :)17:51
cs-shadowAlso, see https://gitlab.com/BuildStream/website/issues/10#note_12793224317:51
*** ChanServ sets mode: +o tristan17:54
tristancs-shadow, ah I saw something about that whiz passed me in my inbox, I'll try to find out17:55
cs-shadowthanks!17:55
*** max has quit IRC17:58
tristancs-shadow, how about the tox stuff, are you going to work on it ?18:11
tristanummm put another way; I would be delighted to give that branch love myself if you are setting it aside and it was a sort of holiday distraction ;-)18:12
tristanlooks like you did make progress on it18:23
*** raoul has quit IRC18:27
*** jonathanmaw has quit IRC18:38
*** rdale has quit IRC18:44
cs-shadowtristan: https://gitlab.com/BuildStream/buildstream/merge_requests/1027 should be ready for review, it's in WIP state as the docker-images change would happen first.18:45
cs-shadowonce you (and others) are happy with this MR, i can make the corresponding change to the docker-images18:45
tristancs-shadow, I was just looking at it18:45
tristanmade a comment18:45
cs-shadowI still wish to make some more improvements to it, but I think we can do those in increments18:46
* cs-shadow just notices the commets18:46
tristanlemme make sure my lookover is thorough18:46
tristanI accidentally commented on your FIXME18:46
tristanwhich I guess needs the docker image change first18:46
cs-shadownp :)18:46
cs-shadowyep, there's a corresponding MR for that, which is also ready to go (barring one change that I need to do)18:47
tristanI think we should rename requirements.in -> requirements.txt18:47
tristanand rename the current requirements.txt -> constraints.txt18:48
tristanand remove all of the stuff from setup.py18:48
tristanbut maybe that can be a separate thing18:48
tristanI could try to cleanup setup.py in some way after18:48
tristanoh you did it too !18:49
cs-shadowhmm, I was thinking to follow the same convention for all requirements file. I had proposed .in for unconstrained and .txt for constrained.18:49
cs-shadowWe can also go with .txt for unfrozen, and something like  `.frozen.txt` if you'd like to keep requirements.txt for this purpose18:49
tristanWith the same trick yeah that is fine18:49
cs-shadowmy only concern is that the same extension for all of the requirement files to avoid confusion18:49
tristancs-shadow, is that the convention ?18:50
tristanThat is fine by me then18:50
tristanI find it to be a strange convention; the ".in" to me says that it is input for a file that needs preprocessing in the build18:50
cs-shadowyeah, it's not very obvious. I have seen different conventions in different places so can't say if this is followed everywhere. I know that some tools like https://github.com/jazzband/pip-tools try to promote this convention but there's no real consistency in the wild about this18:52
cs-shadowI am personally not attached to the .in extension so long as we change it for all the files18:53
cs-shadowdo `.txt` and `.frozen.txt`sound better to you?18:53
tristanI see18:54
tristancs-shadow, so what I understand is there are two approaches, and the doc says that basically "requirements.txt is the input of tools, requirements.in is the input to generate requirements.txt"18:56
tristancs-shadow, the other approach is to have both be the input of tools18:56
tristanam I onto something ?18:56
cs-shadownot sure, I think in our case the .in serves a dual purpose - it's an input to tools and is also the input for requirements.txt18:57
tristancs-shadow, basically; by putting the output of pip-compile or pip freeze into requirements.txt, then you are advertising that these are the specific requirements18:57
tristancs-shadow, the other approach is to have requirements.txt be more loosely defined, and add a constraints file which can be combined (it might add only some constraints, or it might be a full pip freeze)18:58
cs-shadowi mean we already have both the files, it's just about the naming I suppose, right?18:59
tristanI think so yes19:00
cs-shadowwe have these files at the moment: https://gitlab.com/BuildStream/buildstream/tree/bf5f9ef4d9b45ef10dc0f2c2e9db9edd7534f646/tools19:00
tristanwell19:00
tristancs-shadow, the result is technically different too, if you have both a requirements.txt and a constraints.txt, then you have a slightly more complex install I guess19:01
tristanso using the "requirements.in -> requirements.txt -> python environment install" road means there is only one file ever19:01
tristancs-shadow, I guess also when people install with pip, they will by default use the requirements.txt ?19:02
tristanI think this is confusing to me because setuptools is confusing19:02
tristanIt is possible to have no setup.py and have a requirements.txt I think19:03
tristancs-shadow, lets drop this; I'm having a hard time to know what is better at this point, and we can just change it later19:03
cs-shadowtristan: yes, that's also possible. But, at this point, _nothing_ is changing for people using setup.py19:05
cs-shadowthe txt files are used by tox, and nothing else19:05
cs-shadownothing in standard python tooling recognizes requirements.txt as special, so I won't expect people's workflow to change because of this. The only thing is that the package maintainers will have to look at the `.in` file for requirements19:06
cs-shadowwe can always iterate on it as you said :)19:06
tristancs-shadow, yeah just realized it was not worth focusing on :)19:07
cs-shadowtristan: cool, let me know if you'd like me to update anything else :)19:08
tristancs-shadow, I just looked through it, found one important spot I think19:15
cs-shadowtristan: thanks! good catch about INTEGRATION_CACHE, I missed that19:15
tristancs-shadow, we probably want it in docs too, maybe one day we'll merge INTEGRATION_CACHE and BST_SOURCE_CACHE, that is stupidly redundant19:16
tristan(probably I did that)19:16
tristancs-shadow, besides that, I think we want to prepare an image, and make this branch use the new image, and remove the FIXME parts where we install non-python stuff19:17
tristanthen land it19:17
tristancs-shadow, once this lands, I'll try to remove the sdist step from gitlab CI, since tox does it for us, and change the overnight tests to also run through tox19:19
tristan(I'll *try* to ditch the whole ./unpack.sh thing if I can delegate it all to tox)19:20
cs-shadowtristan: definitely, I'll fix the FIXME before landing.19:20
cs-shadowtristan: I'd like to ditch the unpacking business as well. I didn't do it just yet as other jobs (like post-test) use it19:20
tristanyeah19:21
cs-shadowbut that should be achievable with tox, I'd hope19:21
tristanonly in coverage and overnight tests19:21
tristanyes certainly19:21
tristanJust tricky to get the invocation to be nice19:21
*** kapil___ has quit IRC19:22
tristanI wonder if there is a set of default "environments" for tox19:22
tristanso that when you just run "tox", it will only run the tests, unless you do "tox -e docs"19:22
* tristan runs `tox` on cs-shadow's branch :)19:23
cs-shadowyep, that's exactly what envlist does19:23
tristanhmm19:24
cs-shadowhttps://gitlab.com/BuildStream/buildstream/blob/bf5f9ef4d9b45ef10dc0f2c2e9db9edd7534f646/tox.ini#L2 -> since docs isn't listed there, it shouldn't get run unless you do `-e docs`19:24
*** tristan has quit IRC19:27
*** tristan has joined #buildstream19:27
*** ChanServ sets mode: +o tristan19:28
tristancs-shadow, pytest: error: unrecognized arguments: --codestyle19:28
cs-shadowtristan: sorry, I messed up a rebase, it's not listed as a requirement on my branch. Let me fix that quickly19:29
tristantlater[m], still around ?19:33
tristanor too late for you ?19:33
tlater[m]tristan: Incidentally, yes19:33
tristantlater[m], Do you have time to fix the commit message on https://gitlab.com/BuildStream/buildstream/merge_requests/1031 so that it doesn't close the wrong issue, and merge it ?19:34
tristan:)19:34
tlater[m]Haha, sure, I wanted to write a test first19:35
tlater[m]But I suppose I can merge it for now and add a test later19:35
tristantlater[m], yeah a test would be welcome19:35
cs-shadowtristan: if you try the new version of my branch, you should hopefully get better results than last time19:49
*** juergbi has quit IRC19:51
*** juergbi has joined #buildstream19:54
* cs-shadow needs to catch his train now19:54
* cs-shadow tristan: I'll update the docker-images later tonight and un-WIP the tox MR for one final review tomorrow19:54
tristancs-shadow, cool :)19:55
tristancs-shadow, codestyle not fixed ?19:56
* tristan wanted to run it :)19:56
cs-shadowtristan: should be, are you on 03e50fdf04801f8086cd1a710f8bd0136534d635 ?19:57
tristancs-shadow, yes19:57
tristanand I have no codestyle in my requirements afaics19:58
tristanwait I do19:58
cs-shadowah! i think i know19:58
cs-shadowtry adding -r to your tox invocation19:58
tlater[m]Might need to pip install again19:58
cs-shadowtox won't automatically recreate the environment if the requirements file changes19:58
* tristan runs `git clean -xdff`19:59
* cs-shadow feels we should add something `-r`/`--recreate` to the docs as this may trip people up 20:00
cs-shadow*something about20:00
tristancs-shadow, is it easier to teach that or to teach `git clean -xdf`20:00
tristanor -xdff, not sure if double force is needed in this case to bypass .gitignore20:01
tristanI think so20:01
cs-shadowi feel like `git clean -fdx` has the potential to destroy people's work if they are not cautious20:01
tristanYeah, I dont know, I know it is destructive but prefer only needing to know it hehe20:02
cs-shadowsidenote: `-ff` should only be needed if you want it to remove git repos20:02
tristanah20:02
tristanmaybe that's why I always use it with BuildStream20:02
tristan(git repos we create at test time)20:03
tristantests are running now, that did it20:03
tlater[m]tristan: Hrm, I can't get this test written up as quickly as I'd hoped. I'll defer that to later this week, do you still want me to merge?20:04
tristanthey take about 40 min here20:04
tristantlater[m], yeah20:04
* cs-shadow really needs to run, will catch you later :)20:04
tristancs-shadow, later :)20:04
tlater[m]tristan: Fixed the message, set to merge :) I'm also finishing up for the night, o/20:06
*** alatiera_ has quit IRC20:06
tristan\o20:16
*** vidal72[m] has joined #buildstream20:42
*** milloni has joined #buildstream21:06
*** milloni has quit IRC21:10
*** milloni has joined #buildstream21:11
*** mimaz has quit IRC21:24
*** tristan has quit IRC21:48
*** tristan has joined #buildstream21:48
*** mimaz has joined #buildstream21:53
*** max has joined #buildstream21:54
*** mimaz has quit IRC21:59
*** mieszko has joined #buildstream22:15
gitlab-br-botBenjaminSchubert opened issue #842 (File with invalid characters prevents Windows from checking out master) on buildstream https://gitlab.com/BuildStream/buildstream/issues/84222:18
*** mieszko has quit IRC22:38
*** tristan has quit IRC22:39
*** mieszko has joined #buildstream22:56
*** mieszko has quit IRC23:18
*** mieszko has joined #buildstream23:19
*** mieszko has quit IRC23:23

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!