IRC logs for #buildstream for Thursday, 2019-06-06

*** xjuan has joined #buildstream02:11
*** xjuan has quit IRC02:43
*** tristan has quit IRC05:55
*** tristan has joined #buildstream06:26
*** xjuan has joined #buildstream06:30
*** xjuan has quit IRC06:43
jennisbenschubert, thanks for all the lint fixes08:07
jennisIt'd be nice if we could disable "redefined-outer-name" at a directory level08:08
jennisAnd also, are *all* of the "unused-import" disables for cli really necessary? I think for some modules, we perhaps don't need them. i.e. when we explicitly invoke `cli.foo()`  (would be my guess) we should avoid adding this disable comment if we don't need it08:09
benschubertjennis: no worries. We sadly can't disable at a per-directory level, I've checked and am subscribed on the pylint issue about that08:11
benschubertFor the unused-import, I haven't found a place where we could not put it :/08:11
jennisAhh that's a shame, I just asked on #pylint too, but if there is an open issue I guess its not possible08:12
jennisMhmm, so even for modules where we explicitly invoke it, pylint is complaining?08:12
benschubertYep, because the name is shadowed by the function parameter08:13
benschubertso the import looks like unused08:13
benschuberthttps://github.com/PyCQA/pylint/issues/618 for the issue08:13
jennisMight be worth linking that issue in the commit message?08:14
jennisAlso, the issue about no blanket disables08:15
benschubertjennis: we had it already linked in the first commit about tests and linting, I'm not sure it's good relinking on every linting change :) But if you'd rather have it, I can add it08:21
*** adds68 has joined #buildstream08:21
jennismhmm I think for the sake of git blame, if someone goes, "why the hell are these disables in every file", it'd be nice for them to immediately see the issue08:25
jennisWhich is exact thought process I just went through08:25
benschubertsure08:26
benschubertI'll add this08:26
*** raoul has joined #buildstream08:28
*** ikerperez has joined #buildstream08:33
gitlab-br-botaevri approved MR !1378 (bschubert/pylint-fixes->master: Ensure pylint runs in some tests paths) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137808:37
jennisbenschubert, from #pylint: jennis: maybe a pylintrc in the directory? The doc mentions "If the current working directory is in a Python module, Pylint searches up the hierarchy of Python modules until it finds a pylintrc file" but I don't know whether that's the cwd, the root directory provided to pylint, or the directory it's currently working on08:38
jennis(aka the directory containing the files it's processing at the moment)08:38
jennisRegarding a blanket disable ^08:39
benschubertjennis: the problem is you can't compose two configs08:39
benschubertso you defacto  end up with two different pylint files08:39
jennisawh08:39
benschubertjennis: https://gitlab.com/BuildStream/buildstream/merge_requests/1222 for reference08:40
benschubertthat's where we decided to disable per file instead08:40
gitlab-br-botaevri approved MR !1377 (bschubert/pylint-artifactcache->master: Ensure pylint runs in tests/artifactcache) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137708:40
jenniscool, thanks for pointing that out08:42
Kinnisonbenschubert: can we use a makefile to combine them?08:42
benschubertKinnison: I'm not sure that's worth the hack08:43
Kinnisonfair08:43
*** pointswaves has joined #buildstream08:44
*** jonathanmaw has joined #buildstream08:46
jennistristan, are you around? Would you be able to respond to the final review comments on !1344? There is one comment (regarding Queue.PENDING) that I'm not clear on how you want documented08:46
gitlab-br-botMR !1344: WIP: Push based pipeline https://gitlab.com/BuildStream/buildstream/merge_requests/134408:46
gitlab-br-botjennis approved MR !1376 (bschubert/pylint-integration->master: Ensure pylint runs in tests/integration) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137608:50
jennisbenschubert, I probably jumped the gun on that ^, I should say, subject to the upstream issues in the commit messages08:51
benschubertjennis: about !1376, could you double check that the new message is fine for you?08:51
jennismhmm small nit from me: "https://github.com/PyCQA/pylint/issues/618 about this." doesn't read so well08:52
jennisPerhaps "See upstream issue: https://github.com/PyCQA/pylint/issues/618"08:53
gitlab-br-botjennis approved MR !1376 (bschubert/pylint-integration->master: Ensure pylint runs in tests/integration) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137608:55
jennisCool, thanks08:55
benschubertI'll do the same on the other two PRs08:55
jennisOk, let me know when you've pushed and I'll re-review08:56
*** pointswaves has quit IRC08:56
benschubertjennis: ndone for !137709:02
benschubertmarge seems to have died again?09:05
jennis:@09:14
*** phildawson_ has joined #buildstream09:14
gitlab-br-botjennis approved MR !1377 (bschubert/pylint-artifactcache->master: Ensure pylint runs in tests/artifactcache) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137709:15
jennisbenschubert, apparently she's not dead09:20
benschubertjennis: for https://gitlab.com/BuildStream/buildstream/merge_requests/1378 I added it on the MR description, so it will be on the merge commit, goodf for you?09:25
jennisbenschubert, sec09:27
jennisyeah nice idea09:28
gitlab-br-botjennis merged MR !1378 (bschubert/pylint-fixes->master: Ensure pylint runs in some tests paths) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137809:28
jennisAhh09:28
jennisWHOOPS09:28
jennismerge != approve09:28
jennismy bad, I'll accept my public flogging09:29
*** CTtpollard is now known as tpollard09:32
* tpollard flogs jennis 09:35
*** lachlan has joined #buildstream09:44
*** lachlan has quit IRC09:52
*** lachlan has joined #buildstream09:59
jennishttps://gitlab.com/BuildStream/buildstream/merge_requests/1376#note_178493881 Marge is broken from the inside again10:02
gitlab-br-botjuergbi opened issue #1042 (Use buildbox-casd for CAS access) on buildstream https://gitlab.com/BuildStream/buildstream/issues/104210:07
*** lachlan has quit IRC10:14
gitlab-br-botdanielsilverstone-ct approved MR !1379 (bschubert/optimize-loader-types->master: Optimize _loader/types.py) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137910:16
gitlab-br-bottpollard approved MR !1374 (aevri/spawk->master: Rename (spawn, fork) -> 'start process') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137410:16
*** lachlan has joined #buildstream10:16
Kinnisonsigh10:16
tpollard?10:17
*** lachlan has quit IRC10:32
jennisso tests/sourcecache/push.py::test_push_fail keeps failing and we're getting orange pipelines10:42
jennisI've run this test locally inside the same docker container we're using for "tests-fedora-update-deps" and it passes :/10:43
jennisNot sure if anyone has an idea about this10:43
tpollardgot a link to a failed console?10:44
jennisEvery orange pipeline :P10:44
jennisbut here's one: https://gitlab.com/BuildStream/buildstream/-/jobs/22635365510:44
jennisLooks like it's just trying to equate a long string to an object10:45
jennisIt doesn't look like the "in" syntax is doing what we'd expect, but I could be wrong10:45
tpollard:S10:50
tpollarddidn't we have another problem recently with that docker image10:51
jennisI'm not sure10:52
tpollardit's ringing a bell10:52
*** lachlan has joined #buildstream10:53
tpollardI can't find anything, but I definitely remember recently looking at the definition of it for some reason10:55
tpollardsomething something an update to grpc which changed some error api10:59
tpollarddoesn't that ring a bell raoul?10:59
raoulyep that sounds right11:01
raoulgrpc gives a different error in the updated version so test fails11:01
raoulyou might not have updated your docker image or something which could be causing it to pass locally.11:01
laurenceWe're seeing some issues with marge-bot at the moment - one is that there's a bug causing random intermittent failures11:02
jennisThis is looking pretty consistent11:03
laurencehttps://github.com/smarkets/marge-bot/issues/18811:03
laurenceraised upstream ^11:03
laurenceand the other is that it's managed by Codethink and would be better if this project looked after it11:03
laurenceat the same time i think it would be wise to help 'spread the load' of managing everything infra related away from just relying solely on valentind11:03
laurenceand increase the bus factor11:04
tpollardjennis: https://irclogs.baserock.org/buildstream/%23buildstream.2019-05-30.log.html#t2019-05-30T10:56:45 the conversation was here11:04
laurenceI shall email the list as well, but any volunteers for this?11:04
jennistpollard, juergbi, raoul, so it turns out the assert is triggering because the std.err is actually different11:32
jennisI wanted to know whether it'd be ok to make the check less strict11:32
jennisRight now, we check for: "Failed to initialize remote http://localhost:34687: Connect Failed"11:33
gitlab-br-botmarge-bot123 merged MR !1376 (bschubert/pylint-integration->master: Ensure pylint runs in tests/integration) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137611:33
jennisWhereas this test is failing because: Failed to initialize remote http://localhost:34687: failed to connect to all addresses11:33
juergbiah, gRPC error message change?11:33
juergbidefinitely sounds fine to accept either11:34
jennisSo failed to connect to all addresses != connect failed11:34
jennisOk, I'll make it less strict11:34
tpollardyes accept either, else we'll get this when the other pipelines bump to the same version of grpc11:37
*** bilelmoussaoui has joined #buildstream11:38
gitlab-br-botjennis opened MR !1381 (jennis/fix_failing_test->master: Don't assert gRPC messages which may change) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138111:45
*** bilelmoussaoui has quit IRC11:47
jennisjuergbi, tpollard ^11:51
jennisI didn't go with accepting either just incase it changes again...11:53
gitlab-br-bottpollard approved MR !1381 (jennis/fix_failing_test->master: Don't assert gRPC messages which may change) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138111:55
tpollardWe're only bothered that it fails to connect with the given remote ip, the rest of the message is irrelevant imo11:56
gitlab-br-botraoul.hidalgocharman approved MR !1381 (jennis/fix_failing_test->master: Don't assert gRPC messages which may change) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138112:00
gitlab-br-botmarge-bot123 merged MR !1373 (aevri/defensive_send_message->master: _scheduler/jobs: refactor, defensive send_message) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137312:15
gitlab-br-botwillsalmon approved MR !1375 (bschubert/site-consolidation->master: Remove tests/testutils/site.py and move everything to buildstream/testing/_utils/site.py) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137512:53
*** tpollard has quit IRC13:12
*** tpollard has joined #buildstream13:15
*** ChanServ sets mode: +o tristan13:25
tristanjonathanmaw, Gah, I totally overlooked your email today, I saw it come in late last night and intended to give it attention :-/13:25
* tristan makes a note to not forget tomorrow13:26
jonathanmaw:)13:26
benschubertHey tristan in !1375, I consolidated all the sites.py files for testing. Would you see a reason not to make it public? A few plugin projects redefine them, I think we would gain by making this public13:27
*** lachlan has quit IRC13:29
tristanbenschubert, seems odd to me to make public, if things are well segmented13:30
benschuberttristan: the HAS_SANDBOX for example is requried by many13:30
benschubertsame for IS_WSL/Linux/Windows13:31
tristanbenschubert, I mean... repo has their own different set of dependencies right ? it wouldnt make sense for BuildStream to have knowledge about third party plugin repo dependencies13:31
benschuberttristan: I didn't mean adding things from other repos13:31
tristanbenschubert, for what is common, yeah that seems to make sense13:31
benschubertbut I think that BuildStream should open the common stuff :)13:31
tristanI didn't look through the patch, just judging the book by its cover ;-)13:31
benschubertOk, so not expose everything (Git, Bzr, etc) but open Has_sandbox and MACHINR_ARCH, etc ?13:32
tristanbenschubert, it make sense only in testing/ ? or are there things that a plugin should know about at runtime and not at test time ?13:32
benschubertonly test time13:32
benschubertthat's in testings/_utils/site currently13:33
tristanRight, sounds sensible to me :)13:33
benschubertand I'd be for putting at least some of it in testing/site13:33
benschubertOk! I'll add a new PR splitting those then13:33
benschubertthanks!13:33
tristanbenschubert, sorry I didn't get around to that last night, saw you left a comment on IRC13:33
* tristan has been booting VMs all day13:34
benschubertah no worries, it was mainly for information purposes :)13:34
*** lachlan has joined #buildstream13:37
benschubertjennis: was https://gitlab.com/BuildStream/buildstream/merge_requests/1377 good for you?13:40
jennisbenschubert, one sec13:40
gitlab-br-botjennis approved MR !1377 (bschubert/pylint-artifactcache->master: Ensure pylint runs in tests/artifactcache) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137713:41
jennisbenschubert, yes13:41
gitlab-br-botmarge-bot123 merged MR !1381 (jennis/fix_failing_test->master: Don't assert gRPC messages which may change) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138113:59
tpollardwoop13:59
*** tristan has quit IRC14:26
gitlab-br-botmarge-bot123 merged MR !1374 (aevri/spawk->master: Rename (spawn, fork) -> 'start process') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137414:33
*** tristan has joined #buildstream14:54
laurenceThere's an old ticket lying around in #nosoftware related to creation of a sub-group for infrastructure - https://gitlab.com/BuildStream/nosoftware/alignment/issues/4215:02
laurenceanyone mind if i create this?15:02
laurencevalentin and I have started to write down all of the infra connected to the project15:03
laurencewe need somewhere public to store it (apart from credentials, obviously)15:03
gitlab-br-botmarge-bot123 merged MR !1375 (bschubert/site-consolidation->master: Remove tests/testutils/site.py and move everything to buildstream/testing/_utils/site.py) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137515:04
laurenceMaybe just using the wiki would be enough, actually15:05
gitlab-br-botaevri opened (was WIP) MR !1380 (aevri/job_msg_enum->master: Use enums for Job-related constants) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138015:51
gitlab-br-botmarge-bot123 merged MR !1377 (bschubert/pylint-artifactcache->master: Ensure pylint runs in tests/artifactcache) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137715:52
gitlab-br-botraoul.hidalgocharman approved MR !1380 (aevri/job_msg_enum->master: Use enums for Job-related constants) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138016:22
*** xjuan has joined #buildstream16:24
gitlab-br-botBenjaminSchubert approved MR !1380 (aevri/job_msg_enum->master: Use enums for Job-related constants) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138016:32
*** raoul has quit IRC16:38
*** xjuan has quit IRC16:48
gitlab-br-botmarge-bot123 merged MR !1379 (bschubert/optimize-loader-types->master: Optimize _loader/types.py) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/137916:48
*** pointswaves has joined #buildstream17:10
*** shibu has joined #buildstream17:11
*** shibu has quit IRC17:13
*** lachlan has quit IRC17:14
gitlab-br-botmarge-bot123 merged MR !1380 (aevri/job_msg_enum->master: Use enums for Job-related constants) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138017:17
*** jonathanmaw has quit IRC17:22
*** lachlan has joined #buildstream17:28
*** pointswaves has quit IRC17:33
*** pointswaves has joined #buildstream17:36
*** pointswaves has quit IRC17:39
*** pointswaves has joined #buildstream17:40
*** pointswaves has quit IRC17:41
*** xjuan has joined #buildstream17:41
*** pointswaves has joined #buildstream17:42
*** lachlan has quit IRC17:44
*** pointswaves has quit IRC17:47
*** jswagner has joined #buildstream18:57
gitlab-br-botBenjaminSchubert opened MR !1383 (bschubert/optimize-extract-depends-node->master: rewrite _extract_depends_from_node in Cython and optimize) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/138320:26
*** xjuan has quit IRC21:04
*** xjuan has joined #buildstream21:50

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!