IRC logs for #buildstream for Wednesday, 2019-10-16

« Tuesday, 2019-10-15 Index Thursday, 2019-10-17 »
*** rdale has quit IRC03:35
*** rdale has joined #buildstream03:36
*** slaf has quit IRC07:04
*** slaf has joined #buildstream07:06
*** slaf has joined #buildstream07:06
*** slaf has joined #buildstream07:07
*** slaf has joined #buildstream07:07
Kinnisonbenschubert: !1644 👍07:33
*** traveltissues has joined #buildstream07:38
benschubertKinnison: thanks!08:25
*** tiagogomes has joined #buildstream08:40
*** santix has joined #buildstream08:43
*** toscalix has joined #buildstream08:48
gitlab-br-botmarge-bot123 merged MR !1643 (aevri/oldgit->master: testing/.../site: windows-friendly HAVE_OLD_GIT) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164308:52
*** bochecha has joined #buildstream08:56
*** jonathanmaw has joined #buildstream09:20
*** lachlan has quit IRC09:43
*** lachlan has joined #buildstream09:59
*** lachlan has quit IRC10:47
*** lachlan has joined #buildstream10:52
*** santix has quit IRC10:55
*** santix has joined #buildstream10:56
gitlab-br-bottlater opened MR !1645 (tlater/artifactserver-casd->master: Refactor casserver.py: Stop relying on the buildstream-internal `CASCache` implementation) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164511:59
*** santix has quit IRC12:46
gitlab-br-bottraveltissues opened issue #1171 (Review use of `shutil.rmtree`) on buildstream https://gitlab.com/BuildStream/buildstream/issues/117112:53
*** tiagogomes has quit IRC12:56
*** tiagogomes has joined #buildstream12:56
*** santix has joined #buildstream12:59
tlater[m]traveltissues: Do you have any advice for getting mypy to recognize grpc things as types?13:10
tlater[m]For context: https://gitlab.com/BuildStream/buildstream/-/jobs/32257435513:11
traveltissuesf2f13:24
benschuberttlater[m]: we have them ignored normally in the setup.cfg becasue it is a pain :)13:25
benschubertiff we want types, something akin to https://gitlab.com/BuildStream/buildstream/commit/cdf902eda8c2ff551b4b958a95460c942bcb2397 could be done13:25
tlater[m]benschubert: They're not actually relevant, I was misunderstanding the "grpc things" as constructors13:26
tlater[m]We have type aliases for the things those methods return13:26
tlater[m]Notably, typing.SourceRef and directory.Directory13:26
tlater[m]Hrm, no, they aren't those things after all13:32
* tlater[m] will give on type hinting these particular objects13:32
tlater[m]Shame, it really made it more clear what this code was doing13:32
gitlab-br-botcs-shadow approved MR !1644 (bschubert/yaml-tidying->master: Some tidying up of our yaml API) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164414:13
gitlab-br-botmarge-bot123 merged MR !1644 (bschubert/yaml-tidying->master: Some tidying up of our yaml API) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164414:16
benschubertWould people be fine with me tagging a new dev release? ^ was a breaking change on the YAML API and we haven't got a release in 4 weeks14:32
*** lachlan has quit IRC14:43
* juergbi wouldn't mind14:47
*** lachlan has joined #buildstream14:50
gitlab-br-botBenjaminSchubert opened MR !1646 (bschubert/1.91.1->master: NEWS: update release notes for version 1.91.1) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164614:53
gitlab-br-bottraveltissues approved MR !1646 (bschubert/1.91.1->master: NEWS: update release notes for version 1.91.1) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164614:54
benschubertjuergbi: ^ am I forgetting anything in the last 4 weeks? :)14:54
tlater[m]It's already been 4 weeks?14:54
benschubertyup...14:54
juergbibenschubert: traveltissues: maybe mention that incremental workspace builds have temporarily been disabled?14:56
benschubertGood point14:59
benschubertupdated14:59
benschubertah workspaces now work with RE right?15:00
juergbithey should but I think it hasn't been verified yet15:04
benschubertok then I won't add it in the NEWS15:05
benschubertgood like that for you?15:05
tlater[m]juergbi: refs/heads still seems to be used by the buildstream client to remove artifacts15:06
tlater[m]Or did I misunderstand that conversation between you and benschubert?15:07
tlater[m]Admittedly, that's the only use I can see at a glance, but it's a rather important one15:08
gitlab-br-botjuergbi approved MR !1646 (bschubert/1.91.1->master: NEWS: update release notes for version 1.91.1) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164615:08
tlater[m]Oh, no, bunch of other uses that are hidden behind _refpath()15:08
*** lachlan has quit IRC15:09
juergbitlater[m]: artifact protos are not stored in that directory, though15:09
juergbiwhat am I missing?15:09
juergbiartifactcache/sourcecache might (ab)use a CASCache helper, but they don't actually use cas/refs/heads, afaict15:10
tlater[m]juergbi: In stream some functions call CASCache.delete to delete artifacts15:11
tlater[m]E.g. stream.py:72215:11
juergbithat's ArtifactCache.remove()15:11
tlater[m]This will refer to refs/heads/artifact15:12
juergbino15:12
*** lachlan has joined #buildstream15:12
tlater[m]Ah, right15:12
tlater[m]:|15:12
juergbiwe should remove that abused call15:12
* tlater[m] wonders how he got that function15:12
* tlater[m] figures he'll just rip apart any method that relies on refs/heads and sees what breaks15:13
tlater[m]Hm, a lot of things brea15:17
tlater[m]k15:17
tlater[m]juergbi: Looks like I found it because ArtifactCache.remove calls CASCache.remove15:26
juergbiyes but it overrides basedir15:26
juergbiwhich is why I wrote 'abuse' above15:26
tlater[m]It's enough to remove the default value for that15:27
tlater[m]Do you think it would make sense to just call down to _remove_ref instead of going through the indirection?15:27
juergbitlater[m]: I think it would be cleaner to make remove_ref a utils function15:29
juergbiit's a relatively generic thing. although, not sure what the best name would be15:29
tlater[m]Hrm15:30
juergbiand it doesn't actually use `self`15:30
juergbiif it wasn't for the server backward compatibility, I might move it to BaseCache15:30
tlater[m]I suppose that would work for `contains` as well, or perhaps a testutil15:30
tlater[m]juergbi: The server doesn't call that at all anymore15:30
tlater[m]It's using its own implementation15:30
juergbiah, then let's just move it to BaseCache15:31
juergbior do you have a better suggestion?15:31
tlater[m]No, that seems good15:31
traveltissuesjuergbi, are you ok with the changes on !164015:41
gitlab-br-botMR !1640: Workspaces via sourcecache MR 2 https://gitlab.com/BuildStream/buildstream/merge_requests/164015:41
*** santix has quit IRC15:44
*** santix has joined #buildstream15:55
gitlab-br-botmarge-bot123 merged MR !1646 (bschubert/1.91.1->master: NEWS: update release notes for version 1.91.1) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164616:10
gitlab-br-botjuergbi approved MR !1640 (traveltissues/1140-softreset->master: Workspaces via sourcecache MR 2) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164016:19
traveltissuesty16:21
traveltissuesyes, it was why i wanted to avoid that flag, because of the number of places it needed to be added16:28
juergbitraveltissues: I was initially expecting it to be a Loader constructor parameter, so we wouldn't have to pass it around, but that doesn't fit with how/when we're constructing the Laoder16:30
juergbitheoretically we could have a method on the Workspace object to temporarily hide all workspaces instead16:30
juergbibut such state is also not ideal16:31
traveltissuesyes16:31
traveltissuesunfortunately the callstack is fairly complicated16:31
juergbi*Workspaces object16:31
juergbiyes, which is why it's tempting to make it somewhat stateful instead16:32
juergbibut in general I prefer the current stateless approach16:32
tlater[m]aevri: ooi, how does !1638 break multiprocessing spawn?16:32
gitlab-br-botMR !1638: WIP: cascache: refactor, extract CASDProcessManager https://gitlab.com/BuildStream/buildstream/merge_requests/163816:32
* tlater[m] was hoping to use it instead of rolling his own CASDProcessManager in `casserver.py`16:33
juergbiit's also not actually failing the CI spawn test job16:36
aevritlater: aha, re-use, I like it :) the manager itself is fine, it's because cascache pickles it as a bool, instead of the object itself. I'll need to make the manager picklable instead.16:36
aevrijuergbi: that's the much bigger problem this uncovered, I've been looking at that this afternoon.16:36
juergbiaevri: what part of the manager is needed in the subprocess?16:37
juergbifor subprocesses buildbox-casd would be a sibling, so I'm surprised they need anything from the processmanager16:37
aevriit's actually never been testing the spawn path, I'm trying to figure out how much needs to be fixed :S16:37
juergbioh :-/16:37
aevriyeah16:37
tlater[m]Fair enough16:38
aevriI get the impression that as long as we can share the connection string, then it'll all work out. For now I'm focused on the 'not testing any spawning' thing :)16:38
tlater[m]Yeah, that's definitely higher priority ;) Ta aevri16:39
aevrihehe sure thing16:39
juergbiah, connection string. that makes sense16:42
*** santix has quit IRC16:49
*** adds68 has quit IRC17:16
*** ikerperez has quit IRC17:16
*** qinusty has quit IRC17:16
*** jonathanmaw has quit IRC17:20
*** tiagogomes has quit IRC17:22
*** lachlan has quit IRC17:22
*** lachlan has joined #buildstream17:27
gitlab-br-botmarge-bot123 closed issue #1140 (Follow-up from "workspaces via sourcecache 1" - workspace.init_workspace should raise a SourceError) on buildstream https://gitlab.com/BuildStream/buildstream/issues/114017:30
gitlab-br-botmarge-bot123 merged MR !1640 (traveltissues/1140-softreset->master: Workspaces via sourcecache MR 2) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164017:30
benschuberthttps://gitlab.com/BuildStream/bst-plugins-experimental/merge_requests/48 for adapting to 1.91.1 :)17:36
*** traveltissues has quit IRC17:38
*** narispo has joined #buildstream18:24
*** narispo has quit IRC18:30
*** narispo has joined #buildstream18:30
*** lachlan has quit IRC18:41
*** lachlan has joined #buildstream18:48
*** bochecha_ has joined #buildstream18:56
*** bochecha_ has quit IRC18:58
*** toscalix has quit IRC19:02
*** lachlan has quit IRC19:08
*** lachlan has joined #buildstream19:10
*** narispo has quit IRC19:12
*** narispo has joined #buildstream19:12
*** narispo has quit IRC19:16
*** narispo has joined #buildstream19:16
*** narispo has quit IRC19:40
*** narispo has joined #buildstream19:40
*** lachlan has quit IRC20:06
*** narispo has quit IRC20:09
*** narispo has joined #buildstream20:10
*** narispo has quit IRC21:50
*** narispo has joined #buildstream21:50
*** narispo has quit IRC22:00
*** narispo has joined #buildstream22:01
*** narispo has quit IRC22:45
*** narispo has joined #buildstream22:45
*** narispo has quit IRC23:00
*** narispo has joined #buildstream23:00
*** narispo has quit IRC23:13
*** narispo has joined #buildstream23:14
*** narispo has quit IRC23:17
*** narispo has joined #buildstream23:18
« Tuesday, 2019-10-15 Index Thursday, 2019-10-17 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!