| *** tristan has quit IRC | 02:35 | |
| *** nimish2711 has joined #buildstream | 02:41 | |
| *** tristan has joined #buildstream | 03:05 | |
| *** nimish2711 has quit IRC | 04:53 | |
| *** nimish2711 has joined #buildstream | 05:39 | |
| *** nimish2711 has quit IRC | 05:44 | |
| *** nimish2711 has joined #buildstream | 05:46 | |
| *** nimish2711 has quit IRC | 06:29 | |
| *** nimish2711 has joined #buildstream | 06:49 | |
| *** benschubert has joined #buildstream | 07:51 | |
| coldtom | if anyone's interested, i spent some time this weekend procrastinating uni work by building BuildStream in BuildStream: http://othko.xyz/posts/bst-in-bst/ | 08:07 |
|---|---|---|
| Kinnison | coldtom: did you use skullone[m]'s stuff as a basis? | 08:22 |
| Kinnison | either way, nice job. +1 | 08:23 |
| coldtom | Kinnison: nope, I wasn't aware of any other stuff in that direction | 08:23 |
| Kinnison | https://gitlab.com/BuildStream/buildstream-integration/ | 08:24 |
| Kinnison | was a start | 08:24 |
| Kinnison | *please* feel free to improve/update that with the results of your weekend efforts :D | 08:25 |
| *** Guest62 has joined #buildstream | 08:27 | |
| *** rdale has joined #buildstream | 08:32 | |
| *** raoul has joined #buildstream | 08:35 | |
| *** tpollard has joined #buildstream | 08:59 | |
| *** jonathanmaw has joined #buildstream | 09:14 | |
| *** nimish2711 has quit IRC | 09:23 | |
| *** Guest62 has quit IRC | 09:25 | |
| *** lachlan has joined #buildstream | 09:36 | |
| *** lachlan has quit IRC | 09:40 | |
| *** lachlan has joined #buildstream | 09:42 | |
| *** lachlan has quit IRC | 09:45 | |
| *** tristan has quit IRC | 09:52 | |
| *** lachlan has joined #buildstream | 09:53 | |
| jennis | coldtom, nice blog post | 09:56 |
| *** tristan has joined #buildstream | 09:58 | |
| *** lachlan has quit IRC | 10:04 | |
| *** lachlan has joined #buildstream | 10:09 | |
| *** lachlan has quit IRC | 10:15 | |
| *** lachlan has joined #buildstream | 10:32 | |
| *** Guest62 has joined #buildstream | 10:43 | |
| *** lachlan has quit IRC | 10:44 | |
| *** lachlan has joined #buildstream | 10:46 | |
| *** lachlan has quit IRC | 10:51 | |
| *** lachlan has joined #buildstream | 10:53 | |
| *** nimish2711 has joined #buildstream | 10:55 | |
| *** lachlan has quit IRC | 11:06 | |
| *** toscalix has joined #buildstream | 11:18 | |
| *** lachlan has joined #buildstream | 11:20 | |
| *** toscalix has quit IRC | 11:27 | |
| *** nimish2711 has quit IRC | 12:08 | |
| *** nimish2711 has joined #buildstream | 12:09 | |
| *** nimish2711 has quit IRC | 12:41 | |
| *** nimish2711 has joined #buildstream | 12:52 | |
| *** toscalix has joined #buildstream | 12:54 | |
| *** Guest62 has quit IRC | 13:03 | |
| *** Guest62 has joined #buildstream | 13:04 | |
| *** Guest62 has quit IRC | 13:08 | |
| *** lachlan has quit IRC | 13:10 | |
| *** Guest62 has joined #buildstream | 13:19 | |
| *** lachlan has joined #buildstream | 13:29 | |
| *** Guest62 has quit IRC | 13:29 | |
| *** Guest62 has joined #buildstream | 13:31 | |
| *** lachlan has quit IRC | 13:34 | |
| tpollard | urk https://gitlab.com/BuildStream/buildstream/-/jobs/192624130 | 13:35 |
| Kinnison | that doesn't sound good | 13:35 |
| juergbi | tpollard: that's a flaky test issue we see occasionally. just retry | 13:40 |
| juergbi | (asyncio issue) | 13:40 |
| *** lachlan has joined #buildstream | 13:43 | |
| *** toscalix has quit IRC | 13:48 | |
| jennis | juergbi, I'm able to cd into the tmp dir where the test fails and build with remote execution but I'm not able to get the tests passing | 14:13 |
| jennis | My guess is that it's not seeing the user config | 14:13 |
| juergbi | jennis: what failure are you seeing? | 14:14 |
| jennis | NOTIMPLEMENTED | 14:14 |
| jennis | or something like that | 14:14 |
| jennis | sec | 14:14 |
| jennis | [00:00:00][77965fc3][ build:autotools/amhello.bst ] FAILURE Failed to query action cache: StatusCode.UNIMPLEMENTED (Method not found!) | 14:16 |
| jennis | Do you need an action cache env variable too? | 14:16 |
| jennis | Our gitlab-ci.yml file suggests not | 14:17 |
| *** lachlan has quit IRC | 14:19 | |
| *** lachlan has joined #buildstream | 14:23 | |
| jennis | Not having any luck with gitlab-ci/buildgrid.compose.yml either | 14:29 |
| *** lachlan has quit IRC | 14:30 | |
| *** lachlan has joined #buildstream | 14:34 | |
| juergbi | jennis: in my setup I have all three service (execution, action cache, CAS) on the same endpoint | 14:44 |
| jennis | Do you have another container running as a BuildStream remote cache then? | 14:45 |
| juergbi | I'm not using containers for this | 14:45 |
| juergbi | Just a single bgd instance + a worker + bst-artifact-server | 14:46 |
| jennis | I see, but it should be possible | 14:46 |
| jennis | The compose image we're using just spins up exactly that, right? | 14:46 |
| jennis | compose manifest* | 14:46 |
| *** lachlan has quit IRC | 14:47 | |
| juergbi | I think that has two different containers, one for execution service + action cache and one for CAS | 14:49 |
| juergbi | (+ worker[s]) | 14:50 |
| *** lachlan has joined #buildstream | 14:51 | |
| jennis | No idea what's going on | 15:02 |
| jennis | I can build remotely if I don't go via the testsuite | 15:02 |
| jennis | If I am building with remote execution and `bst --cache-buildtrees always build whateva.bst`, should I expect my local cache to contain buildtrees? | 15:03 |
| jennis | And vice versa, if I `bst build whateva.bst` I should expect that my local cache is free of buildtrees (providing the build passed) | 15:04 |
| *** phildawson has joined #buildstream | 15:19 | |
| *** phildawson has quit IRC | 15:20 | |
| *** lachlan has quit IRC | 15:20 | |
| juergbi | jennis: yes, that's what I'd expect | 15:20 |
| *** lachlan has joined #buildstream | 15:24 | |
| jennis | Interesting... | 15:31 |
| jennis | juergbi, have you seen this error before from the bot? | 15:34 |
| jennis | 2019-04-08 16:33:50,835:[ buildgrid.bot.tenant][ERROR]: [Errno 2] No such file or directory: 'buildbox': 'buildbox | 15:34 |
| *** lachlan has quit IRC | 15:37 | |
| *** lachlan has joined #buildstream | 15:42 | |
| juergbi | jennis: that would mean buildbox-fuse is not installed | 15:50 |
| raoul | Should read services (bytestream Read and CAS BatchReadBlobs) be updating mtimes for blobs? I've realised as I've only done an update_tree_mtime for an artifact files, so other parts may expire sooner, even if they've been read since | 16:10 |
| raoul | Could just make the artifact service update files mtimes but I think it'd make sense to update times when we read them | 16:11 |
| juergbi | raoul: we should do it in the artifact service, imo | 16:16 |
| juergbi | as we want to keep those files around even if some happen to be cached already on the cache (and thus, the cache doesn't call Read/BatchReadBlobs) | 16:16 |
| juergbi | theoretically, the server could have different policies for different file types. however, I think for now we best keep it simple | 16:17 |
| raoul | Yeah wasn't sure if we'd maybe want to update mtimes for when we check if blobs are on a server too, but can just put it all in the artifact service for now | 16:18 |
| *** nimish2711 has quit IRC | 16:20 | |
| *** lachlan has quit IRC | 16:20 | |
| juergbi | raoul: with the planned separation between artifact service and CAS, the timestamp update should be triggered indirectly by the artifact service calling GetMissingBlobs() on the CAS server | 16:25 |
| juergbi | but in the first step bst-artifact-server will not use remote CAS, so it can do it internally | 16:25 |
| juergbi | eh, *FindMissingBlobs() | 16:26 |
| raoul | juergbi, I thought the plan would be to have the artifact service just return the artifact protos, and then it'd be up to clients to call GetMissingBlob on the CAS server? That way if we don't use logs or buildtrees they'd be allowed to expire | 16:27 |
| juergbi | it could be done by the client, but then the client would essentially define the retention policy on the CAS server | 16:29 |
| juergbi | and that doesn't sound ideal | 16:29 |
| juergbi | and artifact service likely has higher bandwidth and lower latency to the CAS server, so FindMissingBlobs() server to server would likely be more efficient | 16:30 |
| raoul | how would it then deal with expiring blobs that aren't used? I thought this was something we wanted to eventually do | 16:31 |
| juergbi | which blobs do you mean? parts of the payload/files, or buildtree/logs? | 16:32 |
| *** lachlan has joined #buildstream | 16:32 | |
| raoul | buildtree/logs | 16:33 |
| juergbi | for buildtree/logs, the artifact server could potentially have configuration whether the timestamp update / FindMissingBlobs() should include buildtree and/or logs | 16:33 |
| juergbi | for an artifact server, I don't think we want to make this client driven for the individual parts | 16:34 |
| juergbi | i.e., I expect server side configuration to be more useful in practice, but probably depends on the organization/project | 16:35 |
| raoul | Right, that makes sense, I guess a lot of this will be something to think about in more detail later | 16:35 |
| raoul | anyway I'll add bits to update the blobs from the artifact service | 16:35 |
| raoul | cheers juergbi :) | 16:35 |
| juergbi | yes, and it may be sensible to support timestamp update on Read() in the CAS server but maybe as configuration option | 16:36 |
| juergbi | I'd prefer not requiring clients to call FindMissingBlobs() for timestamp updates, though (as that call is not really needed on the client side for downloads) | 16:37 |
| *** Guest62 has quit IRC | 16:45 | |
| jennis | juergbi, eventually got to the bottom of it... Martin's compose manifest spins up a separate container (on another port) for the storage service, our testsuite expects this service to be at the same endpoint as AC and the execution service | 16:49 |
| *** raoul has quit IRC | 17:08 | |
| *** benschubert has quit IRC | 17:18 | |
| *** lachlan has quit IRC | 17:42 | |
| *** lachlan has joined #buildstream | 17:43 | |
| *** jonathanmaw has quit IRC | 17:51 | |
| *** jonathanmaw has joined #buildstream | 17:52 | |
| *** jonathanmaw has quit IRC | 18:01 | |
| *** lachlan has quit IRC | 19:00 | |
| *** lachlan has joined #buildstream | 19:15 | |
| *** lachlan has quit IRC | 19:16 | |
| *** rdale has quit IRC | 21:18 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!