| *** jjardon has quit IRC | 01:08 | |
| *** skullone[m] has quit IRC | 01:08 | |
| *** dineshdb[m] has quit IRC | 01:08 | |
| *** tchaik[m] has quit IRC | 01:08 | |
| *** mrmcq2u[m] has quit IRC | 01:08 | |
| *** lchlan has quit IRC | 01:08 | |
| *** waltervargas[m] has quit IRC | 01:08 | |
| *** ssssam[m] has quit IRC | 01:08 | |
| *** lchlan has joined #buildstream | 01:08 | |
| *** abderrahim[m] has quit IRC | 01:08 | |
| *** benschubert has quit IRC | 01:08 | |
| *** pro[m] has quit IRC | 01:08 | |
| *** benschubert has joined #buildstream | 01:08 | |
| *** Trevinho[m] has quit IRC | 01:08 | |
| *** m_22[m] has quit IRC | 01:08 | |
| *** Demos[m] has quit IRC | 01:09 | |
| *** awacheux[m] has quit IRC | 01:09 | |
| *** jjardon[m] has quit IRC | 01:09 | |
| *** mattiasb has quit IRC | 01:09 | |
| *** doras has quit IRC | 01:09 | |
| *** cgmcintyre[m] has quit IRC | 01:09 | |
| *** nielsdg has quit IRC | 01:09 | |
| *** tlater[m] has quit IRC | 01:09 | |
| *** theawless[m] has quit IRC | 01:09 | |
| *** reuben640[m] has quit IRC | 01:09 | |
| *** albfan[m] has quit IRC | 01:09 | |
| *** kailueke[m] has quit IRC | 01:09 | |
| *** krichter[m] has quit IRC | 01:09 | |
| *** connorshea[m] has quit IRC | 01:09 | |
| *** jjardon has joined #buildstream | 01:10 | |
| *** ChanServ sets mode: +o jjardon | 01:10 | |
| *** skullone[m] has joined #buildstream | 01:11 | |
| *** dineshdb[m] has joined #buildstream | 01:23 | |
| *** tchaik[m] has joined #buildstream | 02:21 | |
| *** mrmcq2u[m] has joined #buildstream | 02:46 | |
| *** waltervargas[m] has joined #buildstream | 03:02 | |
| *** ssssam[m] has joined #buildstream | 03:11 | |
| *** pro[m] has joined #buildstream | 03:37 | |
| *** abderrahim[m] has joined #buildstream | 03:47 | |
| juergbi | benschubert: that was useful: https://gitlab.com/BuildGrid/buildbox/buildbox-common/merge_requests/118 | 04:26 |
|---|---|---|
| juergbi | I haven't verified yet that this indeed fixes the issue, but I'd expect it to | 04:27 |
| *** m_22[m] has joined #buildstream | 04:39 | |
| *** Trevinho[m] has joined #buildstream | 04:55 | |
| *** jjardon[m] has joined #buildstream | 05:12 | |
| *** awacheux[m] has joined #buildstream | 05:17 | |
| *** Demos[m] has joined #buildstream | 05:18 | |
| *** mattiasb has joined #buildstream | 05:39 | |
| *** doras has joined #buildstream | 06:07 | |
| *** nielsdg has joined #buildstream | 06:08 | |
| *** cgmcintyre[m] has joined #buildstream | 06:09 | |
| *** tlater[m] has joined #buildstream | 06:17 | |
| benschubert | juergbi: cheers I'll try as soon as I'm the office! | 06:24 |
| *** theawless[m] has joined #buildstream | 07:43 | |
| *** reuben640[m] has joined #buildstream | 07:54 | |
| *** albfan[m] has joined #buildstream | 08:09 | |
| *** kailueke[m] has joined #buildstream | 08:09 | |
| *** krichter[m] has joined #buildstream | 08:10 | |
| *** connorshea[m] has joined #buildstream | 08:12 | |
| *** rdale has joined #buildstream | 08:25 | |
| *** traveltissues has joined #buildstream | 08:25 | |
| *** phildawson_ has joined #buildstream | 09:04 | |
| gitlab-br-bot | juergbi opened MR !1591 (juerg/casd->master: casremote.py: Limit request size for batch download and upload) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1591 | 09:14 |
| juergbi | benschubert: I've verified that the buildbox-common MR fixes the issue here with a test case | 09:15 |
| benschubert | juergbi: oh awesome! | 09:15 |
| juergbi | and !1591 should fix #1129 | 09:15 |
| gitlab-br-bot | Issue #1129: Error when src-push to a remote: Message larger than max https://gitlab.com/BuildStream/buildstream/issues/1129 | 09:15 |
| benschubert | let me try this right now :) | 09:15 |
| juergbi | also, with !1591 the buildbox-common bug can't ever be triggered because buildbox won't have to split the gRPC messages as now the buildstream client will already split requests to casd | 09:16 |
| juergbi | i.e., with !1591 none of the two issues should occur anymore even with the old buildbox-common | 09:17 |
| gitlab-br-bot | traveltissues approved MR !1591 (juerg/casd->master: casremote.py: Limit request size for batch download and upload) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1591 | 09:19 |
| benschubert | juergbi: awesome, that means we don't need to make a new release for the other? | 09:23 |
| juergbi | correct | 09:23 |
| benschubert | So what was happening is one client would fail with a request too big, and all subsequent ones would fail because it was using the same client context? | 09:23 |
| juergbi | the buildbox-common issue was triggered if the request from bst to casd was bigger than buildbox's message size limit (1 MB) but smaller than the default gRPC core limit of 4 MB | 09:25 |
| juergbi | i.e., this case triggers message splitting in casd, however, the message splitting had a bug in that it reused the same ClientContext and thus, the second message failed with the assertion failure you saw | 09:26 |
| benschubert | I see | 09:26 |
| juergbi | I think the assertion failure aborted casd, although I haven't verified that | 09:27 |
| benschubert | I'll retry the build see if everything succeeds then :) | 09:27 |
| benschubert | I can confirm it was killing it | 09:27 |
| juergbi | I haven't verified this via fdo-sdk build yet, should probably do this as well | 09:27 |
| *** jonathanmaw has joined #buildstream | 09:33 | |
| benschubert | jjardon: is the flathub remote very flacky usually? https://gitlab.com/BuildStream/buildstream/-/jobs/291588211 I tried a dozen times... | 09:55 |
| tpollard | benschubert: it's not too uncommon https://gitlab.com/BuildStream/buildstream/issues/1098 | 09:56 |
| gitlab-br-bot | traveltissues approved MR !1550 (tpollard/notificationhandler->master: Stream - Scheduler notification handler) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1550 | 09:57 |
| tpollard | although it looks like we're correctly handing the error messages now :) | 09:59 |
| tpollard | as in, we're actually getting the error out, not just a BUG about it being out of sync | 10:00 |
| benschubert | Yep, but we don't get the 3 retries I've seen with other sources, that seems weird | 10:00 |
| jjardon | benschubert: we had same issue with bst-1 branch. 100% reproducible, while the bst-1.2 was 100% of the times fine | 10:00 |
| jjardon | In the end the "fix" was to use Debian9 instead fedora30 for the ci job | 10:01 |
| benschubert | jjardon: thanks I'll try that fix :) | 10:01 |
| benschubert | it should be the last thing before we can have the nightly tests pass again | 10:02 |
| jjardon | Nice!! :) | 10:02 |
| juergbi | Debian 9 ostree uses libsoup as HTTP backend (HTTP 1.x) while Fedora ostree uses curl as HTTP backend (HTTP/2), afaict | 10:02 |
| benschubert | juergbi: oh that would explain the 'http2 message out of sync" | 10:03 |
| gitlab-br-bot | tpollard opened MR !1592 (tpollard/sandboxmessage->master: _message.py: Use bool for sandbox Message() parameter) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1592 | 10:06 |
| *** cs-shadow has joined #buildstream | 10:07 | |
| benschubert | juergbi: one "cheap" test for the pushing to remote caches would be to add a local remote artifactserver on the nightly tests, would slow them down a bit, but would allow us to check a more real scenario, what do you think? | 10:07 |
| tpollard | we still have a spare cache server hosted on digitalocean currently | 10:09 |
| tpollard | but that wouldn't be local obviously :) | 10:09 |
| juergbi | benschubert: could make sense. I did add a test to my MR, though, so at least basic build and push for an element with many files is now tested | 10:09 |
| benschubert | juergbi: great, yeah, we'll see, let's get the tests fixed first :D | 10:10 |
| benschubert | juergbi: also, I *think* the issue is fixed with the update to BUildStream, I have seem to have gone further | 10:10 |
| benschubert | tpollard: I would just like to make sure we have to push ewverything every time :) | 10:11 |
| gitlab-br-bot | BenjaminSchubert approved MR !1591 (juerg/casd->master: casremote.py: Limit request size for batch download and upload) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1591 | 10:11 |
| juergbi | benschubert: ok, I wanted to try it here locally as well but I'm still hitting the ostree background thread issue | 10:11 |
| tpollard | benschubert: yep, definitely worth the slight overhead imo | 10:11 |
| juergbi | wondering why noone else sees this | 10:11 |
| juergbi | maybe I need to recreate my venv | 10:11 |
| benschubert | juergbi: are you running on fedora? | 10:11 |
| juergbi | no | 10:11 |
| juergbi | but I tried with both ostree/libsoup and ostree/curl | 10:12 |
| benschubert | then I have no idea, I'm using ubuntu 19.04 and it seems to work :) | 10:12 |
| juergbi | I tried with ostree 2019.1 and 2019.3 (latest) | 10:12 |
| juergbi | Uubntu 19.04 also has ostree 2019.1 | 10:12 |
| benschubert | juergbi: https://gitlab.com/snippets/1893654 this is roughly my setup | 10:13 |
| juergbi | ta | 10:14 |
| gitlab-br-bot | coldtom opened MR !1593 (coldtom/filter-element-improvements->master: plugins/elements/filter.py: Allow passing integration commands through, fail if dep is a stack element) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1593 | 10:22 |
| gitlab-br-bot | marge-bot123 closed issue #1129 (Error when src-push to a remote: Message larger than max) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1129 | 10:29 |
| gitlab-br-bot | marge-bot123 merged MR !1591 (juerg/casd->master: casremote.py: Limit request size for batch download and upload) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1591 | 10:29 |
| jjardon | bochecha: Is buildstream 1.4.1 already in F30? I wonder if there is some kind of bug at https://repology.org/project/buildstream/versions | 11:00 |
| *** phil has joined #buildstream | 11:01 | |
| *** phildawson_ has quit IRC | 11:02 | |
| gitlab-br-bot | marge-bot123 merged MR !1550 (tpollard/notificationhandler->master: Stream - Scheduler notification handler) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1550 | 11:17 |
| jennis | cs-shadow, I've opened !1590 which makes source checkout a lot more like artifact checkout. This was in response to a UI thread that you wrote last November (:O), would you be able to review? | 11:18 |
| gitlab-br-bot | MR !1590: Ensure `source checkout` is symmetric to `artifact checkout` https://gitlab.com/BuildStream/buildstream/merge_requests/1590 | 11:18 |
| cs-shadow | jennis: thanks! will have a look now | 11:19 |
| bochecha | jjardon: only in updates-testing for now | 11:22 |
| bochecha | jjardon: I'll be able to push it to stable in 5 days | 11:23 |
| jjardon | Ah, then makes sense, thanks! | 11:23 |
| bochecha | (packages need to stay a week in testing by policy) | 11:23 |
| bochecha | unless people test it and give it a +1 | 11:23 |
| bochecha | then it can go to stable right away | 11:23 |
| *** phil has quit IRC | 11:42 | |
| gitlab-br-bot | traveltissues opened MR !1594 (traveltissues/typo->master: Fix typo in pipeline msg detail) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1594 | 11:54 |
| gitlab-br-bot | tpollard approved MR !1594 (traveltissues/typo->master: Fix typo in pipeline msg detail) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1594 | 11:56 |
| gitlab-br-bot | traveltissues opened MR !1595 (traveltissues/none-childdata->master: Return early from tracking queue if no result) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1595 | 12:02 |
| traveltissues | ty tpollard | 12:06 |
| gitlab-br-bot | marge-bot123 merged MR !1594 (traveltissues/typo->master: Fix typo in pipeline msg detail) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1594 | 12:35 |
| * tlater[m] really wants to make the master pipeline fail to test if his docs fixes work | 12:40 | |
| tlater[m] | I think that may be a bad desire. | 12:40 |
| *** phil has joined #buildstream | 12:49 | |
| gitlab-br-bot | marge-bot123 merged MR !1592 (tpollard/sandboxmessage->master: _message.py: Use bool for sandbox Message() parameter) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1592 | 13:14 |
| *** phildawson_ has joined #buildstream | 13:20 | |
| *** phil has quit IRC | 13:21 | |
| jennis | cs-shadow, tlater[m], tpollard, I've taken your review comments on !1584 on board and the MR has now changed significantly | 13:29 |
| gitlab-br-bot | MR !1584: WIP: Add UI section in CONTRIBUTING https://gitlab.com/BuildStream/buildstream/merge_requests/1584 | 13:29 |
| tpollard | will give it a gander | 13:35 |
| benschubert | juergbi: https://gitlab.com/snippets/1893722 that happened 3 times then went away :'D | 14:23 |
| juergbi | benschubert: hm, that happens if casd isn't ready within 15s. either startup failed or it's _really_ slow | 14:33 |
| benschubert | Ouch, ok, I'll open a ML thread about logging for casd, I think it's becoming useful :D | 14:34 |
| juergbi | we should probably also add a is_alive() check to provide a better error message in case casd terminates unexpectedly | 14:43 |
| benschubert | agreed, I 'll add this now :) | 14:44 |
| traveltissues | any strong feelings about !1595? | 14:45 |
| cs-shadow | jennis: just to verify that I'm not looking at the wrong thing, you haven | 14:52 |
| cs-shadow | you haven't already added the design principles from the ML threads, right? | 14:52 |
| *** bochecha has quit IRC | 14:58 | |
| gitlab-br-bot | BenjaminSchubert opened MR !1596 (bschubert/casd-better-reporting->master: [cascache] Check whether local casd is dead when grpc returns UNAVAILABLE) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1596 | 15:09 |
| benschubert | juergbi: ^ | 15:11 |
| jennis | no cs-shadow, I think we're going to do the design principles after a "this is what the frontend should look like" email from tlater[m] | 15:15 |
| jennis | well, it'd probably make sense to | 15:15 |
| gitlab-br-bot | juergbi opened MR !1597 (juerg/cache-usage->master: cascache.py: Fix cache usage monitor on Python older than 3.7) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1597 | 15:16 |
| juergbi | benschubert: commented | 15:17 |
| juergbi | traveltissues: !1597 should fix #1131 | 15:17 |
| gitlab-br-bot | Issue #1131: Error loading elements for freedesktop-sdk https://gitlab.com/BuildStream/buildstream/issues/1131 | 15:17 |
| benschubert | juergbi: thanks a lot! I haven't tested it manually, I wonder if it might be worth adding a test starting the cascache, killing the process and then trying to connect? | 15:18 |
| juergbi | exercising the error path would definitely be nice :) | 15:19 |
| juergbi | but if it's not as it simple as it sounds for some reason, I wouldn't block the MR on it | 15:19 |
| juergbi | at least a quick manual test would be good, though | 15:19 |
| cs-shadow | jennis: thanks, that makes sense. Just wanted to confirm | 15:20 |
| benschubert | juergbi: sure! | 15:22 |
| gitlab-br-bot | BenjaminSchubert approved MR !1597 (juerg/cache-usage->master: cascache.py: Fix cache usage monitor on Python older than 3.7) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1597 | 15:22 |
| juergbi | ta | 15:23 |
| gitlab-br-bot | BenjaminSchubert approved MR !1595 (traveltissues/none-childdata->master: Check result is not None in tracking queue done()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1595 | 15:23 |
| traveltissues | ty benschubert juergbi | 15:23 |
| gitlab-br-bot | traveltissues approved MR !1597 (juerg/cache-usage->master: cascache.py: Fix cache usage monitor on Python older than 3.7) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1597 | 15:31 |
| traveltissues | juergbi, i've tested this locally now, thanks | 15:32 |
| juergbi | ta | 15:32 |
| juergbi | it's now also covered by CI | 15:33 |
| jennis | benschubert, https://gitlab.com/BuildStream/buildstream/merge_requests/1590#note_214866002, are you suggesting that we `seek()` after we've added all of the files? | 15:55 |
| jennis | Not sure what this gains over `close()` | 15:55 |
| benschubert | jennis: not sure why we are closing the file there, but if that is something we would rather not do, seek() would work | 15:56 |
| jennis | Without closing the file I was getting EOF errors when trying to extract the tarball | 15:58 |
| jennis | Ensuring we close the tarfile after we've added all of the files prevented this | 15:59 |
| benschubert | and seek() on the tarfile should have the same effect :) | 15:59 |
| jennis | But is there any advantage to using seek() over close()? | 15:59 |
| benschubert | we avoid closing/reopening? If we are reopening anyways then no | 16:00 |
| gitlab-br-bot | marge-bot123 merged MR !1595 (traveltissues/none-childdata->master: Check result is not None in tracking queue done()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1595 | 16:08 |
| *** phil has joined #buildstream | 16:09 | |
| *** phildawson_ has quit IRC | 16:10 | |
| juergbi | benschubert: hm, thinking about it, I'm actually surprised that it mostly works (except for returncode 0 despite SIGKILL) even though this is typically running in a job child process and waitpid() doesn't allow waiting for sibling processes (buildbox-casd is a child of the main process) | 16:18 |
| juergbi | looking at the full job log, there are definitely still cases where we get the gRPC UNAVAILABLE error | 16:19 |
| juergbi | so I think it works only if SIGCHLD was already processed by the main process at the point the job child process was forked | 16:20 |
| juergbi | although I would expect even more gRPC UNAVAILABLE errors in that case. maybe I'm still missing something | 16:21 |
| benschubert | mmh good point, I'll put this MR back to WIP then | 16:30 |
| benschubert | any idea where to start debugging? | 16:30 |
| benschubert | or what the bugfix should be? | 16:42 |
| juergbi | benschubert: not sure. it seems to work at least sometimes and is just an error message improvement, so it might still be better to merge it than not to | 16:45 |
| juergbi | don't know how much time makes sense to invest to make it work in all cases | 16:45 |
| benschubert | that's a fair point. I'm happy to merge it as is if you are happy with it, I'm not entirely comfortable with that part of the code :) | 16:46 |
| *** traveltissues has quit IRC | 17:00 | |
| gitlab-br-bot | marge-bot123 closed issue #1131 (Error loading elements for freedesktop-sdk) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1131 | 17:08 |
| gitlab-br-bot | marge-bot123 merged MR !1597 (juerg/cache-usage->master: cascache.py: Fix cache usage monitor on Python older than 3.7) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1597 | 17:08 |
| *** jonathanmaw has quit IRC | 17:10 | |
| *** narispo has joined #buildstream | 18:06 | |
| *** phil has quit IRC | 18:07 | |
| benschubert | Artifact servers don't push data from junctions anymore correct? | 19:30 |
| benschubert | ah, found the doc, sorry :) | 19:31 |
| *** rdale has quit IRC | 20:03 | |
| *** narispo has quit IRC | 21:26 | |
| *** narispo has joined #buildstream | 21:26 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!