| *** iker has joined #buildstream | 07:03 | |
| *** tiagogomes has joined #buildstream | 07:10 | |
| *** toscalix has joined #buildstream | 07:52 | |
| *** finn has joined #buildstream | 07:56 | |
| finn | valentind, do you have an example of the flatpak plugin? | 07:57 |
|---|---|---|
| *** phildawson has joined #buildstream | 08:10 | |
| valentind | finn, in freedesktop-sdk, there are several elements in elements/flatpak-images | 08:42 |
| finn | thanks, I'll have another look tonight | 08:54 |
| *** tpollard has joined #buildstream | 09:05 | |
| *** jonathanmaw has joined #buildstream | 09:06 | |
| gitlab-br-bot | buildstream: merge request (juerg/cas-batch->master: _artifactcache/cascache.py: Use BatchReadBlobs) #813 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/813 | 09:11 |
| *** lachlan has joined #buildstream | 09:29 | |
| *** dtf has quit IRC | 09:45 | |
| *** lachlan has quit IRC | 09:46 | |
| *** lachlan has joined #buildstream | 09:50 | |
| *** finn_ has joined #buildstream | 09:56 | |
| *** lachlan has quit IRC | 09:56 | |
| *** finn has quit IRC | 09:57 | |
| *** lachlan has joined #buildstream | 10:08 | |
| *** cs-shadow has joined #buildstream | 10:10 | |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 10:10 |
| gitlab-br-bot | buildstream: merge request (willsalmon/outOfSourecBuild->master: Out of source builds) #776 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/776 | 10:20 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 10:23 |
| toscalix | jmac: ticket #664 is it in your radar for the short term? | 10:27 |
| jmac | toscalix: Yes, I'm working on it now. I have a fix, it just needs lots of testing | 10:29 |
| jmac | Building freedesktop-sdk takes many hours | 10:29 |
| toscalix | thanks | 10:30 |
| toscalix | this ticket and the cache being filled are our pain points this week | 10:31 |
| *** dtf has joined #buildstream | 10:41 | |
| gitlab-br-bot | buildstream: merge request (juerg/cas-batch->master: _artifactcache/cascache.py: Use BatchReadBlobs) #813 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/813 | 10:46 |
| gitlab-br-bot | buildstream: merge request (mablanch/630-remote-execution-reconn->master: Handle connection losses during remote build execution) #806 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/806 | 10:49 |
| *** tiagogomes_ has joined #buildstream | 10:52 | |
| *** tiagogomes has quit IRC | 10:53 | |
| *** tiagogomes has joined #buildstream | 10:54 | |
| *** tiagogomes_ has quit IRC | 10:55 | |
| *** tiagogomes_ has joined #buildstream | 10:57 | |
| *** tiagogomes has quit IRC | 10:58 | |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 11:06 |
| toscalix | Reminder: please add the topics you want to discuss during the gathering in this page https://wiki.gnome.org/Projects/BuildStream/Events#Proposed_topics | 11:12 |
| toscalix | please add yourself to the participants list if you are attending | 11:12 |
| toscalix | I just added the Gathering to the calendar event | 11:13 |
| gitlab-br-bot | buildstream: merge request (juerg/cas-batch->master: _artifactcache/cascache.py: Use BatchReadBlobs) #813 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/813 | 11:18 |
| valentind | juergbi, I tried fill up a cache server. I found something that I do not think is right. | 11:55 |
| valentind | juergbi, while uploading objects, we check for disk space and prune if needed. Before we set a ref. So that means that what is being uploaded which is probably unreachable yet will be removed. | 11:55 |
| valentind | Typically, when the cache is full, I do a push, I get no error. But when I pull it back, then it complains about missing files. | 11:56 |
| juergbi | valentind: yes, that sounds like an issue | 12:01 |
| juergbi | to handle concurrent uploads, we need a more robust solution | 12:02 |
| *** lachlan has quit IRC | 12:02 | |
| valentind | juergbi, I see 2 ways to fix it. Either we make an LRU on objects instead of refs. And remove automatically refs that use them when they are removed. | 12:03 |
| valentind | Or... actually I am not sure how to solve it otherwise. | 12:04 |
| juergbi | the latter part sounds expensive | 12:04 |
| valentind | We would need to change the protocol. | 12:04 |
| juergbi | in what way? | 12:05 |
| valentind | We need a way to have transactions. | 12:05 |
| juergbi | that would be a huge REAPI change/addition | 12:05 |
| valentind | juergbi, we need it fixed on 1.2 branch. | 12:06 |
| valentind | Do you see any other way to fix that? | 12:06 |
| juergbi | I was thinking that we keep LRU of refs but refrain from removing objects that have a recent mtime | 12:06 |
| valentind | It is urgent. Gnome and Freedesktop sdk depend on it. | 12:06 |
| valentind | how recent? | 12:06 |
| juergbi | and let FindMissingBlobs update the mtime | 12:06 |
| juergbi | maybe something like an hour? | 12:07 |
| juergbi | not ideal that we need a fixed time for this but not sure how to handle this otherwise | 12:07 |
| valentind | I am afraid some push might take more than an hour. | 12:07 |
| juergbi | maybe even a few hours to be on the safe side | 12:07 |
| *** jonathanmaw_ has joined #buildstream | 12:07 | |
| juergbi | yes, was just thinking | 12:08 |
| *** jonathanmaw has quit IRC | 12:08 | |
| juergbi | maybe 6 hours or so? that should be sufficient time for large uploads and yet small enough to not be an issue for minimal cache size | 12:08 |
| juergbi | could also extend it to 24 hours. would probably still be fine | 12:09 |
| valentind | juergbi, I can try that. | 12:09 |
| juergbi | great, ta, the FindMissingBlobs part is clear to you? | 12:10 |
| valentind | juergbi, yes, I just touch all digests. | 12:10 |
| juergbi | yes | 12:10 |
| valentind | And then in cas.prune I ignore recent digests. | 12:10 |
| valentind | juergbi, I think you still need to find a better solution for 1.4. | 12:11 |
| juergbi | valentind: what issues do you expect? | 12:13 |
| *** lachlan has joined #buildstream | 12:14 | |
| valentind | juergbi, not being able to upload much for a while after a lot was uploaded at once. | 12:14 |
| valentind | juergbi, we can still keep the mtime thing. But if the client could send a request saying that all it uploaded is safe after sending all the refs, that would help unlocking objects. | 12:15 |
| juergbi | valentind: i.e., you expect the disk quota to be smaller than the working set of a 6h window? | 12:15 |
| valentind | I do not want to have expectations from what the users do. | 12:16 |
| juergbi | will think about it some more | 12:20 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 12:20 |
| juergbi | valentind: btw: while the pruning before the ref is an issue, is this really the cause for #609? i.e., wouldn't it delete too much instead of too little? | 12:21 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 12:22 |
| *** lachlan has quit IRC | 12:22 | |
| valentind | juergbi, I am not sure. I suppose the cache gets stuck full for a while. Nobody sees the error. | 12:22 |
| valentind | But then suddenly there is concurrent push. | 12:22 |
| valentind | 2 request for beginning of upload of big objects. They both call _clean_up_cache. And both think there is enough space. | 12:23 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 12:24 |
| valentind | And then they both upload. But the sum of the two object is bigger than the space remaining. | 12:24 |
| valentind | I suppose we need to _clean_up_cache at each request. | 12:25 |
| valentind | Or fallocate the file. | 12:25 |
| valentind | Like fallocate first, and then _clean_up_cache. | 12:26 |
| valentind | To avoid race condition. | 12:26 |
| juergbi | we already _clean_up_cache on every request | 12:28 |
| valentind | It seems it does it only when "resource_name" is None. | 12:29 |
| valentind | juergbi, you can confirm there is no way that _ByteStreamServicer.Write cannot be called concurrently? | 12:30 |
| valentind | Because we do many "out.write" on the temporary file. | 12:30 |
| valentind | juergbi, does not the iterator of request_iterator yield? | 12:33 |
| juergbi | valentind: it can be called concurrently | 12:34 |
| juergbi | the server is actually multi-threaded | 12:34 |
| juergbi | however, we also have a 2 GB buffer | 12:34 |
| juergbi | we could probably keep track of reserved space, though, or indeed use fallocate | 12:35 |
| juergbi | I don't think it's very likely that this is the cause for the issue, though, given the 2 GB buffer and we probably don't have too many concurrent uploaders | 12:36 |
| juergbi | (a single upload job writes files one by one) | 12:36 |
| *** bochecha has joined #buildstream | 12:37 | |
| Nexus | I cam across the issue where if there is a `project.conf` anywhere above the test suite when running `tests/format/project.py::test_missing_project_conf ` the test will fail. From what i can see there are 2 options to fix this, 1: do a check for project.conf out of bounds before the test, and if it is found, skip the test. 2: Add an optional argument to the `_ensure_project_dir()` function to allow | 12:38 |
| Nexus | you to limit the number of times it cycles through directories, for the sake of testing | 12:38 |
| Nexus | anyone got any thoughts of additional options? | 12:39 |
| gitlab-br-bot | buildstream: merge request (bochecha/blessings->bst-1.2: _frontend/status.py: Remove leftover blessings import) #821 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/821 | 12:42 |
| skullman | Nexus: rather than a limit to the number of times it'll go up, you could set a directory it won't attempt to traverse up out of, which at least could see some use if it defaults to $HOME | 12:45 |
| valentind | juergbi, CI has many uploaders at once. | 12:45 |
| valentind | Usually 4 per branch because 4 architectures. | 12:45 |
| valentind | Then it is not uncommon to have more than 5 branches building in the same time. | 12:45 |
| valentind | So 20 uploaders is not impossible. | 12:46 |
| juergbi | ok but that would still require an average blob (not artifact) size of 100 MiB and all starting upload at almost the same time | 12:48 |
| juergbi | not impossible but it doesn't sound very likely. well, unless we're creating image files or similar. maybe fdo sdk does create such large files | 12:50 |
| valentind | llvm is 43M | 12:50 |
| valentind | And we do have vm images. | 12:50 |
| juergbi | single file? | 12:50 |
| juergbi | vm images could certainly trigger such issues | 12:51 |
| valentind | I am not sure about gnome though. | 12:51 |
| juergbi | in any case, fallocate does sound sensible, if it's easily doable from Python | 12:53 |
| valentind | juergbi, there is a weird behavior I do not understand. With "df" I look at the space available. | 12:53 |
| valentind | And at each upload it goes double size. | 12:53 |
| valentind | And then comes back down. | 12:53 |
| valentind | So, it seems the temporary file is copied rather than linked. | 12:54 |
| juergbi | that's actually indeed the case | 12:55 |
| valentind | Should not we cound double size when calling _clean_up_cache? | 12:55 |
| juergbi | we use add_object() which normally does this to ensure regular user files (which may be concurrently modified) are not linked into the CAS, avoiding corruption | 12:57 |
| juergbi | however, in the case where the CAS server is creating and writing the file, we shouldn't need this precaution and could thus avoid the double write | 12:57 |
| juergbi | we still need to verify the digest but we could do this during the upload | 12:58 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 12:59 |
| *** lachlan has joined #buildstream | 13:04 | |
| gitlab-br-bot | buildstream: merge request (bochecha/blessings->bst-1.2: _frontend/status.py: Remove leftover blessings import) #821 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/821 | 13:06 |
| gitlab-br-bot | buildstream: merge request (mablanch/630-remote-execution-reconn->master: Handle connection losses during remote build execution) #806 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/806 | 13:06 |
| bochecha | juergbi: with my MR merged, do you think there should be a new release quickly? | 13:17 |
| bochecha | as it is, `pip install` 1.2.1 is broken :-/ | 13:17 |
| juergbi | ah, because it's not in setup.py :/ | 13:17 |
| juergbi | for people updating it presumably works because the dependency will still be installed, but new installations will likely fail | 13:21 |
| bochecha | exactly | 13:22 |
| juergbi | might indeed warrant a new release. not ideal that tristan is away | 13:22 |
| bochecha | (it even fails on upgrades in some systems, like rpm-ostree which removes unnecessary deps, and thus removes blessings when bst is updated) | 13:23 |
| *** toscalix_ has joined #buildstream | 13:26 | |
| *** toscalix_ has quit IRC | 13:27 | |
| *** lachlan has quit IRC | 13:29 | |
| gitlab-br-bot | buildstream: merge request (juerg/rebuild->master: element.py: Fix cache check in non-strict mode) #822 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/822 | 14:01 |
| *** lachlan has joined #buildstream | 14:02 | |
| jjardon | Hi, if I do `bst shell base/opus.bst` I get a shell, but if I do `bst shell --build base/opus.bst` I get the following error: https://paste.gnome.org/p5o7xazo6 ; is this the expected behavior? | 14:05 |
| juergbi | skullman: I'd appreciate a review of !822. fixes a regression introduced by the failed build caching support | 14:06 |
| skullman | juergbi: looking now | 14:06 |
| juergbi | jjardon: if those are indeed not cached, yes | 14:06 |
| juergbi | bst shell without --build doesn't stage build-only dependencies | 14:06 |
| jjardon | juergbi: ah! ok. So how I can force it to be cached? | 14:07 |
| jjardon | I would expect `bst shell --build base/opus.bst` to automatically cache build deps? | 14:07 |
| juergbi | bst build --all base/opus.bst should work, however, it's a bit overkill | 14:08 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 14:08 |
| juergbi | as it will also ensure all indirect build dependencies are available | 14:08 |
| juergbi | jjardon: iirc, there was a recent discussion that we're missing a command to only build what's needed for a build shell | 14:08 |
| jjardon | mmm, that ring a bell; bochecha was that you? | 14:09 |
| jjardon | juergbi: thanks for the build --all base/opus.bst ; I will try that for now | 14:09 |
| bochecha | jjardon: yes | 14:09 |
| bochecha | that was me | 14:09 |
| bochecha | tristan asked me to send a proposal to the mailing-list, and I completely forgot :) | 14:09 |
| jjardon | bochecha: did you open an issue by any chance? | 14:10 |
| bochecha | the idea was to remove --all and to instead have a --deps which does like the one for bst pull | 14:10 |
| bochecha | and maybe then add a suboption to --deps, in addition to none/all | 14:10 |
| bochecha | jjardon: I didn't, because tristan said this is the kind of proposal which should go through a discussion on the mailing-list | 14:11 |
| jjardon | I would like `bst shell --build base/opus.bst` to do all this automatically | 14:11 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 14:11 |
| jjardon | I'm actually saying to bst I want a sysroot to build this element; so I'd expect everything the element needs to be included | 14:11 |
| jjardon | in what situations `bst shell --build base/opus.bst` would be useful if not? | 14:12 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 14:15 |
| juergbi | jjardon: we currently don't implicitly start a build session on 'bst shell' and 'bst checkout'. if you think we should change that, we probably should discuss this on the list | 14:17 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 14:17 |
| gitlab-br-bot | buildstream: merge request (juerg/rebuild->master: element.py: Fix cache check in non-strict mode) #822 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/822 | 14:18 |
| skullman | juergbi: reviewed | 14:18 |
| bochecha | jjardon: agreed that automatically doing this would be **very** nice, but in general bst doesn't do anything automatically… so the next best thing is to be able to build just the needed deps :) | 14:18 |
| jjardon | juergbi: sure; if I understand this correctly that is expected when running `bst shell base/opus.bst`, but if I explicity add the `--build` option, I think is clear that what I want is to have everything there so I can build? | 14:19 |
| *** raoul has joined #buildstream | 14:19 | |
| jjardon | I'm trying to imagine a case when you do `bst shell --build base/opus.bst` but you dont actually want the needed things to build in the shell | 14:20 |
| juergbi | jjardon: well, you could say the same for bst shell without --build, i.e., it's clear you want everything so that you can run it | 14:20 |
| juergbi | or bst checkout | 14:21 |
| bochecha | jjardon: to be fair, I can't imagine a case where I'd do `bst shell base/opus.bst` and not want everything to be built either :) | 14:21 |
| bochecha | juergbi: how about adding a --auto-build option to shell and checkout? | 14:21 |
| juergbi | i.e., I don't see a reason to distinguish between these cases. either we always build implicitly or never | 14:21 |
| juergbi | could be an option (or could also be the other way round, auto build by default) | 14:21 |
| jjardon | the docs says: "Use the –build option to create a temporary sysroot for building the element instead." but then when I run that I can not build? It's a bit confusing :) | 14:21 |
| juergbi | true, it doesn't fit in any sensible workflow | 14:22 |
| *** lachlan has quit IRC | 14:22 | |
| *** laowei has joined #buildstream | 14:27 | |
| *** ikerperez has joined #buildstream | 14:29 | |
| *** iker has quit IRC | 14:30 | |
| gitlab-br-bot | buildstream: merge request (jonathan/source-mirror-project-refs->master: tests: Add regression test for mirroring with project.refs) #823 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/823 | 14:31 |
| *** abderrahim has quit IRC | 14:47 | |
| *** abderrahim has joined #buildstream | 14:48 | |
| *** lachlan has joined #buildstream | 15:01 | |
| *** lachlan has quit IRC | 15:08 | |
| *** lachlan has joined #buildstream | 15:11 | |
| *** lachlan has quit IRC | 15:19 | |
| *** tiagogomes has joined #buildstream | 15:26 | |
| *** tiagogomes_ has quit IRC | 15:27 | |
| *** finn_ has quit IRC | 15:33 | |
| *** lachlan has joined #buildstream | 15:37 | |
| *** finn has joined #buildstream | 15:38 | |
| *** tiagogomes_ has joined #buildstream | 15:47 | |
| *** tiagogomes has quit IRC | 15:48 | |
| *** tiagogomes_whostolemyidentity has joined #buildstream | 15:55 | |
| juergbi | bochecha: 1.2.2 release is out, hopefully I didn't miss anything | 15:56 |
| *** tiagogomes_ has quit IRC | 15:56 | |
| juergbi | jjardon: can you please update the topic? | 15:56 |
| *** ikerperez has quit IRC | 15:58 | |
| *** jjardon changes topic to "BuildStream 1.2.2 is out ! | https://gitlab.com/BuildStream/buildstream | Docs: https://buildstream.gitlab.io/buildstream | IRC logs: https://irclogs.baserock.org/buildstream | Mailing List: https://mail.gnome.org/mailman/listinfo/buildstream-list | Roadmap: https://wiki.gnome.org/Projects/BuildStream/Road" | 15:59 | |
| *** toscalix has quit IRC | 16:02 | |
| gitlab-br-bot | buildstream: issue #607 ("Non strict build causes "Missing artifact"") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/607 | 16:02 |
| gitlab-br-bot | buildstream: merge request (juerg/rebuild->master: element.py: Fix cache check in non-strict mode) #822 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/822 | 16:02 |
| *** tiagogomes_whostolemyidentity has quit IRC | 16:05 | |
| *** tiagogomes has joined #buildstream | 16:07 | |
| juergbi | cs-shadow: would it be possible for you to upload 1.2.2 to PyPI? | 16:10 |
| cs-shadow | juergbi: sure, i'll do it in just a few minutes | 16:12 |
| juergbi | thanks | 16:12 |
| bochecha | ha! I wanted to update the Fedora package, but can't… because it downloads from PyPI :P | 16:13 |
| *** lachlan has quit IRC | 16:13 | |
| bochecha | cs-shadow: can you ping me when you've managed to find the time, so I proceed with the package? :) | 16:13 |
| *** finn has quit IRC | 16:17 | |
| cs-shadow | bochecha: sure, i'll ping you in a bit | 16:18 |
| *** finn has joined #buildstream | 16:18 | |
| bochecha | thanks | 16:18 |
| gitlab-br-bot | buildstream: merge request (mac_fixes->master: WIP: Implement compatibility fixes for MacOSX and WSL Blocks #411 and #412") #726 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/726 | 16:22 |
| gitlab-br-bot | buildstream: merge request (richardmaw/push-failed-build-regression->master: tests: Add regression test for pushing failed builds) #824 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/824 | 16:22 |
| *** tiagogomes has quit IRC | 16:33 | |
| *** tiagogomes has joined #buildstream | 16:34 | |
| gitlab-br-bot | buildstream: issue #595 ("BUG: Missing Artifact when checking out with --no-strict") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/595 | 16:36 |
| cs-shadow | bochecha: juergbi: https://pypi.org/project/BuildStream/ is now updated | 16:38 |
| *** lachlan has joined #buildstream | 16:38 | |
| *** tiagogomes has quit IRC | 16:38 | |
| gitlab-br-bot | buildstream: merge request (jonathan/source-mirror-project-refs->master: tests: Add regression test for mirroring with project.refs) #823 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/823 | 16:40 |
| bochecha | juergbi: I wonder though, how did the CI actually pass on Tristan's cherry-pick, since it couldn't work? | 16:41 |
| bochecha | I guess nothing tests frontend/_app.py? | 16:42 |
| juergbi | bochecha: it is tested, the issue is that the CI image used includes blessings | 16:44 |
| juergbi | or at least that's what I suspect | 16:45 |
| cs-shadow | juergbi: that's correct. It's kind of a chicken-and-egg problem with dependencies. We wanted to them to be pre-installed in the image so that we don't waste time on installing them during tests. But this method is bound to cause problems anytime we wish to change the dependencies | 16:48 |
| bochecha | juergbi: ah, right | 16:49 |
| juergbi | yes, tricky to handle | 16:49 |
| bochecha | in a previous project, what we did is that the deps were pre-installed in the image | 16:51 |
| bochecha | but when building the image, we'd clone the repo to find the deps to install | 16:51 |
| bochecha | this way, as long as the image was regularly updated, it had a relatively up-to-date list of dependencies | 16:52 |
| *** lachlan has quit IRC | 16:52 | |
| cs-shadow | bochecha: that's exactly what we do but the issue is that if you clone the repo, you will never get the changes that are in-flight, which is why our CI didn't catch this because the images got rid of blessings only after they were merged in this repo | 16:52 |
| bochecha | cs-shadow: doh, obviously /o\ | 16:53 |
| * bochecha needs a bit more sleep | 16:53 | |
| cs-shadow | the exact same problem also happens if we want to add a new dep :) | 16:53 |
| * cs-shadow isn't aware of a "clean" solution for this | 16:54 | |
| juergbi | save deps list in CI image. in CI job check whether deps match. if they don't, clear out dependencies from base image and install them from the new list? | 16:57 |
| juergbi | doesn't sound very nice, though | 16:57 |
| gitlab-br-bot | buildstream: merge request (jmac/stop-caching-vdirs->master: WIP: Stop caching virtual directories if get_directory is used.) #818 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/818 | 17:00 |
| gitlab-br-bot | buildstream: merge request (jmac/stop-caching-vdirs->master: Stop caching virtual directories if get_directory is used.) #818 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/818 | 17:01 |
| cs-shadow | juergbi: yeah, something like that could work but still is hairy. For example, if we just have one teststuite image, then it will either break on master or on the PR updating the deps. Maybe we can enforce that each dependency update has to come with an accompanying PR to docker-images? Would that be acceptable? | 17:01 |
| gitlab-br-bot | buildstream: merge request (jmac/stop-caching-vdirs->master: Stop caching virtual directories if get_directory is used.) #818 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/818 | 17:02 |
| gitlab-br-bot | buildstream: issue #674 ("utils.safe_copy() fails to copy *to* a location where leading directories do not exist") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/674 | 17:03 |
| gitlab-br-bot | buildstream: merge request (jonathan/source-mirror-project-refs->master: tests: Add regression test for mirroring with project.refs) #823 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/823 | 17:10 |
| gitlab-br-bot | buildstream: merge request (jmac/stop-caching-vdirs->master: Stop caching virtual directories if get_directory is used.) #818 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/818 | 17:11 |
| juergbi | cs-shadow: as that's not very frequent, it might be acceptable | 17:12 |
| jmac | #674 is interesting. I'm trying to write down all the rules for file import at the moment. | 17:12 |
| cs-shadow | juergbi: okay, i'll try to see if we can implement something like this easily | 17:14 |
| *** alatiera_ has joined #buildstream | 17:31 | |
| *** raoul has quit IRC | 17:32 | |
| *** lachlan has joined #buildstream | 18:11 | |
| *** toscalix has joined #buildstream | 18:15 | |
| *** lachlan has quit IRC | 18:22 | |
| *** lachlan has joined #buildstream | 18:54 | |
| gitlab-br-bot | buildstream: issue #632 ("CAS server: Implement BatchReadBlobs") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/632 | 19:28 |
| *** bochecha has quit IRC | 19:42 | |
| *** lachlan has quit IRC | 20:00 | |
| *** jonathanmaw_ has quit IRC | 20:09 | |
| *** toscalix has quit IRC | 20:22 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!