| gitlab-br-bot | push on buildstream@interactive-terminal (by Jürg Billeter): 1 commit (last: _sandboxbwrap.py: Restore terminal after exit of interactive child) https://gitlab.com/BuildStream/buildstream/commit/1431e545adf73a5b35a2b28bcc4c542fd29001dc | 07:08 |
|---|---|---|
| gitlab-br-bot | buildstream: merge request (interactive-terminal->master: _sandboxbwrap.py: Restore terminal after exit of interactive child) #62 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/62 | 07:08 |
| *** jonathanmaw has joined #buildstream | 08:59 | |
| *** tiagogomes has joined #buildstream | 09:03 | |
| *** ssam2 has joined #buildstream | 09:19 | |
| gitlab-br-bot | push on buildstream@sam/artifactcache-preflight-check (by Sam Thursfield): 1 commit (last: Check for write access to remote artifact cache early on in the pipeline) https://gitlab.com/BuildStream/buildstream/commit/cb54f531b705dcc41b9a0bdae51efa327f6f0dc5 | 09:22 |
| gitlab-br-bot | buildstream: merge request (sam/artifactcache-preflight-check->master: Check for write access to remote artifact cache early on in the pipeline) #57 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/57 | 09:22 |
| *** tlater has joined #buildstream | 09:25 | |
| ssam2 | juergbi, wow I'm really curious now what is going wrong with my testing process | 09:27 |
| ssam2 | I can see my branch working, but i must be testing the wrong thing as it clearly doesn't work ... | 09:27 |
| gitlab-br-bot | push on buildstream@sam/artifactcache-preflight-check (by Sam Thursfield): 1 commit (last: Check for write access to remote artifact cache early on in the pipeline) https://gitlab.com/BuildStream/buildstream/commit/8427fbc70c12408be87e4870f63d74dc980dc9c1 | 09:28 |
| juergbi | yes, i figured there must have been something gone wrong while testing | 09:28 |
| gitlab-br-bot | buildstream: merge request (sam/artifactcache-preflight-check->master: Check for write access to remote artifact cache early on in the pipeline) #57 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/57 | 09:28 |
| juergbi | another comment, now it's working for me | 09:29 |
| ssam2 | i am testing the correct file, so I guess i'm just not triggering the errors somehow | 09:30 |
| ssam2 | does it only happen when you push ? | 09:30 |
| ssam2 | ah I guess it does. I just hadn't got that far :-) | 09:30 |
| juergbi | the first comments were about issues as part oft he connectivity test, i.e., you should have hit that in any case with the right config | 09:31 |
| *** locallycompact has joined #buildstream | 09:32 | |
| ssam2 | nah | 09:33 |
| gitlab-br-bot | push on buildstream@sam/artifactcache-preflight-check (by Sam Thursfield): 1 commit (last: Check for write access to remote artifact cache early on in the pipeline) https://gitlab.com/BuildStream/buildstream/commit/2b700e8b618a36763a95722c0a29cfe44c4321e8 | 09:33 |
| ssam2 | bah | 09:33 |
| ssam2 | even | 09:33 |
| gitlab-br-bot | buildstream: merge request (sam/artifactcache-preflight-check->master: Check for write access to remote artifact cache early on in the pipeline) #57 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/57 | 09:33 |
| ssam2 | well we get there in the end at least... | 09:33 |
| juergbi | ssam2: have you seen my last comment? | 09:34 |
| ssam2 | yes... i don't quite follow | 09:35 |
| ssam2 | is the comment in the right place? | 09:35 |
| ssam2 | oh the line it discusses is hidden by gitlab | 09:35 |
| *** locallycompact has quit IRC | 09:35 | |
| juergbi | the relevant line is a few lines down | 09:36 |
| juergbi | yes, i couldn't point to the correct line as it wasn't modified | 09:36 |
| ssam2 | ok | 09:36 |
| ssam2 | still waiting for my bst to push as it's starting from scratch | 09:36 |
| juergbi | ok | 09:36 |
| juergbi | i haven't tested URL remotes, only SCP-style, but i expect URLs to work fine as well | 09:38 |
| *** locallycompact has joined #buildstream | 09:45 | |
| tlater | Is it possible to remount / in a chroot? | 09:58 |
| jonathanmaw | juergbi: hrm, after trying out various other tests in buildstream-tests, I think it might just be dpkg-build elements that fail to be checked out. | 10:00 |
| jonathanmaw | (the dpkg-build test is currently at https://gitlab.com/jonathanmaw/buildstream-tests/tree/jonathan/dpkg-build-test) | 10:02 |
| jonathanmaw | I can't see any particular way that the cache key could be changed by the addition of dynamic public metadata | 10:03 |
| gitlab-br-bot | push on buildstream@sam/artifactcache-preflight-check (by Sam Thursfield): 1 commit (last: Check for write access to remote artifact cache early on in the pipeline) https://gitlab.com/BuildStream/buildstream/commit/cd33dd37936cf19cde2377dc8f1b835a1ff5558e | 10:03 |
| gitlab-br-bot | buildstream: merge request (sam/artifactcache-preflight-check->master: Check for write access to remote artifact cache early on in the pipeline) #57 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/57 | 10:03 |
| jonathanmaw | but that sounds like the most obvious way it could go wrong. | 10:03 |
| juergbi | jonathanmaw: how exactly does checkout fail? | 10:07 |
| juergbi | it can't find the ref in the cache? | 10:07 |
| jonathanmaw | the full error is Failure staging dependencies, with the message "ERROR: dpkg_build element at dpkg-build-test.bst [line 1 column 0]: Missing artifact 33482b6f" | 10:08 |
| jonathanmaw | looking in the ostree, there's nothing with 33482b6f in, and a bit of judicious debug messaging added shows that it doesn't add anything with that key. | 10:08 |
| jonathanmaw | though when building the element, the logging for each stage identifies it with 33482b6f | 10:09 |
| juergbi | jonathanmaw: what key does it show in the logs when building the package? | 10:09 |
| jonathanmaw | juergbi: here's the full build stdout https://pastebin.com/3ftp5M7e | 10:10 |
| juergbi | ta | 10:10 |
| juergbi | hm, so there it displays the same key, seemingly adding that to the cache | 10:11 |
| jonathanmaw | juergbi: the refs it actually commits to ostree are script-test/dpkg-build-test/e96d7fa71067f7b382f7efec282a55e7d0c8a3a3aeee4ab2359be5b6b34b835c (strong) and script-test/dpkg-build-test/432e452e12236a652bf90b627d362af828b6fd3637cc37efebcb60de37326868 (weak) | 10:12 |
| juergbi | i assume you're running in strict mode (default) | 10:12 |
| jonathanmaw | I don't know how to activate the other mode, so probably | 10:13 |
| jonathanmaw | now that I've figured out how to enable non-strict, I certainly haven't enabled it. | 10:14 |
| juergbi | will try to reproduce this here and take a look | 10:16 |
| juergbi | ssam2: do you want to keep the last 3 lines of the commit message? other than that i think it's looking good now | 10:16 |
| ssam2 | hahaha, i really did well on this one | 10:17 |
| ssam2 | maybe i should stick to conference organisation from now on :-) | 10:17 |
| juergbi | you want to test whether anyone actually reviews things? ;) | 10:17 |
| gitlab-br-bot | push on buildstream@sam/artifactcache-preflight-check (by Sam Thursfield): 1 commit (last: Check for write access to remote artifact cache early on in the pipeline) https://gitlab.com/BuildStream/buildstream/commit/14a8b5aa28db973243b58a9ecb1ac9b26afa0994 | 10:17 |
| gitlab-br-bot | buildstream: merge request (sam/artifactcache-preflight-check->master: Check for write access to remote artifact cache early on in the pipeline) #57 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/57 | 10:18 |
| ssam2 | yeah it was all a test :-) | 10:18 |
| gitlab-br-bot | buildstream: merge request (sam/artifactcache-preflight-check->master: Check for write access to remote artifact cache early on in the pipeline) #57 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/57 | 10:29 |
| gitlab-br-bot | push on buildstream@master (by Jürg Billeter): 1 commit (last: Check for write access to remote artifact cache early on in the pipeline) https://gitlab.com/BuildStream/buildstream/commit/14a8b5aa28db973243b58a9ecb1ac9b26afa0994 | 10:29 |
| gitlab-br-bot | buildstream: issue #50 ("Loading pipeline is impossibly slow with > 50 elements") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/50 | 10:37 |
| ssam2 | i've opened that to track the current variant resolution issue | 10:37 |
| ssam2 | jjardon[m], I cancelled https://gitlab.com/baserock/definitions/-/jobs/23491831 as it seems to have hit that issue | 10:40 |
| ssam2 | had been spinning for 4 days :-) | 10:40 |
| jjardon[m] | ssam2: cheers | 10:41 |
| jjardon[m] | ssam2: I wanted to try the branch again as seems some problems with the protocol got solved in latest buildstream, but I triggered that early today? | 10:43 |
| ssam2 | oh right, I was misreading | 10:43 |
| ssam2 | still, it was clearly stuck | 10:43 |
| ssam2 | the artifact cache should now work again, I deleted the existing artifacts from ostree.baserock.org | 10:44 |
| ssam2 | the problem was that the artifact format changed but we don't version it yet | 10:44 |
| juergbi | and there was a protocol limitation which is fixed now | 10:47 |
| ssam2 | oh yes, i keep forgetting that | 10:48 |
| juergbi | jonathanmaw: i can reproduce the issue, looking into it | 11:07 |
| juergbi | install-commands: | 11:11 |
| juergbi | - - "for pkg in <PACKAGES>; do\n cp -a debian/${pkg}/* /buildstream/install\n\ | 11:11 |
| juergbi | - done" | 11:11 |
| juergbi | + - "for pkg in test; do\n cp -a debian/${pkg}/* /buildstream/install\ndone" | 11:11 |
| juergbi | BuildElement.commands is modified, which is part of get_unique_key() | 11:12 |
| juergbi | jonathanmaw: not sure what the best way to fix this is. it would be pretty simple to calculate the key before calling the plugin's assemble() method and thus, avoid any side-effects of that | 11:23 |
| juergbi | it doesn't feel 100% clean, though, maybe there is a better solution | 11:23 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 3 commits (last: element.py/sandbox.py: Add message helpers to sandboxes) https://gitlab.com/BuildStream/buildstream/commit/87e744c0c22a4b2ff111148cdb85a195ac0f7c1d | 11:26 |
| jjardon[m] | juergbi: thanks for the fix btw! :) | 11:28 |
| juergbi | np, thanks for reporting the issue :) | 11:28 |
| gitlab-br-bot | push on buildstream@key-calc (by Jürg Billeter): 1 commit (last: element.py: Calculate effective cache key before calling plugin methods) https://gitlab.com/BuildStream/buildstream/commit/cc08185235a0375cf5488e2f21be064f41cfb5ba | 11:29 |
| juergbi | jonathanmaw: my key-calc is a quick and dirty implementation of what i mentioned and the dpkg test passes with this | 11:29 |
| juergbi | key-calc branch... | 11:30 |
| jonathanmaw | ta juergbi, I'll give it a go | 11:32 |
| *** tlater has quit IRC | 13:03 | |
| *** tlater has joined #buildstream | 13:06 | |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 3 commits (last: _sandboxchroot.py: Fix chroot sandbox susp/cont/term) https://gitlab.com/BuildStream/buildstream/commit/ddb46bd0a2bdbc0e36fbcf4082d651f8dbbae01c | 14:18 |
| *** tlater has quit IRC | 14:30 | |
| *** tlater has joined #buildstream | 14:32 | |
| jonathanmaw | juergbi: key-calc is working for me, and I can't suggest a better approach (mostly because my source-code diving has concluded that it's impossible for key-calc to solve the problem, so I'm not qualified to offer an alternative) | 14:41 |
| jonathanmaw | AIUI, an element's Scope.BUILD does not include itself, and _get_cache_key_for_build() solely uses cache keys from its build-dependencies | 14:42 |
| juergbi | yes (and their runtime dependencies) | 14:42 |
| juergbi | we should probably check with tristan what he thinks about this issue | 14:43 |
| juergbi | could clean up the key-calc branch a bit but the main question is whether we want to stay with this approach or not | 14:44 |
| jonathanmaw | juergbi: I think ideally we'd want to know what's caused it, but at this point I've spent the best part of 2 days and it seems impossible to me. | 14:50 |
| juergbi | jonathanmaw: modifying self.commands in the dpkg plugin triggers a change in the cache key. didn't you see my explanation above? | 14:51 |
| jonathanmaw | ah, I see. | 14:52 |
| juergbi | an alternative solution would be to have two sets of 'commands', one read from the file, one effectively used (similar to how we have two sets of public data) | 14:54 |
| jonathanmaw | juergbi: yep, I vaguely recall an original_commands field in dpkg_build | 14:54 |
| juergbi | i haven't been involved much on the plugin side, though, so it might be better for you or tristan to come up with alternative solutions | 14:55 |
| jonathanmaw | I'll see about overriding the get_unique_key method for dpkg_build elements | 14:55 |
| jonathanmaw | then I can make sure the cache key doesn't contain anything that changes during build. | 14:55 |
| juergbi | ok | 14:57 |
| tlater | juergbi: To port the artifact cache I need to rewrite it to use tarfiles instead of ostree. In your opinion, would it be a good idea to split the current artifactcache class to have something that's less tied to ostree first? | 15:18 |
| juergbi | tlater: changing the API for a better fit with multiple backends will likely make sense. however, not sure whether it should really be a split of the ArtifactCache class, maybe it would be sufficient to move some parts to the Element class (or wherever appropriate) | 15:20 |
| juergbi | i haven't given it much thought yet, though, a split could make sense as well | 15:20 |
| juergbi | in any case, please make sure to keep refactoring in separate commit(s) | 15:21 |
| *** jonathanmaw has quit IRC | 16:05 | |
| *** jonathanmaw has joined #buildstream | 16:08 | |
| *** tiagogomes has quit IRC | 16:08 | |
| *** jonathanmaw has quit IRC | 16:15 | |
| *** locallycompact has quit IRC | 16:18 | |
| *** jonathanmaw has joined #buildstream | 16:28 | |
| *** tlater has quit IRC | 16:43 | |
| *** jonathanmaw has quit IRC | 16:43 | |
| *** jonathanmaw has joined #buildstream | 16:43 | |
| *** jonathanmaw has quit IRC | 17:17 | |
| *** ssam2 has quit IRC | 17:31 | |
| *** set has joined #buildstream | 22:19 | |
| *** set has quit IRC | 22:22 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!