| *** narispo has quit IRC | 00:13 | |
| *** narispo has joined #buildstream | 00:13 | |
| *** narispo has quit IRC | 04:13 | |
| *** narispo has joined #buildstream | 04:13 | |
| *** narispo has quit IRC | 04:16 | |
| *** narispo has joined #buildstream | 04:17 | |
| *** hasebastian has joined #buildstream | 04:41 | |
| *** traveltissues has joined #buildstream | 07:48 | |
| *** benschubert has joined #buildstream | 08:04 | |
| *** phildawson has joined #buildstream | 09:01 | |
| benschubert | juergbi: any idea what the expected format of the proto is for https://gitlab.com/BuildStream/buildstream/-/blob/master/src/buildstream/storage/_casbaseddirectory.py#L158 ? I get a DecodeError when running with bb-run-bwrap and the content of it is : '/cross/lib\n' | 09:02 |
|---|---|---|
| juergbi | benschubert: it needs to be a serialized (binary) Directory protobuf message | 09:03 |
| juergbi | are you doing something manually or do you see an issue in normal operation? | 09:03 |
| benschubert | I am just trying to build a project so everything normal :) | 09:03 |
| benschubert | Let me push a reproduction, give me 5 minutes | 09:04 |
| juergbi | so the object file matching the digest is purely a text file? | 09:04 |
| juergbi | ok | 09:04 |
| benschubert | and yep seems like it :( | 09:04 |
| benschubert | juergbi: https://gitlab.com/BenjaminSchubert/bst-testbed if you do: docker-compose run bst build bootstrap/build/debugedit-build.bst I would expect the error to be reproduced :) | 09:08 |
| benschubert | juergbi: if you have any idea where I should dig, let me know :) | 09:10 |
| juergbi | benschubert: you're not using buildbox-fuse? | 09:12 |
| benschubert | Ah no, would that be better? | 09:12 |
| juergbi | without it, buildbox-casd needs to run as a different uid, otherwise you have no hardlink protection | 09:12 |
| juergbi | (there is an issue open about detecting this misconfiguration) | 09:13 |
| benschubert | ah good point | 09:13 |
| juergbi | let me know whether you still see the issue with buildbox-fuse (or proper uid separation) | 09:14 |
| benschubert | yep, I've just setuid'd buildbox-casd :) will re-try | 09:14 |
| juergbi | my plan is to recommend buildbox-fuse as default setup when we switch to buildbox-run | 09:14 |
| juergbi | (for linux) | 09:15 |
| *** toscalix has joined #buildstream | 09:15 | |
| juergbi | you may also have to clear the cache if it became corrupted, given that you bind the cache directory into the container | 09:16 |
| benschubert | So the fuse layer would be on the buildbox-run side and not casd right? (I think I remembered the reverse, just want to refresh my memory) | 09:16 |
| *** santi has joined #buildstream | 09:16 | |
| benschubert | yep I will | 09:16 |
| juergbi | buildbox-casd is the process that starts buildbox-fuse | 09:17 |
| juergbi | however, it does so on (Stage) request from buildbox-run | 09:17 |
| benschubert | Ok, so you actually install the three? | 09:17 |
| juergbi | yes, buildbox-fuse no longer does any sandboxing | 09:17 |
| juergbi | so you still need buildbox-run-bubblewrap | 09:17 |
| benschubert | Ah gotcha, so it's a third system. How do I tell buildbox-casd that it exist? Found automagically? | 09:18 |
| juergbi | yes, it's used if found in path | 09:18 |
| juergbi | as it's preferred over hardlinking | 09:19 |
| benschubert | awesome. and does that also mean it can be used with userchroot on linux? Or does userchroot restriction prevent us from doing so? | 09:19 |
| juergbi | I actually haven't tried | 09:19 |
| juergbi | I suspect it won't work as the permission requirements may conflict | 09:19 |
| benschubert | Yep makes sense, thanks :) | 09:20 |
| juergbi | but haven't looked into it further. do you see a use case for that combination? | 09:20 |
| benschubert | cases when you cannot use user namespaces :) | 09:20 |
| juergbi | the main advantage I see for userchroot on Linux is to use the same setup across platforms | 09:20 |
| benschubert | That is one, the other is when you have access to fuse so might want the speed benefits, but can't use user namespaces (e.g. kernel too old) and thus cannot use bwrap | 09:21 |
| juergbi | right, could be interesting there | 09:21 |
| juergbi | a possible alternative is to support bwrap without user namespaces | 09:22 |
| juergbi | it does require bwrap to be installed setuid root, though | 09:22 |
| benschubert | but you need setuid then, which is another requirement :) | 09:22 |
| benschubert | yep | 09:22 |
| juergbi | btw: my buildbox-fuse testing has been limited to very recent kernels + whatever we use in CI | 09:23 |
| *** tpollard has joined #buildstream | 09:25 | |
| benschubert | Also for the setuid of casd, I need both to share a group and have ~/.cache/buildstream/ and ~/.cache/buildstream/cas to be owneed by this group in 0o777 right? | 09:27 |
| juergbi | benschubert: yes, well 0o750 is the minimum | 09:41 |
| juergbi | as buildstream should not directly write into CAS directories anymore | 09:42 |
| benschubert | uh? I am confused: | 09:44 |
| benschubert | - Buildbox-casd has been setuid buildboxcasd | 09:44 |
| benschubert | - ~/.cache/buildstream is created with 0750 (buildstream:buildstream) | 09:44 |
| benschubert | - ~/.cache/buildstream/cas is created with 0750 (buildstream:buildstream) | 09:44 |
| benschubert | And casd can't create directories or files in ~/.cache/buildstream/cas. Same if mode it 0770. | 09:44 |
| benschubert | and I am running as 'buildstream:buildstream' | 09:45 |
| juergbi | benschubert: the cas directory should be owned by the casd user | 09:49 |
| juergbi | and run in the same group as buildstream | 09:50 |
| benschubert | does that mean buildstream itself still creates ~the cas directory? | 09:50 |
| juergbi | it still creates it if it doesn't exist but it won't change the permissions if it already exists | 09:51 |
| juergbi | with 0770 or 0777 it should also work if the cas directory is owned by buildstream, but the idea is that buildbox-casd has exclusive write access to the cas directory tree, so chown to casd user makes sense, imo | 09:52 |
| juergbi | we will obviously need good documentation for that and maybe also further fixes | 09:53 |
| *** hasebastian has quit IRC | 09:53 | |
| *** traveltissues has quit IRC | 09:54 | |
| juergbi | my current focus with regards to buildbox is preparing for the switch from the internal bwrap sandboxing backend to buildbox-run-bubblewrap + buildbox-fuse, and the more complex setup is not required in this case | 09:54 |
| benschubert | ok I'll go for this one for now then :) | 09:55 |
| benschubert | https://gitlab.com/BuildStream/buildstream/-/blob/master/src/buildstream/_cas/cascache.py#L72 should not be done though right? :) | 09:55 |
| juergbi | correct, I think we can and should remove this now | 09:56 |
| benschubert | ok I'll make a PR, thanks :) | 09:56 |
| benschubert | same for the logs? Or should we actually move the logs outside of this directory? | 09:56 |
| benschubert | I guess it should be in buildstream/logs/cas ? | 09:56 |
| benschubert | since buildstream still owns them | 09:56 |
| *** traveltissues has joined #buildstream | 09:57 | |
| juergbi | benschubert: yes, moving out makes sense | 09:58 |
| juergbi | logs/cas has a potential for conflicts, though, afaict | 09:58 |
| juergbi | a possible alternative would be to move the logging mechanism to buildbox-casd itself | 09:59 |
| juergbi | not sure whether that makes more sense. haven't thought about it yet. | 09:59 |
| benschubert | how would you envision this? The nice thing with handling it ourselves is that we can post the logs to the last casd run | 10:00 |
| benschubert | so to buildstream/cas-logs :) | 10:00 |
| *** lachlan has joined #buildstream | 10:04 | |
| *** lachlan has quit IRC | 10:25 | |
| *** lachlan has joined #buildstream | 10:47 | |
| *** lachlan has quit IRC | 11:11 | |
| *** toscalix has quit IRC | 11:27 | |
| *** toscalix has joined #buildstream | 11:27 | |
| *** toscalix has quit IRC | 11:28 | |
| *** lachlan has joined #buildstream | 11:31 | |
| gitlab-br-bot | BenjaminSchubert opened MR !1832 (bschubert/stricter-cas-soc->master: Don't create Buildstream-related directories in the cas - owned directory) on buildstream https://gitlab.com/BuildStream/buildstream/-/merge_requests/1832 | 11:59 |
| benschubert | juergbi: ^ for stopping the creation of those directories | 11:59 |
| *** lachlan has quit IRC | 12:12 | |
| *** lachlan has joined #buildstream | 12:15 | |
| gitlab-br-bot | abderrahimk opened MR !1833 (abderrahim/etag->master: _downloadablefilesource.py: don't download the file if etag matches) on buildstream https://gitlab.com/BuildStream/buildstream/-/merge_requests/1833 | 12:31 |
| *** lachlan has quit IRC | 12:31 | |
| benschubert | juergbi: I might be missing something: I've got it almost to work, but now my buildbox-casd creates a unix socket with 755 permissions, meaning my buildstream can't read to it anymore. What am I missing? :/ | 12:36 |
| juergbi | benschubert: ah, that's https://gitlab.com/BuildGrid/buildbox/buildbox-casd/-/merge_requests/127 | 12:37 |
| juergbi | it should work with `umask 002` | 12:37 |
| benschubert | ok, so setting umask in /etc/profile would fix it, great thanks! | 12:38 |
| *** lachlan has joined #buildstream | 12:42 | |
| *** lachlan has quit IRC | 12:51 | |
| *** santi has quit IRC | 14:05 | |
| *** santi has joined #buildstream | 14:06 | |
| *** lachlan has joined #buildstream | 14:08 | |
| jjardon | I think latest bst tag has not been published? https://gitlab.com/BuildStream/buildstream/-/jobs/475306096 | 14:29 |
| *** lachlan has quit IRC | 14:38 | |
| *** lachlan has joined #buildstream | 14:50 | |
| juergbi | jjardon: oh, forgot to bump the version in setup.py | 14:52 |
| *** phildawson has quit IRC | 15:00 | |
| *** phildawson has joined #buildstream | 15:00 | |
| *** lachlan has quit IRC | 15:03 | |
| benschubert | juergbi: https://gitlab.com/BuildStream/buildstream/-/merge_requests/1832#note_307080435 do you think _cas would be acceptable? | 15:07 |
| juergbi | benschubert: a bit unusual but might be a reasonable compromise | 15:09 |
| benschubert | juergbi: thanks! pushed and addressed all your comments, can you have another look please? | 15:24 |
| *** lachlan has joined #buildstream | 15:30 | |
| *** traveltissues has quit IRC | 15:54 | |
| juergbi | benschubert: fyi, you haven't pushed another update | 16:33 |
| benschubert | juergbi: oops sorry about that. fixed | 16:34 |
| *** traveltissues has joined #buildstream | 16:49 | |
| benschubert | juergbi: https://gitlab.com/BuildStream/buildstream/-/merge_requests/1832#note_307250656 good point there. I'd be in favor of 'assert' that it's not None when casd is not none? | 16:49 |
| juergbi | benschubert: that's fine by me as well. however, this would still require fixup in at least one test, afaict | 16:50 |
| benschubert | yup will do :) | 16:50 |
| *** traveltissues has quit IRC | 16:52 | |
| *** cs-shadow has joined #buildstream | 16:53 | |
| *** santi has quit IRC | 17:22 | |
| *** santi has joined #buildstream | 17:24 | |
| *** phildawson has quit IRC | 17:27 | |
| *** phildawson has joined #buildstream | 17:34 | |
| *** lachlan has quit IRC | 17:40 | |
| *** lachlan has joined #buildstream | 17:41 | |
| gitlab-br-bot | abderrahimk opened MR !1834 (abderrahim/fuse3->master: mount: don't pass nonempty option) on buildstream https://gitlab.com/BuildStream/buildstream/-/merge_requests/1834 | 17:52 |
| gitlab-br-bot | abderrahimk opened MR !1835 (abderrahim/options->master: _project.py: resolve options before running the final assertions) on buildstream https://gitlab.com/BuildStream/buildstream/-/merge_requests/1835 | 17:54 |
| *** tpollard has quit IRC | 18:23 | |
| *** santi has quit IRC | 18:48 | |
| *** santi has joined #buildstream | 18:49 | |
| *** santi has quit IRC | 18:52 | |
| *** lachlan has quit IRC | 19:13 | |
| *** cs-shadow has quit IRC | 19:23 | |
| *** phildawson has quit IRC | 20:08 | |
| *** phildawson has joined #buildstream | 20:09 | |
| *** phildawson has quit IRC | 20:12 | |
| *** benschubert has quit IRC | 21:29 | |
| *** traveltissues has joined #buildstream | 22:31 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!