| *** tristan has joined #buildstream | 06:23 | |
| *** jude has joined #buildstream | 06:58 | |
| *** ssam2 has joined #buildstream | 08:27 | |
| *** jonathanmaw has joined #buildstream | 08:40 | |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: _sandboxbwrap.py: Cleanup debris left behind by bwrap) https://gitlab.com/BuildStream/buildstream/commit/11a6b4e095a4d92be83bac622454228311f2556d | 08:45 |
|---|---|---|
| *** tiagogomes has quit IRC | 08:59 | |
| *** tiagogomes has joined #buildstream | 09:15 | |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: scriptelement.py: Cosmetic fix; use element names in messages.) https://gitlab.com/BuildStream/buildstream/commit/d17d99446436334190e3328c4ffdc05ab12892b0 | 09:28 |
| tristan | jjardon[m], can you remind me the branch you were using to test the baserock conversion ? | 09:37 |
| tristan | jonathanmaw, I think we missed a spot with the script element | 09:37 |
| jonathanmaw | ? | 09:37 |
| tristan | jonathanmaw, i.e. I think you updated the build-gnome branch of buildstream-tests, which is an example branch of a converted system | 09:37 |
| tristan | But I have a feeling we may have missed something in the other ? | 09:37 |
| tristan | Or no, I think I'm wrong | 09:38 |
| tristan | I think the baserock conversions dont include a script element automatically | 09:38 |
| tristan | and the only script elements were the initramfs which needs to be added manually after a baserock system autoconversion | 09:38 |
| ssam2 | yes | 09:40 |
| ssam2 | the scripts are only in the defs2bst README | 09:40 |
| tristan | Ah | 09:41 |
| ssam2 | jjardon 's branch is jjardon/bst2 of baserock/definitions.git | 09:41 |
| tristan | jonathanmaw, we should update the defs2bst readme to indicate the appropriate new script element syntax | 09:41 |
| tristan | jjardon[m], for what its worth, your autoconversions will now work with master buildstream and master ostree | 09:42 |
| tristan | Aarch64 has now built 27 out of 135 steps of meta-gnome-apps-tested.bst, without failing once ! | 09:43 |
| tristan | Nice | 09:43 |
| tristan | All of those crazy problems are conveniently swept under the rug for later | 09:43 |
| tristan | But setuid binaries are going to be a problem, I suspect | 09:44 |
| tristan | So, actually if anyone has some opinion on this it would be nice to not be the only one thinking about it | 09:46 |
| tristan | The crux of the issue is explained here: https://mail.gnome.org/archives/ostree-list/2017-June/msg00017.html | 09:46 |
| tristan | Basically, we currently have *some* limitations due to not building as root | 09:47 |
| tristan | A.) No static device nodes | 09:47 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: _artifactcache.py: Use compressed repositories.) https://gitlab.com/BuildStream/buildstream/commit/70914c8192e23de0d73e844670672c9eb0abc461 | 09:47 |
| tristan | B.) No "trusted" xattrs | 09:47 |
| tristan | C.) No assignment of files to arbitrary UIDs | 09:47 |
| tristan | That seems to be about it | 09:47 |
| ssam2 | A is pretty much ignorable right ? nothing modern should need those | 09:48 |
| ssam2 | but B and C are a pain | 09:48 |
| tristan | But, now with user mode checkouts of ostree (which are still currently completely munging permission bits entirely, but we're looking into changing that) | 09:48 |
| tristan | We additionally have the problem that we cannot checkout suid binaries in the sandbox, and the worst part is, it's a valid concern. | 09:49 |
| tristan | I'm currently working around this by *not* using user mode checkouts, but this just wont work with juergbi's artifact sharing code | 09:49 |
| ssam2 | why do we need suid binaries in the sandbox? surely we only need them in the final system ? | 09:49 |
| tristan | ssam2, Well two things: we create an image with mkfs.ext2 filesys.img -d ${sysroot} | 09:50 |
| tristan | Userspace utilities which allow you to change permissions of things inside an already created image are limited | 09:50 |
| tristan | We might have it for ext2 with genext2fs | 09:51 |
| ssam2 | fakeroot is an unholy mess, but pretty powerful | 09:51 |
| tristan | ssam2, asides from this; there is also the fact that, in order to manually do the job, it means you need to carry that information *beside* the fs tree | 09:51 |
| tristan | So it's a convoluted steaming pile of horse shit | 09:51 |
| tristan | If I have to go recording metadata about "this special little file wants to be suid in a deployment" | 09:52 |
| tristan | then that needs mention, somewhere | 09:52 |
| tristan | ssam2, right, and yocto does this with pseudo | 09:52 |
| tristan | which may be better | 09:53 |
| ssam2 | more or less the same I think | 09:53 |
| tristan | but they are both unholy | 09:53 |
| ssam2 | my gut feeling is it'd be great to avoid such hacks | 09:53 |
| tristan | Yes | 09:53 |
| ssam2 | but if we want to be able to create the entire OS filesystem locally as non-root, options are pretty limited ... | 09:53 |
| tristan | So I'm asking the question: If I want to create something which wants to have a setuid root /usr/bin/sudo, Am I willing to own a setuid /usr/bin/sudo myself to get the job done ? | 09:54 |
| tristan | What do others think ? How bad is this ? | 09:54 |
| tristan | I'm sure nobody in the room has a multi-user linux system, or if they do; the other users of the same machine are trustable | 09:55 |
| tristan | We are all admins on our machine | 09:55 |
| tristan | I raised in the email, the possibility that, your automated build machine is also hosting stuff with apache | 09:55 |
| ssam2 | thing is, at this point you might as well run the deploy step as root | 09:55 |
| ssam2 | safety concerns are more or less the same either way, so i prefer whichever is the simplest approach | 09:56 |
| tristan | and for some crazy reason, you are hosting a cgi script which can hijack your builder user's privileges by accessing an suid file owned by the builder user ? | 09:56 |
| tristan | ssam2, I disagree that these are the same | 09:56 |
| tristan | ssam2, running as a regular user means that you can be near 100% sure that running a build (which is filesys intensive stuff) never damages your system | 09:57 |
| tristan | Also it means that you can run the build in the context that you dont have root privileges on a system where you have a login | 09:58 |
| tristan | Owning an suid file is different | 09:58 |
| ssam2 | well, yes OK | 09:58 |
| ssam2 | then I guess as long as the safety concerns are communicated, it's fine | 09:58 |
| tristan | Right, now I think this could be a moving/progressing issue for BuildStream that we dont necessarily have to solve "Right now" | 09:59 |
| ssam2 | but if running as a regular user, how do you end up with a setuid $sandbox/usr/bin/sudo in the first place? | 09:59 |
| persia | I have both multiuser machines where I don't trust users to behave properly and access to machines where I am not admin, as a counterexample to the one-user-one-admin model. | 09:59 |
| tristan | ssam2, any user is allowed to own an suid binary | 09:59 |
| tristan | ssam2, it means that if you own an suid sudo, others who have permission to execute it can gain *YOUR* privileges through it | 10:00 |
| ssam2 | ahh, ok | 10:00 |
| tristan | ssam2, they cannot escalate to root privileges, only yours | 10:00 |
| persia | tristan: For clarity, do you imagine tracking what must be setuid (and for which uid) as metadata, and adjusting at assembly-time? | 10:00 |
| tristan | persia, thats what I was referring to as "A steaming pile of horse shit" above. | 10:01 |
| tristan | a convoluted mess | 10:01 |
| tristan | Some measure of virtualization is preferable to that | 10:01 |
| persia | Yes. I just wasn't sure if that was acceptance of the mess or argument against it. | 10:01 |
| tristan | I personally want an excuse to keep things simple | 10:02 |
| tristan | Now interestingly, it's also possible that for many use cases, like for instance developers booting a GNOME system, never *really* need the end product to have an suid sudo installed | 10:02 |
| tristan | So a stop-gap solution until we find something better, is to let the user decide if they want to block ownership of suid binaries in their sandbox, blocked by default | 10:03 |
| tristan | But allow it in the case that the user wants to build a production system | 10:03 |
| jonathanmaw | tristan: https://gitlab.com/BuildStream/defs2bst/merge_requests/2 should update the readme for defs2bst accordingly. | 10:04 |
| tristan | But "better" solutions currently include yocto pseudo (an LD_PRELOAD library which stores the real permissions in an sqlite database) | 10:04 |
| tristan | Or fakeroot | 10:04 |
| tristan | Both are arguably horrible hacks, but powerful | 10:04 |
| tristan | ssam2, So it actually doesnt end here; because *as it stands*, even if *we* decide that it's okay to let the user, even under only *some* circumstances, own suid binaries | 10:07 |
| tristan | ssam2, Currently ostree wont let us do it | 10:07 |
| tristan | So I need to make the argument that we need that possibility, if we are to have any escape route | 10:07 |
| ssam2 | is this is a matter of principle for OSTree or a matter of technical difficulty? | 10:08 |
| persia | tristan: Does the same ostree issue apply to setgid binaries or sticky-bit directories? | 10:08 |
| tristan | persia, as for your counter example, sure... but how would you answer the question: If you want to build and deploy a production system which contains a setuid /usr/bin/sudo, are you willing to "own" an suid binary in order to do so ? | 10:09 |
| tristan | ssam2, principle | 10:09 |
| ssam2 | right. In which case the question is whether OSTree wants to be able to hold a "real" OS image, or not | 10:09 |
| tristan | persia, setgid yes, sticky bit I have no idea, but I'm quite sure no | 10:09 |
| persia | tristan: Yes, because I don't have any *development* environments where I don't have admin over the environment (VM or machine). | 10:09 |
| ssam2 | if the answer is no, we're going to have to work around it somehow as we need something that can hold a real OS image | 10:09 |
| ssam2 | so I guess the case for OSTree is, do you want us to have to work around your policy? | 10:10 |
| tristan | ssam2, So correction (if you read the emails you will get it)... OSTree is perfectly capable currently except for the user mode | 10:10 |
| persia | There are all sorts of production needs (or even single-user machine needs) that require setgid and sticky, and in both cases, the majority of applications are for non-root ownership. | 10:10 |
| tristan | ssam2, here is the conundrum, I can use "bare" repository mode (which is normally intended for root / ostree-as-host) | 10:10 |
| tristan | ssam2, and I will checkout everything exactly as I committed it | 10:11 |
| tristan | ssam2, if I committed as a regular user, it will come out belonging to me | 10:11 |
| tristan | ssam2, but if I do that, and then push those artifacts to an archive-z2 artifact server, and ANOTHER user tries to checkout those commits... | 10:11 |
| tristan | then boom, if the other user doesnt also have UID 1000, they're screwed | 10:12 |
| tristan | If I use the user-mode checkouts, then I can own them automatically with the checking-out-user's UID | 10:12 |
| tristan | But ostree will munge the permission bits, thinking "This is for running software in a container, regular users who run software in a container should never have suid/sgid binaries" | 10:13 |
| persia | If "regular users who run software in a container should never have sgid binaries" is true, then one should never run games in containers, which doesn't entirely seem ideal. | 10:15 |
| persia | (most games are sgid "games", where "games" is a non-0 non-1000 gid) | 10:15 |
| tristan | persia, I'm not sure that is correct | 10:16 |
| tristan | persia, for instance, if I run a game in a flatpak under bubblewrap, everything in there is uid 0, it's a container running a process | 10:16 |
| tristan | there is no real reason in a user namespace to have multiple users, afaict | 10:17 |
| tristan | or maybe there is a runtime user and an app user ? | 10:17 |
| tristan | but not sure what that buys the container anyway | 10:17 |
| persia | Games are usually setgid games as a means of setting partial protections on some files vs. other files, to reduce the chance of the user overwriting certain things. Core datafiles and executables are usually owned by 0, so the setgid binary can't cause issues with them. Sorting this with only namespaces feels less granular, but maybe? | 10:19 |
| tristan | persia, its certainly less granular, but the needs are less granular in a container I think | 10:22 |
| tristan | maybe that wont always be true, though. | 10:22 |
| tristan | persia, also remember that you probably have an setgid binary that does not belong to you on the system | 10:23 |
| tristan | which means by design, someone who executes that binary, gains privilege of that group for the duration of that process | 10:24 |
| tristan | This is different from owning a setgid binary *yourself* | 10:24 |
| tristan | which, if others are allowed to execute it, may escalate their privileges *through you* | 10:25 |
| persia | Yes. I would only want to own a sgid binary on a development system, where I expected there to be no other users. Never in production. | 10:26 |
| ssam2 | It seems like there's a contradiction in what we're trying to do .. we want to preserve the system exactly as is (with exact UIDs and permission bits), but then check it out a bit differently to how it was (same permission bits but not the same UIDs) | 10:29 |
| persia | Although, for sgid, vs. suid, I don't think the owning user is as important, as long as the owning group is set appropriately (used, for example, with sgid directories in shared team filesystems, where users creating new directories (owned by them) cause the directories to also be owned by the group and sgid. | 10:30 |
| persia | ssam2: Ideally, yes. And we can't rely on name mapping either (some systems have default names like "games" or "daemon" that are used to determine local uid/gid at install-time). | 10:30 |
| ssam2 | Can artifacts have files owned by arbitrary UIDs and GIDs or not? | 10:32 |
| tristan | ssam2, Right now, as a hack, files in artifacts are all owned by the user; and checkouts are performed without user-mode | 10:41 |
| tristan | ssam2, that prevents artifact sharing | 10:41 |
| tristan | ssam2, normally, artifacts should have all files belonging to UID 0, and checkouts by the regular user would have them owned by the regular user | 10:42 |
| tristan | ssam2, with ybd, everything belongs to UID 0 | 10:42 |
| tristan | for instance, we use python TarFile to override the UID and make sure it's always 0 | 10:42 |
| tristan | (but then we run as root so it makes no difference) | 10:42 |
| ssam2 | that doesn't seem right, because in real OSes there can be files owned by arbitrary UIDs and GIDs ... | 10:42 |
| ssam2 | (not | 10:43 |
| tristan | actually maybe you are right | 10:43 |
| ssam2 | well, not arbitrary as they are defined in /etc/passwd, but you get what I mean | 10:43 |
| tristan | ssam2, but to be honest; that's not happening with current baserock defs | 10:44 |
| ssam2 | right | 10:44 |
| ssam2 | in the past it was possible, because we required the build and deploy to run as root | 10:44 |
| ssam2 | there is at least one package which does chown to root as part of `make install` -- i don't like that, but it's there | 10:45 |
| tristan | Right so to add a group or a user to /etc/passwd and /etc/group, requires no special permissions really | 10:45 |
| tristan | Yes, chown to root always works, in bubblewrap as well as in chroot | 10:45 |
| tristan | ssam2, What typically happens is that at runtime, we tell systemd to run this process/daemon as "pulseaudio" user | 10:46 |
| tristan | for example | 10:46 |
| ssam2 | I see | 10:46 |
| tristan | thats all defined in the service files and whatnot | 10:46 |
| ssam2 | there's also systemd-sysusers and the sysusers.d format which seems kinda nice | 10:47 |
| tristan | I do find it odd though | 10:47 |
| tristan | It seems that daemons and whatnot require owning files under their user | 10:47 |
| tristan | Not sure that all of this is really magically taken care of at boot time | 10:48 |
| ssam2 | especially if you don't have systemd | 10:48 |
| tristan | right | 10:48 |
| tristan | Now with bubblewrap, we *could* support arbitrary UID/GIDs, somehow, I think, it's complicated | 10:49 |
| tristan | in a user namespace you are allowed to remap multiple uid/gids | 10:49 |
| tristan | but it's very contorted if the build system has to know what the build content is going to want | 10:49 |
| persia | Much of the setuid/setgid/sticky mangling is done for reasons that could sensibly remap to namespaces in a rearchitected system, but the transition requires fairly deep understanding of the rationale behind each of the historical choices and probably a flag day. | 10:50 |
| persia | Well, that or CAPABILITIES | 10:50 |
| tristan | persia, I think a key point here is that the assumption with containers and namespaces, is that A.) A regular user owns some files, and B.) That same user will enter a virtual environment inside those files they own | 10:51 |
| tristan | A container can keep the processes being run contained, but it cannot hide itself from the surrounding system | 10:52 |
| persia | tristan: I don't trust that assumption, really. | 10:52 |
| tristan | Otherwise, what is the purpose of the container ? | 10:52 |
| persia | So, it covers cases like group www. | 10:52 |
| persia | It is less useful for cases like group pulse or group audio with jack. | 10:53 |
| persia | and actively false for cases like user mail | 10:53 |
| tristan | persia, right so, inside a more complex container, one can remap multiple UIDs and such; I'm not familiar with all of these container use cases | 10:54 |
| persia | It *might* work for group games, but it might not. | 10:54 |
| persia | I'm just listing magic uid/gid use cases. | 10:54 |
| persia | I haven't thought through how to map those in containers. | 10:54 |
| tristan | But, I could envision some cases where for instance, a container that requires admin privileges to *setup*, will allow remapping of some UIDs and mounting of some real system directories which contain files with special capabilities | 10:55 |
| tristan | persia, But in those cases, those special files with special capabilities, or suid files, dont ever belong to a regular user | 10:55 |
| tristan | that would break security outside of the container | 10:55 |
| tristan | Ok so anyway, all of this aside, the problem *right now* is user mode checkouts and sacrificing the ability to create systems with any suid binaries at all | 10:57 |
| persia | tristan: At least for group games, there are plenty of sgid binaries. | 10:57 |
| persia | Also for group audio | 10:57 |
| persia | I can't think of any non-root setuid binaries off the top of my head. | 10:58 |
| ssam2 | I think the choices are (a) bend OStree to our will, or (b) store ownership and permission info separately to the artifacts | 11:03 |
| ssam2 | and (b) breaks down into either fakeroot hacks, or manually specifying permission & ownership info in the .bst files | 11:03 |
| persia | Do we undestand our will enough for (a)? | 11:04 |
| ssam2 | but (a) has the issue that.. even if we bend OSTree to our will, we still can't necessarily check things out as a regular user | 11:04 |
| persia | The (real) need to potentially remap ownership without reliable naming in /etc/password makes me think (a) is potentially very difficult. | 11:04 |
| ssam2 | and (b) kinda has the same issue that at some point, you have to either avoid the kernel when producing the final system images, or be priviliged so the kernel lets you produce a filesystem with arbitrary permissions | 11:05 |
| * tristan adds at least one more comment to: https://github.com/ostreedev/ostree/issues/907 | 11:09 | |
| * tristan gets back from long meeting | 12:02 | |
| tristan | ssam2, persia; So with regards to both of these... first; "our will" in the short term, is we want reliable permission bits but we only really need to create systems with files owned by UID 0, because "we have ways" at boot time | 12:03 |
| tristan | Longer term, this can be enhanced | 12:03 |
| tristan | We should make some separation of what is needed right now for the basics and what would be good to have longer term | 12:04 |
| persia | tristan: For setuid, OK. For setgid, do you imagine using ways also? Lastly, how do we track when we need ways? | 12:04 |
| * persia thinks 99% of sticky bits is done by admins anyway, so less important from a system building perspective | 12:04 | |
| tristan | suid & sgid are permissions | 12:04 |
| tristan | We really, really should be able to get this "right now" even if it is at the expense of owning an suid binary to do so | 12:05 |
| tristan | Longer term, creating files which belong to other uids is something we should be able to do | 12:05 |
| tristan | yocto does this _and_ the suid bits with the pseudo LD_PRELOAD thing, but I hope something better emerges | 12:06 |
| tristan | But, this leaves us in a bit of a predicament for suid, even in the case that we can have an ostree which lets us create them; we should disable it by default | 12:07 |
| persia | To my mind, the blocker is tracking. It isn't hard to get the right assignments (either the build system thinks it is root and does it or the build system tries it anyway and we trap the error), but it is hard to keep this data available throughout the dataflows involved before delivery to the end state (either user unpack or system deployment). | 12:07 |
| tristan | persia, right; but this to my mind, is something that can be thought of in the longer term nice to haves; I hope, as long as ostree grants us this wish to be able to checkout suid files as a regular user | 12:08 |
| tristan | So at the very least, hopefully we dont have to think about this. | 12:08 |
| tristan | wow | 12:10 |
| tristan | On the bright side | 12:10 |
| persia | Ah, so short-term, artifacts are created with setuid/setgid stuff all configured properly, and if someone checks it out in user-mode, this is flattened, and this is an ostree wishlist to resolve? | 12:10 |
| tristan | My Aarch64 builds are looking _exactly the same_ as my intel builds | 12:10 |
| persia | That's not good. The ISAs differ enough that you should see at least filesize differences. | 12:10 |
| jjardon[m] | tristan: amazing; thanks. Anything I have to take in account / change in that branch? | 12:10 |
| tristan | persia, not the binaries | 12:10 |
| persia | If they truly produce the same thing, then either we need to be more aggressive about arch-any vs. arch-all, or we need another way to track artifact hashes. | 12:11 |
| * jjardon[m] reads channel history | 12:11 | |
| tristan | persia, the success rate, everything that succeeds on intel, succeeds on aarch64 | 12:11 |
| * persia is relieved | 12:11 | |
| persia | Oh, of course. We fixed that a couple years ago. | 12:11 |
| persia | Last I checked, there were only a few remaining bootstraps needing doing (LUAJit, OPC, etc.) | 12:11 |
| persia | But nothing in the conventional C-based toolchain | 12:12 |
| tristan | jjardon[m], nope, except maybe ensuring that the gitlab-ci builds ostree master and sets up GI_TYPELIB_PATH/LD_LIBRARY_PATH to ensure we use the master ostree ? | 12:12 |
| tristan | persia, and I think you misunderstand a bit about checkouts in user mode; the thing is: We _need_ to checkout in user mode, _always_ | 12:13 |
| tristan | In every single build sandbox, we need to checkout in user mode | 12:13 |
| persia | Then I don't understand the difference you are making between "nice to have" and "need this immediately" | 12:13 |
| tristan | Because UID/GID ownership of files, is part of the checkout data | 12:13 |
| tristan | sigh | 12:13 |
| tristan | This is frustratingly difficult to explain | 12:13 |
| persia | I may not need to understand, of course :) | 12:14 |
| tristan | persia, ok so we can have it one way, or we can have it the other way, but we can not have it both ways. | 12:14 |
| tristan | I can commit and checkout data to an OSTree repo (in "bare" mode), and exactly the same thing that goes in, comes out | 12:14 |
| tristan | This is fine, for the regular user, or any user. | 12:14 |
| tristan | BUT | 12:14 |
| tristan | If I want to share that artifact with ANOTHER user | 12:15 |
| persia | Right, mappings. | 12:15 |
| tristan | Then that user probably doesnt have my UID | 12:15 |
| tristan | So permission denied | 12:15 |
| persia | Also setgid stuff goes wonky in that case. | 12:15 |
| tristan | Which is why we need user mode checkouts, but we need user mode checkouts which _only_ munge the UID/GID to the active user | 12:15 |
| persia | Um, for UID, I can see how that might work. | 12:16 |
| tristan | Right now, they strip suid/sgid bits | 12:16 |
| persia | For GID, I don't see how it works at all. | 12:16 |
| tristan | Not sure about what you mean by "might work" | 12:16 |
| persia | One possible way for it to work (there are many potential implementations). Essentially, I can comprehend how to go about writing detailed requirements for a system that munged uids as described. | 12:17 |
| tristan | If I try to create a file that belongs to user "persia" on my machine, it will only succeed because user "tristan" and user "persia" both happen to be UID 1000 | 12:17 |
| persia | I cannot comprehend how to do such a thing for GIDs. | 12:17 |
| tristan | Because almost everyone on their laptop is UID 1000 | 12:17 |
| tristan | Now dont start putting the sandbox into the equation :) | 12:18 |
| tristan | and UID mapping, that'll just confuse things | 12:18 |
| tristan | When I enter the sandbox, *my* UID becomes UID 0, and in the build environment everything is UID 0, inside the sandbox | 12:18 |
| persia | Even ignoring the sandbox, if we cannot trust there to be entries with shared names in /etc/group, I don't understand how I could construct anbything setgid and have you unpack it properly. | 12:18 |
| tristan | but in real life, it's UID "tristan" | 12:18 |
| tristan | We are _only_ talking about ostree here | 12:18 |
| tristan | First I checkout data from ostree (which belongs to user "tristan"), run a build in a sandbox, then I exit the sandbox, and the results again belong to UID "tristan" | 12:19 |
| jjardon[m] | tristan: ok, I will try to play with it today | 12:19 |
| tristan | Now I commit the results (belonging to UID "tristan") into the ostree artifact cache | 12:19 |
| tristan | persia, because inside the sandbox, everything is UID 0, when I go ahead and create a rootfs image, everything in that image is also UID 0 | 12:20 |
| tristan | So then an image file gets committed as UID "tristan", with filesystems whos contents all contain stuff that is UID 0 | 12:21 |
| * persia does not consider that related to the discussion, and suspects conflation of UID and GID. | 12:21 | |
| persia | If GID is not considered now, then I have no point, other than the it ought have been :) | 12:21 |
| tristan | Ok all the files in OSTree are UID and GID "tristan" | 12:22 |
| tristan | When we enter the sandbox, the sandboxed processes see everything as UID/GID 0 | 12:22 |
| persia | Except user GIDs are almost never used for setgid, making that example uninteresting. | 12:23 |
| tristan | The point here is we're only talking about OSTree, we can have it one way, or the other, but not both | 12:23 |
| persia | It's non-root system gids that are the interesting case. | 12:23 |
| persia | Which need mapping. | 12:23 |
| persia | And the target user may belong to many groups, but the target user's primary group is almost certainly the incorrect value for the group for the setgid file. | 12:23 |
| tristan | I dont care right now about the meaningfulness of setuid and setgid bits on files, only inasumuch as that they are significant data on a file | 12:24 |
| tristan | Lets not get into the meaningfullness and usefulness of them | 12:24 |
| tristan | they are file metadata | 12:24 |
| persia | Yes. | 12:24 |
| tristan | Now: If I use "bare" repo mode and no "user mode checkout", then they are always recorded and checked out and never lost. | 12:25 |
| persia | But, in the setuid case, it is typially safe to assume the owner of the file should have been UID0, so one has some comprehension of the correct setting to use in a sandbox. | 12:25 |
| tristan | But so is the _ownership_ of the file. | 12:25 |
| tristan | which is _always_ UID "tristan" | 12:25 |
| tristan | Because I invoked buildstream | 12:25 |
| persia | In the setgid case, it is not safe to assume the group of a file should be group 0 (or group 1000), so one has to track the *name* of the group in order to capture the permission. | 12:25 |
| tristan | So it will _always_ work for me | 12:25 |
| persia | For UID, yes, it will work for you. | 12:26 |
| persia | For GID, it's more complex. | 12:26 |
| tristan | But when you check out the artifacts I want to share with you, and continue to build the other half of my build on your machine | 12:26 |
| tristan | Permission denied | 12:26 |
| persia | Yes. | 12:26 |
| tristan | _unless_ your user is also UID 1000 | 12:26 |
| persia | Sure. That's fixable with fairly trivial mapping solutions. | 12:26 |
| tristan | Now, OSTree has "bare-user" mode and "user mode checkouts", which allow you to checkout the filesystem tree as your user | 12:27 |
| tristan | *BUT*, if you exercise that feature, it will strip away the suid/sgid bits on any checked out files for you | 12:27 |
| tristan | So you lose data | 12:27 |
| persia | But I'm more interested in the issue when something needs to be GID 123 to execute properly, and gets checked out on a system with UID 0, UID 1000, GID 0, GID 1000, and GID 178 (which would be the map, but we need to know why). | 12:27 |
| tristan | Now: I'm trying to convince OSTree maintainers to have some API, method to allow me to checkout files as the active user, but *not* munge any permission bits. | 12:28 |
| persia | Do I understand that you could cheat, and check them out in bare-user mode inside a sandbox. and get full permissions (for now)? | 12:30 |
| persia | Except that you'd need some way of mapping "tristan" as UID 1000 to "persia" as UID 1001? | 12:31 |
| persia | Otherwise I'd get "permission denied" if I tried to do anything? | 12:31 |
| tristan | First of all, I dont want to ever do that, it will make the engine complex | 12:32 |
| tristan | Artifact staging and collection happens outside of the sandbox | 12:32 |
| tristan | In order to prepare a sandbox or collect output from it | 12:32 |
| * persia was imagining an epheremal sandbox used only to lie about UID to ostree | 12:32 | |
| persia | Err, ephemeral | 12:33 |
| tristan | Yeah, baggage to carry, really dont like it | 12:33 |
| tristan | On the other hand, I'm looking at one or two lines of code in ostree which decide to munge SUID/SGID bits | 12:34 |
| tristan | not so much baggage | 12:34 |
| tristan | Also, I'm not really entirely convinced that it would work | 12:34 |
| persia | I suspect the best thing for me to do is wait until those are adjusted, and then explore the setgid edge cases :) | 12:34 |
| tristan | Or even if it's possible | 12:35 |
| persia | It is possible, if the unpacking user has the right capabilities or access to a constrained environment with elevated capabilities. After unpacking, the unpacked results could be used for privilege escalation (as you discussed before). It might not actually make a working system, but it might be close enough for development testing. | 12:36 |
| tristan | Right, for a privileged user | 12:37 |
| tristan | But we've avoided the need of being root for any element processing so far; yocto builds systems fine without ever being root, yuck | 12:37 |
| tristan | right ? | 12:38 |
| tristan | Now, currently actually the ostree code is wrong and being fixed anyway: | 12:39 |
| tristan | https://github.com/ostreedev/ostree/pull/908/commits/5fb31fde40a87b76560fe5f0c28ffe697f110731 | 12:39 |
| tristan | The previous lack of suid/sgid stripping was a mistake | 12:40 |
| tristan | The OR'ing of 0755 is something nobody knows why the hell they were ever doing that to begin with | 12:40 |
| tristan | But if I at _least_ have my way with that patch (last comment here: https://github.com/ostreedev/ostree/issues/907) | 12:41 |
| tristan | Then _only_ the SUID/SGID will be munged, and we deal with that separately; hopefully I can get _some_ kind of patch in for it | 12:41 |
| * tristan is losing hair over this :-S | 12:41 | |
| ssam2 | given that the existing best practice is Fakeroot, we can safely say it's a hard problem | 12:43 |
| persia | +1 | 12:45 |
| persia | In Debian, in addition to use of fakeroot, there is elaborate additional machinery used to ensure gid mappings for setgid stuff. | 12:45 |
| tristan | indeed its a hard problem | 12:47 |
| tristan | But we can very well live with a situation without SUID/SGID munging in user mode checkouts and have a _lot_ of the desired functionality | 12:48 |
| tristan | before having to wallow in fakeroot/pseudo territory | 12:48 |
| tristan | we only really need that for creation of files owned by arbitrary uids/gids | 12:49 |
| tristan | afaics, fakeroot also virtualizes static dev nodes, but noone needs that | 12:49 |
| tristan | and root capability xattrs are not really that common | 12:49 |
| tristan | also assignment of daemons' file ownership is something I believe systemd can manage at boot time, so for a minority of systems which actually dare run systemd </snark> we dont even really need the uid/gid bits | 12:51 |
| * tristan has to go out for dinner | 12:52 | |
| tristan | I'll try to not think about this tomorrow and punt the artifact sharing problem to next week if I can manage | 12:52 |
| tristan | But I really wanted to land that | 12:52 |
| *** tristan has quit IRC | 12:57 | |
| *** jude has quit IRC | 13:31 | |
| *** jude has joined #buildstream | 13:34 | |
| *** jude has quit IRC | 14:37 | |
| *** luisa has joined #buildstream | 14:38 | |
| luisa | Look how I'm fucking online → http://best-sexchat.top | 14:40 |
| *** luisa has quit IRC | 14:41 | |
| *** jude has joined #buildstream | 15:17 | |
| *** jude has quit IRC | 16:04 | |
| *** jude has joined #buildstream | 16:20 | |
| *** jude has quit IRC | 16:32 | |
| *** jonathanmaw has quit IRC | 16:48 | |
| *** ssam2 has quit IRC | 17:04 | |
| *** jude has joined #buildstream | 19:34 | |
| *** jude has quit IRC | 19:48 | |
| *** jude has joined #buildstream | 20:05 | |
| *** jude has quit IRC | 20:46 | |
| *** albfan[m] has quit IRC | 21:24 | |
| *** mattiasb has quit IRC | 21:24 | |
| *** jjardon[m] has quit IRC | 21:24 | |
| *** waltervargas[m] has quit IRC | 21:24 | |
| *** persia has quit IRC | 21:24 | |
| *** juergbi has quit IRC | 21:24 | |
| *** ironfoot has quit IRC | 21:24 | |
| *** persia has joined #buildstream | 21:25 | |
| *** juergbi has joined #buildstream | 21:25 | |
| *** ironfoot has joined #buildstream | 21:25 | |
| *** irc.poop.nl sets mode: +o ironfoot | 21:25 | |
| *** albfan[m] has joined #buildstream | 21:40 | |
| *** jjardon[m] has joined #buildstream | 21:45 | |
| *** mattiasb has joined #buildstream | 21:46 | |
| *** Abu has joined #buildstream | 21:54 | |
| *** Abu has quit IRC | 21:55 | |
| *** mattiasb has quit IRC | 22:00 | |
| *** jjardon[m] has quit IRC | 22:00 | |
| *** albfan[m] has quit IRC | 22:00 | |
| *** waltervargas[m] has joined #buildstream | 22:59 | |
| *** albfan[m] has joined #buildstream | 23:22 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!