| *** bochecha has quit IRC | 00:05 | |
| *** xjuan has quit IRC | 00:17 | |
| *** foddo3 has joined #buildstream | 01:27 | |
| *** leopi has joined #buildstream | 03:15 | |
| *** leopi has quit IRC | 03:28 | |
| *** leopi has joined #buildstream | 05:06 | |
| *** ernestask[m] has left #buildstream | 05:12 | |
| *** lebster has joined #buildstream | 05:17 | |
| *** tristan has joined #buildstream | 06:07 | |
| gitlab-br-bot | buildstream: merge request (tpollard/386-cherrypick->bst-1.2: widget.py: Limit failure summary to currently failing elements (#386)) #589 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/589 | 06:32 |
|---|---|---|
| tristan | juergbi, Around ? | 06:34 |
| juergbi | yes | 06:34 |
| tristan | https://gitlab.com/BuildStream/buildstream/issues/38#note_87327672 seems to indicate that you intend to postpone fixing the permissions regression until a day that we can handle the most difficult parts, like setuid, arbitrary uid and xattrs | 06:35 |
| tristan | This is not good, at all | 06:35 |
| tristan | We need to open a separate, closable issue which does not block on rocket science | 06:35 |
| tristan | I am in the process of opening, and/or "dressing up" the deficit issues imposed by the early cycle landing of things | 06:36 |
| juergbi | tristan: being able to set group permission without actually being able to specify group id doesn't make sense to me | 06:36 |
| tristan | juergbi, it's a huge step backwards; right now you may need to do some hacks to create new groups and chowns before shipping a bootable system, but you can at least rely on the mode bits being carried through | 06:37 |
| tristan | before this regression | 06:37 |
| tristan | Lets move forward and keep focused on this | 06:38 |
| tristan | I don't think this is acceptable at all, and was really unhappy to accept the CAS merge at GUADEC because of this | 06:38 |
| juergbi | so we should add a temporary solution for permission bits and then might need to introduce something completely different for the rest? | 06:38 |
| tristan | Your reassurance that we can work around CAS made me think that we will fix this | 06:38 |
| juergbi | yes, of course, we will fix this, but we need to keep the whole picture in mind | 06:39 |
| tristan | that's conflating quite a bit, as far as I can see; CAS pertains to only storage of the metadata | 06:39 |
| tristan | I.e. if we can store the metadata of permissions, we should store UID/GID, xattrs, everything in the same storage | 06:39 |
| tristan | The mode bits do not rely on the sandbox having anything fancy, though | 06:40 |
| tristan | In a future where we address the more difficult issues, is there a reason why we would want to change the way we store file metadata ? | 06:40 |
| juergbi | well, with hardlinks mode bits are still problematic, and also were with OSTree | 06:40 |
| juergbi | (with FUSE that's fixed) | 06:41 |
| tristan | The FUSE in buildbox thing ? | 06:41 |
| juergbi | yes | 06:41 |
| tristan | Will that land in 1.3 ? | 06:42 |
| juergbi | well, it's not implemented but FUSE avoids the hardlink issue | 06:42 |
| juergbi | at least optional support for buildbox I definitely want to land in 1.3 | 06:42 |
| tristan | I'm a bit unhappy about not being able to backport the fix to 1.2, to be honest | 06:42 |
| tristan | Since I am not aware enough of what security concerns might arise from not having this information in flatpak SDKs and runtimes | 06:43 |
| tristan | I still don't know if 1.2 is going to blow up in our face because of this regression | 06:44 |
| tristan | And that is a very hard question to answer | 06:44 |
| juergbi | yes, it's hard to answer but I still don't expect any serious issues | 06:44 |
| juergbi | in environments where everything is single user | 06:45 |
| juergbi | i.e., it doesn't matter what group/other permissions you set if there is only one UID (and that's the owner of the file) | 06:45 |
| tristan | We need a plan B for 1.4 (and possibly even 1.2), in case we cannot really land buildbox transition | 06:45 |
| juergbi | Extending CAS to cover the permission bits like OSTree would likely be relatively simple | 06:46 |
| juergbi | I just don't want to jump the gun and add something where we might want a different solution for the big picture | 06:47 |
| tristan | During 1.2, I should note that it is quite intended to also generate bootable systems, well; we already have been in freedesktop-sdk and in other places | 06:47 |
| tristan | So flatpaks, while being the only thing (which we are *aware* of) that is "in production", is not the only concern | 06:47 |
| juergbi | yes, but as you mentioned above, the fully bootable system currently anyway needs undocumented workarounds | 06:47 |
| juergbi | adding a few chmods there is not impossible | 06:48 |
| tristan | juergbi, I understand the sentiment, but we need to live within the release cadence, and sometimes that requires compromise | 06:48 |
| juergbi | yes, if there are actual users that are impacted, we need a quicker fix that may not be ideal | 06:48 |
| tristan | And this is a slippery slope, we should not wait for perfect in order to at least do what is possible now | 06:49 |
| juergbi | but so far we're not aware of anyone like that | 06:49 |
| juergbi | that's not the same thing | 06:49 |
| tristan | juergbi, No, don't think of it that way, think of it the opposite way | 06:49 |
| juergbi | I'm saying the permission bits are hardly useful alone | 06:50 |
| juergbi | i.e., it's far away from a 'good enough but not perfect' solution | 06:50 |
| tristan | Here is a GNOME analogy: It's not because Nokia sold out to microsoft that we can throw stable API out the window; instead, it is only by providing this stability that others will buy in | 06:50 |
| tristan | It's not 'good enough', no, but it is most certainly better than "only executable or not" | 06:52 |
| juergbi | I'd say barely better | 06:53 |
| tristan | For a project which started out with the primary target being "Create custom bootable linux systems from scratch", because "If we can do that, we can cover all the other build cases", this is pretty embarrassing | 06:53 |
| juergbi | let's just say that when we look into this (and we should do this very soon), we first spend a few minutes thinking about the big picture instead of immediately ignoring everything outside the permission bits regression | 06:55 |
| juergbi | we should provide some solution for #38 soon anyway, even if it's just a documented way of handling the limitations | 06:56 |
| juergbi | I was hoping that we would get to that quite a while ago | 06:56 |
| tristan | juergbi, I understand, but please understand that this is a hard fight for me when we allow things to regress like this | 06:56 |
| juergbi | but the discussion is currently not whether to merge CAS or not, it's merged already, assuming you don't want to back it out... | 06:57 |
| juergbi | so we have to discuss what to do next | 06:57 |
| tristan | juergbi, while you might be personally very aware of what this could break and not, we also have to hold the line on issues like: https://gitlab.com/BuildStream/buildstream/merge_requests/581 | 06:57 |
| *** noisecell has joined #buildstream | 06:57 | |
| tristan | juergbi, So on one hand, I have people who misunderstand our priorities and goals, and have to design a format that will work for the ultimate plan | 06:58 |
| tristan | On the other hand, I have this regression | 06:58 |
| juergbi | that needs thinking of the whole picture (UIDs, GIDs, etc.) independent of the permission bits regression | 06:58 |
| tristan | juergbi, not *that* much, this is format we're talking about in 581, and it will still make sense to express mode bits separately from ownership bits in the format | 06:59 |
| juergbi | I'm not sure about that | 07:00 |
| juergbi | e.g., where would you store file capabilities if we were to support that? | 07:00 |
| tristan | Separately, of course | 07:00 |
| tristan | Actually, I think you dont mean store, I think you mean *express* | 07:00 |
| juergbi | store as in .bst | 07:01 |
| tristan | But I also dont think we'd want to express that part in the context of a plugin which stages a file | 07:01 |
| juergbi | maybe not but the same could be considered for permission bits outside the executable bit | 07:01 |
| tristan | File capabilities and xattrs are probably better expressed by calling setcap in a script, and carried through the artifact metadata | 07:01 |
| juergbi | possibly, but the same could be said for permission bits | 07:02 |
| juergbi | and uid/gid | 07:02 |
| juergbi | all of those are likely very uncommon | 07:02 |
| juergbi | (in this context) | 07:02 |
| tristan | Well, this is not the only case, because we also now have tar, local, zip, etc to think about; tar actually should ideally just work transparently | 07:04 |
| tristan | but currently I'm unsure of what it's behavior will be, asides from dropping the xattrs and ownership bits when run as a regular user | 07:04 |
| juergbi | there are both situations there | 07:04 |
| juergbi | for sources you often don't want uid/gid bits | 07:04 |
| tristan | juergbi, the point being that if we have a way to express permissions in sources, we should have the same way applied wherever it appears in the format | 07:05 |
| tristan | We both want the same thing. You want to only implement the ultimate solution for #38... I want to only implement the ultimate solution in the format. | 07:06 |
| tristan | So there will be gaps, and if there are gaps, that is fine; so long as everyone is on the same page of what the end product is going to be | 07:06 |
| tristan | It's better IMO to create a format which conveniently expresses the permission bits, acknowledging that some of these parts dont work because of the regression | 07:07 |
| tristan | Than to change formats along the way | 07:07 |
| juergbi | sure, as long as it's about aspects that are already clear | 07:07 |
| tristan | For local source, I'm thinking a similar behavior as proposed in 581 will work, with a possibility to have a list of glob patterns which single out specific files to be staged and set permissions on those | 07:08 |
| tristan | I do not think it sensible to expose only the executable bit through the format, when it's clear that all of the bits are important; and that permission bits are clearly different things than ownership bits and xattrs | 07:10 |
| tristan | <juergbi> we should provide some solution for #38 soon anyway, even if it's just a documented way of handling the limitations | 07:12 |
| tristan | juergbi, that also is disappointing, a documented way of handling the limitations is something we should have right now, in the *absence* of fixing #38 | 07:12 |
| tristan | Which still needs to remain our goal | 07:12 |
| tristan | Honestly, other tooling has solved this problem, we need to solve it to ensure we are relevant | 07:13 |
| juergbi | yes, I wanted to have this already a long time ago as well | 07:13 |
| juergbi | we should definitely have some reasonable support for permissions/ownership. explicit annotations may be part of the solution, though, imo | 07:14 |
| juergbi | (at least in case of setuid / file capabilities, I strongly prefer explicit annotations to implicit magic) | 07:15 |
| *** toscalix has joined #buildstream | 07:17 | |
| tristan | juergbi, explicit annotations in places are necessary, but is precisely what we want to strive to avoid | 07:23 |
| tristan | We've seen this before, in rpm spec files, in hacky scripts which run before deploying a firmware after building a system (I've seen these scripts in at least 3 organizations so far, and is always a huge hack that is hard to keep in sync and desirable to remove from production pipelines) | 07:24 |
| tristan | We're trying to achieve something better than the status quo, while we are doing great in other areas; we need to keep our focus on this end game as well | 07:25 |
| juergbi | tristan: it depends. I've also seen some horrible scripts before. however, as I mentioned, for setuid I as a distro maintainer would want to be aware of every single setuid file installed in the system. i.e., I would actually dislike the automatism there (because it could lead to security issues) | 07:28 |
| juergbi | regular permission bits are typically much less of an issue (although blindly accepting world-writable files might also not be ideal) | 07:29 |
| tristan | juergbi, That is *easily* handled with a semantic similar to `tar -tvv` | 07:29 |
| tristan | It can even be tested for | 07:29 |
| juergbi | I don't understand how manual listing of a tar archive helps here... | 07:30 |
| tristan | in the first place, when building a system; we have to first trust the upstream maintainers, of packages like PAM and openssl | 07:30 |
| tristan | to have put a lot of effort into their build scripts to install something sensible | 07:30 |
| juergbi | trusting upstream maintainers of security-critical packages is one thing | 07:31 |
| tristan | after that we tweak the system, only after a lot of burdensome research; knowledge which is better suited to the designers of the upstream software | 07:31 |
| juergbi | but allowing random app developers to install setuid root binaries is completely different | 07:31 |
| tristan | Random app developers have always been able to setuid on make install, it doesnt mean that they do so | 07:32 |
| *** noisecell has quit IRC | 07:32 | |
| tristan | I think that is a separate conversation, really | 07:32 |
| juergbi | on my system there are less than 30 setuid binaries in total, so the overhead of explicit annotation is definitely worth it to me, but maybe that's just me | 07:32 |
| juergbi | I guess so | 07:33 |
| tristan | Those 30 special cases add up, with setcaps, with intricate descisions of which binaries need which user/group/other bits, they all come together | 07:33 |
| tristan | Even if we were talking about one place to make those 30 exceptions; however, I don't believe we are | 07:34 |
| tristan | Because of the pipeline nature of BuildStream, we should be able to trust that when I depend on an element which made those 30 exceptions; those attributes should carry through in the data | 07:34 |
| juergbi | yes, I'm not saying we only want explicit annotations, just wanted to mention that some explicit annotations can even be beneficial and we shouldn't focus too much on 'magic' | 07:35 |
| tristan | (we shoulnt have to make those exceptions more than once) | 07:35 |
| juergbi | the main point is that it needs to be reasonably convenient and maintainable | 07:35 |
| tristan | juergbi, I think that filesystem data is essentially at the core of our bread and butter, and we need to be able to carry all these bits through in the artifacts - i.e. even if you want to explicitly set permissions for files installed by GDM or PAM, those should be applied in the corresponding elements | 07:45 |
| tristan | in the abstract, we are just a mechanism which processes elements with filesystem data in and out | 07:45 |
| juergbi | yes, I agree that we definitely would want annotations in the corresponding elements | 07:45 |
| tristan | We should not complicate things by saying that `chmod` run in the install commands is not carried forward, and that we need to run special "things" which read annotations from public data that an element exposes | 07:46 |
| juergbi | however, we may not be able to represent everything in the sandbox, which is a bit of an issue | 07:46 |
| tristan | Right, however; other build systems have managed to do it, the technique of `pseudo` and `fakeroot` are a bit too ugly for us; but I don't think this is impossible | 07:48 |
| tristan | I.e. first of all, we definitely need to accept that a setcap binary in a sandbox will not really have the capabilities it purports to have on the host system | 07:49 |
| juergbi | to some extent that actually works in a user namespace (as much as the sandbox allows) | 07:49 |
| *** finn has joined #buildstream | 07:49 | |
| juergbi | the biggest issue is uid/gid | 07:49 |
| juergbi | with a suitably configured linux host, I think we can handle this with subuid/subgid. however, this raises the bar for host system requirements | 07:50 |
| tristan | Right, and the only issue is recording it, pseudo uses LD_PRELOAD but we probably need a ptrace solution | 07:50 |
| *** bochecha has joined #buildstream | 07:51 | |
| *** bochecha has joined #buildstream | 07:51 | |
| juergbi | one possibility I mentioned on gitlab is the combination of seccomp and ptrace. pure ptrace would be way too slow | 07:51 |
| juergbi | for just recording, the overhead for this may be acceptable, however, if we also want stat() to return the proper uid, this could be an issue | 07:52 |
| tristan | yeah, we definitely need that | 07:52 |
| juergbi | also, it's far from ideal that we have to pay the overhead for both FUSE and ptrace | 07:52 |
| tristan | Unfortunately, even if we only need read-back in a few cases, it would be horrible if the user ever had to know that and choose | 07:53 |
| juergbi | one option could be to implement two solutions. one with subuid/subgid (requires suitable host config but is fast) and the other is ptrace (works on any linux but is slow) | 07:53 |
| juergbi | I expect the ptrace support with read-back to have a noticeable overhead | 07:53 |
| juergbi | and thus really want the faster subuid/subgid support | 07:53 |
| juergbi | (or LD_PRELOAD but that's not a good fit) | 07:54 |
| juergbi | but the ptrace support is also the most complicated one, and it's the one we wouldn't actually recommend... | 07:54 |
| tristan | If we could run bwrap with LD_PRELOAD and have that override apply to what runs in the container, that'd be fine, but it's not possible | 07:55 |
| juergbi | yes, it would require cooperation by the project, we couldn't do it transparently | 07:56 |
| tristan | rather, we need fuse-like support for overriding the syscall table in a tree of processes, without context switching | 07:56 |
| juergbi | the only other option I could think of would be running a Linux VM | 07:56 |
| * tristan has to eat lunch at this point | 07:57 | |
| *** tristan has quit IRC | 08:00 | |
| *** noisecell has joined #buildstream | 08:07 | |
| *** leopi has quit IRC | 08:08 | |
| *** noisecell has quit IRC | 08:11 | |
| *** ernestask has joined #buildstream | 08:12 | |
| * paulsherwood reads backscroll, and realises that folks are using 'stable' is if it is a real thing | 08:21 | |
| *** mohan43u has quit IRC | 08:23 | |
| *** mohan43u has joined #buildstream | 08:26 | |
| *** leopi has joined #buildstream | 08:28 | |
| *** bochecha has quit IRC | 08:30 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/331_include->master: WIP: Add support for include in project.conf) #471 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/471 | 08:48 |
| *** leopi has quit IRC | 08:53 | |
| *** leopi has joined #buildstream | 09:00 | |
| tiagogomes | fakeroot_ng uses ptrace. However when I looked at it it didn't support all the arches baserock cared about | 09:05 |
| juergbi | tiagogomes: do you know anything about the performance overhead offhand? | 09:07 |
| tiagogomes | juergbi, no sorry, I didn't run any benchmark at it | 09:08 |
| *** noisecell has joined #buildstream | 09:19 | |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: WIP: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 09:19 |
| *** WSalmon has quit IRC | 09:37 | |
| *** WSalmon has joined #buildstream | 09:37 | |
| gitlab-br-bot | buildstream: issue #482 ("Decide whether to add type hints to coding standard") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/482 | 10:14 |
| qinusty | adds68: RE Manifect - Is the desire for this manifest to be produced during a build and elements, refs etc? | 10:18 |
| adds68 | qinusty, erm i think it would be better to just call bst on a project with an option to produce the manifest, producing one every build is probably wasteful | 10:24 |
| qinusty | Does #freedesktopsdk have a manifest I can take a look at? | 10:24 |
| gitlab-br-bot | buildstream: merge request (mablanch/448-autocompletion-broken-defaults->master: WIP: Resolve "Autocompletion defaults to the project's root before the element path") #592 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/592 | 10:41 |
| *** bochecha has joined #buildstream | 10:50 | |
| gitlab-br-bot | buildstream: merge request (mablanch/448-autocompletion-broken-defaults->master: WIP: Resolve "Autocompletion defaults to the project's root before the element path") #592 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/592 | 10:59 |
| Nexus | I'm making changes to someone else's MR, should i append a commit onto their branch? | 11:16 |
| gitlab-br-bot | buildstream: merge request (jmac/virtual_directories->master: Abstract directory class and filesystem-backed implementation) #445 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/445 | 11:16 |
| tlater | Nexus: Ask someone if they're happy with that ;p | 11:16 |
| Nexus | not had to do this before so i'm not sure what our policy is | 11:16 |
| Nexus | tlater: they aren't here | 11:16 |
| Nexus | thta's why i'm doing it | 11:17 |
| tlater | Seems fine by me then, they can always revert it if need be. | 11:17 |
| Nexus | kk | 11:17 |
| Nexus | ty | 11:17 |
| gitlab-br-bot | buildstream: merge request (richardmaw/cache-fail->master: Store failed builds in the cache) #475 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/475 | 11:18 |
| tiagogomes | Hrm, fixing ttps://gitlab.com/BuildStream/buildstream/issues/195 will disallow side-by-side projects as freedesktop-sdk has | 11:19 |
| tlater | Hrmmmmm... I'm trying to import an OS tarball, which of course contains device nodes | 11:23 |
| tlater | Those can't be stored in CAS | 11:23 |
| tlater | Is there any way to remove everything in /dev before creating my artifact in an import element? | 11:23 |
| tlater | I suppose I might need a manual element, but just in case... | 11:23 |
| Kinnison | I thought juergbi had done something to put non-CAS-recordable content alongside in the bst cache? | 11:23 |
| tlater | Kinnison: I get an exception telling me that /dev/null has an unsupported file type | 11:24 |
| Kinnison | oh dear | 11:24 |
| Kinnison | Is this only when using a remote cache? | 11:24 |
| Kinnison | Or with the local CAS based cache? | 11:24 |
| tlater | Kinnison: Local cache, can't cache the artifact. | 11:25 |
| Kinnison | blerp | 11:25 |
| * Kinnison guesses juergbi needs to weigh in | 11:25 | |
| tlater | Kinnison: I'm 90% sure I'll need to create a manual element and remove the device nodes, frankly. This is how things have worked for as long as I can remember | 11:27 |
| tlater | The error message is nicer now, though | 11:27 |
| Kinnison | tlater: Yeah, the manual element approach is probably easiest | 11:28 |
| * Kinnison supposes elements which want to create device nodes in the final output will probably have to do so with integration scripts or somesuch | 11:28 | |
| tlater | Yep, I'll probably have to do this as part of my init scripts | 11:29 |
| tlater | Though I think that's normal anyway | 11:29 |
| gitlab-br-bot | buildstream: merge request (mablanch/448-autocompletion-broken-defaults->master: WIP: Resolve "Autocompletion defaults to the project's root before the element path") #592 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/592 | 11:30 |
| * Kinnison used to have an initramfs based distro at a previous job where the linuxrc started by creating device nodes for the most part (though I think null and console were needed or the kernel was upset) | 11:31 | |
| jjardon | Hi! Would it be posible to add https://gitlab.com/BuildStream/buildstream/issues/524 to the blocker list? That issue is currently blocking us to use 1.1.4 | 11:32 |
| Kinnison | That was easy to do though because it was a stock cpio of the device nodes | 11:32 |
| *** finn has quit IRC | 11:33 | |
| juergbi | tlater: regular users normally can't create device nodes (I believe this will be supported in user namespaces with 4.18), I'm confused why you even get that far | 11:36 |
| juergbi | tlater: or are you running bst as root? | 11:36 |
| tlater | juergbi: Extracting a tarball | 11:36 |
| tlater | Oh, yeah, hell I am! | 11:36 |
| * tlater should fix that | 11:36 | |
| juergbi | ah ok | 11:36 |
| juergbi | tar can't normally even extract those | 11:36 |
| tlater | Hrm, I'll need custom tar flags then | 11:37 |
| * tlater checks if BuildStream supports this highly eccentric case yet | 11:37 | |
| juergbi | we should probably make sure that bst tar behavior is always the same, no matter what user | 11:37 |
| tlater | juergbi: This is why we test as a normal user in CI ;) | 11:38 |
| juergbi | yes, that's definitely good but occasionally you have someone running bst as root ;) | 11:39 |
| tlater | Yeah, I suppose this could be a problem for actual projects that don't test as thoroughly as we do. | 11:39 |
| tlater | juergbi: Do you have suggestions for how I could extract a tarball with device nodes in it? | 11:41 |
| tlater | I could do it on the command line, but BuildStream doesn't handle this case well... | 11:41 |
| tlater | Hmm | 11:41 |
| tlater | I guess I could use a file source and manually extract it | 11:42 |
| juergbi | tlater: how do you manually extract it? | 11:44 |
| juergbi | btw: there is no need to have device nodes in tarballs / persistent file systems anymore these days | 11:44 |
| tlater | juergbi: I know, but I'm grabbing alpine's tarball | 11:45 |
| tlater | juergbi: tar -xf <tarball> -X dev/* would work, I think | 11:47 |
| tlater | If I used `kind: remote` instead of `kind: tar` | 11:48 |
| persia | juergbi: None at all? I thought there was still a need for /dev/zero, /dev/console, and a couple others (either in storage or in initramfs). | 12:14 |
| juergbi | persia: devtmpfs mount covers whole /dev | 12:16 |
| juergbi | and it's mountable early enough | 12:17 |
| Kinnison | I think you can ask the kernel to automount it after it unpacks the initramfs | 12:32 |
| persia | It's an ordering issue. I had thought that a very minimal set of device nodes was required to run init, but if one can mount early enough, it doesn't matter. | 12:35 |
| Kinnison | CONFIG_DEVTMPFS_MOUNT: Automount devtmpfs at /dev | 12:36 |
| Kinnison | Though apparently that may not affect initramfs booting | 12:36 |
| * Kinnison hrms | 12:36 | |
| Kinnison | Oh it means that initramfsen need to sort /dev on the target themselves | 12:37 |
| *** mohan43u has quit IRC | 12:48 | |
| *** leopi has quit IRC | 12:49 | |
| *** mohan43u has joined #buildstream | 12:51 | |
| persia | Increasingly not the ideal channel for the discussion, but generally the initramfs can mount devtmpfs (or whatever), making that easy. The tricky bit is launching userspace processes in initramfs before /dev is populated at all. A common example is init scripts that want to redirect output to /dev/null or /dev/console before those exist. | 12:53 |
| *** finn has joined #buildstream | 12:54 | |
| *** xjuan has joined #buildstream | 12:55 | |
| *** mohan43u has quit IRC | 12:55 | |
| *** mohan43u has joined #buildstream | 12:58 | |
| *** leopi has joined #buildstream | 13:03 | |
| *** mohan43u has quit IRC | 13:03 | |
| *** mohan43u has joined #buildstream | 13:07 | |
| *** mohan43u has quit IRC | 13:11 | |
| *** mohan43u has joined #buildstream | 13:14 | |
| gitlab-br-bot | buildstream: merge request (mablanch/447-stack-trace-checkout->master: Handle checkout failure for unbuilt elements) #590 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/590 | 13:18 |
| toscalix | should we consider a "race condition" a bug by default? | 13:26 |
| noisecell | if you want the tool to be deterministic, I would say yes? | 13:27 |
| paulsherwood | it depends, imo. it could be deliberate as a feature of the algorithm/design | 13:28 |
| paulsherwood | (i'm thinking of ybd's randomised build order... which guarantees races) | 13:29 |
| paulsherwood | but most of the time it's almost certainly a bug | 13:29 |
| flatmush | generally you'd only use the term race condition if it's unplanned | 13:31 |
| paulsherwood | oh... | 13:32 |
| * paulsherwood googles | 13:32 | |
| * paulsherwood now agrees with flatmush | 13:33 | |
| paulsherwood | toscalix: yes :-) | 13:33 |
| persia | toscalix: I think it depends on the underlying problem. If it introduces nondeterminism, then I think it is a bug. If it just reorders log files, I think it doesn't matter. | 13:33 |
| flatmush | is a reordered log file not an example of non-determinism? | 13:34 |
| persia | Or, rather, if it introduces *critical* nondeterminism. Being fair about it, disordered logs are nondeterministic, but that happens with any parallel build process and is largely unavoidable if builds should happen quickl;y. | 13:34 |
| paulsherwood | yup, imo | 13:34 |
| persia | flatmush: It is, although my brain seems to have had an incorrect semantic mapping for "nondeterminism" when I wrote the first sentenc. | 13:35 |
| toscalix | thanks. I will label the ticket then as bug | 13:35 |
| gitlab-br-bot | buildstream: merge request (mablanch/448-autocompletion-broken-defaults->master: WIP: Resolve "Autocompletion defaults to the project's root before the element path") #592 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/592 | 13:56 |
| *** toscalix has quit IRC | 14:06 | |
| *** leopi has quit IRC | 14:06 | |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: WIP: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 14:08 |
| *** toscalix has joined #buildstream | 14:08 | |
| gitlab-br-bot | buildstream: issue #531 ("Fetch jobs resume despite "terminate" option on CTRL-C") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/531 | 14:10 |
| *** leopi has joined #buildstream | 14:12 | |
| *** Phil has quit IRC | 14:14 | |
| gitlab-br-bot | buildstream: merge request (relative_workspaces->master: WIP: Patch for issue #191 support relative workspaces) #504 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/504 | 14:15 |
| *** leopi has quit IRC | 14:21 | |
| *** eth2 has joined #buildstream | 14:27 | |
| valentind | It takes an awful amount of time for bst to start because it tries getting the size of the artifact directory. | 14:29 |
| tlater | valentind: We have a ticket for that open | 14:32 |
| tlater | I think jonathanmaw will be looking at that next week (he's on vacation atm) | 14:32 |
| valentind | tlater, OK, thank you. | 14:32 |
| valentind | So I suppose until next week, I have to clean my artifact directory. | 14:33 |
| tlater | :( | 14:35 |
| tlater | Sorry about that, I wanted to spend more time optimizing it, but the branch was already huge at that point. | 14:35 |
| valentind | No worries. | 14:40 |
| toscalix | hopefully we will have a known issues page soon where, at least, we will be able to prevent users and tell them workarounds until problems that are known get fixed | 14:41 |
| tlater | Oh, that would be very useful. I've also been accidentally re-stating known issues as of late, it's hard to keep up. | 14:42 |
| toscalix | but let's keep in mind that the default action when something like this happens is to revert the changes. It is important that the changes are small enough so revert them does not becomes a drama | 14:43 |
| persia | One solution I've seen work well elsewhere is to merge a minimal implementation, followed immediately by opening a new WIP branch with improvements, etc. | 14:45 |
| persia | This helps stall reversion by clearly communicating to all that the issues are being addressed. | 14:45 |
| laurence | so in this case do you propose to revert the change? maybe we cannot because it was a huge branch. but this is effecting users, sadly | 14:45 |
| persia | This depends on most users not using the primary integration branch though. | 14:46 |
| laurence | tlater, what's the ticket that captures this technical debt? | 14:47 |
| toscalix | laurence, this is where the maintainer rules if there is no other process. | 14:47 |
| laurence | toscalix, I notice we have a label named 'needs backport' - is that new? | 14:47 |
| laurence | seems a good idea :) | 14:47 |
| tlater | laurence: needs backport is ages old ;p | 14:48 |
| laurence | tlater, ah | 14:48 |
| tlater | Only one issue had it afaik. | 14:48 |
| laurence | tlater, just rarely used | 14:48 |
| laurence | toscalix, understood | 14:48 |
| toscalix | I cannot say of we should revert the change or not. That requires a carefull evaluation. | 14:48 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: WIP: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 14:48 |
| laurence | toscalix, should we also have a label for 'technical debt' at least ? | 14:49 |
| laurence | I believe a label is enough to capture those issues | 14:49 |
| toscalix | I have concerns about putting a label with such a broad meaning | 14:49 |
| toscalix | we would end up with that label in so many places... | 14:49 |
| toscalix | it would become useless at some point. If it requires further explanation, I would not do it | 14:50 |
| toscalix | if there is a technical debt, you create another task explaining what should be done, why and when | 14:50 |
| laurence | ok, but i am keen to devise some way of capturing the 1.4 technical debt | 14:50 |
| Nexus | my tests are hanging on `tests/frontend/workspace.py::test_build[strict-git]` | 14:50 |
| Nexus | anyone else getting this? | 14:50 |
| laurence | sooner rather than later, since we are developing a lot at the moment and the earlier we start the better | 14:51 |
| laurence | I can help by providing what I think the list is | 14:51 |
| laurence | we just need to capture in gitlab | 14:51 |
| toscalix | create another tasks and assign 1.4 milestone, link the task to the one being closed | 14:51 |
| tlater | laurence: Sorry that took so long, the issue is #466 | 14:51 |
| laurence | tlater, cheers | 14:51 |
| cs_shadow | hi, who should I contact for the credentials of the "buildstream" docker hub account? | 14:52 |
| tlater | That would be me | 14:52 |
| adds68 | Hi, has anyone seen this error when trying to contact the cache? | 14:53 |
| adds68 | [--:--:--][][] WARNING Failed to initialize remote https://testcache.codethink.co.uk:11002: Method not found! | 14:53 |
| tlater | Oh, wait | 14:53 |
| tlater | The actual buildstream account? | 14:53 |
| tlater | ssssam[m] probably still owns those. | 14:53 |
| cs_shadow | tlater: yeah, the actual buildstream one. Apparently the CI one doesn't have enough permissions other than push | 14:54 |
| tlater | It certainly doesn't, but I believe I gave you an account with enough permissions to do anything? | 14:54 |
| tlater | If not; I definitely need to now, heh | 14:55 |
| cs_shadow | tlater: no, I do not have any special creds | 14:55 |
| tlater | cs_shadow: Let me see what I can do | 14:56 |
| cs_shadow | tlater: thanks | 14:57 |
| qinusty | nope adds68. I've never tried it over a network though | 14:59 |
| adds68 | juergbi, hey i don't suppose you have seen the error i am seeing from the cas before ^^ ? | 14:59 |
| adds68 | qinusty, hm yea it did work, but since the land and switching to 1.1.4 i now see this error | 14:59 |
| juergbi | adds68: hm, no, I don't think I've seen that before | 15:00 |
| adds68 | :( | 15:01 |
| adds68 | no logs reported from systemd either | 15:01 |
| adds68 | juergbi, oh wait, there is an error now: SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE. | 15:02 |
| qinusty | How are you starting the CAS server? | 15:07 |
| adds68 | bst-artifact-server --port 11002 --server-key /here/ --server-cert /here/ --client-certs /home/artifacts/authorized.crt --enable-push /path/to/repo | 15:11 |
| adds68 | qinusty, ^ | 15:11 |
| tlater | cs_shadow: Do you have a docker account? | 15:12 |
| gitlab-br-bot | buildstream: merge request (mablanch/448-autocompletion-broken-defaults->master: WIP: Resolve "Autocompletion defaults to the project's root before the element path") #592 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/592 | 15:14 |
| cs_shadow | tlater: I can create one, give me a minute | 15:15 |
| tlater | In fact, does anyone else here have a valid docker account? iirc it took me a little while to make one. | 15:15 |
| tlater | I'm not going to be around much after today, so it would be good to have someone else with admin rights. | 15:16 |
| cs_shadow | tlater: here's my profile: https://hub.docker.com/u/csshadow/ | 15:17 |
| tlater | Ah, jjardon is also an owner | 15:18 |
| tlater | cs_shadow: I'm adding you as an admin, you're doing most of the maintainer work these days anyway | 15:19 |
| tlater | You should now be able to do anything you like here: https://hub.docker.com/u/buildstream/dashboard/ | 15:19 |
| cs_shadow | tlater: thanks very much | 15:20 |
| *** edb has joined #buildstream | 15:21 | |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: WIP: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 15:32 |
| flatmush | Does buildstream isolate the network when sandboxing? | 16:06 |
| tlater | flatmush: It attempts to | 16:07 |
| * tlater has not asserted how effective it is at doing so | 16:07 | |
| gitlab-br-bot | buildstream: issue #532 ("bst cache responds: "Failed to initialize remote https://testcache.codethink.co.uk:11002: Method not found!"") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/532 | 16:08 |
| flatmush | ok, we're just seeing odd inconsistencies that would make it seem like it's accessing the network, but not sure yet, will keep you posted | 16:09 |
| adds68 | Has anyone seen this error when configuring/using the cache? SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE. | 16:16 |
| qinusty | Didnt see the ping adds68 :( Sorry. I assume your /here/ for server crt and key actually points to the crt and key files? | 16:25 |
| qinusty | as opposed to a directory | 16:25 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 16:25 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 16:27 |
| adds68 | qinusty, yea correct | 16:31 |
| adds68 | qinusty, from further investigation it could be an ssl issue | 16:31 |
| adds68 | due to certificates | 16:31 |
| gitlab-br-bot | buildstream: merge request (richardmaw/cache-fail->master: Store failed builds in the cache) #475 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/475 | 16:34 |
| laurence | qinusty, are you looking for some extra info on the YAML indentation issue from adds68 ? https://gitlab.com/BuildStream/buildstream/issues/470 | 16:55 |
| *** noisecell has quit IRC | 16:55 | |
| gitlab-br-bot | buildstream: merge request (mablanch/448-autocompletion-broken-defaults->master: Fix autocompletion for elements in sub-folders) #592 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/592 | 17:04 |
| *** phildawson has quit IRC | 17:06 | |
| *** tlater has quit IRC | 17:08 | |
| *** aiden has quit IRC | 17:08 | |
| *** bethw has quit IRC | 17:08 | |
| *** milloni has quit IRC | 17:09 | |
| *** bethw has joined #buildstream | 17:11 | |
| *** aiden has joined #buildstream | 17:11 | |
| *** milloni has joined #buildstream | 17:12 | |
| *** tlater has joined #buildstream | 17:13 | |
| gitlab-br-bot | buildstream: issue #76 ("Cache failed builds") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/76 | 17:17 |
| gitlab-br-bot | buildstream: merge request (richardmaw/cache-fail->master: Store failed builds in the cache) #475 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/475 | 17:17 |
| *** leopi has joined #buildstream | 17:36 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/331_include->master: WIP: Add support for include in project.conf) #471 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/471 | 17:57 |
| *** cholcombe has joined #buildstream | 18:37 | |
| cholcombe | <+SP9002_@efnet> so, he wants the win. so we're just gonna get lunch or something, then hes gonna push me to the ground and tap my ass with his foot so he can claim he "kicked my ass" tbh im going along with it becase I dont wanna lose any teeth | 18:37 |
| *** cholcombe has quit IRC | 18:37 | |
| laurence | at least his patter was semi-interesting.... | 18:44 |
| laurence | Nexus, right so with the merging of caching of failed builds - https://gitlab.com/BuildStream/buildstream/merge_requests/475 | 18:45 |
| laurence | We need to file an extra issue to account for the technical debt we have accrued | 18:45 |
| laurence | as there's an issue with artifacts not being pushed | 18:46 |
| *** leopi has quit IRC | 18:51 | |
| *** xjuan has quit IRC | 19:21 | |
| *** jennis2 has joined #buildstream | 19:55 | |
| *** jennis has quit IRC | 19:57 | |
| *** ernestask has quit IRC | 20:38 | |
| *** bochecha has quit IRC | 22:33 | |
| *** edb has quit IRC | 22:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!