| *** bochecha has joined #buildstream | 08:11 | |
| bochecha | tristan: hi, for #78, do we go with adding a new `exclude:` to the `compose` element config? I'd be happy to take a stab at implementing that :) | 08:18 |
|---|---|---|
| *** jonathanmaw has joined #buildstream | 08:30 | |
| *** tlater has joined #buildstream | 08:56 | |
| *** ssam2 has joined #buildstream | 08:59 | |
| *** jude has quit IRC | 09:32 | |
| *** jonathanmaw_ has joined #buildstream | 09:33 | |
| *** jonathanmaw has quit IRC | 09:34 | |
| *** jude has joined #buildstream | 09:48 | |
| *** jonathanmaw_ has quit IRC | 10:14 | |
| *** jonathanmaw has joined #buildstream | 10:15 | |
| gitlab-br-bot | buildstream: issue #77 ("Add a `bst build --no-pull` mode") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/77 | 11:24 |
| gitlab-br-bot | buildstream: issue #77 ("Add a `bst build --no-pull` mode") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/77 | 12:22 |
| ssam2 | bah, i didn't mean to do that | 12:23 |
| ssam2 | well, i did, but then I changed my mind again | 12:23 |
| gitlab-br-bot | buildstream: issue #77 ("Add a `bst build --no-pull` mode") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/77 | 12:23 |
| *** tlater has quit IRC | 12:28 | |
| *** tlater has joined #buildstream | 13:35 | |
| *** tristan has quit IRC | 14:29 | |
| *** tlater has quit IRC | 14:33 | |
| *** tlater has joined #buildstream | 14:36 | |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 1 commit (last: Make the sandboxchroot chroot in the same process) https://gitlab.com/BuildStream/buildstream/commit/5ad569d78c0c277710fc894dbb03562f60776037 | 14:50 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 14:51 |
| *** bochecha has quit IRC | 14:51 | |
| *** tristan has joined #buildstream | 15:01 | |
| *** ChanServ sets mode: +o tristan | 15:01 | |
| tristan | ssam2, yeah; I was thinking about that yesterday and then thought that --no-pull doesnt make much sense | 15:01 |
| tristan | for the proposed use case | 15:02 |
| tristan | A '--no-pull' option would seem to imply it's not going to pull any artifacts; which might mean; even if I have some things to build; dont use the artifact share to download any dependencies | 15:02 |
| tristan | so, it wouldnt have been the right name anyway | 15:02 |
| ssam2 | yeah, i'm not particularly attached to that design | 15:04 |
| ssam2 | and yeah I see what you mean, --no-pull is not even unambiguous! | 15:05 |
| tristan | yeah I knew something stinked about that heh | 15:06 |
| tristan | ssam2, fwiw I really have to figure out what to do with conditionals | 15:07 |
| tristan | i.e. this https://mail.gnome.org/archives/buildstream-list/2017-August/msg00007.html | 15:08 |
| tristan | anyway I have a variantless design brewing in the background | 15:08 |
| ssam2 | interesting | 15:09 |
| tristan | maybe I should writeup a tentative design and send to list ? | 15:09 |
| ssam2 | yeah, please do | 15:09 |
| tristan | thing is time is tight and I need something fast but rather get some feedback | 15:09 |
| * tristan wonders what is bochecha's time zone | 15:10 | |
| ssam2 | i very much agree with your principal of wanting to avoid combinatorial explosion | 15:10 |
| tristan | variants gave us a kind of control on limiting it; or a system that encourages less possibilities | 15:10 |
| tristan | I think | 15:11 |
| ssam2 | yeah | 15:11 |
| ssam2 | but I also see the issue that you can only have one variant, which might be /too/ limiting | 15:11 |
| tristan | Even granular element to element constraints limits the amount of possibilities | 15:11 |
| ssam2 | very hard to get right | 15:11 |
| tristan | and yeah, imaging granular variable * element constraints; ... exponentially more complex to get right than just one variant | 15:12 |
| ssam2 | is automatic resolution something that's actually needed? | 15:12 |
| tristan | So I wonder; what would a world look like with A.) Pre determined conditional statements ... B.) Variants resolved after but still existing ? | 15:13 |
| ssam2 | A) is for things like --enable-debug ? | 15:13 |
| tristan | I.e. our argument for nuking variants is that it isnt flexible enough to handle all cases | 15:13 |
| tristan | but does that mean it should be removed ? | 15:13 |
| ssam2 | i see your point | 15:13 |
| tristan | ssam2, yeah that's the same question I'm asking; what would it look like ? what would I be doing ? | 15:14 |
| ssam2 | we need to collect the various use cases for this together I guess | 15:14 |
| tristan | it *seems* to me that variants for different ways of compiling things or different features enabled, but other conditionals in place for "if os == bsd" or "if level = debug", together might still make sense ? | 15:15 |
| tristan | ssam2, one example I think juergbi raised was... | 15:15 |
| tristan | A.) gtk with wayland, B.) gtk with x11, C.) gtk with wayland and pulseaudio, D.) gtk with wayland and alsa, E.) .... smaller combinatorial explosion ... | 15:16 |
| tristan | ssam2, I'm not hugely concerned with that though; while it does imply some redundant typing; overall you only support the variants you use | 15:17 |
| ssam2 | yeah | 15:17 |
| tristan | ssam2, another thing to consider is; with an alternative to variants; I want to completely get rid of "arch" and host/target arch | 15:17 |
| ssam2 | and you can build GTK+ with both Wayland and X11 backends | 15:17 |
| tristan | Right | 15:17 |
| ssam2 | hmm, yeah would be OK with that | 15:17 |
| ssam2 | the issue with 'arch' is that if you want to do something for all 64-bit arches, for example, you have to manually list each 64-bit arch and copy and paste the same config for each one | 15:18 |
| tristan | So if we gave the user a way A.) to specify what "options" can be set in project.conf ... and B.) A way to configure those options at build time | 15:18 |
| ssam2 | the flip side of this of course... is that shell already has conditionals | 15:19 |
| tristan | We can use that to completely remove arch conditionals | 15:19 |
| tristan | ssam2, so yeah; there can also be other ways of matching variables with conditionals | 15:19 |
| tristan | you could have glob or regex matches but I dont feel like it | 15:20 |
| tristan | 's something to encourage | 15:20 |
| tristan | I.e. "just because your symbolic machine ABI name ends in the characters 6 and 4, does not a 64bit platform make" | 15:21 |
| ssam2 | ha, i agree | 15:21 |
| tristan | Unless you happen to also be the person who named all of your platforms | 15:21 |
| ssam2 | (although this case it's Baserock, so we actually do name them all ourselves) | 15:21 |
| tristan | right | 15:21 |
| ssam2 | it's more that having to list the same config for each arch in turn, rather than being able to say "for this set of arches, do this" | 15:22 |
| tristan | so not sure what | 15:22 |
| tristan | I was thinking something like if/else and switch/case | 15:22 |
| tristan | switch/case/default | 15:22 |
| ssam2 | ok | 15:22 |
| tristan | ssam2, in a list; where conditions are resolved in the sequence described by the element (the order that the element declares conditions) | 15:23 |
| tristan | you have multiple vars now in play, it's important that the element gets to decide which conditions to process first | 15:23 |
| ssam2 | yes that needs to be clear | 15:23 |
| tristan | I also thought of further parsing the yaml | 15:24 |
| tristan | I'm not perfectly sold on the "everything must be parsable and represented in true YAML form" standpoint | 15:25 |
| tristan | so i.e. a conditional could have a single attribute like "ifeq(option_name, value)" | 15:25 |
| tristan | rather than 3 obnoxious lines of YAML | 15:26 |
| ssam2 | yeah I agree on that front; I mean Ansible embeds most of a templating framework and gets away with it | 15:26 |
| ssam2 | my philosophical feeling on this is that once you add conditionals, you don't actually have a fully serialized data structure any more | 15:27 |
| ssam2 | the underlying data structure is actually the combination of all possible values of each of those conditions; and what you are writing is "syntactic sugar" effectively | 15:28 |
| tristan | Well sure | 15:28 |
| ssam2 | in Baserock definitions it really is a serialized data format, there are no conditionals, but of course that brings other issues such as we have 20 different copies of linux.morph | 15:28 |
| tristan | Even when you parse the data structure it still requires input (specification of options) to resolve to anything meaningful | 15:28 |
| tristan | but not sure where that conversation is going :) | 15:28 |
| ssam2 | i'm basically agreeing saying do whatever you want with the YAML syntax :-) | 15:29 |
| tristan | Heh | 15:29 |
| tristan | Ok so I'll try to make a writeup | 15:30 |
| ssam2 | as anyone who wants to see the real data will have to do some extra processing themselves, regardless of how conditionals are specified | 15:30 |
| tristan | Has anyone had the pleasure of trying out the latest feature https://buildstream.gitlab.io/buildstream/completion.html#completion ? | 15:30 |
| tristan | By the way ? | 15:30 |
| tristan | tlater, regarding the cross platform work; my concern with that Platform class is that it will become very branchy and difficult to track what is going on | 15:32 |
| tristan | tlater, adding just one additional sandbox or artifact cache implementation from here will make it more complex | 15:33 |
| tristan | a lot I think | 15:33 |
| tristan | tlater, I also dont like how things are resolve on the fly | 15:34 |
| tristan | like... what happens when I build 500 modules and half way through I install bwrap on my host | 15:34 |
| tristan | I might stop using chroot ? | 15:34 |
| tristan | So my suggestion is to have one Platform *interface* that everything calls into, and to have one check at load time to decide which Platform *implementation* to use | 15:35 |
| tristan | and then the fallback Platform implementation just uses chroot + tarballs, and the regular linux backend uses ostree & bwrap | 15:36 |
| tristan | And if we really want to support chroot + ostree, that would be a different backend | 15:36 |
| tlater | Alright, should this still be decided be what is available on the system or should it just assume linux = ostree + bwrap? | 15:37 |
| tlater | *by what is available | 15:37 |
| tristan | That part can easily be changed; I dont mind starting with a switch on the reported OS and then asserting presence of required tooling for that OS | 15:38 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 3 commits (last: artifactcache.py/tarcache.py: Remove cache get_kind) https://gitlab.com/BuildStream/buildstream/commit/3ee9cd1e6d9cdee08cc8f2ba0ad228646a7061e9 | 16:20 |
| *** xjuan has joined #buildstream | 16:35 | |
| *** jonathanmaw has quit IRC | 17:05 | |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 3 commits (last: Make the sandboxchroot chroot in the same process) https://gitlab.com/BuildStream/buildstream/commit/b8dbd72e54c504121e4aa9dce8a40df442035bbf | 17:08 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 17:08 |
| *** ssam2 has quit IRC | 17:20 | |
| *** tlater has quit IRC | 17:26 | |
| *** tlater has joined #buildstream | 17:29 | |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 17:34 |
| tlater | tristan: Are you still about? | 17:38 |
| tristan | tlater, yep | 18:02 |
| *** tlater` has joined #buildstream | 18:06 | |
| *** tlater has quit IRC | 18:06 | |
| gitlab-br-bot | push on buildstream@compose-exclude (by Tristan Van Berkom): 2 commits (last: element.py: Changed artifact staging apis to have an 'exclude' argument) https://gitlab.com/BuildStream/buildstream/commit/3c88a48daee90818fdd1f5e2199238e106d2aa6c | 18:13 |
| tlater` | tristan: I assume by 'restoring the environment' in your comment on the sandboxchroot you only mean chrooting back to the original root? | 18:15 |
| *** tlater` is now known as tlater | 18:15 | |
| tristan | tlater`, that and the current working directory | 18:16 |
| tristan | tlater, to be honest juergbi's suggestion to use the preexec function sounds nice because you dont have to do any of that | 18:16 |
| tlater | The issue with that is the great red warning in the documentation that says not to use it. | 18:17 |
| tristan | the cwd or root or anything in your envp, if we ever touch any of that in our processes we should revert it quickly after | 18:17 |
| tristan | ybd was using a chdir() context manager but I made sure to use cwd argument in Popen() everywhere so as to avoid any nasty side effects of environment modifying | 18:18 |
| juergbi | i can't say much about python's preexec_fn but in C you'd definitely do it there (between fork and exec) | 18:19 |
| tristan | This one "The preexec_fn parameter is not safe to use in the presence of threads in your application. The child process could deadlock before exec is called. If you must use it, keep it trivial! Minimize the number of libraries you call into." ? | 18:19 |
| tlater | Yeah | 18:19 |
| tristan | tlater, yeah I would ignore that | 18:19 |
| tristan | there are no threads | 18:20 |
| tristan | first off | 18:20 |
| juergbi | another option could be to use the 'chroot' binary instead of os.chroot(), i.e., push 'chroot' and the root dir in front of the argv | 18:20 |
| juergbi | however, i would go with preexec_fn unless you're aware of a specific issue with that | 18:21 |
| tristan | juergbi, I rather prefer this to dealing with escaping the commands for that | 18:21 |
| tlater | juergbi: That was the original implementation, for some reason that didn't work (something about nested 'sh -c's) | 18:21 |
| juergbi | ah, argv handling can be painful, especially with abstraction layers | 18:22 |
| tristan | right | 18:22 |
| juergbi | on the low level this would be trivial as you don't need sh -c outside chroot | 18:22 |
| juergbi | so you could literally just prepend two strings to argv | 18:22 |
| tlater | Unfortunately the plugins have full control over that :/ | 18:22 |
| juergbi | the chroot binary may have restrictions regarding cwd, though | 18:23 |
| tristan | right the sandbox gives you an environment to execute something | 18:23 |
| tristan | you decide to execute sh with it | 18:23 |
| juergbi | the sandbox impl. could still modify argv, couldn't it? | 18:23 |
| juergbi | chroot /sysroot sh -c '...' should be fine | 18:24 |
| juergbi | as i mentioned, i would anyway prefer preexec_fn, though | 18:24 |
| tlater | I'll use preexec_fn then, though I don't think restoring the environment is that terrible if that does run into issues. | 18:25 |
| tristan | preexec_fn is the right thing | 18:25 |
| juergbi | restoring chroot might not work on netbsd, iirc, in case that's relevant | 18:26 |
| tristan | the documentation is just like; exposing an interrupt handler callback to a web JS programmer and saying: please dont start a word processor here, things will likely break | 18:26 |
| tristan | "If you must use it, keep it trivial! Minimize the number of libraries you call into." | 18:27 |
| tristan | Like, really, calling into libraries in a preexec_fn ? | 18:27 |
| juergbi | present a gtk dialog asking whether you really want to exec the process? ;) | 18:28 |
| tristan | What probably happened, is some idiot started calling APIs from a threading library in the preexec_fn; after fork and before exec | 18:28 |
| tristan | right, and then run the dialog in a thread | 18:28 |
| juergbi | the warning is warranted in the sense that it's indeed fragile in case the original process is already multi-threaded | 18:29 |
| tristan | "just let me know after I finish execing that this is what you wanted" | 18:29 |
| juergbi | due to locks | 18:29 |
| tristan | sure | 18:29 |
| tristan | But even then, in C I think it would not really be | 18:29 |
| tristan | I suspect python api level locks | 18:29 |
| juergbi | and some libraries might use background threads without the application developer being aware of it | 18:29 |
| tristan | that is just wrong though | 18:30 |
| juergbi | not so sure about that | 18:30 |
| juergbi | fork with a background thread holding the malloc lock? | 18:30 |
| tristan | I remember a 10+ year open bug on seriously fucked libgnomeui and bonobo libs | 18:30 |
| juergbi | you try to malloc in the child? | 18:30 |
| tristan | they were initializing threads at random thats right | 18:30 |
| tristan | Well anyway, I guess I have my opinions; libraries which spawn threads should certainly make that clear, and thread lifecycles should be tied to lifecycles of things exposed in the API | 18:31 |
| tristan | what libgnomeui was doing is rather orthogonal now that I think of it, that had to do specifically with glib and how we had initialization setup | 18:32 |
| tristan | (was still horrible to see though) | 18:33 |
| tristan | juergbi, good point though I hadnt considered that | 18:35 |
| tristan | forks before exec in threads may even have issues at the C library level | 18:35 |
| tristan | certainly even there chroot is the kind of thing I would expect to be meant to work | 18:36 |
| gitlab-br-bot | push on buildstream@compose-exclude (by Tristan Van Berkom): 1 commit (last: element.py: Fixup last commit) https://gitlab.com/BuildStream/buildstream/commit/4623cc8f7529875038cf212cf3896a551b4cbcda | 18:39 |
| *** tlater has quit IRC | 18:43 | |
| *** tlater has joined #buildstream | 18:45 | |
| juergbi | yes, definitely. chroot is a straight syscall, doesn't require any allocations or any other substantial userspace work (in C) | 18:51 |
| tlater | I'll just avoid opening gtk dialogs for now... | 18:51 |
| juergbi | hehe | 18:52 |
| tlater | tristan: On ignoring an ImportError, that's there to avoid the ostree source, which crashes if ostree is not installed. It's probably not the best way to do that, but I'm not sure how to do that otherwise - unless the platform should also be in charge of loading plugins. | 18:55 |
| tristan | tlater, which comment is that | 18:57 |
| tristan | about what exactly | 18:57 |
| tristan | try / except for importerror is okay, I think there must be something else to that comment | 18:57 |
| tlater | I'm not sure how to link them... | 18:58 |
| tristan | tlater, for instance, I recall one thing similar where; various things had a comment about the import error | 18:58 |
| tlater | Ah | 18:58 |
| tlater | https://gitlab.com/BuildStream/buildstream/merge_requests/81#note_38547984 | 18:58 |
| tristan | and then I found a comment about the import error | 18:58 |
| tristan | where one might have been in the past | 18:58 |
| tlater | Yeah, that's all gone now | 18:58 |
| tristan | but had been removed, but the comment remained, and I commented that that should be gone | 18:58 |
| * tristan opens link | 18:58 | |
| tlater | This is a different comment | 18:58 |
| tristan | comment about set_root_read_only() ? | 18:59 |
| tlater | No, one above that | 18:59 |
| tlater | Just scrolls to the wrong part of the page | 19:00 |
| tristan | Ahhhh | 19:00 |
| tristan | THAT one | 19:00 |
| tlater | Yeah, definitely should have had a comment at least | 19:00 |
| tristan | Yeah that should be solved | 19:00 |
| tristan | yeah should have at least | 19:00 |
| tristan | so how would we solve that ? | 19:00 |
| tlater | It should also depend on the platform | 19:01 |
| tristan | tlater, for now just write a comment that not all plugins succeed to load on every platform and configuration and that we should do something more elaborate around there | 19:01 |
| tristan | I'm fine with not blocking the branch on that detail anyway | 19:01 |
| tlater | Alright | 19:01 |
| tristan | but long term we should save some context about it so we can do better error reporting | 19:01 |
| tristan | I guess, maybe we'll forget if nobody asks for it | 19:02 |
| tlater | I could put a warning message there, though that might scare users | 19:02 |
| tristan | currently it will just bail out with something like "no such source kind `ostree`" | 19:02 |
| tristan | if you try to use one | 19:02 |
| tristan | yeah dont | 19:02 |
| tristan | it would be nice if it were an error, and if we only ever tried to load plugins on demand | 19:04 |
| tristan | but we dont load them on demand and the API doesnt want us to, so lets leave it for now :) | 19:04 |
| tlater | Another # FIXME that will certainly be fixed at some point in the future | 19:06 |
| tlater | certainly | 19:06 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 3 commits (last: Make the sandboxchroot chroot in the same process) https://gitlab.com/BuildStream/buildstream/commit/6ccd10685196306ff1f41e7e48d74a2713c4c44a | 19:10 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 19:10 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 1 commit (last: _plugincontext.py: Add comment on loading ostree source) https://gitlab.com/BuildStream/buildstream/commit/2e46c642fa17ed4d29271ffa85396627bac03b82 | 19:14 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 19:14 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 4 commits (last: Make the sandboxchroot chroot in the same process) https://gitlab.com/BuildStream/buildstream/commit/e7a6cfc21363c7e4af56a3e47e23d6e1a38a0707 | 19:15 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 19:15 |
| tlater | tristan: On this comment: https://gitlab.com/BuildStream/buildstream/merge_requests/81#note_38547427 | 19:19 |
| tlater | I was experimenting with compression types, that can go, I'll just stick to bz2 - the fallback is there in case we can't find a working tar command though, should we just ignore that edge case? | 19:19 |
| tlater | ^ That might happen on an AIX system that doesn't have gtar. | 19:20 |
| tlater | Or has a weird implementation of gtar, I guess | 19:21 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 4 commits (last: Make the sandboxchroot chroot in the same process) https://gitlab.com/BuildStream/buildstream/commit/2db9ad5764ac684a0ba901dafaab151e2d903a57 | 19:35 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 19:36 |
| tristan | tlater, keeping the python tar implementation sounds sensible for that I guess, but yeah please have them agree on the compression :) | 19:41 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 1 commit (last: tarcache.py: Remove compression options) https://gitlab.com/BuildStream/buildstream/commit/d82331362459eece8e37b81b40cad8acb99a7130 | 19:43 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 19:43 |
| gitlab-br-bot | buildstream: issue #79 ("Load plugins on demand") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/79 | 19:52 |
| tlater | tristan: Last one for now I suppose: https://gitlab.com/BuildStream/buildstream/merge_requests/81#note_38548665 | 20:01 |
| tlater | I think I must have misunderstood why sandbox read-onlyness matters to hardlinks, would you mind elaborating? | 20:01 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch compose-exclude | 20:03 |
| gitlab-br-bot | push on buildstream@compose-exclude (by Tristan Van Berkom): 3 commits (last: plugin.py: Added detail parameter to Plugin.timed_activity()) https://gitlab.com/BuildStream/buildstream/commit/ccddb1e827d9f2fe38974d3d331682312679893e | 20:03 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 4 commits (last: Split monolithic _platform.py into multiple platform implementations) https://gitlab.com/BuildStream/buildstream/commit/b284a86422d5d2a2c22a22940b8bdcb1e584b83e | 20:13 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 20:15 |
| tristan | tlater, it matters because if have a read-write location which exposes an artifact, and it's files are hardlinked, writing files corrupts the artifact cache | 20:27 |
| tristan | tlater, so root-read-only-ness is not sufficient; it assumes that artifacts are only staged at the root | 20:28 |
| tristan | but in script elements they are often staged in a separate location, or a debian packaging element might stage an artifact independently | 20:29 |
| tristan | of the base | 20:29 |
| tlater | Ah, alright, I understand | 20:30 |
| tristan | ultimately artifacts appear where elements mark them with Sandbox.mark_directory() | 20:30 |
| tlater | So anything that's staged needs to be read-only? | 20:31 |
| tristan | currently there is a note that says only the root *can* be readonly, but this could be changed if that's better | 20:31 |
| tristan | tlater, no, an element must be allowed to do it's thing if thats what it wants | 20:32 |
| tristan | tlater, instead; if there is no fuse solution; then files must be copied if they are going to a location that is read-write | 20:32 |
| tristan | Then again, it's true that root-read-only-ness, at least; is allowed to change across invocations | 20:33 |
| tristan | hmmm | 20:33 |
| tristan | tlater, so what is it that we said about running integration commands ? | 20:34 |
| * tristan thinks he has a photo of the whiteboard we had | 20:34 | |
| tlater | They are only run when the root is read-only, but obviously that doesn't work | 20:34 |
| tristan | of course not | 20:34 |
| laurenceurhegyi | I have that photo if you need it tristan and tlater... | 20:35 |
| tristan | I have it here; I dont know if you've seen this one laurenceurhegyi | 20:36 |
| tlater | I have a copy too | 20:36 |
| tristan | "If an element is going to run s-i, then the sandbox must be staged with copies" (must be able to know if s-i commands need to run) | 20:36 |
| tristan | BuildElement needs a "no-integrate" we had a point on | 20:37 |
| gitlab-br-bot | push on buildstream@cross_platform (by Tristan Maat): 2 commits (last: tarcache.py: Remove compression options) https://gitlab.com/BuildStream/buildstream/commit/8b238ef39d6e4ae88d0767ee239cf9276d8cdca5 | 20:37 |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: WIP: Cross platform) #81 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 20:37 |
| tristan | I can see it right there, "Change semantics of "root-read-only" for the lifetime of a sandbox | 20:38 |
| tristan | " | 20:38 |
| tristan | But I dont know what we were thinking :-/ | 20:39 |
| tristan | How does that work ? | 20:39 |
| tristan | Oh | 20:39 |
| tlater | It might have been Friday afternoon after a tough week mistakes | 20:39 |
| tristan | It means that on linux we have to live with read-write root | 20:39 |
| tristan | at build time | 20:39 |
| tristan | Which I guess is livable, but undesirable because it raises no errors in the case that accidentally happens | 20:40 |
| tlater | Why do we have to live with a read-write root again? | 20:41 |
| tristan | Well, we need to run integration commands right ? | 20:41 |
| tristan | So we need read-write root for that | 20:41 |
| tlater | Unless we have no-integrate | 20:41 |
| tristan | Normally we discard read-write root after that point | 20:41 |
| tristan | Right, but what about the normal case :) | 20:42 |
| tristan | Hmmm, I very much dont like that people will need to use things differently to optimize for their platform | 20:42 |
| tristan | i.e. no-integrate and inserting integration rounds as compose elements or such as needed throughout the stack | 20:43 |
| tristan | if you consider a full system | 20:43 |
| tristan | Also we might need to cheat with ldconfig | 20:44 |
| tlater | There's no real way around that though, you'll always have to copy on AIX/solaris and take the hit or manage integration yourself. | 20:44 |
| tristan | it could be reduced splitting things up; for instance `rm /etc/ld.so.cache && ldconfig` is safe to run | 20:46 |
| tristan | but of course its not safe cause you cant error out if the user makes a mistake | 20:46 |
| tristan | so scratch that | 20:46 |
| tristan | (although we will probably *want* a special case like that for ldconfig) | 20:47 |
| tristan | tlater, anyway so no-integrate is something a user can use to turn off integration right ? | 20:48 |
| tristan | Alright well, it's a wart | 20:48 |
| tristan | tlater, so in this case is there no way that the platform/sandbox can handle running root-read-only on a per invocation instead of per sandbox basis anymore ? | 20:50 |
| tlater | I'm not entirely sure when individual invocations happen right now | 20:53 |
| tristan | I mean when the sandbox is run | 20:53 |
| tristan | a command or something | 20:54 |
| tristan | each time the root could be readwrite or readonly, now it has to be readwrite or readonly all the time for the sake of other platforms | 20:54 |
| tristan | can we do it in a way that that doesnt happen ? | 20:55 |
| tlater | I don't see why we couldn't | 20:56 |
| tlater | W | 20:56 |
| tlater | hat was the original reason we wanted it to be read-only? | 20:56 |
| tlater | Sorry that I'm being slow right now, it's been a while... | 20:57 |
| tlater | But as far as I understand we'd like to avoid individual invocations touching other artifacts staged into the sandbox | 20:58 |
| tlater | No, I'm lost -.- | 20:59 |
| *** xjuan has quit IRC | 21:00 | |
| *** tlater has quit IRC | 21:21 | |
| gitlab-br-bot | buildstream: issue #73 ("Open workspace should be able to check out new branch of git source") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/73 | 21:39 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 3 commits (last: plugin.py: Added detail parameter to Plugin.timed_activity()) https://gitlab.com/BuildStream/buildstream/commit/ccddb1e827d9f2fe38974d3d331682312679893e | 22:05 |
| gitlab-br-bot | buildstream: issue #78 ("Domain inclusion seem to not work") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/78 | 22:05 |
| *** tristan has quit IRC | 22:36 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!