| *** adds68 has joined #buildstream | 00:40 | |
| *** alatiera has quit IRC | 01:40 | |
| *** kapil___ has joined #buildstream | 02:46 | |
| *** nimish has quit IRC | 03:20 | |
| *** lantw44 has quit IRC | 07:01 | |
| *** lantw44 has joined #buildstream | 07:05 | |
| *** xjuan has joined #buildstream | 07:35 | |
| *** tristan has joined #buildstream | 07:57 | |
| *** xjuan has quit IRC | 08:28 | |
| *** xjuan has joined #buildstream | 08:29 | |
| *** xjuan has quit IRC | 08:38 | |
| nielsdg | I don't know if this is the correct place to put this, but I just had a very weird bug: if I open a bst shell for a GTK+ application on a system that doesn't have icon-hicolor-theme installed, I get some really weird stuff (no characters being drawn/no icons found) when I run the application inside that shell | 08:55 |
|---|---|---|
| nielsdg | So I don't know if it means GNOME should add a dependency for icon-hicolor-theme in its dependencies for any GTK+ app | 08:56 |
| *** ChanServ sets mode: +o tristan | 09:03 | |
| tristan | nielsdg, It is surprising to me that the host system could have any effect on this | 09:03 |
| tristan | nielsdg, If I run an app in a shell and the shell doesnt have the icon theme, then I equally get missing icons even if my host system has them | 09:04 |
| nielsdg | tristan: it seems to be something else though | 09:04 |
| tristan | nielsdg, Here is an example of what I did to make glade work in the shell: https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/master/elements/world/glade.bst | 09:05 |
| juergbi | is D-Bus passed through and the theme is taken from there somewhere? | 09:05 |
| tristan | i.e. cantarell fonts and adwaita makes it work | 09:05 |
| nielsdg | tristan: Indeed, that might be it | 09:05 |
| tristan | juergbi, hmmm have to look at the project.conf to see what it does now, but I doubt it | 09:06 |
| juergbi | was just a wild guess | 09:06 |
| tristan | We poke a hole for the pulseaudio socket and for the dri | 09:06 |
| tristan | I mean it is possible :) | 09:06 |
| juergbi | DBUS_SESSION_BUS_ADDRESS: '$DBUS_SESSION_BUS_ADDRESS' | 09:06 |
| tristan | right there :) | 09:06 |
| * tristan forgets what precisely that was done for | 09:07 | |
| tristan | but surely it helps | 09:07 |
| juergbi | the theme issue might indeed just be due to the dependency | 09:09 |
| *** alatiera has joined #buildstream | 09:16 | |
| benschubert | Can soemone tell me where the yaml file for plugins is loaded? I couldn't find it :/ | 09:23 |
| tristan | benschubert, it's only for elements | 09:23 |
| tristan | benschubert, it should be parsed in element.py | 09:24 |
| benschubert | ok thanks! I'll look in there | 09:24 |
| tristan | __init_defaults() | 09:24 |
| benschubert | tristan: thanks a lot! | 09:27 |
| *** jennis has joined #buildstream | 09:28 | |
| tristan | juergbi, now would be a good time to talk about symlinks :) | 09:32 |
| juergbi | :) | 09:32 |
| juergbi | have you already had a chance to read the description/comments of !1140 (and possibly !1139)? | 09:32 |
| gitlab-br-bot | MR !1140: WIP: Do not resolve or mangle symlinks during staging https://gitlab.com/BuildStream/buildstream/merge_requests/1140 | 09:32 |
| gitlab-br-bot | MR !1139: Return all directories in list_relative_paths() https://gitlab.com/BuildStream/buildstream/merge_requests/1139 | 09:33 |
| tristan | Somehow I think we're rehashing something that has been discussed so many times before | 09:33 |
| tristan | ok lemme freshen up on those for a sec... | 09:33 |
| juergbi | there have been various issues in connection with symlinks over time, so there was certainly some discussion | 09:36 |
| nielsdg | tristan: Thanks, adding those as runtime dependencies solved it :) | 09:36 |
| juergbi | I've linked a few of them | 09:36 |
| tristan | nielsdg, :) | 09:39 |
| jjardon | nielsdg: please remember to send a patch if this is using gnome-build-meta :) | 09:41 |
| nielsdg | jjardon: Already planned, after work :-) | 09:41 |
| jjardon | tristan: hey, any chance to have a new bst-1.2.x release soonish? | 09:42 |
| tristan | jjardon, lets do it; I have a dinner at 8pm so I don't think I can do it today | 09:42 |
| jjardon | Sure, let me know if you need any help releasing the tarball in gnome servers | 09:43 |
| tristan | jjardon, Ok I'll put on my TODO 2 things for tomorrow morning, both release the next 1.2.x, and add a section to CONTRIBUTING.rst about the release process | 09:43 |
| tristan | jjardon, you, juergbi and I all have that access so shouldnt be an issue :) | 09:44 |
| *** raoul has joined #buildstream | 09:45 | |
| juergbi | tristan: I actually already removed the unused (and buggy) list_dirs=False support, !1138, hope you don't mind | 09:48 |
| gitlab-br-bot | MR !1138: Symlink fixes https://gitlab.com/BuildStream/buildstream/merge_requests/1138 | 09:48 |
| tristan | juergbi, Ok so put some comments down on the both... | 09:55 |
| tristan | juergbi, nah, since it was unused it makes sense to remove deadcode | 09:56 |
| tristan | juergbi, Ok so, I think that the majority of problems here are stemming from lack of absolute symlink support and more importantly, the assumption that it might be okay to apply split rules after initially staging an artifact | 09:57 |
| tristan | split rules are parsed on the element and supported in the staging routines explicitly to stage portions of the artifact, but we are failing because we are expecting that those locations are unaffected by staging | 09:57 |
| tristan | juergbi, Anyway, for all of those comments, it doesnt necessarily mean we should not change it of course | 09:59 |
| tristan | From a coding perspective, it would be nice to treat symlinks like anything else | 10:00 |
| tristan | From a user perspective, it would be nice to continue to use a %{prefix} of "/opt/foo" in all my elements, even if I later decide that /opt/foo is a symlink to /usr/local/foo | 10:01 |
| tristan | Maybe that is unnecessary | 10:02 |
| tristan | I'm pretty sure I've seen this setup used before, on rpm/mic based firmwares (where the software is compiled into prefix /opt/vendor, but the data for /opt/vendor is stored in a different real path) | 10:03 |
| juergbi | tristan: regarding split rules, that might indeed make sense but current compose.py doesn't follow this approach | 10:04 |
| juergbi | right now compose stages all dependencies without splitting, then runs the integration commands and computes the split manifest after integration | 10:04 |
| tristan | Right, that definitely appears to be a grave mistake which went unnoticed until people started raising issues | 10:05 |
| juergbi | I was also thinking that it would be nice if we could simply stage respecting the split domains and then drop the more complex manifest calculation | 10:05 |
| tristan | juergbi, Only if integration commands are supposed to be run | 10:05 |
| juergbi | well, yes | 10:05 |
| tristan | Otherwise it should only stage the selected data, it is specifically for that reason too | 10:05 |
| tristan | cause we can't expect integration to work on a subset | 10:05 |
| juergbi | yes, that's a potential concern | 10:06 |
| juergbi | regarding /opt/foo being a symlink: I think such system installations should indeed be valid, however, I don't think it's a concern inside BuildStream sandboxes | 10:07 |
| juergbi | or rather, that will still work. e.g., freedesktop-sdk replaces /usr/shares/locale with a symlink to the localization extension point for flatpak images, or something along those lines | 10:07 |
| juergbi | and that seems to still work fine with my branch | 10:08 |
| juergbi | as moving the directory and replacing it with a symlink is done in an integration command | 10:08 |
| tristan | Well | 10:08 |
| tristan | You can always depend on an element which runs an integration command which does that kind of thing | 10:08 |
| juergbi | exactly, that's what fd-sdk does and that will continue to work in any case | 10:09 |
| tristan | But, the philosophy behind the current approach is that we should avoid as much as possible any fragmentation of knowledge | 10:09 |
| tristan | Especially, we should avoid any kind of master post processing hacky script which "runs at the end" where all the confusing unexplainable special sauce ends up | 10:09 |
| tristan | because not everybody comments perfectly about everything | 10:10 |
| *** bochecha has joined #buildstream | 10:10 | |
| tristan | This symlink seems most appropriately done by the element which installs the libc locales thing right ? | 10:10 |
| juergbi | well, the thing is that they only want the symlink for the flatpak image | 10:11 |
| juergbi | more regular system images wnat regular /usr/share/locales | 10:11 |
| juergbi | that's why it makes sense as an integration command there | 10:11 |
| tristan | That is what project options are for | 10:11 |
| juergbi | would also be a possibility but reduce build reuse | 10:11 |
| tristan | And no, I mean regardless of the the ultimate decision, I definitely do not think it makes more sense to have an element at the end "do stuff" just because we are building in flatpak mode | 10:12 |
| juergbi | I think it really depends on what 'stuff' is being done | 10:13 |
| juergbi | special treatment for flatpak extension points is acceptable in dedicated integration packages, imo | 10:14 |
| juergbi | that's a bit like a deployment process | 10:14 |
| juergbi | or rather, that is deployment. not just a bit like | 10:15 |
| tristan | I don't really agree with treating one filesystem permutation as special or different from another | 10:16 |
| juergbi | it's not special | 10:16 |
| juergbi | but doing that at the end is not bad per se, imo | 10:17 |
| juergbi | you first build stuff and then you deploy the thing | 10:17 |
| juergbi | (or prepare deployment image, whatever you want to call it)\ | 10:17 |
| tristan | So, if you were "only ever building a flatpak runtime", you could just build glibc locales in the right place to begin with, and never have to understand it in some "do all hack and patch" element at the end | 10:18 |
| tristan | But since we can build many different things, we instead build one way all the way up to a point, and start mashing up the output later on for reverse dependency branches which target different things | 10:19 |
| tristan | it feels disgusting | 10:19 |
| juergbi | it could be compared to someone wanting to install locales on a different filesystem | 10:19 |
| juergbi | maybe not something that useful for standard installations but who knows, root partition could be size limited | 10:19 |
| tristan | I mean, I think you probably have convinced me based on difficulty to order your dependencies correctly to produce the desired results, and weak messaging to the user to understand the consequences | 10:20 |
| tristan | But I wont get behind the post processing of things at the end being a good/desirable thing | 10:20 |
| juergbi | I know where you're coming from, with some projects having huge badly maintained scripts of fixing up permissions etc | 10:21 |
| tristan | Of course the option always remains open to do so, but being able to abolish these things by providing an experience that is as lfs and seamless as possible is a goal | 10:22 |
| tpollard | in l579 of cli.py we check 'not _cached_success' to state we're mounting in a failed build, however if I switch that to '_cached_failure' it doesn't catch | 10:22 |
| tpollard | now it is cached, else cached_success would return false straight away, so I'm not sure what the issue is | 10:22 |
| * tpollard assumes something is astray in _get_build_result | 10:25 | |
| tristan | tpollard, sounds like anyway | 10:29 |
| *** jonathanmaw has joined #buildstream | 10:29 | |
| tpollard | yep, I wonder why not _cached_success was used in the first place tbh | 10:30 |
| tristan | tpollard, oh btw, I send in a proposal for a paper at FOSSASIA, and considering going there anyway, if I recall you were interested in going | 10:30 |
| tpollard | tristan: I am indeed hoping to go :) | 10:30 |
| tristan | I might go anyway even if she can't squeeze me in | 10:30 |
| tristan | but I'll let you know if I get a talk there it would be cool :) | 10:31 |
| tristan | tpollard, git blame, I think you might ask skullman (?) about caching failed builds | 10:31 |
| tristan | juergbi, Ok so anyway I do get the complication of symlinks and we've been struggling with them since the beginning | 10:32 |
| tristan | juergbi, it seems that it might be the right tradeoff | 10:33 |
| tristan | juergbi, if I understand, your branch will flatly disallow staging of files across existing symlink boundaries ? | 10:33 |
| tristan | I do think it is a shortcoming, but maybe we can safely say that there is no way to allow that while giving the user appropriate messaging when they make a mistake | 10:34 |
| juergbi | yes, if any parent path component from the file list is currently a non-directory, it will bail out | 10:34 |
| juergbi | (same for a directory in the file list) | 10:34 |
| tristan | juergbi, alright then :) | 10:37 |
| juergbi | :) | 10:38 |
| tristan | juergbi, On a similar topic, especially with everything that has been discussed at the gathering... I think it is paramount that we start on a porting guide immediately | 10:38 |
| juergbi | you're also happy with the list_relative_paths() change or is there anything else I can check in that aspect? | 10:38 |
| tristan | And that we carefully update the porting guide with every breaking change | 10:38 |
| juergbi | makes sense | 10:39 |
| tristan | juergbi, honestly regarding the list_relative_paths() API, I am happy if tests pass and existing projects build | 10:39 |
| juergbi | the list in NEWS is hopefully complete but at least some points will require explanation how to port | 10:39 |
| juergbi | ok, great | 10:39 |
| tristan | juergbi, And I am happy that is remains the *single* focal point for traversing directories | 10:40 |
| tristan | that is sort of the point of list_relative_paths(), ensuring that nobody is calling os.walk() by themselves | 10:40 |
| juergbi | CASBasedDirectory has its own, of course, but that matches the behavior now | 10:40 |
| tristan | I suppose we will always need both, even if elements are only allowed to access sandbox paths through CASBasedDirectory | 10:41 |
| tristan | juergbi, but maybe list_relative_paths() belongs on FileSystemBasedDirectory ? | 10:41 |
| tristan | Anyway, that is rather besides the point | 10:42 |
| tristan | I recall pulling my hair out when YBD had two different functions for traversing filesystems | 10:42 |
| tristan | and would like to keep the rest of my hair :D | 10:42 |
| *** tristan has quit IRC | 10:43 | |
| *** tristan has joined #buildstream | 10:43 | |
| juergbi | FileBasedDirectory simply calls utils.list_relative_paths() right now. we could consider removing the utils filesystem functions from the public API and only supporting the 'virtual' Directory API | 10:43 |
| *** ChanServ sets mode: +o tristan | 10:43 | |
| juergbi | I'm anyway planning on working towards moving everything to our Directory API starting next week, such that we can have the optimal pure CAS path in the future, where we can avoid real filesystem access for staging (for remote execution or local buildbox) | 10:44 |
| tristan | Right | 10:44 |
| juergbi | but one step at a time :) | 10:44 |
| * tristan has to run to dinner... | 10:45 | |
| juergbi | enjoy, I may start merging, then | 10:45 |
| gitlab-br-bot | jennis merged MR !1101 (jennis/refactor_artifact_log->master: Refactor artifact log command) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1101 | 10:48 |
| *** tristan has quit IRC | 10:48 | |
| *** lachlan has joined #buildstream | 10:50 | |
| jmac | Is there a list of built-in element types with a summary anywhere? | 11:15 |
| benschubert | jmac: ls plugins/elements | grep .py? | 11:18 |
| benschubert | buildstream/plugins/elements | 11:18 |
| juergbi | jmac: and there is https://docs.buildstream.build/core_plugins.html | 11:19 |
| jmac | Yeah, I just thought it'd be useful to have a list with a summary in it | 11:20 |
| jmac | Also it's confusing to refer to them as 'plugins' when they come supplied with Buildstream | 11:21 |
| juergbi | in a way, yes | 11:21 |
| juergbi | most of them are planned to move out of the core repository, though | 11:21 |
| *** lachlan has quit IRC | 11:36 | |
| *** lachlan has joined #buildstream | 11:40 | |
| gitlab-br-bot | valentindavid merged MR !1132 (valentindavid/pull-chmod-bug-1.2->bst-1.2: buildstream/_artifactcache/cascache.py: Set 0644 rights to pulled files) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1132 | 11:41 |
| *** bochecha has quit IRC | 11:46 | |
| *** lachlan has quit IRC | 11:47 | |
| gitlab-br-bot | jonathanmaw opened issue #911 (Buildstream's scheduler spends a long time creating subprocesses) on buildstream https://gitlab.com/BuildStream/buildstream/issues/911 | 11:52 |
| juergbi | jonathanmaw: was this on WSL or Linux? | 11:53 |
| jonathanmaw | juergbi: the profile I used was on Linux | 11:55 |
| juergbi | oh, that's quite surprising to me | 11:55 |
| jonathanmaw | on my WSL run, it took much longer, but was a dramatically smaller fraction of total time | 11:55 |
| juergbi | ok, I guess I/O is the main issue on WSL | 11:55 |
| jonathanmaw | on that one, definitely, it was a HDD over USB | 11:56 |
| jonathanmaw | though linux copes with that much latency much better than WSL does, too | 11:56 |
| juergbi | Linux's aggressive dentry caching might make a big difference | 11:57 |
| * juergbi wonders why tests-wsl is running for my MR even though it's set to manual | 11:59 | |
| jonathanmaw | that was my fault | 12:00 |
| jonathanmaw | I started it manually so I could check that the new WSL runner I set up was working | 12:00 |
| juergbi | ah ok, no problem | 12:02 |
| *** lachlan has joined #buildstream | 12:02 | |
| *** solid_black has joined #buildstream | 12:22 | |
| *** eadnjunior has joined #buildstream | 12:23 | |
| *** raoul has quit IRC | 12:34 | |
| gitlab-br-bot | juergbi merged MR !1139 (juerg/list-all-directories->master: Return all directories in list_relative_paths()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1139 | 12:42 |
| gitlab-br-bot | juergbi opened (was WIP) MR !1140 (juerg/symlinks2->master: Do not resolve or mangle symlinks during staging) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1140 | 12:46 |
| *** nimish has joined #buildstream | 12:54 | |
| *** nimish has quit IRC | 12:58 | |
| *** nimish has joined #buildstream | 12:59 | |
| *** nimish has quit IRC | 13:03 | |
| *** nimish has joined #buildstream | 13:06 | |
| *** solid_black has quit IRC | 13:07 | |
| *** nimish has quit IRC | 13:14 | |
| *** nimish has joined #buildstream | 13:14 | |
| juergbi | if anyone wants to review !1140 before it's merged, please do so by tomorrow noon | 13:19 |
| *** raoul has joined #buildstream | 13:30 | |
| tpollard | juergbi: I've changed it to the 'tristate', hopefully it's in an agreeable format now :) https://gitlab.com/BuildStream/buildstream/merge_requests/1135/ | 13:36 |
| juergbi | great, thanks, will take a look | 13:37 |
| tpollard | as per usual I've lost the CI/rebase race | 13:37 |
| * tpollard rebases | 13:37 | |
| gitlab-br-bot | BenjaminSchubert opened MR !1147 (bschubert/cleanup-local-state->master: Cleanup MetaElement local state) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1147 | 13:48 |
| *** lachlan has quit IRC | 14:47 | |
| gitlab-br-bot | juergbi closed issue #896 (Optional creation of buildtrees) on buildstream https://gitlab.com/BuildStream/buildstream/issues/896 | 14:50 |
| gitlab-br-bot | juergbi merged MR !1135 (tpollard/896->master: Optional creation of buildtrees) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1135 | 14:50 |
| tpollard | woop | 14:51 |
| gitlab-br-bot | phildawson approved MR !1147 (bschubert/cleanup-local-state->master: Cleanup MetaElement local state) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1147 | 14:59 |
| *** kapil___ has quit IRC | 15:06 | |
| *** lachlan has joined #buildstream | 15:19 | |
| jmac | I'm trying to use the docker source plugin on OSX, and it's creating a cache directory which has 555 permissions, which it then fails to delete | 15:33 |
| jmac | As per https://pastebin.com/WeWKiTVU | 15:33 |
| *** lachlan has quit IRC | 15:35 | |
| *** lachlan has joined #buildstream | 15:52 | |
| gitlab-br-bot | cs-shadow opened MR !1148 (chandan/dot-graph->master: contrib/bst-graph: Add script to print graph in DOT format) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1148 | 16:12 |
| gitlab-br-bot | BenjaminSchubert merged MR !1147 (bschubert/cleanup-local-state->master: Cleanup MetaElement local state) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1147 | 16:13 |
| *** lachlan has quit IRC | 17:15 | |
| *** ipatch has left #buildstream | 17:21 | |
| jmac | ^ tarfile apparently. The tar contains permissions for '.'. | 17:22 |
| *** lachlan has joined #buildstream | 17:32 | |
| *** raoul has quit IRC | 17:32 | |
| *** nimish has quit IRC | 17:33 | |
| gitlab-br-bot | jmacarthur opened issue #912 (Tar file extract can break our temporary directories) on buildstream https://gitlab.com/BuildStream/buildstream/issues/912 | 17:37 |
| *** tpollard has quit IRC | 17:48 | |
| *** slaf has quit IRC | 18:04 | |
| *** lachlan has quit IRC | 18:24 | |
| *** lachlan has joined #buildstream | 18:40 | |
| *** nimish has joined #buildstream | 18:49 | |
| *** jonathanmaw has quit IRC | 18:56 | |
| *** lachlan has quit IRC | 19:05 | |
| *** slaf has joined #buildstream | 19:10 | |
| *** nimish has quit IRC | 20:27 | |
| *** alatiera has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!