| *** benschubert has quit IRC | 00:58 | |
| *** nimish has joined #buildstream | 02:02 | |
| *** tristan has joined #buildstream | 02:52 | |
| *** nimish has quit IRC | 02:52 | |
| *** xjuan has quit IRC | 02:53 | |
| *** xjuan has joined #buildstream | 02:54 | |
| *** xjuan has quit IRC | 03:46 | |
| *** tristan has quit IRC | 05:33 | |
| doras[m] | What is the meaning of "(>)" in split rules? | 08:50 |
|---|---|---|
| *** toscalix has joined #buildstream | 09:00 | |
| juergbi | doras[m]: that appends to the list of split rules, see https://docs.buildstream.build/format_intro.html#list-append | 09:02 |
| doras[m] | Thanks, juergbi. | 09:06 |
| *** sambishop has joined #buildstream | 09:46 | |
| gitlab-br-bot | jennis closed issue #879 (Hidden commands are still shown within our documentation) on buildstream https://gitlab.com/BuildStream/buildstream/issues/879 | 09:52 |
| *** benschubert has joined #buildstream | 09:53 | |
| benschubert | tristan: that seems good to me! I will update in the next day our internal setup with this and let you know :) | 09:53 |
| benschubert | I mean, let you know if other steps would be necessary | 09:54 |
| *** raoul has joined #buildstream | 09:58 | |
| *** solid_black has joined #buildstream | 10:13 | |
| *** jonathanmaw has joined #buildstream | 10:18 | |
| doras[m] | I don't get "filter". It includes unrelated stuff that appear to come from my filtered element's dependecies. | 10:18 |
| jennis | doras[m], was this when you did a checkout? | 10:23 |
| gitlab-br-bot | phildawson opened (was WIP) MR !1057 (phil/848-plugin-deprecation-warnings->master: plugin.py: Add API to allow plugins to raise deprecation warnings) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1057 | 10:23 |
| jennis | by default, checkout will include runtime dependencies, you'll need to specify `--deps none`, if you want to see exactly what that filter element's artifact contains | 10:24 |
| *** bochecha has joined #buildstream | 10:27 | |
| doras[m] | I see. Now it's... empty. | 10:28 |
| *** lachlan has joined #buildstream | 10:29 | |
| doras[m] | Strangely enough, '--deps none' always gives me empty directories. | 10:31 |
| doras[m] | ... even for normal elements. I'm using buildstream version 1.2.3, if it matters. | 10:34 |
| *** sambishop has quit IRC | 10:36 | |
| *** lachlan has quit IRC | 10:36 | |
| doras[m] | Looks like 1.2.3 doesn't include this: https://gitlab.com/BuildStream/buildstream/commit/f89a8ab97bcd8431243624ecc9f1ac227e8559d7 | 10:45 |
| doras[m] | I'll try building the tip of 1.2 myself and see if it works. | 10:47 |
| *** lachlan has joined #buildstream | 10:47 | |
| *** sambishop has joined #buildstream | 10:52 | |
| *** lachlan has quit IRC | 10:56 | |
| tpollard | I'm trying to build builstream via buildstream-integration | 10:56 |
| tpollard | I have bst-external installed, but keep getting 'Failed to load Source plugin 'git_tag': The 'buildstream-external' distribution was not found and is required by the application' | 10:56 |
| tpollard | it's coming up under pip | 10:57 |
| doras[m] | I had to use pip3 specifically | 10:58 |
| tpollard | doras[m]: ahh, that gem | 11:00 |
| * tpollard can't wait for 2.7 to die | 11:00 | |
| tpollard | ty | 11:00 |
| doras[m] | ;) | 11:00 |
| *** sambishop has quit IRC | 11:02 | |
| *** sambishop has joined #buildstream | 11:08 | |
| *** lachlan has joined #buildstream | 11:12 | |
| *** lachlan has quit IRC | 11:19 | |
| raoul | related to the local cache discussion earlier, looking at my MR !1100 and talking to juergbi yesterday, it looks like moving the cache quota functions from the artifact cache into the CAS cache might be a good move. | 11:22 |
| gitlab-br-bot | MR !1100: root cache directory https://gitlab.com/BuildStream/buildstream/merge_requests/1100 | 11:22 |
| raoul | it doesn't solve above issues, and wont take sources into account (or only staged ones with the source cache), but I think it's a sensible plan of action | 11:23 |
| raoul | I imagine tristan has thoughts on this but he doesn't seem to be around | 11:23 |
| *** rdale has quit IRC | 11:28 | |
| *** lachlan has joined #buildstream | 11:36 | |
| *** lantw44_ has joined #buildstream | 11:42 | |
| *** lantw44_ has quit IRC | 11:43 | |
| *** lantw44_ has joined #buildstream | 11:49 | |
| *** lantw44_ has quit IRC | 11:49 | |
| *** rdale has joined #buildstream | 11:50 | |
| *** lantw44_ has joined #buildstream | 11:53 | |
| jennis | doras[m], I thought that fix was backported, did you manage to get this working? | 11:54 |
| jennis | mhmm, backported but not released, so it seems? | 11:57 |
| doras[m] | Backported but not released, yes. | 11:58 |
| doras[m] | jennis, it does seem to work with the tip of bst-1.2. | 12:00 |
| doras[m] | However, my filter element still includes its dependencies with '--deps none' | 12:00 |
| *** brlogger has joined #buildstream | 12:02 | |
| jennis | To be clear, your filter element build-depends on some other element, this element depends on another element, and you're seeing things that you know to belong to the artifact of the 'other' element within the filter element's artifact? | 12:05 |
| doras[m] | Yes. The "know to belong" part isn't certain. I do see unrelated build products. | 12:08 |
| jennis | and if you checkout (--deps none) the element which your filter build depends on, these files are not present? | 12:09 |
| doras[m] | Yes. | 12:11 |
| jennis | ahh :/ jonathanmaw may have more of an idea regarding this, definitely sounds like a bug to me | 12:12 |
| WSalmon | jennis, phildawson i think i have resolved all the issues on https://gitlab.com/BuildStream/benchmarks/merge_requests/12 if you would like to take another look :D | 12:12 |
| doras[m] | jennis, jonathanmaw, I created an "empty" filter element which only build-depends on the other element without setting anything in "include" or "exclude" (I basically commented out the entire "config" part). Checking out this "empty" filter with '--deps none' creates a folder containing 1.2GB of files (basically its entire runtime?). When I check out the element my "empty" filter build-depends on with '--deps none', I get 57 MB of files | 12:25 |
| doras[m] | only containing that element's files. | 12:25 |
| jennis | right, because I've just tried to replicate this issue with an include, and I've not received any of the dependencies artifacts in the filter's artifact | 12:25 |
| doras[m] | jennis, which version of buildstream have you used? | 12:26 |
| jennis | bst-1.2 | 12:27 |
| doras[m] | Very odd. I guess there's another variable here. | 12:27 |
| jennis | https://paste.gnome.org/pzgptn3xc | 12:27 |
| doras[m] | Hold on, I'll try this too. | 12:28 |
| jennis | So I've basically taken a project from our examples there (tests/element/filter/basic/) and added in a 'base.bst' element which include.bst depends on | 12:29 |
| *** alatiera has joined #buildstream | 12:32 | |
| jennis | doras[m], now, with an empty includes, I'm only checking out artifacts which are declared in the split rules (in this case, just foo and bar) | 12:34 |
| doras[m] | Exact same experience with your example, jennis. I'll try to modify it to be more similar to my setup and see if I can reproduce the issue. | 12:40 |
| jjardon | Can I have a review of https://gitlab.com/BuildStream/website/merge_requests/109 , please? | 12:40 |
| jennis | doras[m], sure ok. Thanks! | 12:41 |
| tpollard | jjardon: I'd approve, but I don't have the ability | 12:42 |
| *** lachlan has quit IRC | 12:46 | |
| *** lantw44 has quit IRC | 12:49 | |
| *** lantw44 has joined #buildstream | 12:50 | |
| *** raoul has quit IRC | 12:50 | |
| doras[m] | jennis, I think I managed to reproduce it. https://pastebin.com/raw/iJatrAsr | 13:04 |
| doras[m] | The trick is giving same name to the split rule in base.bst and input.bst. | 13:05 |
| doras[m] | In this case, I used "foo". | 13:06 |
| doras[m] | The issue: "base-system" is included in the filter element but not in the element that it filters. In other words, the filter is not a subset of its "parent" like it's supposed to be. | 13:07 |
| doras[m] | Not sure if it means anything, but I also see the same result in ~/.cache/buildstream/artifacts/extract/bst-filter-test/output-include/<hash>/files | 13:22 |
| juergbi | doras[m]: runtime dependencies of the filtered element are indeed included | 14:00 |
| juergbi | you could change the base.bst dependency in input.bst to 'type: build' to avoid that | 14:01 |
| juergbi | would that work for you? | 14:01 |
| doras[m] | juergbi, but that makes a filter element not a subset of the element it filters. Doesn't it effectively turns the filtered element's runtime dependencies into a "build product" of the element? That makes no sense. | 14:11 |
| doras[m] | turn* | 14:11 |
| *** nimish has joined #buildstream | 14:14 | |
| doras[m] | My use case is filtering systemd to libsystemd like every distribution does to solve a cyclic dependency with dbus in the freedesktop sdk. I don't want to package all of systemd's runtime dependencies in libsystemd, and I also can't change systemd's runtime dependencies since it needs them to, well, run. | 14:16 |
| juergbi | doras[m]: won't you use a second filter element for the rest of systemd? | 14:19 |
| juergbi | i.e., one that excludes libsystemd | 14:20 |
| juergbi | in that case you could specify the other runtime dependencies there | 14:20 |
| juergbi | that said, I'm not sure whether there is a use case for the current implementation that includes the runtime dependencies. don't remember any particular discussion about it | 14:21 |
| doras[m] | I could split it and I probably will, but unless I missed something, that doesn't change the fact that libsystemd would *be* "systemd plus all of its runtime dependencies", which in this case is the most of the freedesktop runtime. | 14:24 |
| juergbi | ah, using unmodified systemd.bst from fdo-sdk, yes, that wouldn't work | 14:26 |
| *** lachlan has joined #buildstream | 14:27 | |
| doras[m] | A filter could, perhaps, inherit the runtime dependencies of its filtered element and make them its own runtime dependencies (probably only if it runtime-depends on its filterted element), but it really shouldn't consume them unconditionally. | 14:29 |
| *** raoul has joined #buildstream | 14:29 | |
| juergbi | jonathanmaw: your proposal for the Filter element stated 'Only stages the direct build-dependency, skipping indirect dependencies' https://gitlab.com/BuildStream/buildstream/issues/214#note_59947315 | 14:32 |
| juergbi | however, the actual implementation includes indirect dependencies (runtime dependencies of the build dependency) | 14:32 |
| juergbi | was this necessary for something or might this have been accidental? | 14:32 |
| juergbi | see discussion above with doras[m] | 14:32 |
| juergbi | `self.dependencies(Scope.BUILD)` includes runtime dependencies of build dependencies. maybe that wasn't clear when writing the code | 14:34 |
| juergbi | recurse=False would disable that | 14:34 |
| jonathanmaw | juergbi: if it's including indirect dependencies, then that's unintentional | 14:34 |
| juergbi | ok, thanks for the info | 14:34 |
| juergbi | doras[m]: can you please file a bug? should be easy to fix | 14:34 |
| doras[m] | juergbi: sure. Thanks! | 14:35 |
| juergbi | you can also try it yourself, using self.dependencies(Scope.BUILD, recurse=False) in filter.py:assemble() | 14:36 |
| doras[m] | I'll try it and send a pull request if it works. | 14:38 |
| doras[m] | Or... merge request. Gitlab. | 14:38 |
| *** nimish has quit IRC | 14:49 | |
| *** nimish has joined #buildstream | 14:49 | |
| *** nimish has quit IRC | 14:54 | |
| *** nimish has joined #buildstream | 14:54 | |
| *** sambishop has quit IRC | 14:57 | |
| juergbi | that would be great | 14:59 |
| doras[m] | juergbi: that did it! Thanks again. I'll open a MR. | 14:59 |
| juergbi | :) | 14:59 |
| tpollard | cs-shadow: I reworked the interactive option to use choice based off your feedback, does it seem ok to you? https://gitlab.com/BuildStream/buildstream/merge_requests/1050 Hoping to get this merged today | 15:03 |
| cs-shadow | tpollard: thanks! I'll have a look in just a bit | 15:04 |
| tpollard | cs-shadow: there's also the comment around the implementation/semantics of https://gitlab.com/BuildStream/buildstream/merge_requests/1024/ (obviously WIP, so won't be merging today!) | 15:04 |
| tpollard | cheers! :) | 15:04 |
| doras[m] | juergbi: will it require invalidation of cached filter elements to fix existing artifacts? If so, is there any method to force this from the source? Some kind of revision I can change? | 15:06 |
| juergbi | doras[m]: we should change get_unique_key() | 15:07 |
| juergbi | maybe add 'indirect-dependencies': False | 15:08 |
| doras[m] | Got it. I'll test it as well. | 15:12 |
| *** sambishop has joined #buildstream | 15:15 | |
| abderrahim[m] | juergbi: doras There is also BST_ARTIFACT_VERSION https://docs.buildstream.build/buildstream.element.html#buildstream.element.Element.BST_ARTIFACT_VERSION | 15:28 |
| juergbi | yes but that forces an invalidation of all artifacts | 15:28 |
| juergbi | shouldn't use that for plugin-specific changes | 15:28 |
| juergbi | ah, no | 15:29 |
| juergbi | I was thinking of core artifact version | 15:29 |
| juergbi | I forgot about the per-plugin version | 15:29 |
| juergbi | it doesn't appear to have been used by any plugins so far | 15:30 |
| juergbi | but we could indeed use it here, thanks for the reminder | 15:30 |
| *** lantw44 has quit IRC | 15:35 | |
| *** lantw44 has joined #buildstream | 15:35 | |
| gitlab-br-bot | phildawson opened (was WIP) MR !1075 (phil/plugin-testing-api->master: Expose basic api for testing external plugins.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1075 | 15:35 |
| *** lantw44 has quit IRC | 15:36 | |
| WSalmon | juergbi, who has merge rights to benchmarking? jennis and phildawson have done quite a long review of my mr https://gitlab.com/BuildStream/benchmarks/merge_requests/12 and I now need to work out who to butter up to actualy get it merged | 15:36 |
| *** lantw44 has joined #buildstream | 15:36 | |
| WSalmon | jonathanmaw, lachlan ^ you both seem to have MR rights, could you take a look? | 15:40 |
| * jonathanmaw peeks | 15:41 | |
| WSalmon | ta | 15:42 |
| jennis | doras[m], apologies I was preoccupied, looks like you (and juergbi) have solved the issue? | 15:43 |
| *** sambishop has quit IRC | 15:46 | |
| *** tristan has joined #buildstream | 15:49 | |
| *** sambishop has joined #buildstream | 15:51 | |
| jonathanmaw | WSalmon: looks good to me. | 15:51 |
| jonathanmaw | lachlan: does the benchmark use bstgen's command-line to do the benchmarks? | 15:51 |
| lachlan | Not at the minute, but they are in the pipeline - so to speak | 15:54 |
| *** nimish has quit IRC | 15:54 | |
| * jonathanmaw hits the big merge button | 15:54 | |
| *** nimish has joined #buildstream | 15:55 | |
| gitlab-br-bot | jonathanmaw opened MR !1108 (jonathan/wsl-tests->master: Add pre-merge tests that use a WSL runner) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1108 | 16:04 |
| jjardon | tristan: Hey, Is there a chance to make another 1.2.x release soonish? there are several in the bst-1.2 branch It would be nice they were in a official release | 16:10 |
| *** ChanServ sets mode: +o tristan | 16:18 | |
| tristan | jjardon, Good point, lets make one and a 1.3.1 together, at the gathering, after the session which decides the number of the dev release | 16:18 |
| tristan | jjardon, I wanted to make one before christmas :-S | 16:18 |
| jjardon | tristan: coolio; Christmas present :) | 16:19 |
| abderrahim[m] | there are also some fixes in master I'd like to see in a 1.2.x release | 16:23 |
| gitlab-br-bot | jennis opened (was WIP) MR !1088 (jennis/add_new_profile_topic->master: Add new 'scheduler' and 'load-selection' profiling topics) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1088 | 16:27 |
| *** nimish has quit IRC | 16:30 | |
| *** nimish has joined #buildstream | 16:30 | |
| jjardon | abderrahim[m]: do you have a list somewhere? | 16:31 |
| abderrahim[m] | no | 16:31 |
| abderrahim[m] | , | 16:32 |
| abderrahim[m] | bu | 16:32 |
| abderrahim[m] | t | 16:32 |
| abderrahim[m] | btu | 16:32 |
| abderrahim[m] | but I can make one | 16:32 |
| abderrahim[m] | sorry | 16:32 |
| jjardon | abderrahim[m]: we have a gathering next week, so I think that feedback would be very valuable | 16:32 |
| jjardon | tristan: Does the gathering have a wiki? | 16:33 |
| laurence | jjardon, so far only a page on the wiki for registration, would you like a page where you can add more, like notes and so on? | 16:33 |
| laurence | happy to set it up | 16:33 |
| laurence | i suspect we'll use google docs on the day on something similar for collaborative note taking | 16:34 |
| jjardon | abderrahim[m]: maybe add the list at https://wiki.gnome.org/Projects/BuildStream/Events/Gathering-201901 | 16:34 |
| Kinnison | laurence: either that, or an etherpad or similar yeah | 16:34 |
| jjardon | or send an email to the list | 16:34 |
| jjardon | Kinnison: when is the last day for the performance results you asked in the list? I was thinking on generate them over the weekend | 16:36 |
| Kinnison | jjardon: Ideally yesterday, but I'll be happy to receive more data to process on Monday | 16:36 |
| * Kinnison has been processing the results yesterday and today, but will have plenty to do on Monday too anyway | 16:37 | |
| Kinnison | :-D | 16:37 |
| jjardon | Kinnison: oh, sorry about that. I will try to have something by Monday | 16:40 |
| jjardon | Kinnison: did anyone from GNOME/freedesktop-sdk send something to you already? | 16:40 |
| Kinnison | jjardon: valentind did | 16:40 |
| Kinnison | jjardon: He has a monster computer :-D | 16:40 |
| jjardon | valentind: did you use your personal computer or the runners? | 16:40 |
| jjardon | Kinnison: we have several monster computers, yes :) | 16:41 |
| Kinnison | Unless the runners are water-cooled Xeons with 24G of RAM and 1T NVMe systems, he used a personal computer | 16:41 |
| jjardon | wow | 16:42 |
| jjardon | ok then :) | 16:42 |
| abderrahim[m] | maybe I should do with a modest computer :) | 16:42 |
| Kinnison | jjardon: Smeone else used a similar NVMe, but with 64G of ram and a Ryzen threadripper | 16:43 |
| Kinnison | jjardon: terrifyingly powerful system | 16:43 |
| Kinnison | abderrahim[m]: the more results, the merrier | 16:43 |
| jjardon | Kinnison: 48 Xeon cores / 251G RAM here :) | 16:45 |
| tpollard | unless you're highly parallelised, a desktop threadripper is probably faster | 16:50 |
| *** nimish has quit IRC | 16:50 | |
| jonathanmaw | I've got a MR for adding pre-merge tests that use WSL (!1108). Since the worker is a windows VM, I don't have any docker config to generate the workers, and instead have scripts for creating and configuring a VM stored in a personal repo (https://gitlab.com/jonathanmaw/wsl-setup-script). Should I store these scripts somewhere on the buildstream project? | 16:50 |
| *** nimish has joined #buildstream | 16:50 | |
| jjardon | tpollard: probably, yes. But those servers are running several builds at the same time so maybe the difference is not so much | 16:51 |
| *** toscalix has quit IRC | 16:52 | |
| jjardon | 2 x Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz | 16:52 |
| jjardon | I guess we need to test :) | 16:52 |
| * tpollard requests a 2950X | 16:54 | |
| lachlan | re: benchmarking, I'm restricting runs on the main laptop to master branch only (this prevents results being compromised by dev branch pushes). I'm also putting restrictions in on merges - and these will now require approval (with some consideration of what is actually running on the benchmarking at the time). | 17:01 |
| *** tpollard has quit IRC | 17:02 | |
| Kinnison | jjardon: phwoar | 17:06 |
| gitlab-br-bot | cs-shadow approved MR !1050 (tpollard/829->master: Download buildtrees on demand for bst shell --use-buildtree) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1050 | 17:09 |
| gitlab-br-bot | cs-shadow closed MR !1050 (tpollard/829->master: Download buildtrees on demand for bst shell --use-buildtree) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1050 | 17:09 |
| gitlab-br-bot | cs-shadow reopened MR !1050 (tpollard/829->master: Download buildtrees on demand for bst shell --use-buildtree) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1050 | 17:10 |
| *** nimish has quit IRC | 17:20 | |
| *** nimish has joined #buildstream | 17:21 | |
| *** sambishop has quit IRC | 17:23 | |
| *** solid_black has quit IRC | 17:26 | |
| *** nimish has quit IRC | 17:46 | |
| *** nimish has joined #buildstream | 17:46 | |
| *** nimish has joined #buildstream | 17:46 | |
| *** nimish has joined #buildstream | 17:47 | |
| *** tristan has quit IRC | 17:49 | |
| *** bochecha has quit IRC | 17:50 | |
| gitlab-br-bot | jennis merged MR !1088 (jennis/add_new_profile_topic->master: Add new 'scheduler' and 'load-selection' profiling topics) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1088 | 17:59 |
| *** tristan has joined #buildstream | 18:03 | |
| *** ChanServ sets mode: +o tristan | 18:03 | |
| tristan | oh damn | 18:03 |
| tristan | Anyone else looking at this https://gitlab.com/BuildStream/buildstream/pipelines ? | 18:03 |
| *** rdale has quit IRC | 18:04 | |
| tristan | They are taking longer and longer... | 18:04 |
| cs-shadow | did all our caches disappear? | 18:05 |
| tristan | No I've noticed something... ssssstrangeeeeeee | 18:06 |
| tristan | This is a recent job from today, took 62 min (the pipeline took like 83 min) https://gitlab.com/BuildStream/buildstream/-/jobs/151312947 | 18:07 |
| tristan | hmmm nope, the trend is not matching up | 18:09 |
| tristan | Yesterday 1 hour pipeline https://gitlab.com/BuildStream/buildstream/-/jobs/151312947 | 18:09 |
| tristan | cs-shadow, What I was looking at was exactly the cache, and what I suspect is that it is somehow growing | 18:09 |
| tristan | I cannot explain numbers like 32G and 21G used /cache partition | 18:10 |
| tristan | maybe there are other things on /cache than just our cache, cause ours should be only a couple GB | 18:11 |
| tristan | Both tests show the slowest test is test_autotools_build at around 3min, that definitely hit the cache and did not download an sdk with ostree | 18:13 |
| tristan | the file counts of cache look constant so the cache is not randomly growing | 18:14 |
| *** nimish has quit IRC | 18:17 | |
| cs-shadow | hmm, do you know if there's a way to browse the cache in the same way one can browse the job artifacts in GitLab? | 18:17 |
| *** nimish has joined #buildstream | 18:17 | |
| tristan | The recent debian 9 from today which completed in 62min "1288 passed, 3 skipped in 2557.41 seconds", which is 42min | 18:18 |
| *** jonathanmaw has quit IRC | 18:19 | |
| tristan | The test itself is taking almost twice as long as it should, and the gitlab setup/upload overhead is up to 20min | 18:19 |
| tristan | cs-shadow, you can always push a branch with a test .gitlab-ci.yml | 18:19 |
| tristan | cs-shadow, other than that I don't know how to look at it | 18:20 |
| cs-shadow | fair enough, i'll try that | 18:20 |
| tristan | jjardon, ^^^^^^^^ any ideas ? | 18:20 |
| tristan | lemme look at a successful pipeline from like 1 or 2 weeks ago, when it was running better | 18:21 |
| tristan | the first < 30min pipeline I can find | 18:21 |
| *** lachlan has quit IRC | 18:21 | |
| cs-shadow | tristan: https://gitlab.com/BuildStream/buildstream/pipelines/43948961 | 18:21 |
| *** nimish has quit IRC | 18:22 | |
| *** nimish has joined #buildstream | 18:23 | |
| tristan | cs-shadow, for the win at https://gitlab.com/BuildStream/buildstream/-/jobs/149673013 | 18:23 |
| tristan | well, 33min | 18:24 |
| tristan | nice you found 28 | 18:24 |
| tristan | oh yeah | 18:24 |
| tristan | cs-shadow, "cache/: found 44565 matching files" vs "cache/: found 588166 matching files" | 18:25 |
| tristan | cs-shadow, I started with this suspicion and it keeps coming back :) | 18:26 |
| tristan | We must have broken something with tox, causing something accumulative to end up in the cache | 18:26 |
| tristan | It might my change to the artifacts directory in integration mode | 18:26 |
| tristan | must be that | 18:27 |
| *** raoul has quit IRC | 18:27 | |
| tristan | cs-shadow, I'm pretty sure 6286d8209 is at the root of this | 18:27 |
| tristan | I wonder why it's not getting deleted properly | 18:27 |
| *** bochecha has joined #buildstream | 18:29 | |
| gitlab-br-bot | tristanvb opened MR !1109 (tristan/gitlab-test->master: .gitlab-ci.yml: Adding some diagnostics) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1109 | 18:34 |
| tristan | oops | 18:34 |
| cs-shadow | heh, I just created https://gitlab.com/BuildStream/buildstream/commits/channdan/test too, couldn't even spell my own name correct | 18:35 |
| cs-shadow | it's Friday I guess | 18:35 |
| tristan | So !1106 makes it so that the cache quota is treated as `min(quota, available_space - headroom(2GB))`, issues a warning if the quota doesnt fit in the available space, and an error if the quota exceeds total storage space on the volume | 18:42 |
| gitlab-br-bot | MR !1106: _artifactcache.py: Don't require the quota to be available on disk. https://gitlab.com/BuildStream/buildstream/merge_requests/1106 | 18:42 |
| tristan | Question: I guess we should make the headroom configurable with this right ? | 18:42 |
| tristan | cache: { headroom: 2G } in buildstream.conf, with default of quota staying at infinity | 18:43 |
| tristan | juergbi, cs-shadow, any thoughts anyone ? | 18:43 |
| * tristan supposes we can go ahead with it for now as it is solving problems, leave configurability separate | 18:48 | |
| tristan | Oh were "downloading the mega huge cache" https://gitlab.com/BuildStream/buildstream/-/jobs/151363432 | 18:49 |
| cs-shadow | tristan: we have these things in the cache: https://gitlab.com/BuildStream/buildstream/-/jobs/151363426 | 18:50 |
| cs-shadow | (as many as GitLab's CI can show in one job) | 18:50 |
| tristan | wat ?! | 18:52 |
| tristan | cs-shadow, any idea what /builds/BuildStream/buildstream/cache/integration-cache/artifacts/extract itself comes from ? | 18:52 |
| cs-shadow | i meant the GitLab cache. I did a `find cache` whose output is more than what GitLab's CI can show (I should've thought about that ) | 18:53 |
| tristan | Oh | 18:53 |
| tristan | yeah but I think I got it | 18:53 |
| tristan | first of all, a change like the one I did in the mentioned commit... requires that we manually zap the gitlab cache | 18:54 |
| tristan | Because normally the artifacts/ directory is removed at the end of the run, but this time artifacts/ was not | 18:54 |
| tristan | or, no... that doesnt make any sense does it ? | 18:54 |
| tristan | the artifacts/ directory should not be in the cache because conftest.py creates/removes it every session | 18:55 |
| tristan | But then, maybe something is hardcoding the artifacts directory ? | 18:55 |
| cs-shadow | should we only cache `artifacts/sources` ? | 18:55 |
| cs-shadow | sorry i meant `cache/soruces` | 18:56 |
| tristan | yeah | 18:56 |
| tristan | cs-shadow, that is already the case, though | 18:56 |
| tristan | https://gitlab.com/BuildStream/buildstream/blob/master/conftest.py#L69 | 18:57 |
| tristan | conftest.py is a pytest recognized file which inserts some global things into the test environment, the integration_cache() is a session wide fixture, which creates and removes the artifacts directory | 18:58 |
| cs-shadow | we definitely have artifacts in the cache based on my find command | 18:59 |
| tristan | yes | 19:00 |
| tristan | Ok, so lets run a tox with --integration locally, and see if it leaks anything behind | 19:00 |
| tristan | But I doubt it | 19:01 |
| tristan | I think we need to zap that cache and have it redownload the first time around, and the problem will be solved | 19:01 |
| tristan | But... I wonder if bst-1.2 and master share the same caches | 19:01 |
| tristan | If gitlab is smart enough to recognize the difference, or if we need to namespace the caches with more than CI_JOB_NAME | 19:02 |
| *** nimish has quit IRC | 19:02 | |
| tristan | sharing of caches across disparate branches could make a nasty test environment | 19:03 |
| *** nimish has joined #buildstream | 19:03 | |
| cs-shadow | i would expect that the caches are shared as I am not sure if GitLab differentiates between bst-1.2 and any other feature branch. And we do see that feature branch share cache with master | 19:04 |
| tristan | yeah | 19:04 |
| tristan | I'm wondering how we could tune that | 19:05 |
| tristan | We would want to use the closest recognized origin branch point, like if we recognized 1.2 and master, anything branched off of anything branched off of 1.2 or master would use that cache respectively | 19:06 |
| cs-shadow | if you only want this behavior for special release branches, we can simply make a note of changing the cache key for each release branch | 19:06 |
| tristan | but that's not really a thing | 19:06 |
| cs-shadow | in their `.gitlab-ci.yml` | 19:06 |
| tristan | Right, we could have a magic number | 19:06 |
| tristan | That's not a bad idea | 19:06 |
| cs-shadow | or the name of the relase :) | 19:06 |
| tristan | Haha yeah okay | 19:07 |
| tristan | still this doesn't make much sense | 19:08 |
| tristan | If 1.2 and master are both cleaning up their artifact directories after tests, then neither of them would negatively pollute the cache | 19:09 |
| tristan | they just share the same sources, which is all we really cache | 19:09 |
| *** nimish has quit IRC | 19:11 | |
| *** nimish has joined #buildstream | 19:18 | |
| *** tristan has quit IRC | 19:35 | |
| *** nimish has quit IRC | 19:38 | |
| *** nimish has joined #buildstream | 19:38 | |
| *** toscalix has joined #buildstream | 19:58 | |
| valentind | jjardon for the buildstream performance benchmark I used my computer. | 20:03 |
| valentind | And yes it is a watercooled Xeon, 24GB with 1TB NVMe. | 20:04 |
| *** nimish has quit IRC | 20:08 | |
| *** nimish has joined #buildstream | 20:09 | |
| *** nimish has quit IRC | 20:26 | |
| *** tristan has joined #buildstream | 20:43 | |
| *** ChanServ sets mode: +o tristan | 20:49 | |
| tristan | This shows that after having run a pipeline which gets rid of `cache/artifacts`, the next run starts with a clean cache again: https://gitlab.com/BuildStream/buildstream/-/jobs/151406530 | 20:49 |
| tristan | Maybe just having run that pipeline will be enough | 20:49 |
| *** bochecha has quit IRC | 20:57 | |
| *** bochecha has joined #buildstream | 20:58 | |
| tristan | I suspect that running a pipeline on this https://gitlab.com/BuildStream/buildstream/commit/2d8fe599cdcb7987da24751d8710121a11e04ab2 is what initially triggered this leakage into the cache | 20:59 |
| tristan | Well, we learned something - anything the tests leave behind unintentionally will accumulate in the cache, and slow things down over time | 21:00 |
| tristan | (or anything inside the INTEGRATION_CACHE directory) | 21:00 |
| valentind | tristan, I recently change the size of the builders because we were getting out of disk. | 21:02 |
| valentind | Is that related? | 21:02 |
| valentind | The integration cache was big. | 21:02 |
| tristan | valentind, yeah, I am talking about the cause of the integration cache being big | 21:05 |
| tristan | valentind, there is a cache/integration-cache/artifacts directory in the cache which should not be there | 21:05 |
| tristan | This one's been creating it's cache to upload for 20min now: https://gitlab.com/BuildStream/buildstream/-/jobs/151364099 | 21:06 |
| gitlab-br-bot | tristanvb closed issue #869 (Local cache size calculation is wrong) on buildstream https://gitlab.com/BuildStream/buildstream/issues/869 | 21:18 |
| gitlab-br-bot | tristanvb closed issue #733 (quota config parameter should be the maximum amount of cache allowed, not the minimum) on buildstream https://gitlab.com/BuildStream/buildstream/issues/733 | 21:18 |
| gitlab-br-bot | tristanvb merged MR !1106 (tristan/cache-quota-max-only->master: _artifactcache.py: Don't require the quota to be available on disk.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1106 | 21:18 |
| *** justice has joined #buildstream | 22:07 | |
| tristan | With the cache size fixed by force removal of the artifacts directory, we're down to 48min https://gitlab.com/BuildStream/buildstream/-/jobs/151417333 | 22:08 |
| tristan | it helped, but there is more at work here | 22:08 |
| tristan | So looking back at the good old days: https://gitlab.com/BuildStream/buildstream/-/jobs/148923732 test_autotools_build takes 104 seconds, now it takes 216 seconds: https://gitlab.com/BuildStream/buildstream/-/jobs/151417333 | 22:18 |
| tristan | As it is a test which spends most of it's time in I/O copying files rather than processing elements, I think it's safe to say that the remainder of the performance drop is due to I/O performance on gitlab | 22:19 |
| tristan | or digitalocean | 22:19 |
| tristan | valentind, I think we can reduce the size of the builders again, unless we need big disks for the nightly tests | 22:23 |
| tristan | valentind, after running a pipeline which does `rm -rf cache/integration-cache/artifacts` a bunch of times, seems that the surprise ton of artifacts has gone away | 22:23 |
| doras[m] | CONTRIBUTING.rst suggested that I'd ask for access to the gitlab repo for submitting patches. I'd appreciate it if anyone could grant me access. My username is doraskayo. | 22:45 |
| tristan | doras[m], Done :) | 23:02 |
| doras[m] | tristan: thanks :) | 23:04 |
| *** justice has quit IRC | 23:31 | |
| *** justice has joined #buildstream | 23:34 | |
| gitlab-br-bot | doraskayo opened issue #883 (Filter element unintentionally stages its indirect runtime dependencies recursively) on buildstream https://gitlab.com/BuildStream/buildstream/issues/883 | 23:48 |
| *** justice has quit IRC | 23:51 | |
| cs-shadow | tristan: can you please have a look at !1107 again when you a chance? | 23:55 |
| gitlab-br-bot | MR !1107: Generate man pages using tox & update them https://gitlab.com/BuildStream/buildstream/merge_requests/1107 | 23:55 |
| cs-shadow | *get a chance | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!