| *** nimish has quit IRC | 02:43 | |
| *** bochecha has joined #buildstream | 03:37 | |
| *** mohan43u has joined #buildstream | 06:44 | |
| *** cs-shadow has joined #buildstream | 06:45 | |
| juergbi | hi jmac, I've updated !991 now that symlinks should no longer be an issue. can you please take a look? I've described the changes in an MR comment | 07:56 |
|---|---|---|
| gitlab-br-bot | MR !991: Cache artifacts with virtual directories instead of filesystem. https://gitlab.com/BuildStream/buildstream/merge_requests/991 | 07:56 |
| *** tristan has joined #buildstream | 08:15 | |
| *** tristan has quit IRC | 08:20 | |
| *** tristan has joined #buildstream | 08:23 | |
| juergbi | hi tristan, any thoughts on !968, especially with regards to the point I raised? | 08:26 |
| gitlab-br-bot | MR !968: Allow absolute paths in whitelist https://gitlab.com/BuildStream/buildstream/merge_requests/968 | 08:26 |
| *** ChanServ sets mode: +o tristan | 08:34 | |
| jmac | juergbi: Having a look now | 08:34 |
| juergbi | ta | 08:34 |
| gitlab-br-bot | jmacarthur opened issue #917 (Maximum recursion error while using tar source (upstream Python issue)) on buildstream https://gitlab.com/BuildStream/buildstream/issues/917 | 08:55 |
| *** alatiera has joined #buildstream | 09:06 | |
| *** tristan has quit IRC | 09:06 | |
| *** tristan has joined #buildstream | 09:07 | |
| *** cs-shadow has quit IRC | 09:27 | |
| *** tpollard has joined #buildstream | 09:32 | |
| *** raoul has joined #buildstream | 09:33 | |
| jmac | juergbi: I don't recognise any of this code anymore, sorry. I can still review it, but it's going to be a multi-hour job, and I don't think I'll have enough time for it anytime soon. | 09:37 |
| laurence | jmac, i'm struggling to find your write up of the RE testing on the wiki - is it still there? | 09:39 |
| juergbi | jmac: ok, then just let's leave it as is. it shouldn't have anything controversial. it's actually simpler now due to the symlink handling change in master | 09:40 |
| laurence | jmac, ah found it | 09:41 |
| laurence | https://wiki.gnome.org/Projects/BuildStream/RemoteExecutionTesting | 09:41 |
| *** jonathanmaw has joined #buildstream | 09:42 | |
| gitlab-br-bot | tristanvb merged MR !1155 (tristan/document-release-process->master: CONTRIBUTING.rst: Documenting the release process) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1155 | 09:43 |
| juergbi | tristan: I've replied to your comment | 09:48 |
| *** tristan has quit IRC | 09:52 | |
| *** tristan has joined #buildstream | 09:54 | |
| *** ChanServ sets mode: +o tristan | 09:54 | |
| tristan | juergbi, network seems to keep slipping... replied again | 09:58 |
| juergbi | tristan: it seems like we're still misunderstanding each other. a couple lines below the highlighted regex change, that's actually what the patch does (prepending a slash) | 09:59 |
| juergbi | i.e., there are essentially two changes in the branch | 09:59 |
| juergbi | one to prepend the slash to make proper comparison, matching the documentation | 09:59 |
| juergbi | and another one to keep backward compatibility with whitelists that don't expect absolute paths | 10:00 |
| tristan | juergbi, Yeah I noticed that after making the initial comment | 10:00 |
| juergbi | the first one I think we can all agree on | 10:00 |
| juergbi | the second one is my question | 10:00 |
| juergbi | i.e., do we really want that for master | 10:00 |
| tristan | And I think I included my agreement with you in the last comment :) | 10:00 |
| tristan | Second sentence | 10:00 |
| juergbi | ah ok, your 'this' was a bit ambiguous to me | 10:01 |
| tristan | Ahhh | 10:01 |
| tristan | Sorry :) | 10:01 |
| juergbi | no problem | 10:01 |
| juergbi | valentind: can you please confirm that the highlighted change in !968 is just for backward compatibility? | 10:01 |
| gitlab-br-bot | MR !968: Allow absolute paths in whitelist https://gitlab.com/BuildStream/buildstream/merge_requests/968 | 10:02 |
| gitlab-br-bot | tristanvb opened issue #918 (requirements makefile adds pkg-resources 0.0.0) on buildstream https://gitlab.com/BuildStream/buildstream/issues/918 | 10:19 |
| *** lachlan has joined #buildstream | 10:32 | |
| *** lachlan has quit IRC | 10:40 | |
| *** lachlan has joined #buildstream | 10:41 | |
| laurence | adds68, where does your marge-bot forward email to? | 10:55 |
| laurence | just a handful of core contributors? is the mail useful? | 10:55 |
| adds68 | laurence, its just the standard gitlab emails for a specific MR | 10:56 |
| laurence | ok cheers | 11:00 |
| laurence | juergbi, want to receive emails from marge-bot? tristan, do you ? | 11:00 |
| juergbi | I don't think I normally want these mails as I'll already get notified by merges via my normal GitLab account | 11:01 |
| juergbi | however, if needed for admin or so, I can be a receiver and just automatically filter them out | 11:01 |
| laurence | ok | 11:02 |
| laurence | I'll not do so for now | 11:02 |
| tristan | laurence, What I understand from adds68 is that the merge bot just comments on the merge request; and gitlab as a result sends an email, am I misunderstanding ? | 11:05 |
| tristan | adds68, ^^ ? | 11:07 |
| adds68 | tristan, yea correct | 11:07 |
| adds68 | laurence, tristan see the bottom of this issue as an example: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/merge_requests/681 | 11:08 |
| adds68 | MR* sorry | 11:08 |
| juergbi | tristan: I think laurence's question is about whether you want the emails that are sent to marge-bot's email address | 11:09 |
| tristan | yeah | 11:09 |
| juergbi | and you probably don't (normally) | 11:09 |
| tristan | Err right, we only want to know if something is broken, if it absolutely must have an email address, it would be fine if it were an archive that a buildstream developer could inspect when finding that the merge bot is having trouble | 11:10 |
| gitlab-br-bot | raoul.hidalgocharman approved MR !991 (jmac/cache_artifacts_with_vdir->master: Cache artifacts with virtual directories instead of filesystem.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/991 | 11:15 |
| gitlab-br-bot | juergbi closed issue #787 (Alter Element._cache_artifact so it uses the virtual directory system) on buildstream https://gitlab.com/BuildStream/buildstream/issues/787 | 11:18 |
| gitlab-br-bot | juergbi merged MR !991 (jmac/cache_artifacts_with_vdir->master: Cache artifacts with virtual directories instead of filesystem.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/991 | 11:18 |
| tpollard | https://2019.fossasia.org/event/schedule.html#5216 tristan 's talk is on the schedule :D | 11:22 |
| tristan | Already ! | 11:31 |
| laurence | tristan, juergbi, adds68, ok thanks - I'll have the emails sent to myself, I don't mind deleting them / setting up a filter | 11:33 |
| adds68 | tpollard, get some stickers printed | 11:36 |
| *** lachlan has quit IRC | 11:44 | |
| jmac | If I checkout an element which has a build-depends in it, should I expect to see the contents of the dependency artifact in the dependent element's checkout? | 11:49 |
| Kinnison | jmac: The defaults are runtime dependencies, try --deps build | 11:58 |
| Kinnison | jmac: if you want that | 11:58 |
| jmac | OK, that looks like it's doing something, thanks | 12:00 |
| Kinnison | No probs, confused me a few times too | 12:00 |
| juergbi | Kinnison: you expect build dependencies to be included in a checkout by default, or was it simply confusing but you agree that they shouldn't be included by default? | 12:09 |
| Kinnison | juergbi: I expected only the single element's content | 12:10 |
| Kinnison | juergbi: i.e. 'none' | 12:10 |
| juergbi | ah ok, that I can understand | 12:10 |
| *** nimish has joined #buildstream | 12:13 | |
| *** lachlan has joined #buildstream | 12:13 | |
| *** lachlan has quit IRC | 12:23 | |
| *** lachlan has joined #buildstream | 12:31 | |
| *** lachlan has quit IRC | 12:37 | |
| tristan | so I zapped the gitlab caches on the weekend and they have been rebuilt since then | 12:38 |
| tristan | But pipelines are still taking somewhere from 40min to 60min, instead of ~25/30min | 12:38 |
| juergbi | tristan: might be buildtree-related. for one, multiple buildtree tests are among the 20 slowest, and also other integration tests might take a bit longer with buildtrees | 12:41 |
| juergbi | although, not sure about the latter, given that I don't expect the buildtrees of our tests to be very large | 12:41 |
| gitlab-br-bot | cs-shadow opened MR !1156 (chandan/remove-pkg-resources->master: requirements/Makefile: exclude pkg-resources) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1156 | 12:45 |
| tristan | juergbi, Except I recall that pipelines were in good health at the time we introduced tox | 12:46 |
| tristan | That is the confusing part :-S | 12:46 |
| tristan | Maybe I should try running a pipeline from around then and see if it takes around the same time | 12:46 |
| *** cs-shadow has joined #buildstream | 12:49 | |
| juergbi | tristan: might it be this? conftest.py: Use different artifact directory for integration tests | 12:57 |
| gitlab-br-bot | tristanvb opened MR !1157 (tristan/collect-missing-coverage->master: .gitlab-ci.yml: Some new test environments were added, but not collecting coverage.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1157 | 12:58 |
| tristan | juergbi, it's not supposed to, I mean; that just makes the directories unique per run, so that there are no races when running parallel tests with detox | 12:59 |
| juergbi | but did we clean the artifact caches before? | 12:59 |
| tristan | All I can imagine is that might causing artifact caches to land in the cache | 13:00 |
| tristan | Yes | 13:00 |
| gitlab-br-bot | cs-shadow approved MR !1157 (tristan/collect-missing-coverage->master: .gitlab-ci.yml: Some new test environments were added, but not collecting coverage.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1157 | 13:00 |
| tristan | juergbi, The artifact cache is never supposed to persist in the gitlab cache, only the sources are stored/shared/reused | 13:00 |
| juergbi | I don't mean across CI jobs, I meant across integration tests in a single job | 13:00 |
| tristan | Nope, that they should not | 13:01 |
| tristan | It is one artifact cache for the session | 13:01 |
| juergbi | ok, but with your commit not anymore | 13:01 |
| juergbi | hence possible slowdown | 13:01 |
| juergbi | (but better isolation) | 13:01 |
| tristan | I cannot believe that to be true | 13:01 |
| juergbi | which part exactly? | 13:02 |
| tristan | So what did I miss ? | 13:02 |
| tristan | hehe | 13:02 |
| tristan | juergbi, it is a session wide value, I don't know what black magic caused `scope="session"` to work before, but it should work equally now | 13:03 |
| tristan | 6286d8209a3db7fcf2b490a55044fa4c2be0c007 | 13:03 |
| tristan | i.e. if you check that commit, you will see that we only change the directory name, it is still removed at the end of the session wide fixture | 13:03 |
| juergbi | ah, the fixture is per session, I missed that | 13:03 |
| tristan | yeah I was totally confused at that when writing that commit | 13:04 |
| tristan | but it seems to work the same way as it did before | 13:04 |
| tristan | that should really make the integration tests a *lot* slower if I screwed that up :) | 13:04 |
| juergbi | not sure it would be that drastic, as long as sources are still cached | 13:06 |
| juergbi | independent of why the slowdown happened, maybe we should start running tests in parallel to speed things up, at least when the CI runners are not overbooked | 13:07 |
| juergbi | there is still one integration test (cachedfail) that often fails when running tests in parallel, though | 13:07 |
| juergbi | would be good to fix that in any case, of course | 13:07 |
| juergbi | locally I can run all tests in about 5 min | 13:07 |
| tpollard | woah | 13:07 |
| tristan | yeah I'm impressed | 13:08 |
| tristan | juergbi, each integration test would have to do the import - one optimization I made before is that we had the alpine image named/cached differently in different tests | 13:08 |
| tristan | plus the flatpak runtime | 13:09 |
| tristan | I think that for the integration tests, the slowest ones are the autotools build one because it is the first to create the base runtime artifact | 13:09 |
| tristan | (as I understand, CPU is never a problem, it is I/O on gitlab which is to be feared) | 13:10 |
| juergbi | after the WIP CAS optimizations (incl. source cache), we could probably have that _a lot_ faster | 13:10 |
| juergbi | although, initial import into CAS still needs to happen, of course. should still be faster, though | 13:11 |
| gitlab-br-bot | tristanvb closed issue #918 (requirements makefile adds pkg-resources 0.0.0) on buildstream https://gitlab.com/BuildStream/buildstream/issues/918 | 13:43 |
| gitlab-br-bot | tristanvb merged MR !1156 (chandan/remove-pkg-resources->master: requirements/Makefile: exclude pkg-resources) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1156 | 13:43 |
| Kinnison | cs-shadow: Thanks for a comprehensive update on the dockers | 13:52 |
| cs-shadow | Kinnison: you're welcome :) | 13:52 |
| gitlab-br-bot | matthew-yates opened issue #919 ('bst build <elem>' does not assemble all requried elements in some circumstances) on buildstream https://gitlab.com/BuildStream/buildstream/issues/919 | 13:57 |
| *** tristan has quit IRC | 14:00 | |
| *** phildawson has quit IRC | 14:01 | |
| *** phildawson has joined #buildstream | 14:01 | |
| *** raoul has quit IRC | 14:02 | |
| *** tristan has joined #buildstream | 14:15 | |
| *** ChanServ sets mode: +o tristan | 14:15 | |
| gitlab-br-bot | tristanvb merged MR !1157 (tristan/collect-missing-coverage->master: .gitlab-ci.yml: Some new test environments were added, but not collecting coverage.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1157 | 14:27 |
| *** bochecha has quit IRC | 14:42 | |
| *** raoul has joined #buildstream | 15:03 | |
| gitlab-br-bot | jennis closed issue #234 (Allow viewing build logs from artifacts) on buildstream https://gitlab.com/BuildStream/buildstream/issues/234 | 15:05 |
| raoul | juergbi, after you've looked at the cache dir MR, if you could it'd be good if you had a quick look over the WIP source cache MR (!1124) to check I'm going in the right direction. There's still few bits left like the source refs (how they're named and where the refs should be kept), which need to be done, though could possibly be done in separate MR. | 15:13 |
| juergbi | ok, will do | 15:14 |
| raoul | cheer :) | 15:14 |
| *** nimish has quit IRC | 15:26 | |
| *** nimish has joined #buildstream | 15:36 | |
| jonathanmaw | jjardon: I'm looking into adding the details of the new runner to https://wiki.gnome.org/action/login/Projects/BuildStream/Infrastructure, and it seems to be an Immutable Page to me | 15:51 |
| *** lachlan has joined #buildstream | 15:53 | |
| *** lachlan has quit IRC | 16:06 | |
| gitlab-br-bot | juergbi opened issue #920 (Use virtual directories for staging dependencies) on buildstream https://gitlab.com/BuildStream/buildstream/issues/920 | 16:26 |
| gitlab-br-bot | juergbi opened issue #921 (Drop extract directories) on buildstream https://gitlab.com/BuildStream/buildstream/issues/921 | 16:29 |
| gitlab-br-bot | phildawson opened issue #922 (Make it easy to test BuildStream against external plugins) on buildstream https://gitlab.com/BuildStream/buildstream/issues/922 | 16:39 |
| *** lachlan has joined #buildstream | 16:59 | |
| laurence | jmac, iirc you also had a page on the wiki about symlinks (i think??? or absolute relative paths..?? can't quite remember) | 17:01 |
| laurence | anyway i think it may be obsolete now due to the recent changes | 17:01 |
| jmac | Yes, hang on... | 17:04 |
| jmac | https://wiki.gnome.org/Projects/BuildStream/ImportRules, but yes, as you say, I think the new rules make most of that obsolute | 17:05 |
| jmac | obsolete | 17:05 |
| juergbi | jmac: I think we can also close #817 or was any of that not symlink-related? | 17:06 |
| gitlab-br-bot | Issue #817: We need examples showing the importance of file import ordering https://gitlab.com/BuildStream/buildstream/issues/817 | 17:06 |
| gitlab-br-bot | jmacarthur closed issue #817 (We need examples showing the importance of file import ordering) on buildstream https://gitlab.com/BuildStream/buildstream/issues/817 | 17:07 |
| jmac | Yep, good point | 17:07 |
| juergbi | ta | 17:07 |
| juergbi | regarding ImportRules, we should still document what overwrites/replacements are allowed / what fails, but yes, it should be quite a bit simpler now | 17:07 |
| *** lachlan has quit IRC | 17:23 | |
| *** mlalkaka has joined #buildstream | 17:33 | |
| gitlab-br-bot | jonathanmaw opened MR !1159 (jonathan/wsl-tests-allow-fail->master: Allow WSL tests to run, but they don't block the CI) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1159 | 17:36 |
| *** lachlan has joined #buildstream | 17:50 | |
| *** lachlan has quit IRC | 17:55 | |
| *** mohan43u has quit IRC | 18:01 | |
| *** lachlan has joined #buildstream | 18:02 | |
| gitlab-br-bot | cs-shadow approved MR !1159 (jonathan/wsl-tests-allow-fail->master: Allow WSL tests to run, but they don't block the CI) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1159 | 18:03 |
| *** lachlan has quit IRC | 18:06 | |
| *** raoul has quit IRC | 18:19 | |
| juergbi | valentind: do you mind if I quickly update !968 to remove the backward compatibility bit for master or will you do that? | 18:27 |
| gitlab-br-bot | MR !968: Allow absolute paths in whitelist https://gitlab.com/BuildStream/buildstream/merge_requests/968 | 18:27 |
| *** tristan has quit IRC | 18:28 | |
| gitlab-br-bot | cs-shadow opened MR !1160 (chandan/fix-badge-subprocess-error->master: doc/badges.py: Fix catching of CalledProcessError) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1160 | 18:46 |
| *** jonathanmaw has quit IRC | 18:54 | |
| gitlab-br-bot | jonathanmaw merged MR !1159 (jonathan/wsl-tests-allow-fail->master: Allow WSL tests to run, but they don't block the CI) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1159 | 18:57 |
| *** juanalday has joined #buildstream | 18:58 | |
| *** mlalkaka has quit IRC | 19:28 | |
| *** mlalkaka has joined #buildstream | 19:30 | |
| *** juanalday has quit IRC | 19:47 | |
| gitlab-br-bot | cs-shadow opened MR !1161 (chandan/fix-mapping-import->master: buildstream/_yaml.py: Import Mapping from collections.abc) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1161 | 19:47 |
| *** tristan has joined #buildstream | 19:49 | |
| *** nimish has quit IRC | 19:49 | |
| *** juanalday has joined #buildstream | 20:00 | |
| valentind | juergbi, I do not mind. | 20:10 |
| *** lachlan has joined #buildstream | 20:24 | |
| gitlab-br-bot | cs-shadow merged MR !1160 (chandan/fix-badge-subprocess-error->master: doc/badges.py: Fix catching of CalledProcessError) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1160 | 20:31 |
| *** lachlan has quit IRC | 20:40 | |
| *** mohan43u has joined #buildstream | 20:59 | |
| *** mohan43u has quit IRC | 21:25 | |
| *** mohan43u has joined #buildstream | 21:34 | |
| *** mlalkaka_ has joined #buildstream | 21:36 | |
| *** mlalkaka has quit IRC | 21:37 | |
| *** juanalday has quit IRC | 21:41 | |
| *** tristan has quit IRC | 22:25 | |
| *** mlalkaka_ has quit IRC | 22:25 | |
| *** juanalday has joined #buildstream | 22:52 | |
| *** cs-shadow has quit IRC | 23:40 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!