| *** xjuan has quit IRC | 00:38 | |
| *** bochecha has quit IRC | 01:02 | |
| *** jjardon has quit IRC | 01:15 | |
| *** jjardon has joined #buildstream | 01:15 | |
| *** claro has quit IRC | 02:55 | |
| *** leopi has joined #buildstream | 03:35 | |
| *** tristan has joined #buildstream | 05:23 | |
| *** Prince781 has joined #buildstream | 05:27 | |
| gitlab-br-bot | buildstream: merge request (tristan/finish-backport-of-679->bst-1.2: Tristan/finish backport of 679) #706 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/706 | 05:45 |
|---|---|---|
| *** alatiera_ has joined #buildstream | 05:47 | |
| gitlab-br-bot | buildstream: merge request (tristan/finish-backport-of-679->bst-1.2: Tristan/finish backport of 679) #706 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/706 | 05:55 |
| gitlab-br-bot | buildstream: merge request (valentindavid/inconsistant-workspace->master: Improve error message for deleted open workspaces) #703 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/703 | 06:40 |
| *** skullman has quit IRC | 06:45 | |
| *** csoriano has quit IRC | 06:45 | |
| *** skullman has joined #buildstream | 06:45 | |
| *** csoriano has joined #buildstream | 06:45 | |
| *** ChanServ sets mode: +o tristan | 06:45 | |
| tristan | cs-shadow, pypi.org: tristan, test.pypi.org: tristanvb | 06:45 |
| gitlab-br-bot | buildstream: merge request (tristan/finish-backport-of-679->bst-1.2: Tristan/finish backport of 679) #706 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/706 | 06:57 |
| gitlab-br-bot | buildstream: merge request (tristan/inconsistent-workspace-1.2->bst-1.2: Improve error message for deleted open workspaces) #707 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/707 | 07:07 |
| *** tristan has quit IRC | 07:12 | |
| *** tristan has joined #buildstream | 07:19 | |
| *** ChanServ sets mode: +o tristan | 07:19 | |
| gitlab-br-bot | buildstream: issue #576 (""Inconsitent Pipeline" error when workspace is open") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/576 | 07:45 |
| gitlab-br-bot | buildstream: issue #576 (""Inconsitent Pipeline" error when workspace is open") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/576 | 07:45 |
| gitlab-br-bot | buildstream: merge request (valentindavid/inconsistant-workspace->master: Improve error message for deleted open workspaces) #703 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/703 | 07:45 |
| gitlab-br-bot | buildstream: merge request (tristan/inconsistent-workspace-1.2->bst-1.2: Improve error message for deleted open workspaces) #707 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/707 | 08:00 |
| *** bochecha has joined #buildstream | 08:02 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/extract-expiry->master: Remove artifact extracts when artifact expires in cache) #685 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/685 | 08:02 |
| *** WSalmon has joined #buildstream | 08:03 | |
| gitlab-br-bot | buildstream: merge request (tristan/artifact-expiry-1.2->bst-1.2: Remove artifact extracts when artifact expires in cache) #708 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/708 | 08:05 |
| gitlab-br-bot | buildstream: merge request (tristan/reduce-gitlab-ci->master: WIP: .gitlab-ci.yml: Avoid running tests in post-merge) #709 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/709 | 08:11 |
| tristan | jjardon, Can you have a look at https://gitlab.com/BuildStream/buildstream/merge_requests/709 and tell me if the .gitlab-ci.yml changes make sense to you ? | 08:13 |
| tristan | jjardon, i.e. I'm mostly worried about specifying the magick yaml `<<:` thing more than once in the same dictionary | 08:17 |
| tristan | Also, anyone have an opinion of disabling the tests in post-merge in CI ? | 08:17 |
| valentind | tristan, I think gitlab has some tools to verify syntax. | 08:17 |
| tristan | valentind, well; the pipeline for the MR is running which is a good sign that it didnt choke... but yeah it would be interesting to know more than "didnt choke" | 08:18 |
| tristan | I also have a feeling we should drop the `codequality` job | 08:19 |
| tristan | it was doing cool stuff for a while, but I haven't seen it provide any meaningful feedback in a long time | 08:19 |
| tristan | always fails to load the quality report | 08:19 |
| valentind | tristan, there is this button "CI lint" | 08:20 |
| valentind | If you go to CI/CD -> Pipelines | 08:20 |
| valentind | On the top right. | 08:20 |
| *** toscalix has joined #buildstream | 08:20 | |
| *** noisecell has joined #buildstream | 08:21 | |
| tristan | valentind, ah cool thanks, lets look at that | 08:21 |
| tristan | hmmm, well it looks right | 08:23 |
| tristan | Do we know if that needs to be applied to bst-1.2 as well as master ? | 08:24 |
| tristan | Or will gitlab only care about the .gitlab-ci.yml on master ? no... it wont; as I recall I've run some tests | 08:24 |
| tristan | Has to go on both branches | 08:25 |
| valentind | We need | 08:25 |
| valentind | It reads the current commit's .gitlab-ci.yml. | 08:25 |
| valentind | Not the one on master. | 08:25 |
| tristan | Exactly | 08:25 |
| tristan | So once it's on master and bst-1.2, it will take effect on people's merge requests which have been rebased | 08:26 |
| tristan | was an idiot question :) | 08:26 |
| valentind | It is OK to think out loud. | 08:26 |
| tristan | since we regularly try some hacks in .gitlab-ci.yml :) | 08:26 |
| tristan | Unfortunate that we lose the cute coverage badge with that, though | 08:27 |
| tristan | At least we still get the important part of coverage; i.e. we can see the overall coverage result on a merge request | 08:27 |
| valentind | tristan, sorry for !701. I understood that !679 was not going to be backported. But #584 (which is only the very long wait at start) needed a fix that was exactly one of the commit of that merge request. | 08:32 |
| valentind | The extract directory, which takes much more time to scan than the cas, was scanned only at start. | 08:33 |
| valentind | And that commit stopped from scanning the extract directory. Which was all we needed for #584. | 08:34 |
| valentind | #466 was marked as 1.4 for the scope. | 08:35 |
| valentind | And !679 was a partial fix. | 08:35 |
| *** rdale has joined #buildstream | 08:36 | |
| tristan | valentind, I understand... the bigger problem with that is jonathanmaw merged that weird addition to context, and then fixed it in the following commit, failing to squash them together | 08:41 |
| tristan | The result is bad | 08:41 |
| tristan | Because then we have master and bst-1.2 with a completely different code shape | 08:42 |
| tristan | So any bugfixes which happen later on master in that area will conflict and cause friction | 08:42 |
| valentind | OK | 08:42 |
| valentind | I did not realize it. I asked him to tell me if it was fine to take only that commit to be sure. | 08:43 |
| tristan | So, better to just take the whole MR in this case; if there are issues stemming from this in master we can easily backport fixes | 08:43 |
| valentind | I agree. | 08:43 |
| qinusty | Can I get a review on https://gitlab.com/BuildStream/buildstream/merge_requests/700 from someone who can spare some time? | 09:00 |
| gitlab-br-bot | buildstream: merge request (tristan/artifact-expiry-1.2->bst-1.2: Remove artifact extracts when artifact expires in cache) #708 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/708 | 09:03 |
| *** CTtpollard has quit IRC | 09:05 | |
| *** jonathanmaw has joined #buildstream | 09:06 | |
| *** tpollard has quit IRC | 09:06 | |
| *** tpollard has joined #buildstream | 09:09 | |
| gitlab-br-bot | buildstream: issue #561 ("The artifacts/extract directory is never cleaned") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/561 | 09:10 |
| gitlab-br-bot | buildstream: issue #561 ("The artifacts/extract directory is never cleaned") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/561 | 09:10 |
| gitlab-br-bot | buildstream: merge request (valentindavid/extract-expiry->master: Remove artifact extracts when artifact expires in cache) #685 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/685 | 09:10 |
| gitlab-br-bot | buildstream: merge request (tristan/blessings->master: deps: Specify the minimum version required for blessings) #664 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/664 | 09:12 |
| gitlab-br-bot | buildstream: merge request (tristan/blessings->master: deps: Specify the minimum version required for blessings) #664 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/664 | 09:12 |
| qinusty | coldtom can you explain https://gitlab.com/BuildStream/buildstream/issues/600 a little more? I'm unable to checkout that commit (perhaps it was rebased over?) Which variable was recursive? | 09:14 |
| gitlab-br-bot | buildstream: merge request (tristan/reduce-gitlab-ci->master: WIP: .gitlab-ci.yml: Avoid running tests in post-merge) #709 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/709 | 09:14 |
| gitlab-br-bot | buildstream: merge request (tristan/reduce-gitlab-ci->master: .gitlab-ci.yml: Avoid running tests in post-merge) #709 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/709 | 09:15 |
| coldtom | qinusty: it was rebased over, prefix is recursive as libdir is defined using prefix | 09:16 |
| gitlab-br-bot | buildstream: merge request (tristan/reduce-gitlab-ci-1.2->bst-1.2: .gitlab-ci.yml: Avoid running tests in post-merge (1.2)) #710 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/710 | 09:18 |
| tristan | Any last words before losing post-merge CI ? | 09:19 |
| tristan | i.e. https://gitlab.com/BuildStream/buildstream/merge_requests/709 | 09:20 |
| qinusty | reproduced it coldtom, nearly crashed my pc | 09:22 |
| coldtom | qinusty: hahaha yep, it's quite a spectacular break | 09:22 |
| valentind | What crash? | 09:35 |
| coldtom | valentind: https://gitlab.com/BuildStream/buildstream/issues/600 | 09:35 |
| valentind | qinusty, are you on it? | 09:36 |
| qinusty | Looking into it | 09:36 |
| qinusty | It's just that our variable resolution has no consideration of recursive variables | 09:37 |
| *** toscalix has quit IRC | 09:37 | |
| coldtom | to be fair, you weren't expecting someone to be so foolish as to use them :P | 09:37 |
| valentind | Well, maybe we want to have conditionals in macro expansion at some point, which then would make sense to have recursion. | 09:38 |
| * qinusty shrugs | 09:38 | |
| qinusty | For now, I reckon we need to raise an exception on them. They're unhandled and pretty much instantly freeze up my machine | 09:39 |
| *** toscalix has joined #buildstream | 09:39 | |
| *** toscalix has quit IRC | 09:46 | |
| *** toscalix has joined #buildstream | 09:47 | |
| tristan | What is a recursive variable ?? | 09:51 |
| tristan | qinusty, coldtom that seems like a regression; I'm quite certain we had a check for that in place at some point | 09:52 |
| qinusty | We check for cyclic | 09:52 |
| qinusty | But... It is just checking two lists for equality. | 09:52 |
| coldtom | tristan: I had prefix essentially defined as %{prefix}/foo | 09:53 |
| tristan | coldtom, Ah | 09:53 |
| tristan | Yeah that's not allowed, so we do check for cyclic, but not well enough | 09:53 |
| tristan | That needs to fire an appropriate LoadError() and test in tests/format/variables.py | 09:54 |
| qinusty | Indeed | 09:54 |
| * tristan doesnt see where the test for cyclic is | 09:55 | |
| tristan | I guess we raise the UNRESOLVED_VARIABLE reason for that from Variables._resolve() ? | 09:56 |
| qinusty | Basically tristan, the whole unmatched == last_matched doesn't work because the lists are changing each loop | 09:58 |
| gitlab-br-bot | buildstream: merge request (juerg/cas-1.2->bst-1.2: CAS: Fix resource_name format for blobs) #711 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/711 | 09:59 |
| tristan | qinusty, that "doesn't work" ? or it just doesnt work under this specific circumstance ? | 10:00 |
| tristan | I think that works | 10:00 |
| qinusty | Hmmmm, well all I'm seeing it. Each iteration the list is switching up and elements are moving places | 10:00 |
| coldtom | is there an equivalent to ostree checkout with the cas cache? | 10:00 |
| qinusty | we could compare them as a set? Does the number of elements matter? | 10:00 |
| juergbi | tristan: is 1.1.7 release planned soon? I'd like to get the above backport in before that to fix the CAS URLs (as discussed last week) | 10:01 |
| *** tpollard has quit IRC | 10:01 | |
| *** tpollard has joined #buildstream | 10:03 | |
| tristan | qinusty, it indeed looks a bit wonky, but I think the logic of "So long as there are unmatched variables, try to resolve one... and if resolving one did not have a result, this indicates a cyclic variable which could not be resolved" is fairly sound | 10:04 |
| tristan | rather, it looks to me that resolve_one() is overstepping and resolving more than one | 10:05 |
| tristan | but yeah, looks like that has been a bit broken :-S | 10:05 |
| tristan | juergbi, today | 10:05 |
| valentind | %{prefix}/foo gets resolved to %{prefix}/foo/foo | 10:05 |
| qinusty | And then repeat | 10:06 |
| tristan | valentind, that is a problem, at the point we encounter that we should raise an error right away | 10:06 |
| qinusty | foo/foo/foo/foo/foo | 10:06 |
| juergbi | ok, CI is running for !711 | 10:06 |
| tristan | qinusty, that's what I'm saying | 10:06 |
| qinusty | Basically IF variable = "%{variable}/whatever | 10:07 |
| qinusty | bail out | 10:07 |
| tristan | qinusty, i.e. for *that* case; we should handle it and raise the error while trying to resolve %{prefix} | 10:07 |
| tristan | qinusty, the outer loop will catch the recursive problem properly (or is intended to), when it involves more than one variable | 10:07 |
| tristan | juergbi, it'll need a rebase in... around 10 or 15 min... | 10:08 |
| tristan | but yeah, lets get that in and call it 1.1.7 at that point | 10:09 |
| tristan | I'm not sure I can squeeze anything else into 1.1.7 | 10:09 |
| juergbi | ok | 10:09 |
| coldtom | qinusty: in that approach you need to make sure the recursion is nowhere in the variable, not just at the start | 10:11 |
| qinusty | Indeed | 10:11 |
| juergbi | jjardon: heads up, 1.1.7 will require artifact server update. older clients will still work with the 1.1.7 server but 1.1.7+ clients will not work with 1.1.6 and older servers | 10:11 |
| tristan | qinusty, valentind: I.e. consider; "foo: %{bar}", "bar: %{baz}", "baz: %{bar}" - now that I see it, it's not really recursion | 10:11 |
| tristan | but that is supposed to be handled | 10:11 |
| valentind | Yes. I understood this one. | 10:11 |
| juergbi | jjardon: are you handling server updates or who needs to be informed? | 10:11 |
| tristan | valentind, is it possible to have this recursion, with more than one variable ? | 10:12 |
| gitlab-br-bot | buildstream: merge request (tristan/reduce-gitlab-ci->master: .gitlab-ci.yml: Avoid running tests in post-merge) #709 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/709 | 10:13 |
| valentind | tristan, like that? a=%{b}/c and b=%{a}/d | 10:13 |
| coldtom | tristan: in my actual case it was prefix: foo, bar: %{prefix}/baz, prefix: %{bar}/bas | 10:14 |
| *** ChanServ sets mode: +o jjardon | 10:15 | |
| jjardon | juergbi: I will take care of it, thanks. Can you please update the NEWS file so we do not forget about that? Also, in your sentence: "1.1.7+ clients will not work with 1.1.6 and older servers" only to re-check: Is that clients >= 1.1.7 or > 1.1.7 | 10:16 |
| juergbi | client >= 1.1.7 | 10:17 |
| juergbi | ok, will add a NEWS entry | 10:18 |
| jjardon | tristan: the MR looks ok, but what is the benefit of doing that? | 10:20 |
| tristan | jjardon, it avoids running too many pipelines | 10:20 |
| jjardon | tristan: why is that a problem? | 10:21 |
| tristan | jjardon, right now CI is unbearable; it is averaging at around 1 hour; but with the whole thing running in parallel with redundant pipelines, they can crawl down to 2 hours | 10:21 |
| tristan | jjardon, this is probably the biggest problem with the project now, it should be taking 15min all the time | 10:21 |
| jjardon | tristan: that is not possible, every job is independent | 10:21 |
| jjardon | doesnt matter how many pipelines are running | 10:22 |
| tristan | Yesterday I had one hundred and twenty three minutes and fourty one seconds (!) | 10:22 |
| tristan | So, it's certainly possible | 10:22 |
| jjardon | oh yeah, its possible a job is slow, but that MR is not going to fix that | 10:22 |
| jjardon | every job run in a indeendent machine | 10:23 |
| tristan | jjardon, persia is looking into organizing a meeting, I think he will invite you; we also have to look into getting dedicated machines and drop this "ephemeral" stuff | 10:23 |
| tristan | At which point, it will also be more expensive to run these redundant jobs too | 10:23 |
| jjardon | sure, at that point yes but not in the current config | 10:24 |
| tristan | I'm hoping that happens this week; but worry it might take a very long time :'( | 10:24 |
| tristan | Current situation is unlivable | 10:25 |
| jjardon | well, let's have that meeting today then? | 10:25 |
| jjardon | is there an issue open somewhere to explain the current problem? | 10:25 |
| tristan | I asked persia to coordinate this for me, there is an action item from the meeting minutes | 10:26 |
| tristan | An issue could be filed in nosoftware I guess, if people like issues for that sort of thing | 10:26 |
| tristan | I dont really mind as long as we get it solved | 10:26 |
| jjardon | open and issue please, explain the problem and I will try to provide some solutions there | 10:27 |
| tristan | jaysus | 10:27 |
| tristan | Okay... | 10:27 |
| jjardon | but yeah take in account that MR you created is not going to help current slow jobs at all (I think you already know that but only to confirm) | 10:28 |
| tristan | no appropriate group for that here: https://gitlab.com/BuildStream/nosoftware/ | 10:28 |
| tristan | jjardon, sure, but you came in a few hours too late and spoke up after the last call, I wont bother reverting now | 10:29 |
| tristan | I asked about this last night too before leaving, nobody spoke up | 10:29 |
| jjardon | tristan: sure | 10:29 |
| jjardon | maybe open it at https://gitlab.com/BuildStream/buildstream and label it as "infra" ? | 10:29 |
| tristan | Yeah about to do that | 10:29 |
| tristan | toscalix, maybe you'd like to open a new subgroup called "infrastructure" to avoid crowding the software issue tracker with infra related issues ? | 10:30 |
| toscalix | tristan: let me think a little about it | 10:31 |
| toscalix | infra is "service" and will end up having code (configs) so probably nosoftware subgroup is not the right place | 10:32 |
| toscalix | a new subgroup could be the right place | 10:32 |
| persia | Lots of projects have separate "infra" repos indeed. | 10:33 |
| toscalix | I do not think buildstream repo is the right place. We should not mix development and service and expect the same "gitlab structure" to fit both well | 10:33 |
| toscalix | so right now I would say: create a new subgroup. But let me finish what I am doing and then I will propose it to the list | 10:34 |
| toscalix | maybe somebody else have objections | 10:34 |
| gitlab-br-bot | buildstream: issue #603 ("CI takes a very long time") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/603 | 10:38 |
| tristan | I've filed: https://gitlab.com/BuildStream/buildstream/issues/603 for now | 10:39 |
| tristan | Before we had fancy subgroups, we used the infra label; so went with that to get the red tape out of my hair | 10:39 |
| tristan | But indeed, I think it would be good that since we have all these new subgroups for issues... we create one for this | 10:40 |
| tristan | persia, do you really have no gitlab account ? | 10:40 |
| * tristan surprised he doesnt find persia as a project member | 10:40 | |
| persia | Yes. The authentication registration system hates me. | 10:41 |
| tristan | jjardon, https://gitlab.com/BuildStream/buildstream/issues/603 | 10:41 |
| * persia hasn't tried in a few weeks, so will try again today to see if anything changed to make it work | 10:41 | |
| tristan | oh well, here is another case of juergbi's patch seeming to beat mine to the punch, even though my pipeline started > 30min before | 10:44 |
| valentind | tristan, for #316, it seems to me there are situations with --no-strict and some workspace opened or elements were modified, that we build something and we do not have a strong key. | 10:44 |
| tristan | let's see who wins | 10:44 |
| valentind | So we cannot commit to the strong key. | 10:44 |
| valentind | Even though we can build. | 10:44 |
| jjardon | tristan: cheers, let's identify what is actually slow: I guess we can pick any pipeline as all of them are slow? | 10:46 |
| tristan | valentind, I can see you commented on the issue; I don't have space to discuss this randomly - the most useful thing you can do is capture all of the actual information which lead you to that conclusion very clearly (better yet an isolated test case), and evolve this issue in the tracker | 10:46 |
| valentind | tristan, I am planning to write a test case. | 10:46 |
| tristan | jjardon, My guess is that (A) The SSD is not guaranteed to be on the same physical machine... and (B) the gitlab cache could be made more reliable and persistent | 10:47 |
| tristan | jjardon, just keep in mind that our tests are very I/O intensive; so any slow down between CPU/RAM/SSD is what kills performance | 10:48 |
| jjardon | tristan: it has been always this slow (~2h) or something changed recently ? | 10:49 |
| qinusty | It depends on the time of day in my experience | 10:49 |
| qinusty | Sometimes it's as fast as 30m | 10:49 |
| jjardon | looking at https://gitlab.com/BuildStream/buildstream/pipelines is normally ~1h | 10:51 |
| jjardon | It was always been around that time to run the tests? | 10:52 |
| cs-shadow | i remember it being ~30 mins | 10:52 |
| jjardon | Have anyone checked if the gitlab cache is actually working? ie cache/integration-cache is not empty? Is this meant to be big? | 10:59 |
| *** jonathanmaw_ has joined #buildstream | 11:02 | |
| *** jonathanmaw has quit IRC | 11:02 | |
| gitlab-br-bot | buildstream: merge request (tristan/reduce-gitlab-ci-1.2->bst-1.2: .gitlab-ci.yml: Avoid running tests in post-merge (1.2)) #710 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/710 | 11:04 |
| gitlab-br-bot | buildstream: merge request (juerg/cas-1.2->bst-1.2: CAS: Fix resource_name format for blobs) #711 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/711 | 11:05 |
| tristan | jjardon, it was down to 15min at one point in the past when ssssam[m] migrated integration tests to use the alpine linux tarball | 11:09 |
| tristan | jjardon, it went up to averaging an hour probably shortly after adding the flatpak example | 11:09 |
| *** jonathanmaw_ is now known as jonathanmaw | 11:09 | |
| jjardon | tristan: probably that's the main problem: it has to download the artifacts every single time | 11:11 |
| tristan | jjardon, no artifacts | 11:11 |
| tristan | well, *gitlab* artifacts maybe you mean ? | 11:11 |
| tristan | We don't cache artifacts in the gitlab cache | 11:11 |
| tristan | Only sources | 11:11 |
| jjardon | tristan: setup wise nothing has change since ssssam[m] did that, so I guess you do much more stuff now | 11:12 |
| jjardon | tristan: wherever is needed to build the flatpak example | 11:12 |
| tristan | https://gitlab.com/BuildStream/buildstream/blob/master/doc/examples/flatpak-autotools/elements/base/sdk.bst | 11:15 |
| tristan | jjardon, freedesktop SDK 1.4 | 11:15 |
| tristan | jjardon, I'm not sure that it's redownloading every time; the fact that it needs to run the import step and create the artifact can be the bottleneck | 11:16 |
| tristan | but at least last week we had redownloads happening, otherwise we would not have hit that problem with the autotools tarball from ftp.gnu.org | 11:17 |
| gitlab-br-bot | buildstream: merge request (Qinusty/600-recursive-variables->master: WIP: Add cyclic check within variable resolution) #712 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/712 | 11:18 |
| qinusty | initial fix for recursive variable declaration. https://gitlab.com/BuildStream/buildstream/merge_requests/712 Anyone see an issue with this? | 11:19 |
| jjardon | tristan: that can be one of the bottlenecks indeed (I would say the biggest probably); if the SDK is not being cached you are downloading it every single time | 11:19 |
| * qinusty goes for lunch | 11:19 | |
| tristan | jjardon, note from one of the example pipelines I posted: https://gitlab.com/BuildStream/buildstream/-/jobs/91382655 | 11:25 |
| tristan | That job took 145 minutes... but it says: | 11:26 |
| tristan | Checking cache for tests-fedora-27--4... | 11:26 |
| tristan | Successfully extracted cache | 11:26 |
| tristan | So, it appears that it did get the cache | 11:26 |
| jjardon | tristan: sure, but maybe is empthy | 11:27 |
| tristan | jjardon, if it is cached properly; then the bottleneck is interaction with the SSD, just doing the `ostree checkout` step to sync a few GB to disk can take a very long time | 11:27 |
| tristan | Right, could be, I don't know | 11:27 |
| jjardon | mmm, is a pitty we do not have timestamps there to check if it got stuck in a specific test or something | 11:28 |
| * paulsherwood suppresses a chuckle | 11:28 | |
| tristan | paulsherwood, sure... but we're talking about timestamps issued by pytest here, which doesnt time the individual tests | 11:29 |
| paulsherwood | bug on them them | 11:29 |
| paulsherwood | then | 11:29 |
| paulsherwood | sorry ... that's a bug, on them, then | 11:30 |
| tristan | would be nice, it does time the entire suite - maybe there is an option we didnt opt into even | 11:30 |
| tristan | https://stackoverflow.com/questions/27884404/printing-test-execution-times-and-pinning-down-slow-tests-with-py-test | 11:31 |
| tristan | Looks like we can add --durations=0 to get the completion times of all tests (I think it will be reported after everything completes, not in line) | 11:31 |
| persia | Maybe that should be enabled as soon as? | 11:32 |
| tristan | yeah | 11:32 |
| paulsherwood | can it be turned on for always somehow? | 11:32 |
| tristan | persia, I'm running two full tests on my laptop ... waiting the the first to complete | 11:32 |
| tristan | paulsherwood, it can be enabled in our setup.cfg yes | 11:32 |
| tristan | lemme see what it looks like, just want the first test to complete so I can see how long it takes with my cache populated | 11:33 |
| persia | tristan: For clarity, when I suggest "as soon as" without referent, I very much don't mean "now", but "when it can be done without breaking whatever other things are happening" | 11:33 |
| tristan | persia, yeah but I presume that it's harmless :) | 11:33 |
| tristan | technically I'm just waiting for juergbi's https://gitlab.com/BuildStream/buildstream/merge_requests/711 to pass so I can wrap the release | 11:34 |
| persia | In this context, it might be useful if "pres" was something that had stronger negative semantic value. | 11:34 |
| persia | I'm personally in favour of avoiding production changes until they are tested: if we change that config, is there a way to test the change before it applies to all new tests? | 11:35 |
| tristan | you mean adding the option to pytest, to print the summary of durations of all tests ? | 11:36 |
| tristan | persia, that configuration is a part of our repo, so it is gated on pre-merge CI like everything else | 11:36 |
| persia | Ah, excellent. Yeah, let's queue up that pre-merge test then :) | 11:37 |
| coldtom | hi, is there a way i can checkout an artifact by cache key? analagous to ostree checkout? | 11:37 |
| tristan | coldtom, not yet :'( | 11:38 |
| jmac | I believe finn has an external tool that can do that | 11:38 |
| tristan | coldtom, that is part of https://mail.gnome.org/archives/buildstream-list/2018-July/msg00051.html | 11:38 |
| jmac | I also wrote one at https://gitlab.com/jmacarthur/cas-inspector but I think finn's is better | 11:39 |
| tristan | specing that out into issues is high up on my todo list, but still havent reached it :-S | 11:39 |
| coldtom | that feature will be great! | 11:41 |
| *** solid_black has joined #buildstream | 11:41 | |
| coldtom | getting to the build logs of a given element w/ a given cache key is something i've been missing but didn't realise | 11:41 |
| persia | It's one of the more frequent requests I've seen recently from freedesktop SDK contributors in this channel. | 11:42 |
| persia | Might be worth documenting how to use one of finn's or jmac's tools as a short-term workaround until tristan's proposal is implemented. | 11:43 |
| tristan | persia, indeed, I wouldn't be against that - *however* it requires a nice bold statement that the artifact structure is not guaranteed stable | 11:44 |
| tristan | persia, those tools are basically low level CAS relates afaik (they will get you the whole artifact) | 11:44 |
| tristan | then we run the risk of people consuming the metadata manually instead of using well defined CLI APIs that we can guarantee stability of | 11:45 |
| persia | tristan: Cool, yes, yes, and probably need a bit extra to cover the "recover logs" use case (which is the one I see asked about). | 11:45 |
| persia | Well, yes, but the last alternate workaround I saw involved rebuilding all the packages from source on someone's laptop after clearing and disabling cache. That seems a worse short-term workaround. | 11:46 |
| tristan | I'm seeing a redundancy in the tests here | 11:47 |
| persia | (and problems trying to implement this workaround were part of the "force buildstream to rebuild this package rather than get it from cache" issue) | 11:47 |
| *** toscalix has quit IRC | 11:47 | |
| tristan | integration-cache/sources/tar/sysroot_tarballs_integration_tests_base_v1_x86_64_tar_xz/ and integration-cache/sources/tar/alpine_integration_tests_base_v1_x86_64_tar_xz/ | 11:48 |
| tristan | same sha, but we're caching it twice | 11:48 |
| tristan | I think it means we're using a different alias name in the integration tests, and in the examples | 11:48 |
| *** toscalix has joined #buildstream | 11:48 | |
| tristan | small(ish) optimization is possible there anyway | 11:48 |
| persia | Significant IO reduction potentially, and I continue to believe the slowdown is due to IO limitations (although I'd like to have enough data to know this, rather than simply believing it) | 11:49 |
| toscalix | build times in buildstream master meeting scheduled. Feel free to add more guests yourself ironfoot is invited as optional. Up to him to participate. I have pinged Codethink ops in case they can/want to join to provide advice. In such short notice, I doubt they can but still... | 11:54 |
| toscalix | hummm ... master and release branch | 11:55 |
| gitlab-br-bot | buildstream: merge request (juerg/cas-1.2->bst-1.2: CAS: Fix resource_name format for blobs) #711 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/711 | 11:58 |
| toscalix | moved the meeting into the buildstream calendar, for the record | 12:00 |
| tristan | valentind, tests/integration/source-determinism.py does not use cli_integration | 12:05 |
| tristan | valentind, which means it does not benefit from the shared cache, and re-downloads the alpine sysroot tarball for each test (ouch) | 12:05 |
| tristan | that'll slow things down significantly | 12:06 |
| jjardon | tristan: you indeed have a different cache for every job: https://gitlab.com/BuildStream/buildstream/blob/master/.gitlab-ci.yml#L4 | 12:11 |
| tristan | jjardon, so I have a different cache for debian-9, etc etc, is that a bad thing ? | 12:15 |
| tristan | or is it unique for every run ? | 12:16 |
| tristan | that commit goes back quite a while, january this year | 12:17 |
| jjardon | Tristan no, not a bad thing | 12:19 |
| jjardon | It means every job it has its own cache (shared between builds) | 12:20 |
| tristan | right, I think that is pretty fine | 12:20 |
| tristan | and must have been intentional | 12:20 |
| tristan | what we do in different jobs can differ, it might be especially important to not mix the linux jobs with the unix one | 12:21 |
| jjardon | Yup | 12:22 |
| *** WSalmon has quit IRC | 12:23 | |
| *** WSalmon has joined #buildstream | 12:23 | |
| gitlab-br-bot | buildstream: merge request (BenjaminSchubert/fix-quota-tests->master: Mock storage space checks for tests.) #702 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/702 | 12:24 |
| persia | One potential issue with all the caches: is there enough total space reserved to store that much? | 12:26 |
| tristan | it looks like less than I suspected... a du -hs of the integration-cache after a run was ~500MB only | 12:26 |
| paulsherwood | shouldn't they cull? | 12:27 |
| tristan | paulsherwood, I don't know how it works exactly, but it seems they automatically append a number to the cache name; so there is definitely some cycling going on there | 12:28 |
| tristan | "Checking cache for tests-fedora-28--4" | 12:29 |
| tristan | the "-4" is added by gitlab | 12:29 |
| paulsherwood | ewww | 12:29 |
| tristan | I think it's anyway needed because one run can cause the cache to change (say, when we download something new) | 12:30 |
| tristan | well, the run took 20min here, but I have to figure out what to do with tests/integration/source-determinism.py, my fix to avoid redundant downloads didn't work | 12:33 |
| tristan | valentind, is there any reason you chose to do that as an integration test ? | 12:33 |
| tristan | valentind, I thought that was about staging, and didnt really require anything else than an import element to test each source | 12:34 |
| tristan | anyway -> 1.1.7 time | 12:34 |
| toscalix | tristan: able to join? | 12:35 |
| tristan | Also bad news, the --durations=0 shows really unuseful information | 12:35 |
| tristan | toscalix, is that now ? I wasn't planning on it really; I kind of punted that to persia in the monthly meeting hoping others would handle the issue | 12:35 |
| tristan | qinusty, you seem to keep popping up as multiple authors in the commit log | 12:39 |
| qinusty | hmm | 12:39 |
| qinusty | How so? | 12:39 |
| * qinusty recently changed email on his commits | 12:40 | |
| qinusty | Also on separate machines which may use different emails. But they should all be registered with my gitlab account | 12:40 |
| tristan | qinusty, it might be from backports | 12:44 |
| tristan | just pointing it out, for instance, see `git shortlog -s 1.1.6...bst-1.2` | 12:44 |
| * qinusty only sees one Josh Smith. Sees both Will and William Salmon | 12:45 | |
| qinusty | Qinusty is there for other patches though I see | 12:46 |
| *** finn has quit IRC | 12:47 | |
| *** finn has joined #buildstream | 12:50 | |
| persia | Consistency is good. One task that ends up frustratingly common is preparing a list of unique contributors, for which inconsistency is confusing. That said, *meaningful inconsistency* is important (e.g. if someone worked at foo, and moves to bar, it is important to track jrdev@foo and jrdev@bar separately; unless jrdev has permission to use jrdev@jrdev.io for their work). | 12:50 |
| * qinusty will look into where his name is changing | 12:51 | |
| *** tristan changes topic to "BuildStream 1.1.7 is out ! | https://gitlab.com/BuildStream/buildstream | Docs: https://buildstream.gitlab.io/buildstream | IRC logs: https://irclogs.baserock.org/buildstream | Roadmap: https://wiki.gnome.org/Projects/BuildStream/Roadmap" | 12:51 | |
| persia | \o/ 1.1.7 | 12:52 |
| gitlab-br-bot | buildstream: merge request (jmac/tempfile-extraction-bug->master: WIP: Correct crash after staging tars with read-only directories) #713 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/713 | 12:53 |
| tristan | cs-shadow, so how about we start testing with 1.1.7 ? | 13:02 |
| tristan | <tristan> cs-shadow, pypi.org: tristan, test.pypi.org: tristanvb | 13:03 |
| tristan | Incase you missed that from this morning ^^^^ | 13:03 |
| cs-shadow | tristan: sounds good. I may have found an issue with our MANIFEST.in that I'm trying to correctly reproduce at the moment | 13:03 |
| tristan | Sure :) | 13:03 |
| cs-shadow | let me just fix that up and then we should be able to do a test release | 13:03 |
| tristan | hmmm, that's unfortunate | 13:04 |
| tristan | I was hoping we'd be testing with the actual 1.1.7 release | 13:04 |
| cs-shadow | oh, we can do that as well. I meant we could do a test release first and then do the real one since we only get one chance to upload a given version | 13:05 |
| tristan | cs-shadow, ah yeah; that's why we have the test.pypi.org | 13:05 |
| cs-shadow | tristan: yeah, by test release i just meant releasing it on test.pypi | 13:06 |
| tristan | cs-shadow, ok well, I'll be out shortly; so let me know if it works :) | 13:06 |
| cs-shadow | then verifying that it installs correctly and then doing the real thing | 13:06 |
| tristan | cs-shadow, will the process modify the release tarball somehow ? | 13:07 |
| tristan | cs-shadow, that is already CI'd to work | 13:07 |
| tristan | cs-shadow, I am going off the assumption that we feed in https://download.gnome.org/sources/BuildStream/1.1/BuildStream-1.1.7.tar.xz to the PyPI stuff | 13:07 |
| tristan | still, of course an additional smoke test doesnt hurt at all | 13:08 |
| cs-shadow | tristan: that should work, just to clarify, it was built using `setup.py sdist`, right? | 13:09 |
| tristan | cs-shadow, yep | 13:10 |
| tristan | cs-shadow, and similar to here: https://gitlab.com/BuildStream/buildstream/blob/master/.gitlab-ci.yml#L35 | 13:10 |
| tristan | cs-shadow, note that all the tests we run, already run against an installation of an sdist we unpack in the first CI step | 13:10 |
| tristan | so tests should pass, and docs should build, from any release tarball | 13:11 |
| cs-shadow | tristan: cool, then we should be able to upload it test.pypi now | 13:12 |
| cs-shadow | tristan: do you want to do that or should i do it? | 13:12 |
| tristan | cs-shadow, I'm pretty much already gone by now | 13:14 |
| cs-shadow | tristan: no worries, i can do that. Do you want me to upload it to actual pypi if things work or should we wait until tomorrow? | 13:15 |
| *** flatmush1 has joined #buildstream | 13:17 | |
| tristan | cs-shadow, well, it's going to be 1.1.7 anyway; so lets say: If you can upload it to test, and then install it locally via pip; and it basically works, then "yes" | 13:17 |
| tristan | cs-shadow, if there is some issue we find, it will be fixed in 1.1.8 | 13:17 |
| cs-shadow | tristan: cool, I'll post my findings on the GitLab issue in that case | 13:17 |
| tristan | cs-shadow, this way we get it off the ground and can land the important docs patches | 13:17 |
| *** flatmush has quit IRC | 13:18 | |
| tristan | and they will mean something right away :) | 13:18 |
| cs-shadow | sounds good | 13:18 |
| * cs-shadow goes back to chasing his heisenbug | 13:18 | |
| *** tristan has quit IRC | 13:29 | |
| *** tristan has joined #buildstream | 13:35 | |
| *** ChanServ sets mode: +o tristan | 13:36 | |
| *** xjuan has joined #buildstream | 13:48 | |
| jmac | I can't seem to run an individual test from the test suite. I'm trying ./setup.py test --addopts '-k tests/frontend/buildtrack.py::test_build_track' as per HACKING.rst but it deselects everything. | 14:01 |
| jjardon | qinusty: do you have a ssh key you can share with me? | 14:12 |
| jjardon | so I can give you access to the new machine for testing CI times | 14:12 |
| qinusty | pm'd | 14:13 |
| *** toscalix has quit IRC | 14:14 | |
| jjardon | qinusty: thanks. So any idea if it is possible to add some timestamps to the build logs? https://gitlab.com/BuildStream/buildstream/issues/603#note_96430154 | 14:15 |
| *** toscalix has joined #buildstream | 14:16 | |
| qinusty | Unsure, I'm looking into profiling tests at the minute | 14:16 |
| qinusty | Which could give us a much better understanding of how tests run both locally and through CI | 14:16 |
| jennis | jmac: ./setup.py test 'path/to/test/foo.py' | 14:16 |
| jjardon | It's a pity this is not provided by gitlab already: https://gitlab.com/gitlab-org/gitlab-ce/issues/22745 | 14:17 |
| qinusty | Indeed | 14:17 |
| jennis | jmac: ./setup.py test 'path/to/test/foo.py::test_foo_bar' | 14:17 |
| jmac | Aha | 14:17 |
| qinusty | jmac perhaps integration? | 14:17 |
| jmac | HACKING.rst needs an update then | 14:17 |
| jennis | yeah I noticed that earlier | 14:17 |
| jennis | The `-k` screws things | 14:17 |
| jennis | right? | 14:17 |
| jmac | Yep | 14:18 |
| jjardon | qinusty: would not be better to have some real numbers first, of what is actually slow? | 14:18 |
| qinusty | Profiling will give us exactly that | 14:18 |
| jjardon | maybe the tests are actually fast but we spend a lot of time getting artifacts | 14:19 |
| jjardon | ah ok | 14:19 |
| jennis | And jmac, I missed an --addopts there... | 14:19 |
| jmac | It's ok, I figured that out | 14:19 |
| *** solid_black_ has joined #buildstream | 14:22 | |
| *** tintou_ has joined #buildstream | 14:22 | |
| *** xjuan has quit IRC | 14:23 | |
| *** solid_black has quit IRC | 14:23 | |
| *** rdale has quit IRC | 14:23 | |
| *** alatiera_ has quit IRC | 14:23 | |
| *** jennis has quit IRC | 14:23 | |
| *** qinusty has quit IRC | 14:23 | |
| *** coldtom has quit IRC | 14:23 | |
| *** adds68 has quit IRC | 14:23 | |
| *** mohan43u has quit IRC | 14:23 | |
| *** tlsa has quit IRC | 14:23 | |
| *** johnward has quit IRC | 14:23 | |
| *** laurence has quit IRC | 14:23 | |
| *** tintou has quit IRC | 14:23 | |
| *** gitlab-br-bot has quit IRC | 14:23 | |
| *** jmac has quit IRC | 14:23 | |
| *** lantw44 has quit IRC | 14:23 | |
| *** Trevinho has quit IRC | 14:23 | |
| *** tintou_ is now known as tintou | 14:23 | |
| *** alatiera_ has joined #buildstream | 14:23 | |
| *** solid_black_ is now known as solid_black | 14:24 | |
| *** rdale has joined #buildstream | 14:25 | |
| *** tlsa has joined #buildstream | 14:25 | |
| *** jmac has joined #buildstream | 14:25 | |
| *** mohan43u has joined #buildstream | 14:26 | |
| *** laurence has joined #buildstream | 14:28 | |
| *** johnward has joined #buildstream | 14:36 | |
| *** xjuan has joined #buildstream | 14:36 | |
| *** lantw44 has joined #buildstream | 14:36 | |
| *** jennis has joined #buildstream | 14:37 | |
| *** qinusty has joined #buildstream | 14:37 | |
| *** adds68 has joined #buildstream | 14:37 | |
| *** coldtom has joined #buildstream | 14:37 | |
| *** gitlab-br-bot has joined #buildstream | 14:39 | |
| gitlab-br-bot | buildstream: issue #604 ("Cache not storing objects, but storing refs") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/604 | 14:39 |
| *** coldtom has quit IRC | 14:41 | |
| *** adds68 has quit IRC | 14:41 | |
| *** qinusty has quit IRC | 14:41 | |
| *** jennis has quit IRC | 14:41 | |
| *** lantw44 has quit IRC | 14:41 | |
| *** xjuan has quit IRC | 14:41 | |
| *** johnward has quit IRC | 14:41 | |
| *** johnward has joined #buildstream | 14:47 | |
| *** qinusty has joined #buildstream | 14:50 | |
| *** jennis has joined #buildstream | 14:50 | |
| *** coldtom has joined #buildstream | 14:50 | |
| *** adds68 has joined #buildstream | 14:50 | |
| *** lantw44 has joined #buildstream | 14:50 | |
| *** xjuan has joined #buildstream | 14:52 | |
| *** leopi has quit IRC | 14:53 | |
| *** Trevinho has joined #buildstream | 14:55 | |
| *** Trevinho has quit IRC | 15:05 | |
| *** Trevinho has joined #buildstream | 15:05 | |
| gitlab-br-bot | buildstream: merge request (jmac/tempfile-extraction-bug->master: WIP: Correct crash after staging tars with read-only directories) #713 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/713 | 15:10 |
| gitlab-br-bot | buildstream: merge request (jmac/tempfile-extraction-bug->master: Correct crash after staging tars with read-only directories) #713 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/713 | 15:11 |
| valentind | tristan, the problem is that I wanted to test that the filesystem was correct in the container. I could have made a plugin specially for that, to report the stat of the files. But I remember the last time where I had a specific plugin for testing, you were against it. | 15:11 |
| gitlab-br-bot | buildstream: merge request (jmac/tempfile-extraction-bug->master: Correct crash after staging tars with read-only directories) #713 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/713 | 15:11 |
| valentind | And import is not good here. It strips information away that I need. | 15:11 |
| *** Trevinho is now known as Trevinyo | 15:12 | |
| *** Trevinyo is now known as _3v1n0_ | 15:12 | |
| jmac | jennis: !713 ^ is ready for review if you have a couple of minutes | 15:13 |
| jennis | jmac, nice, I'll test it now :) | 15:16 |
| *** _3v1n0_ is now known as marco | 15:17 | |
| *** marco is now known as Trevinho | 15:18 | |
| qinusty | https://gitlab.com/BuildStream/buildstream/merge_requests/712 is also out there for people to take a look at, perhaps coldtom | 15:29 |
| coldtom | looks good, qinusty | 15:32 |
| valentind | qinusty, I am not that sure. | 15:34 |
| valentind | Are not there cases where you need to resolve the same variable several time but at different times. | 15:35 |
| valentind | For example you have "some %{A} and some %{B}", where B is defined as "something with %{A}". | 15:35 |
| *** flatmush1 has quit IRC | 15:36 | |
| valentind | Ah maybe I am wrong. | 15:36 |
| coldtom | i think that should be covered by _subst shouldn't it? | 15:37 |
| *** flatmush has joined #buildstream | 15:38 | |
| valentind | Yes. I see. That should work. | 15:40 |
| jennis | jmac, thanks, the patched has fixed my problem! However, I'm concerned about the additional time this takes to run, which I believe will now be the case for *all* imports? | 15:46 |
| jennis | the patch* | 15:46 |
| jennis | I've left a comment on the MR with some numbers | 15:47 |
| jmac | It's entirely possible | 15:49 |
| jmac | Ugh, yeah, that's a massive slowdown | 15:50 |
| *** leopi has joined #buildstream | 15:52 | |
| jmac | Hmm, I might have an idea | 15:53 |
| tristan | valentind, got it, anyway when we fix it to use the cli_integration (avoiding the pointless redundant downloads of the sysroot), then the test case's remove_artifact() thing is broken | 16:01 |
| tristan | valentind, also it is probably already broken anyway as it was not removing the extract dir, so probably falsifying any tests which use that | 16:02 |
| tristan | so we have to make that function use ArtifactCache.remove() | 16:02 |
| cs-shadow | tristan: I wasn't expecting you to be online now but while you are here, we have https://pypi.org/project/BuildStream/ now. | 16:02 |
| cs-shadow | I had to change the tar format to tar.gz as pypi doesn't like tar.xz but other than that, no changes are required | 16:03 |
| tristan | cs-shadow, sounds sensible, anyway when I publish them it is a tar.gz, and the server converts it to tar.xz to publish on the ftp sites/mirrors | 16:04 |
| tristan | so in regular release circumstances, same input | 16:05 |
| cs-shadow | tristan: perfect, I'll add some notes and docs soon | 16:06 |
| jennis | cs-shadow, nice! Would it be an idea to add a link to the documentation under Project Links? | 16:15 |
| jennis | oh, looking at other projects, it doesn't look like that's the done thing | 16:16 |
| jennis | Nevermind me | 16:16 |
| cs-shadow | jennis: that would be neat but I'm not sure how to do that as it's coming from setup.py's "url" field that doesn't look like it accepts a list | 16:17 |
| qinusty | Can anyone point me in the direction for bwrap issues? "bwrap: capset failed: Operation not permitted" | 16:17 |
| jjardon | qinusty: let me fix, sec | 16:17 |
| qinusty | ty | 16:18 |
| jjardon | qinusty: vim /etc/gitlab-runner/config.toml and change "privileged" to "true" | 16:18 |
| jjardon | qinusty: done | 16:18 |
| qinusty | nice :D | 16:18 |
| qinusty | I'll retry jobs then | 16:18 |
| valentind | tristan, I did see some failures with this test. | 16:20 |
| valentind | I always test it with previous code. | 16:20 |
| valentind | tristan, The result of "ls -l" would be different, and the extract would be different. | 16:22 |
| valentind | The extract are indexed by the tree hash, not by the artifact key. So it should work. | 16:22 |
| tristan | valentind, ah maybe the artifact cache will not hit the extract dir unless the content is identical, ever since we changed the extract dir to not be based on the cache key | 16:23 |
| tristan | yeah exactly | 16:23 |
| tristan | still, it is failing | 16:23 |
| valentind | Yes. So that is why it works. But I did not think about it. It would be nice to remove the extracts anyway. | 16:23 |
| tristan | valentind, try using cli_integration like the other integration tests, it will fail while removing artifacts | 16:23 |
| valentind | tristan, are there some logs for the failure are you running it locally? | 16:24 |
| valentind | OK I wil | 16:24 |
| tristan | that has to be fixed anyway to avoid the redundant downloads | 16:24 |
| tristan | I think I want to remove cli_integration and reimplement the tests anyway, but for now that's what distinguishes whether the cli will reuse a shared artifact cache | 16:25 |
| tristan | err, I mean whether it will reuse a cached *source cache* | 16:25 |
| tristan | and share the same artifact cache for the whole test session | 16:25 |
| tristan | that could be done differently, maybe with decorators it would be better | 16:26 |
| tristan | cli_integration also makes an ugly assumption that all the tests use the same project.conf | 16:26 |
| *** solid_black has quit IRC | 16:31 | |
| valentind | tristan, I have a fix. Still testing it. I will push a branch so you can say what you think. | 16:43 |
| valentind | tristan, https://gitlab.com/BuildStream/buildstream/commit/edfa58f3da9c66becb6346cd996a247bdca1c14b | 16:45 |
| *** WSalmon has quit IRC | 16:46 | |
| *** noisecell has quit IRC | 16:49 | |
| *** tpollard has quit IRC | 16:49 | |
| *** WSalmon has joined #buildstream | 16:53 | |
| *** toscalix has quit IRC | 17:05 | |
| valentind | Or https://gitlab.com/BuildStream/buildstream/commit/2f58ac279625190e65994ea403841e870f0316c1 | 17:10 |
| *** finn has quit IRC | 17:14 | |
| *** jonathanmaw has quit IRC | 17:26 | |
| tristan | valentind, looks reasonable and shows what went wrong, stylistically; it would be better I think to have an optional kwarg added to the existing cli.remove_artifact_from_cache(), rather than add a whole new function | 17:32 |
| tristan | then again, it's not that important; the whole thing needs a bit of an overhaul, and we'll want to make a nice public API out of the test utilities for the sake of third party plugins | 17:34 |
| gitlab-br-bot | buildstream: merge request (jmac/tempfile-extraction-bug->master: Correct crash after staging tars with read-only directories) #713 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/713 | 17:39 |
| *** leopi has quit IRC | 17:53 | |
| valentind | tristan, OK, I will put it as keyword. | 18:14 |
| valentind | The freedesktop sdk cache server is broken. | 18:24 |
| valentind | Here is what I see in the logs: | 18:24 |
| valentind | https://paste.gnome.org/pudhy82eo | 18:24 |
| gitlab-br-bot | buildstream: merge request (valentindavid/cli_integration_source_determinism->master: tests/integration/source-determinism.py: Use cli_integration.) #715 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/715 | 18:32 |
| gitlab-br-bot | buildstream: merge request (valentindavid/cli_integration_source_determinism->master: tests/integration/source-determinism.py: Use cli_integration.) #715 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/715 | 18:36 |
| *** rdale has quit IRC | 19:04 | |
| gitlab-br-bot | buildstream: issue #605 ("linux.py uses context message API before initialization") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/605 | 19:18 |
| tristan | valentind, did you upgrade with 1.1.7 ? maybe it related to !711 ? | 19:19 |
| valentind | tristan, Yes I found out. | 19:20 |
| valentind | I upgraded the server. Now both 1.1.6 and 1.1.7 work on it. | 19:20 |
| tristan | valentind, a bit annoying that it failed with a stack trace, though :-S | 19:21 |
| tristan | I was thinking it would just "not work", didn't expect that | 19:21 |
| valentind | Yes. When the server is done, it does not complain. | 19:22 |
| tristan | anyway, sounds like something you cannot fix in the past | 19:22 |
| tristan | good to have it out of the way before 1.2 | 19:23 |
| gitlab-br-bot | buildstream: merge request (valentindavid/cli_integration_source_determinism->master: tests/integration/source-determinism.py: Use cli_integration.) #715 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/715 | 19:25 |
| gitlab-br-bot | buildstream: merge request (jjardon/ci_show_timings->master: .gitlab-ci.yml: Show timing of every test) #716 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/716 | 20:13 |
| gitlab-br-bot | buildstream: merge request (jjardon/ci_show_timings->master: .gitlab-ci.yml: Show timing of every test) #716 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/716 | 20:16 |
| gitlab-br-bot | buildstream: merge request (jjardon/ci_show_timings->master: .gitlab-ci.yml: Show timing of every test) #716 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/716 | 20:18 |
| gitlab-br-bot | buildstream: merge request (jjardon/ci_show_timings->master: .gitlab-ci.yml: Show timing of every test) #716 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/716 | 20:20 |
| *** cs-shadow has quit IRC | 20:29 | |
| *** tristan has quit IRC | 21:07 | |
| gitlab-br-bot | buildstream: merge request (jjardon/ci_show_timings->master: .gitlab-ci.yml: Show timing of every test) #716 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/716 | 21:22 |
| gitlab-br-bot | buildstream: merge request (jjardon/ci_show_timings->master: .gitlab-ci.yml: Show timing of the 20 slowes tests) #716 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/716 | 21:23 |
| gitlab-br-bot | buildstream: merge request (jjardon/ci_show_timings->master: .gitlab-ci.yml: Show timing of the 20 slowest tests) #716 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/716 | 21:24 |
| *** alatiera_ has quit IRC | 21:39 | |
| *** toscalix has joined #buildstream | 22:10 | |
| gitlab-br-bot | buildstream: merge request (chandan/pip-install-instructions->master: doc: Add instructions to install BuildStream via PyPI) #717 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/717 | 22:33 |
| gitlab-br-bot | buildstream: merge request (chandan/pip-install-instructions->master: doc: Add instructions to install BuildStream via PyPI) #717 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/717 | 22:35 |
| gitlab-br-bot | buildstream: merge request (chandan/pip-install-instructions->master: doc: Add instructions to install BuildStream via PyPI) #717 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/717 | 22:44 |
| gitlab-br-bot | buildstream: merge request (chandan/pip-install-instructions->master: doc: Add instructions to install BuildStream via PyPI) #717 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/717 | 22:45 |
| *** xjuan has quit IRC | 23:20 | |
| *** bochecha has quit IRC | 23:30 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!