| *** WSalmon has joined #buildstream | 01:24 | |
| *** WSalmon has quit IRC | 01:28 | |
| *** tristan has joined #buildstream | 04:37 | |
| *** tristan has quit IRC | 05:57 | |
| *** leopi has joined #buildstream | 06:06 | |
| *** tristan has joined #buildstream | 06:07 | |
| *** ChanServ sets mode: +o tristan | 06:08 | |
| *** leopi has quit IRC | 07:05 | |
| *** toscalix has joined #buildstream | 07:39 | |
| *** toscalix has quit IRC | 07:40 | |
| *** toscalix has joined #buildstream | 07:41 | |
| *** WSalmon has joined #buildstream | 08:06 | |
| *** phildawson has joined #buildstream | 08:32 | |
| *** leopi has joined #buildstream | 08:35 | |
| * qinusty notices that the majority of our test time comes from running the tests/examples/ tests | 08:38 | |
| tristan | qinusty, most notably the flatpak example; because it uses the flatpak SDK which is big | 08:45 |
|---|---|---|
| tristan | the freedesktop SDK, rather | 08:45 |
| juergbi | and ftp.gnu.org has issues since last night, pipelines are failing | 08:45 |
| qinusty | Our CI times are hitting upwards of 60 minutes :/ | 08:45 |
| qinusty | I also noticed this juergbi | 08:46 |
| tristan | if that is cached more reliably; and the runners are dedicated (CPU/ram/SSD on the same machine), it should be much faster | 08:46 |
| qinusty | I thought it might have been just last night | 08:46 |
| juergbi | I also thought that we're using a persistent source cache, so not sure why it's redownloading this at all | 08:46 |
| tristan | we are, but... | 08:46 |
| tristan | <jjardon> buildstream uses the ephemeral runners from baserock, as the ones from gitlab.com was not so good | 08:46 |
| tristan | juergbi, I think that is the problem ^^^^ | 08:46 |
| tristan | ephemeral doesnt sound promising... | 08:47 |
| juergbi | but at least at some point it zipped up the source cache at the end and uploaded it to some server | 08:47 |
| tristan | I also expect that if this is the case, we *should* be able to bring down >10hrs build times for freedesktop-sdk/gnome-build-meta... to a mere 4 hours for full rebuilds | 08:48 |
| juergbi | was not optimal but at least it didn't have to contact external sources all the time | 08:48 |
| qinusty | Also | 08:48 |
| tristan | well, we *do* that; just I'm not sure that the downloaded cache zip always has everything | 08:48 |
| jjardon | tristan: if your builds are big and you dont have cache servers then no, its not good | 08:49 |
| tristan | juergbi, i.e. we do put the source cache into the gitlab cache, so that should be working unless it has regressed | 08:49 |
| jjardon | tristan: freedesktop-sdk/gnome-build-meta doesn't take >10 hours for full rebuilds, where have you seen that? | 08:49 |
| jjardon | that would be a bug | 08:50 |
| juergbi | right, I don't know the current state of that. I'm just seeing repeatedly failed pipelines, blocking everything :/ | 08:50 |
| tristan | jjardon, of course; I think the issue we're looking at is rather that we are redownloading sources too often on gitlab | 08:50 |
| laurence | If CI pipelines are taking a long time, we may have an answer - finn and I yesterday acquired a pretty powerful desktop which we're going to set up as a remote execution cluster | 08:50 |
| laurence | this was for testing buildgrid and actually using remote execution | 08:50 |
| laurence | if it can be shared with buildstream CI and help in that regard, great | 08:51 |
| tristan | juergbi, also, could it be that maybe the examples tests are not benefiting from the gitlab shared cache ? | 08:51 |
| juergbi | it's possible, it's been a while since I looked into that shared cache thingy (was before the examples, iirc) | 08:51 |
| tristan | And one more thought; I still think that I/O is too slow - even if we have the freedesktop-sdk cached in gitlab; running the examples tests and corresponding docs builds still needs to generate the artifact from the ostree source | 08:52 |
| tristan | which means a huge copy; which should happen fairly quickly on a real machine | 08:52 |
| jjardon | tristan: if you need to cache sources using gitlab (using the cache: directive in .gitlab-ci.yml) someone needs to setup https://docs.gitlab.com/runner/install/registry_and_cache_servers.html#install-your-own-cache-server | 08:52 |
| tristan | ... but that creation of the artifact will suffer dearly in the case that we have silly nfs mounts and such going on | 08:53 |
| jjardon | AFAIR one was working some time ago, then I think it stops but nobody spend enough time trying to fix it | 08:54 |
| qinusty | Also tristan, TingPing made a good point yesterday about the naming of the import.py module. `from .import import Import` | 08:54 |
| qinusty | It isn't actually valid syntax | 08:54 |
| tristan | jjardon, right; that sounds like something I was under the impression happened many months ago, but maybe didnt happen at all | 08:54 |
| *** jonathanmaw has joined #buildstream | 08:54 | |
| tristan | qinusty, uhhhh, more context ? | 08:55 |
| qinusty | If you wanted to extend the Import kind | 08:55 |
| qinusty | you literally can't import it | 08:55 |
| finn | I do indeed have a new desktop | 08:55 |
| tristan | ohhhh | 08:55 |
| finn | but it's going to require setting up... | 08:55 |
| qinusty | without some fancy jiggery | 08:55 |
| tristan | qinusty, you dont want to extend the import kind; that is not supported anyway | 08:55 |
| qinusty | fair enough | 08:55 |
| qinusty | can't recall why he was importing it, but interesting none the less | 08:56 |
| tristan | deriving from Sources "happens to work" for now; for elements it does not | 08:56 |
| qinusty | I'm struggling to get these examples/junctions.py tests running locally. They're skipping. skipif(not IS_LINUX), but IS_LINUX is True... | 08:59 |
| tristan | qinusty, is it really ? | 08:59 |
| qinusty | According to a python interpreter importing tests/testutils/site.py | 09:00 |
| qinusty | Ah, regardless of whether I can run them. It'll fail because ftp.gnu.org is failing | 09:00 |
| tiagogomes | qinusty, I fall in that hole once. You need `--addopts --integration` | 09:01 |
| tristan | qinusty, I was wondering if something fundamental broke, because of https://gitlab.com/BuildStream/buildstream/issues/385#note_94607485 | 09:01 |
| tristan | qinusty, but it appears that valentind has a much better theory about that issue | 09:01 |
| jjardon | Use https://ftpmirror.gnu.org instead directl | 09:02 |
| jjardon | directly http://ftp.gnu.org/gnu ; it had helped here | 09:02 |
| qinusty | can we patch the test away from ftp.gnu? It's blocking all CI | 09:02 |
| tristan | qinusty, yes please ! | 09:03 |
| tristan | this new issue seems interesting https://gitlab.com/BuildStream/buildstream/issues/583 | 09:07 |
| *** leopi has quit IRC | 09:11 | |
| qinusty | https://gitlab.com/BuildStream/buildstream/merge_requests/665, Now we just wait for CI | 09:12 |
| tristan | qinusty, bst-1.2 also please ? | 09:13 |
| qinusty | Will do :) | 09:13 |
| tristan | thanks :) | 09:13 |
| qinusty | unnecessary :) junctions examples don't exist in 1.2 | 09:14 |
| tristan | qinusty, I presume you tried the new URL with a wget first ? | 09:15 |
| qinusty | I did better, I tested it locally | 09:15 |
| tristan | :) | 09:15 |
| tristan | nice | 09:15 |
| * qinusty notices he's missing an example which ALSO uses gnu | 09:17 | |
| *** solid_black has joined #buildstream | 09:17 | |
| tristan | yup, oops, that'll happen in more than one place indeed | 09:23 |
| tristan | probably once for the integration test, and another time for the docs build | 09:23 |
| qinusty | Okay, I've rebased my other two branches ontop of the ftp.gnu.org fix. | 09:25 |
| qinusty | If https://gitlab.com/BuildStream/buildstream/merge_requests/627 passes CI, I've addressed your comments from yesterday tristan. Artifact version is now 5 and we're adding fatal-warnings to the cache key | 09:27 |
| qinusty | I also fixed #531 in https://gitlab.com/BuildStream/buildstream/merge_requests/662, turns out that jobs would go into the retry logic if you tried to terminate them in some cases. | 09:28 |
| tristan | valentind, thanks for !656, I've merged it and opened https://gitlab.com/BuildStream/buildstream/issues/537 for the backport | 09:30 |
| tristan | luckily, it was on the tip and had passed CI before the gnu.org failures ;-) | 09:30 |
| tristan | qinusty, I think today... I might even catch up to all the notifications in my inbox ! \o/ | 09:31 |
| qinusty | :O | 09:31 |
| tristan | qinusty, Sure; I think we've caught all the important bits by now | 09:34 |
| qinusty | As you say though, there's room for more thought and improvement in the future with fatal-warnings: True | 09:34 |
| tristan | qinusty, I *would* like to have format version bumps, and artifact version bumps + effected tests; in separate commits | 09:34 |
| tristan | but I don't think we have to be super strict about that if it's very inconvenient to change, traceability of the version bumps is still quite in tact the way you have it anyway | 09:35 |
| qinusty | I noticed, which is why I tried to make it visible in the commit message. The problem with separating it into a new commit was that once I've committed the change in the code, the version is directly affected in that commit | 09:36 |
| qinusty | Which is why I thought it needs to go in the commit where the change occurs | 09:36 |
| tristan | nah, in the same branch merge is also fine; but I don't want to nitpick this too much anyway | 09:37 |
| tristan | more importantly, every commit in the branch passes it's own tests | 09:37 |
| qinusty | Ah okay, yes | 09:37 |
| tristan | and that we can trace what happened for every version change (which is fine the way you did it, also) | 09:38 |
| qinusty | Which is why I sometimes include test changes into other commits. To fix breakages | 09:38 |
| qinusty | I have a separate commit for new tests | 09:38 |
| tristan | so, it may be more a matter just of taste to keep things separated | 09:38 |
| qinusty | I can see it both ways | 09:38 |
| tristan | right, when we bump the cache key version, it always comes with an update to tests/cachekey/* in the same commit | 09:38 |
| qinusty | Is there a reason we need to have all those key files which require updating? | 09:39 |
| valentind | Buildstream is very slow because of this cache scanning. I thought there was an issue opened for that. But I do not find it. | 09:39 |
| qinusty | as opposed to validating through generation? | 09:39 |
| tristan | valentind, you mean BuildStream :) | 09:39 |
| tristan | hehe | 09:39 |
| tristan | qinusty, when you bump the cache key version; it prints a very descriptive error message when the cache key test fails | 09:40 |
| tristan | qinusty, that message instructs you how to run tests/cachekey/update.py, so that you don't have to spend time in there | 09:40 |
| qinusty | I noticed :D, but you might not notice that until 60 mintues into CI | 09:40 |
| qinusty | Now I know though, if I bump the cache key version, I have to run update.py | 09:41 |
| tristan | I normally like to assume that after coding a branch and before pushing it to CI, developers have at least run the whole suite once | 09:41 |
| tristan | it only takes 10 or 15min... at least not including the --integration tests | 09:41 |
| * qinusty has started doing that, but previouisly had issues with lint etc | 09:41 | |
| qinusty | it was just easier for me to run through CI | 09:42 |
| qinusty | But I fixed it, ish | 09:42 |
| tristan | valentind, right I think there is no issue open for the initial cache scan | 09:47 |
| valentind | tristan, OK, we should fix that. Because for me it takes maybe 5 to 10 seconds before it prints anything. | 09:48 |
| tristan | valentind, from what I understood, it was pretty much unavoidable, but forgivable because it should normally only happen once and a while | 09:48 |
| tristan | i.e. once the kernel dentry cache is hot, that becomes quite snappy | 09:48 |
| valentind | tristan, it never gets snappy. | 09:48 |
| valentind | Every time, it is slow. | 09:48 |
| tristan | valentind, ouch | 09:48 |
| valentind | It is the thing that looks at the size. | 09:48 |
| tristan | sigh, yes please file an issue :-/ | 09:48 |
| tristan | right | 09:49 |
| valentind | tristan, I have the impression it scans it lots of times. | 09:49 |
| tristan | valentind, that thing should be getting fast results once the dentry cache is hot | 09:49 |
| valentind | Not sure though. | 09:49 |
| tristan | but then, I guess if you have a very huge cache already, it will still take time | 09:49 |
| tristan | a lot of enhancements to ArtifactCache are needed to make that graceful | 09:50 |
| valentind | tristan, actually, I measured 25 seconds. | 09:50 |
| *** tiagogomes has quit IRC | 09:50 | |
| *** tiagogomes has joined #buildstream | 09:51 | |
| tristan | like, when we commit an artifact, the cache should tell us how many bytes precisely the commit has added to the cache | 09:51 |
| valentind | tristan, also, I think they changed something in vte. I do not get any notification anymore in gnome-terminal. Or maybe this is a Debian issue. | 09:51 |
| tristan | same with remove, also; we'd probably want to record that somewhere | 09:51 |
| tristan | valentind, I have never seen the notification actually work, which is part of the reason I wasnt very fond of the patch | 09:51 |
| coldtom | tristan, valentind, if we're talking about the time without output before initialising the pipeline, i've had waits of 30-60s when running the fdo makefile, on each command | 09:52 |
| tristan | valentind, apparently it "works on fedora, because they have a downstream patch applied to vte"... or smth | 09:52 |
| valentind | tristan, Maybe it worked for me on Gentoo, and did not notice it was not on Debian. | 09:52 |
| tristan | valentind, for that issue, I think mcatanzaro will probably respond with his TERM env var value, ... whether he does or not, we probably need to relax the env check a bit | 09:54 |
| * qinusty seems to have broken his bzr installation | 09:54 | |
| valentind | tristan, by the way, 25 seconds. This is with an NVMe Samsung 960PRO. | 09:54 |
| tristan | valentind, can you file the issue for that slowness, milestone it 1.2 and label it "Bug" + "Optimization" + "Urgent", please ? | 09:55 |
| valentind | tristan, OK | 09:55 |
| tristan | "High_Impact" alsoe | 09:55 |
| tristan | *also | 09:55 |
| tiagogomes | Are we talking about https://gitlab.com/BuildStream/buildstream/issues/573 ? | 09:57 |
| tristan | aha ! | 09:58 |
| tristan | valentind, ^^^^^^^ | 09:58 |
| tristan | tiagogomes, nice, you also proposed basically the same solution :) | 09:58 |
| valentind | tristan, maybe. | 10:01 |
| valentind | Well it is just at startup. | 10:02 |
| valentind | So I think there is more. | 10:02 |
| tristan | valentind, yes but it is the same thing happening at startup and periodically | 10:03 |
| valentind | utils._get_dir_size is slow. | 10:04 |
| tristan | valentind, I think in either case, it's desirable to not bloat the issue tracker; treat it as the same thing | 10:04 |
| qinusty | Has anyone else seen issues with the use of bzr within buildstream if their /usr/bin/python is python3? | 10:04 |
| valentind | This is where the time is lost. | 10:04 |
| tristan | valentind, basically yeah, that also gets called; at startup, and also later on | 10:04 |
| valentind | There is one call. | 10:04 |
| valentind | One call takes 25 seconds. | 10:05 |
| tristan | valentind, and that call happens in the cachesizejob... right ? | 10:05 |
| valentind | tristan, "du" takes only 0.8 seconds. | 10:05 |
| tristan | sigh | 10:06 |
| tristan | valentind, ok... so it sounds like you are already hot on the trail of solving it... do you insist to treat the call at startup time as a separate issue as 573 ? I don't understand why | 10:07 |
| valentind | tristan, It is nice if we do not call it too often. But we need to also fix the function itself. There is something weird, that should not take that much time. | 10:08 |
| tristan | So... how about reporting your findings on 573 and working from there ? | 10:09 |
| * tpollard apologises for the formatting on his latest post to the list, Thunderbird is really playing up for me | 10:09 | |
| tiagogomes | valentind, are you doing do in the right directory? ~/cache/artifacts? | 10:09 |
| tiagogomes | And not only on ~/cache/artifacts/cas | 10:09 |
| valentind | tiagogomes, Ahhhh | 10:10 |
| valentind | It counts also the extracts. | 10:10 |
| tiagogomes | yup | 10:10 |
| valentind | It is only 2G more. But takes more time. 12 seconds. | 10:10 |
| tristan | valentind, and the next time you run du, now that you've made the dentry cache hot ? | 10:10 |
| tristan | same 12 seconds ? | 10:11 |
| valentind | du 12 seconds, bst 27 seconds. | 10:12 |
| valentind | tristan, we do 2 stat per file. | 10:12 |
| valentind | Because we call .is_dir | 10:12 |
| valentind | Do not we check for inodes? | 10:13 |
| tiagogomes | I think jonathanmaw did some work on trying to optimize that function | 10:17 |
| valentind | Could we be more aggressive on cleaning up extracts? They seem to take all the time. | 10:18 |
| valentind | Or actually, do we need to scan it. Files are hardlinks. | 10:18 |
| jonathanmaw | tiagogomes: yep, issue 466, though it's also expanded to speeding up loading the pipeline, as well | 10:19 |
| valentind | Seems very difficult to make something faster in python. | 10:19 |
| tristan | eek, we call is_dir() ? yuck | 10:19 |
| tiagogomes | We could always shell out and call du if it makes a significant difference | 10:20 |
| tristan | valentind, as I recall, tiagogomes is taking care of fixing ArtifactCache.remove() to also remove the extract dir | 10:20 |
| valentind | great | 10:20 |
| tristan | we should probably also fix it to not call is_dir, that can be checked with the output of os.lstat() anyway | 10:21 |
| tristan | at least short term improvement | 10:21 |
| valentind | tristan, But I think we should remove them even when we keep the artifact. | 10:21 |
| tristan | valentind, we don't want to re-extract them, either | 10:21 |
| tristan | valentind, as long as we have the artifact, we should have it's extract | 10:22 |
| tristan | lets not start talking about a start up time wipe of the extracts again | 10:22 |
| valentind | tristan, OK, so if we have them removed automatically, can we stop scanning extracts? | 10:22 |
| tristan | another thing is, if we accept that the calculation is a bit "fuzzy", we could... exactly, stop scanning those | 10:23 |
| tristan | removed automatically with the artifact was always supposed to happen, it's an error in tlater's branch that it didnt happen since the beginning | 10:23 |
| valentind | Because it is what takes the most time. And if we say we have extract only if we have the artifact, then we can just not scan, the size should be the same due to hardlinks. | 10:23 |
| tristan | valentind, it will be fuzzy, but I think it's acceptable | 10:24 |
| tristan | valentind, for instance, the dentries cost disk space... and any file which is deduplicated in CAS but has different attributes, will be a copy in the extract dir, not a hardlink | 10:24 |
| valentind | if du takes .8 seconds, I suppose bst will take 1.6 seconds. It a big improvement from 25 seconds. | 10:25 |
| tristan | valentind, I personally feel that fuzzy/imperfect but performing much better, is a good tradeoff until we can do much better | 10:25 |
| tristan | still, if we're doing is_dir(), that's ugly, messy and unprofessional | 10:25 |
| tristan | we should not do that | 10:25 |
| valentind | tristan, It seems to me that now, we ignore hardlinks in the calculation. | 10:25 |
| valentind | I would expect some inode and device pairs saved in a set. | 10:26 |
| tristan | valentind, except I think we already agreed that we should only scan the cas/ subdir and accept fuzzyness | 10:27 |
| valentind | tristan, tried reusing the result stat and there is not much different with calling is_dir. | 10:27 |
| *** gitlab-br-bot has joined #buildstream | 10:27 | |
| valentind | tristan, OK | 10:27 |
| tristan | valentind, i.e. no need to spin developer hours perfecting that; the cas itself should not contain significant hardlinks within it's own deduplicated store | 10:27 |
| *** gitlab-br-bot has quit IRC | 10:28 | |
| valentind | Yes, yes. I prefer we do not scan extracts. | 10:28 |
| tristan | Ultimately; perfecting this should be done how tiagogomes suggests in the issue; the commit/remove APIs should provide the context and we should be able to bookkeep that without scanning I think | 10:29 |
| tristan | or, the underlying CAS implementation could make that transparent | 10:29 |
| tristan | Until doing it "perfectly", an imperfect but performant solution is better | 10:29 |
| tristan | gitlab-br-bot: ping | 10:30 |
| tristan | ironfoot, any updates on that ^^^^ ? | 10:30 |
| valentind | +1 for bookkeeping. | 10:32 |
| valentind | We need locking anyway because of deletion. | 10:32 |
| tristan | yeah | 10:33 |
| tristan | valentind, well, we need it for multi-instance; we guarantee exclusivity at delete time within one instance | 10:33 |
| tristan | but still, we'll need it; I'd just rather put the efforts where they will make more impact; but we should still track this and strive to be better | 10:34 |
| *** gitlab-br-bot has joined #buildstream | 10:46 | |
| cs-shadow | yay! gitlab-br-bot is back :) | 10:46 |
| cs-shadow | tristan: do you have a few moments to talk about SourceTransform? | 10:47 |
| *** gitlab-br-bot has quit IRC | 10:48 | |
| *** gitlab-br-bot1 has joined #buildstream | 10:48 | |
| valentind | tristan, By the way, I am looking around at issues. But if you have something you want me to fix in priority, please tell me. | 10:49 |
| tristan | valentind, I'm still wading through the flood of emails... but did you make a patch for the ostree breakage with source mirroring ? | 10:53 |
| valentind | tristan, yes | 10:53 |
| valentind | !658 | 10:54 |
| tristan | valentind, Can you please capture what was discussed above regarding scanning only the cas/ subdir and make sure the short term approach is documented on 527 ? | 10:54 |
| tristan | cs-shadow, certainly - its a good time | 10:55 |
| tristan | gitlab-br-bot1: welcome back ! | 10:55 |
| valentind | tristan, 527? | 10:55 |
| cs-shadow | tristan: cool, I think I have fixed other outstanding comments except for the `directory` discussion | 10:56 |
| tiagogomes | Would using flock acceptable in BuildStream? I don't know if it is supported on MacOS | 10:56 |
| cs-shadow | tiagogomes: it isn't installed by default on MacOS | 10:56 |
| cs-shadow | tristan: I was thinking that maybe we can preserve the meaning of `directory` as it is and let the individual plugins come up with a new field for "this is where you should transform things" if needed | 10:57 |
| tristan | valentind, the issue that is all about cache size calculations making things slow, which has an expensive perfect solution that we cant do just yet | 10:58 |
| tristan | valentind, the issue that you wanted to have a separate issue for, but I would prefer to just keep one of, do I recall the right number of it ? | 10:58 |
| valentind | Yes, 584 | 10:58 |
| cs-shadow | especially since there is not much user-facing difference between normal and sorucetransoform plugins, I wonder if changing the meaning of `directory` will cause more confusions than it's worth | 10:58 |
| tristan | valentind, sorry I looked up in backscroll and thought I had it right :) | 10:59 |
| tristan | cs-shadow, *yes*, I think what you are suggesting is already what I expressed that I would prefer, right ? | 10:59 |
| tristan | cs-shadow, i.e. anyway a Source cannot read the 'directory' value, it belongs to the core | 11:00 |
| tristan | cs-shadow, so a Source *could not* make a different meaning for it, even if it tried | 11:00 |
| tristan | cs-shadow, what I think should be happening though; is that we should have symmetry in the portion of the directory that we expose to sources which "require the previous source directories" | 11:01 |
| tristan | cs-shadow, I.e. if a `pip` plugin is configured with the directory 'subdir/' it should not only be called at stage time with `/path/to/builddir/subdir`, it should *also* get `/path/to/builddir/subdir` in the previous_sources_dir | 11:03 |
| tristan | i.e. make it consistent, if we configured the source with a subdir, it should only ever see that subdir | 11:03 |
| cs-shadow | tristan: Ah! I see your point now. At first, I thought you were referring to the `directory` attribute of previous sources hence the confusion | 11:04 |
| cs-shadow | this makes more sense | 11:04 |
| tristan | :) | 11:04 |
| cs-shadow | tristan: I'll make `fetch()` and `track()` symmetric to `stage()`. Do you see any other issues for this branch? | 11:05 |
| *** gitlab-br-bot1 has quit IRC | 11:06 | |
| tristan | cs-shadow, I think the biggest one was the possibility of inadvertent tracking, I think that should be more explicit; but... I might be wrong | 11:07 |
| tristan | well | 11:07 |
| tristan | It could definitely be more explicit, but it might not be an actual problem due to the order in which things are done | 11:07 |
| cs-shadow | I have changed that to an assert as if i'm understanding it correctly it should never happen | 11:07 |
| cs-shadow | i.e. all previous sources must be tracked in all cases | 11:08 |
| tristan | cs-shadow, I presume that has changed, but I'll give it another run through; I suspect we're done | 11:08 |
| valentind | Do we have an issue to deal cache with multiple bst running in the same time? | 11:08 |
| tristan | cs-shadow, it will be an awesome feature :) | 11:08 |
| tristan | cs-shadow, "all previous sources must be tracked in all cases" does not match my understanding, but you might just be wording that in a way that confuses me | 11:09 |
| tristan | cs-shadow, all previous sources must be consistent in all cases, seems to be true | 11:09 |
| tristan | cs-shadow, and only when tracking, should everything be tracked; and when fetching, nothing should ever be tracked | 11:10 |
| tristan | cs-shadow, when tracking; it is possible that we *have to fetch*, but *never the inverse* | 11:10 |
| cs-shadow | tristan: yes, that's what I meant. More precisely, that track() will not be called on a source unless it has also been called on all the sources that appear before it | 11:11 |
| *** gitlab-br-bot has joined #buildstream | 11:11 | |
| tristan | valentind, not that I'm aware of; it would be appreciated if you could file that as a regression issue; fallout from local cache cleanup | 11:11 |
| valentind | tristan, OK | 11:11 |
| *** gitlab-br-bot has quit IRC | 11:12 | |
| *** gitlab-br-bot has joined #buildstream | 11:12 | |
| tristan | tiagogomes, I didn't answer your flock() question; do we have to do it right now, or can it be a part of fixing multi-instance ? | 11:12 |
| tiagogomes | multi instance? I am already pulling my hairs due the fact there are multiple parallel fetchers to the cache | 11:12 |
| tristan | But deletion is ensured to be exclusive | 11:13 |
| gitlab-br-bot | buildstream: merge request (artifactcache->master: Artifact Cache) #1 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/1 | 11:13 |
| tristan | tiagogomes, is it crashing when you delete the extract/ directory in ArtifactCache.remove() ? | 11:13 |
| tristan | gitlab-br-bot, \o/ | 11:13 |
| tristan | finally, we merged MR !1 ? | 11:14 |
| tristan | wow | 11:14 |
| tristan | that took a while | 11:14 |
| cs-shadow | tristan: coming back to the directory discussion. Consider this case with pip: | 11:14 |
| cs-shadow | - we have a git source as the first source, that provides a requirements.txt | 11:14 |
| cs-shadow | - second source is pip, with `directory: foo` so that it can put all tarballs in one place | 11:14 |
| cs-shadow | if we append `foo` to `previous_sources_dir` then pip will not have access to the requirements file | 11:14 |
| tristan | cs-shadow, that is exactly what I expect, yes | 11:15 |
| gitlab-br-bot | buildstream: merge request (Qinusty/gnu-mirror->master: Fix CI - ftp.gnu.org unreachable) #665 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/665 | 11:15 |
| qinusty | Welcome back bot :D | 11:15 |
| cs-shadow | tristan: so how do we allow for a plugin to put its output in a subdirectory? | 11:15 |
| gitlab-br-bot | buildstream: merge request (Qinusty/526-fail-on-warnings->master: Configurable Warnings) #627 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/627 | 11:16 |
| tristan | cs-shadow, i.e. I expect that the `pip` plugin has a default "Place where it puts stuff", e.g. `.bst_pip_downloads`... maybe that has to eventually become configurable, but I suppose it might never need to be configurable | 11:16 |
| qinusty | is https://gitlab.com/BuildStream/buildstream/merge_requests/627 ready for merge on pipeline success tristan? | 11:17 |
| tristan | cs-shadow, I do however expect that if you have staged TWO python git repositories into separate subdirectories, using `directory` for both of those git sources AND both of those `pip` plugins... that both will happen, in separate subdirectories | 11:17 |
| tristan | qinusty, Yes :) | 11:18 |
| qinusty | I'll babysit it then :) | 11:19 |
| tristan | remember to keep ticking the "Remove source branch" | 11:19 |
| tristan | since you didn't do it at MR submission time | 11:19 |
| tristan | cs-shadow, do you follow ? | 11:20 |
| cs-shadow | tristan: this is the part that confuses me the most. As a user, I've come to expect `directory` for a source to mean "Place where it puts stuff". I'm not sure what we buy by changing its meaning and then adding a different attribute for what it does for other sources. Couldn't we retain the meaning of `directory` as the place where a plugin should put its output, and let individual plugins come up with something like "base-dir" or | 11:20 |
| cs-shadow | something similar? | 11:20 |
| tristan | cs-shadow, I think it means: "The place where this source does it's stuff" - so far; the only thing a Source has ever done, is "put things" | 11:21 |
| tristan | cs-shadow, Also, I am very much against the concept that you keep coming back to; which is that a source transform has a specific discernable output which ends up at a given location | 11:21 |
| tristan | cs-shadow, I don't think thats what a source transform does, I think it creates additional config files, puts stuff in some places, etc, etc | 11:22 |
| cs-shadow | tristan: fair enough, I wasn't aware of this meaning of `directory`. | 11:22 |
| tristan | i.e. for the cargo/rust scenario, it will probably be adding a config file which informs the rust toolchain where to get it's crates | 11:22 |
| tristan | it's possible that we need to configure where that config file goes, separately to where it puts the downloaded crates | 11:23 |
| cs-shadow | tristan: I do think we will need some way to configure where to put files for most source transform plugins but I'm happy to call it something different than `directory` | 11:25 |
| tristan | cs-shadow, indeed; and what I think is that only a subset of these plugins will have a single place for where things go | 11:27 |
| tristan | cs-shadow, I think in that light; making all of it relative to the `directory`, which is completely opaque and handled by the core, is the correct thing | 11:27 |
| qinusty | tristan is --on-error still a thing? Its in bst --help but showing up as no such option. | 11:27 |
| tristan | what ? | 11:28 |
| qinusty | bst fetch a.bst --on-error | 11:28 |
| cs-shadow | tristan: makes sense. I'll update my branch accordingly and let you know | 11:28 |
| qinusty | Error: no such option: --on-error | 11:28 |
| tristan | qinusty, --on-error is very much a thing, are you specifying it as a bst option ? or maybe you accidentally specified it as an option to a subcommand ? | 11:28 |
| cs-shadow | thanks for clearing it up, it was very useful | 11:28 |
| * qinusty can't seem to get it to behave | 11:29 | |
| * qinusty got it, it must go BEFORE fetch | 11:29 | |
| gitlab-br-bot | buildstream: issue #585 ("Concurent instances of bst should share cache without issues") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/585 | 11:33 |
| tristan | qinusty, right, the options for the main `bst` command can be specified for any of the bst commands, and must come after `bst` and before `<command>`, commands in turn can have options; as well as subgroups, etc | 11:39 |
| qinusty | Yup got it, made the assumption that placement wasn't important D: | 11:39 |
| gitlab-br-bot | buildstream: merge request (willsalmon/CacheExpiryTest->master: Trying to mitigate a file system issue) #595 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/595 | 11:47 |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 11:51 |
| cs-shadow | tristan: I've updated https://gitlab.com/BuildStream/buildstream/merge_requests/568 based on what we just discussed. Please have a look when you get a chance | 12:01 |
| *** jonathanmaw has quit IRC | 12:02 | |
| tristan | tiagogomes, I think you are working on https://gitlab.com/BuildStream/buildstream/issues/561... can you self-assign it please ? | 12:04 |
| tristan | if you are ? :) | 12:04 |
| tristan | I'm wondering what to give to valentind | 12:04 |
| tiagogomes | At the moment I am working on https://gitlab.com/BuildStream/buildstream/issues/573 | 12:05 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_releases->master: Add section about current releases) #661 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/661 | 12:05 |
| tristan | tiagogomes, ah, ok I see | 12:05 |
| *** jonathanmaw has joined #buildstream | 12:05 | |
| tristan | tiagogomes, so then, I was a bit confused; our conversation before regarding calculating only the cas/ subdirectory makes sense for you ? | 12:06 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_releases->master: Add section about current releases) #661 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/661 | 12:06 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_releases->master: Add section about current releases) #661 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/661 | 12:07 |
| tristan | tiagogomes, I feel like 561 is closely related so maybe you want to do that also; but we could hand that to valentind if it will save time | 12:07 |
| tiagogomes | It is related, and conflicts with what I am doing now. On the other hand I will be in holiday next week I am not sure if I can finish both | 12:09 |
| *** jonathanmaw_ has joined #buildstream | 12:11 | |
| tristan | alright then, it's high priority; we've implemented cache cleanup and people have complained regularly about the cache getting too big; we need to fix it | 12:11 |
| tristan | valentind, I was mixed up before, please do take care of ensuring ArtifactCache.remove() also removes the extract directory, issue #561 | 12:12 |
| *** jonathanmaw has quit IRC | 12:12 | |
| *** ironfoot has quit IRC | 12:15 | |
| gitlab-br-bot | buildstream: issue #526 ("Add configuration option to fail on warnings") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/526 | 12:16 |
| gitlab-br-bot | buildstream: merge request (Qinusty/526-fail-on-warnings->master: Configurable Warnings) #627 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/627 | 12:16 |
| *** ironfoot has joined #buildstream | 12:19 | |
| tristan | cs-shadow, I gave it a final run through; and yeah I'm happy with the assert statement | 12:23 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_releases->master: Add section about current releases) #661 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/661 | 12:23 |
| cs-shadow | tristan: thanks! | 12:24 |
| tristan | cs-shadow, I'm not sure why we should make the location of the downloaded packages configurable; it seems that that would cause automation later on to be more complicated to achieve | 12:24 |
| tristan | and I'm not sure I see the benefit of making it configurable | 12:24 |
| tristan | I only added one comment | 12:24 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_releases->master: Add section about current releases) #661 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/661 | 12:24 |
| tristan | about that configurability, and if we do want to keep it, that it should probably only happen at stage() time, not effecting how we cache it | 12:25 |
| cs-shadow | tristan: My only reason was that it seems a bit odd to make up the convention about `.bst_pip_downloads` since `pip` itself provides no such concention | 12:25 |
| cs-shadow | so if you think that documenting it is enough, i can drop that | 12:25 |
| tristan | cs-shadow, I think documenting it is enough until such a time that it's needed, yeah | 12:26 |
| cs-shadow | other than that, the only other reason I see is if someone has a file with the same name in their sources, but that seems a bit unlikely in practice | 12:26 |
| tristan | I think `.bst_pip_downloads` is a name that has a very low risk of already being reserved by a git repository | 12:26 |
| tristan | exactly | 12:26 |
| cs-shadow | cool, I'll remove that. I do not see any strong need for it just yet | 12:27 |
| tristan | cs-shadow, mostly, I'd rather keep the format API surface as small as possible at all times, that is my main motivation | 12:27 |
| tristan | not adding some config until it's proven to be a needed use case is usually a good idea | 12:28 |
| *** toscalix has quit IRC | 12:28 | |
| cs-shadow | tristan: makes sense. reminds me of https://www.martinfowler.com/bliki/Yagni.html :) | 12:29 |
| *** toscalix has joined #buildstream | 12:29 | |
| tristan | cs-shadow, I somehow stumbled on that article last week, coincidentally | 12:30 |
| tristan | not quite as exhilarating as Naggum's perl rant, but not a horrible read :) | 12:32 |
| * cs-shadow googles Naggum's perl rant | 12:32 | |
| tristan | I think I stumbled on it via an article about technical debt, and seeing software development like finance | 12:32 |
| tristan | haha, cs-shadow you are in for a whole series of great rants from the 90s, but I think that one is the best :) | 12:33 |
| cs-shadow | :) | 12:35 |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: WIP: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 12:38 |
| * tristan heads out for the night | 12:39 | |
| tristan | cs-shadow, Let's pull the trigger and merge the source transform stuff... please do it ! | 12:39 |
| tristan | :D | 12:39 |
| cs-shadow | tristan: cool, i have removed the output-dir stuff from user-facing side | 12:40 |
| cs-shadow | I think we are ready | 12:40 |
| tristan | me too | 12:40 |
| tpollard | qinusty: nice to see the configurable warnings merged :) | 12:40 |
| tristan | cs-shadow, I think docs could be more tidy as a result of this, but we can improve them after | 12:40 |
| cs-shadow | tristan: so, shall i hit the green button the tests pass or do you want to have another look? :) | 12:41 |
| cs-shadow | when the tests pass* | 12:41 |
| tristan | cs-shadow, please also make sure there is a new issue filed for having some automation in how the `pip` results are used at build time | 12:41 |
| qinusty | Yes I think it merged within the last hour when CI passed. Let me know how you get on and how the docs read. Primarily in Plugin.py and format_project.rst | 12:41 |
| tiagogomes | The artifact server is not very talkative | 12:41 |
| qinusty | tpollard ^ | 12:41 |
| tristan | cs-shadow, it's fine; lets do it :) | 12:41 |
| cs-shadow | tristan: will do, thanks! | 12:42 |
| cs-shadow | I am seeing failures with ssl handshake errors on the CI when trying to connect of ftpmirror.gnu.org, for example in https://gitlab.com/BuildStream/buildstream/-/jobs/89482064 | 12:48 |
| cs-shadow | has anyone got any ideas as to why this is failing? | 12:48 |
| *** tristan has quit IRC | 12:48 | |
| cs-shadow | interestingly the same tests pass in the "test-unix" pipeline but fail on "test-debian-9" | 12:49 |
| gitlab-br-bot | buildstream: merge request (willsalmon/CacheExpiryTest->master: Trying to mitigate a file system issue) #595 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/595 | 12:50 |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: WIP: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 12:59 |
| *** edb has joined #buildstream | 12:59 | |
| gitlab-br-bot | buildstream: merge request (juerg/cas->master: CAS: Fix resource_name format for blobs) #660 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/660 | 13:02 |
| qinusty | No idea cs-shadow, it seems tempermental | 13:09 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: WIP: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 13:17 |
| gitlab-br-bot | buildstream: merge request (valentindavid/fallback_mirror_ostree->master: Fix ostree repository mirroring) #658 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/658 | 13:22 |
| gitlab-br-bot | buildstream: merge request (valentindavid/fallback_mirror_ostree-1.2->bst-1.2: Fix ostree repository mirroring) #667 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/667 | 13:25 |
| cs-shadow | qinusty: oh well, I'll keep hitting it with the rebuild hammer until it passes | 13:29 |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: WIP: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 13:31 |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 13:34 |
| *** jonathanmaw_ is now known as jonathanmaw | 13:35 | |
| gitlab-br-bot | buildstream: merge request (dp0/513/cas-cache-client-certs->master: Support dynamic client certificates for CAS cache) #594 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/594 | 13:36 |
| gitlab-br-bot | buildstream: merge request (jjardon/pyproject->master: Some minor setuptools improvements) #639 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/639 | 13:43 |
| gitlab-br-bot | buildstream: merge request (Qinusty/531-fetch-retries-on-terminate->master: Prevent jobs retrying on terminate) #662 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/662 | 13:47 |
| gitlab-br-bot | buildstream: merge request (Qinusty/531-fetch-retries-on-terminate->master: Prevent jobs retrying on terminate) #662 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/662 | 13:49 |
| gitlab-br-bot | buildstream: issue #572 ("CAS: URLs used for blob download and upload do not match protocol spec") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/572 | 14:13 |
| gitlab-br-bot | buildstream: merge request (juerg/cas->master: CAS: Fix resource_name format for blobs) #660 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/660 | 14:13 |
| gitlab-br-bot | buildstream: issue #572 ("CAS: URLs used for blob download and upload do not match protocol spec") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/572 | 14:13 |
| qinusty | Anyone fancy reviewing https://gitlab.com/BuildStream/buildstream/merge_requests/662? | 14:14 |
| *** leopi has joined #buildstream | 14:17 | |
| skullman | qinusty: sure | 14:17 |
| gitlab-br-bot | buildstream: merge request (Qinusty/531-fetch-retries-on-terminate->master: Prevent jobs retrying on terminate) #662 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/662 | 14:21 |
| valentind | So what do we do with the issues with gnu server? | 14:24 |
| juergbi | valentind: qinusty changed it to ftpmirror.gnu.org which appears to work | 14:25 |
| valentind | juergbi, is that fix on bst-1.2? | 14:25 |
| juergbi | ah, maybe not, should be trivial to backport | 14:25 |
| qinusty | juergbi, however we have been seeing the occasional ssl error causing one of the 5 distros to work. | 14:25 |
| valentind | qinusty, are you going to backport it? | 14:26 |
| qinusty | Hmmm valentind, I did look at backporting. The junctions example isn't there so that doesn't need backporting. But I didn't check autotools | 14:26 |
| valentind | qinusty, I got 3 errors with autotools. | 14:26 |
| qinusty | backporting now :) | 14:26 |
| valentind | qinusty, thank you | 14:26 |
| skullman | qinusty: ping | 14:28 |
| qinusty | helloo | 14:28 |
| skullman | replied to your MR | 14:28 |
| qinusty | I agree with your points :D I was going to run a few runs of the CI and see how it goes. But you're right we have 0 guarantees. | 14:30 |
| qinusty | Custom source plugin could work, is there much work involved with that? | 14:30 |
| gitlab-br-bot | buildstream: merge request (willsalmon/versionTagRegrex->master: Search for tags with the *.*.* patten for version) #601 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/601 | 14:30 |
| skullman | qinusty: haven't written one before so can't help you there. I've done synchronisation for tests like this before with named pipes, though this results in the tests hanging when they break. | 14:34 |
| gitlab-br-bot | buildstream: merge request (Qinusty/gnu-mirror-backport-1.2->master: Qinusty/gnu mirror backport 1.2) #668 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/668 | 14:34 |
| gitlab-br-bot | buildstream: merge request (Qinusty/gnu-mirror-backport-1.2->bst-1.2: Qinusty/gnu mirror backport 1.2) #668 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/668 | 14:34 |
| qinusty | Apologies for spam, welcome back gitlab-br-bot | 14:35 |
| qinusty | valentind https://gitlab.com/BuildStream/buildstream/merge_requests/668, merge on pipeline completion? | 14:35 |
| qinusty | What makes it worse skullman, is that I think git is caching buildstream so when my test tries to fetch buildstream it's basically instant | 14:38 |
| * qinusty assumes `git clone` does some form of caching | 14:39 | |
| skullman | not usually | 14:39 |
| skullman | and I wouldn't expect the test suite to share a source cash between tests | 14:40 |
| qinusty | https://gitlab.com/BuildStream/buildstream/-/jobs/89514398 seems to indicate a fast git clone | 14:42 |
| skullman | could just have a fast worker and a good connection, buildstream.git isn't huge | 14:46 |
| skullman | git does no caching, and that output says it ran git rather than using a result cached by BuildStream | 14:48 |
| gitlab-br-bot | buildstream: merge request (patch-1->master: install_linux_distro.rst docs: updated to point user to latest stable branch) #657 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/657 | 15:01 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: WIP: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 15:03 |
| gitlab-br-bot | buildstream: merge request (patch-1->master: install_linux_distro.rst docs: updated to point user to latest stable branch) #657 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/657 | 15:08 |
| gitlab-br-bot | buildstream: merge request (Qinusty/531-fetch-retries-on-terminate->master: Prevent jobs retrying on terminate) #662 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/662 | 15:12 |
| qinusty | skullman, I have addressed the issues with a custom source plugin which takes 20s to fetch. I wait 3s before sending SIGINT, this /should/ be enough for the pipeline to set up for one element. (otherwise we have other issues.) | 15:13 |
| jmac | juergbi: I've got a pylint warning after your merge of !660 | 15:13 |
| jmac | W:331,60: Cell variable resource_name defined in loop (cell-var-from-loop) | 15:13 |
| juergbi | :/ pylint's inconsistency across systems is painful | 15:15 |
| juergbi | I don't see the warning here locally and obviously neither do any of the CI systems | 15:15 |
| juergbi | I'm on pylint 2.1.1 here | 15:15 |
| qinusty | same, I'm getting it too juergbi but I get 17 pylint warnings on my system | 15:16 |
| qinusty | python 3.5.3 | 15:16 |
| qinusty | I want to run my linter in a docker container but I get ImportMismatch errors for conftest and haven't looked into fixing them yet | 15:18 |
| jmac | No big problem | 15:18 |
| juergbi | afaict, the code behaves correctly. but we could avoid the warning by moving uuid and resource_name into def request_stream() | 15:19 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: plugins/git.py: Fail if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 15:19 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: plugins/git.py: Warn if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 15:19 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: plugins/git.py: Warn if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 15:20 |
| tpollard | jjardon: MR564 has been updated to support the new configurable fail on warnings, we might be able to land your request for the git plugin now | 15:22 |
| jjardon | tpollard: awesome | 15:22 |
| tpollard | qinusty: thanks for the work on that, I hope I've utilised it as you intended | 15:23 |
| jjardon | tpollard: you plan to backport it to bst-1.2, right? | 15:23 |
| *** leopi has quit IRC | 15:23 | |
| skullman | qinusty: that'll break if the runner is too slow, and take a disproportionately long time to run the test suite on a faster runner. Not ideal, but not too bad a solution. | 15:24 |
| qinusty | It'll fail if the runner is slow, and it'll take 3s~ if the runner is fast and the test passes | 15:25 |
| qinusty | And it's a single test, 20s is quick in comparison to the examples/ tests | 15:25 |
| tpollard | jjardon: it depends if the configurable warnings is to be backported too, if so yes | 15:25 |
| gitlab-br-bot | buildstream: merge request (Qinusty/gnu-mirror-backport-1.2->bst-1.2: Qinusty/gnu mirror backport 1.2) #668 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/668 | 15:25 |
| qinusty | tpollard, I think it missed feature freeze, so probably not | 15:26 |
| skullman | qinusty: good enough for me | 15:28 |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspace_reverse_dependencies->master: Invalidate reverse dependencies to scheduled element with workspace) #669 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/669 | 15:36 |
| gitlab-br-bot | buildstream: merge request (valentindavid/fallback_mirror_ostree-1.2->bst-1.2: Fix ostree repository mirroring) #667 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/667 | 15:37 |
| gitlab-br-bot | buildstream: merge request (valentindavid/fallback_mirror_ostree-1.2->bst-1.2: Fix ostree repository mirroring) #667 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/667 | 15:37 |
| gitlab-br-bot | buildstream: merge request (jmac/cas_virtual_directory->master: CAS-backed virtual directory implementation) #481 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/481 | 15:41 |
| *** leopi has joined #buildstream | 15:46 | |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 15:53 |
| *** finn_ has joined #buildstream | 15:56 | |
| *** finn has quit IRC | 15:57 | |
| gitlab-br-bot | buildstream: merge request (Qinusty/531-fetch-retries-on-terminate->master: Prevent jobs retrying on terminate) #662 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/662 | 16:03 |
| gitlab-br-bot | buildstream: merge request (tpollard/483->master: plugins/git.py: Warn if ref is not in given track) #564 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 16:20 |
| *** toscalix has quit IRC | 16:33 | |
| gitlab-br-bot | buildstream: merge request (Qinusty/message-helpers->master: WIP: Continued work on improving BuildStream messaging API) #670 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/670 | 16:54 |
| gitlab-br-bot | buildstream: issue #587 ("Release BuildStream on PyPi") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/587 | 16:55 |
| *** tpollard has quit IRC | 16:55 | |
| gitlab-br-bot | buildstream: issue #589 ("pip element should install dependencies pulled by pip source") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/589 | 17:00 |
| tiagogomes | [00:00:00][e801946f][build:app.bst ] SUCCESS foo/app/e801946f-build.12744.log | 17:01 |
| tiagogomes | What is the sorcery that makes e801946f appear inside the brackets ^ | 17:01 |
| gitlab-br-bot | buildstream: merge request (jmac/cas_virtual_directory->master: CAS-backed virtual directory implementation) #481 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/481 | 17:03 |
| tiagogomes | Is this the cache key of the element? | 17:03 |
| *** jonathanmaw has quit IRC | 17:13 | |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 17:29 |
| *** finn_ has quit IRC | 17:40 | |
| *** phildawson has quit IRC | 17:40 | |
| *** WSalmon has quit IRC | 17:40 | |
| *** Nexus has quit IRC | 17:40 | |
| *** johnward has quit IRC | 17:40 | |
| *** kailueke[m] has quit IRC | 17:40 | |
| *** awacheux[m] has quit IRC | 17:40 | |
| *** m_22[m] has quit IRC | 17:40 | |
| *** mattiasb has quit IRC | 17:40 | |
| *** cgmcintyre[m] has quit IRC | 17:40 | |
| *** inigomartinez has quit IRC | 17:40 | |
| *** theawless[m] has quit IRC | 17:40 | |
| *** oknf[m] has quit IRC | 17:40 | |
| *** tlater has quit IRC | 17:40 | |
| *** valentind has quit IRC | 17:40 | |
| *** csoriano has quit IRC | 17:40 | |
| *** mablanch has quit IRC | 17:40 | |
| *** skullman has quit IRC | 17:40 | |
| *** cs-shadow has quit IRC | 17:40 | |
| *** tintou has quit IRC | 17:40 | |
| *** abderrahim[m] has quit IRC | 17:40 | |
| *** albfan[m] has quit IRC | 17:40 | |
| *** ssssam[m] has quit IRC | 17:40 | |
| *** TingPing has quit IRC | 17:40 | |
| *** Nexus has joined #buildstream | 17:41 | |
| *** WSalmon has joined #buildstream | 17:41 | |
| *** johnward has joined #buildstream | 17:41 | |
| *** cs-shadow has joined #buildstream | 17:41 | |
| *** phildawson has joined #buildstream | 17:41 | |
| *** finn_ has joined #buildstream | 17:41 | |
| *** csoriano has joined #buildstream | 17:41 | |
| *** tlater has joined #buildstream | 17:41 | |
| *** valentind has joined #buildstream | 17:41 | |
| *** tintou has joined #buildstream | 17:41 | |
| *** mablanch has joined #buildstream | 17:42 | |
| *** skullman has joined #buildstream | 17:45 | |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 18:23 |
| *** kailueke[m] has joined #buildstream | 18:38 | |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: Allow source plugins to access previous sources) #568 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 18:59 |
| *** WSalmon_ has joined #buildstream | 19:09 | |
| *** WSalmon has quit IRC | 19:09 | |
| *** TingPing has joined #buildstream | 19:25 | |
| *** leopi has quit IRC | 19:33 | |
| gitlab-br-bot | buildstream: merge request (tiagogomes/issue-573->master: Reduce IO overhead caused by artifact cache size calculation) #671 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/671 | 19:49 |
| *** abderrahim[m] has joined #buildstream | 20:03 | |
| *** tristan has joined #buildstream | 20:03 | |
| *** ssssam[m] has joined #buildstream | 20:06 | |
| *** albfan[m] has joined #buildstream | 20:14 | |
| gitlab-br-bot | buildstream: issue #381 ("Introduce a new source plugin - `SourceTransform`") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/381 | 20:21 |
| gitlab-br-bot | buildstream: merge request (chandan/sourcetransform->master: Allow source plugins to access previous sources) #568 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/568 | 20:21 |
| *** tristan has quit IRC | 21:27 | |
| *** edb has quit IRC | 22:04 | |
| *** connorshea[m] has quit IRC | 22:56 | |
| *** slaf has quit IRC | 22:56 | |
| *** krichter[m] has quit IRC | 22:56 | |
| *** segfault3[m] has quit IRC | 22:56 | |
| *** dineshdb[m] has quit IRC | 22:56 | |
| *** waltervargas[m] has quit IRC | 22:56 | |
| *** ptomato[m] has quit IRC | 22:56 | |
| *** jjardon has quit IRC | 22:56 | |
| *** persia has quit IRC | 22:56 | |
| *** albfan[m] has quit IRC | 22:56 | |
| *** ssssam[m] has quit IRC | 22:56 | |
| *** abderrahim[m] has quit IRC | 22:56 | |
| *** TingPing has quit IRC | 22:56 | |
| *** kailueke[m] has quit IRC | 22:56 | |
| *** skullman has quit IRC | 22:56 | |
| *** tlater has quit IRC | 22:56 | |
| *** finn_ has quit IRC | 22:56 | |
| *** Nexus has quit IRC | 22:56 | |
| *** csoriano has quit IRC | 22:56 | |
| *** cs-shadow has quit IRC | 22:56 | |
| *** ironfoot has quit IRC | 22:56 | |
| *** bethw has quit IRC | 22:56 | |
| *** rafaelff[m] has quit IRC | 22:56 | |
| *** Demos[m] has quit IRC | 22:56 | |
| *** jjardon[m] has quit IRC | 22:56 | |
| *** pro[m] has quit IRC | 22:56 | |
| *** alatiera has quit IRC | 22:56 | |
| *** asingh_[m] has quit IRC | 22:56 | |
| *** mablanch has quit IRC | 22:56 | |
| *** tintou has quit IRC | 22:56 | |
| *** gitlab-br-bot has quit IRC | 22:56 | |
| *** tiagogomes has quit IRC | 22:56 | |
| *** mohan43u has quit IRC | 22:56 | |
| *** milloni has quit IRC | 22:56 | |
| *** rdale has quit IRC | 22:56 | |
| *** aiden has quit IRC | 22:56 | |
| *** jmac has quit IRC | 22:56 | |
| *** tiagogomes_ has quit IRC | 22:56 | |
| *** jennis has quit IRC | 22:56 | |
| *** adds68 has quit IRC | 22:56 | |
| *** lantw44 has quit IRC | 22:56 | |
| *** qinusty has quit IRC | 22:56 | |
| *** coldtom has quit IRC | 22:56 | |
| *** Trevinho has quit IRC | 22:56 | |
| *** slaf has joined #buildstream | 22:56 | |
| *** albfan[m] has joined #buildstream | 22:56 | |
| *** ssssam[m] has joined #buildstream | 22:57 | |
| *** abderrahim[m] has joined #buildstream | 22:57 | |
| *** TingPing has joined #buildstream | 22:57 | |
| *** kailueke[m] has joined #buildstream | 22:57 | |
| *** skullman has joined #buildstream | 22:57 | |
| *** tlater has joined #buildstream | 22:57 | |
| *** csoriano has joined #buildstream | 22:57 | |
| *** finn_ has joined #buildstream | 22:57 | |
| *** cs-shadow has joined #buildstream | 22:57 | |
| *** Nexus has joined #buildstream | 22:57 | |
| *** ironfoot has joined #buildstream | 22:57 | |
| *** bethw has joined #buildstream | 22:57 | |
| *** rafaelff[m] has joined #buildstream | 22:57 | |
| *** Demos[m] has joined #buildstream | 22:57 | |
| *** jjardon[m] has joined #buildstream | 22:57 | |
| *** pro[m] has joined #buildstream | 22:57 | |
| *** alatiera has joined #buildstream | 22:57 | |
| *** asingh_[m] has joined #buildstream | 22:57 | |
| *** persia has joined #buildstream | 22:57 | |
| *** mablanch has joined #buildstream | 22:57 | |
| *** tintou has joined #buildstream | 22:57 | |
| *** gitlab-br-bot has joined #buildstream | 22:57 | |
| *** tiagogomes has joined #buildstream | 22:57 | |
| *** mohan43u has joined #buildstream | 22:57 | |
| *** milloni has joined #buildstream | 22:57 | |
| *** rdale has joined #buildstream | 22:57 | |
| *** aiden has joined #buildstream | 22:57 | |
| *** jmac has joined #buildstream | 22:57 | |
| *** tiagogomes_ has joined #buildstream | 22:57 | |
| *** jennis has joined #buildstream | 22:57 | |
| *** lantw44 has joined #buildstream | 22:57 | |
| *** adds68 has joined #buildstream | 22:57 | |
| *** coldtom has joined #buildstream | 22:57 | |
| *** qinusty has joined #buildstream | 22:57 | |
| *** Trevinho has joined #buildstream | 22:57 | |
| *** jjardon has joined #buildstream | 22:57 | |
| *** oknf[m] has joined #buildstream | 23:25 | |
| *** theawless[m] has joined #buildstream | 23:26 | |
| *** rdale has quit IRC | 23:50 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!