IRC logs for #buildstream for Monday, 2019-01-21

« Sunday, 2019-01-20 Index Tuesday, 2019-01-22 »
*** nimish has joined #buildstream01:49
*** chill has quit IRC03:21
*** nimish has quit IRC03:21
*** nimish has joined #buildstream03:21
*** nimish has quit IRC03:25
*** mohan43u has quit IRC04:55
*** mohan43u has joined #buildstream04:56
*** alatiera has joined #buildstream08:41
*** toscalix has joined #buildstream08:46
*** alatiera has quit IRC09:05
*** finn has joined #buildstream09:08
*** bochecha has joined #buildstream09:13
*** alatiera has joined #buildstream09:19
*** WSalmon_ has joined #buildstream09:24
*** alatiera has quit IRC09:29
*** alatiera has joined #buildstream09:30
*** alatiera has joined #buildstream09:35
*** rdale has quit IRC09:35
*** benschubert has joined #buildstream09:50
*** rdale has joined #buildstream09:58
*** raoul has joined #buildstream09:59
gitlab-br-botjennis approved MR !1086 (aevri/bst_track_guidance->master: Fixup refs to 'bst track' and 'bst fetch') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/108610:05
*** jonathanmaw has joined #buildstream10:13
*** tpollard has joined #buildstream10:16
*** jonathanmaw has quit IRC10:18
*** lachlan has joined #buildstream10:29
*** jonathanmaw has joined #buildstream10:29
*** jswagner has joined #buildstream10:32
*** nimish has joined #buildstream10:33
jennisjuergbi, thanks for the review of !1045, will make those changes today10:35
gitlab-br-botMR !1045: Move push/pull/checkout to the artifact subcommand group https://gitlab.com/BuildStream/buildstream/merge_requests/104510:35
juergbita10:36
*** nimish has quit IRC10:38
*** nimish has joined #buildstream10:39
*** nimish has joined #buildstream10:39
*** nimish has quit IRC10:54
*** nimish has joined #buildstream10:54
*** raoul_ has joined #buildstream11:00
*** raoul has quit IRC11:01
*** nimish has quit IRC11:09
*** nimish has joined #buildstream11:10
*** alatiera has quit IRC11:13
*** alatiera has joined #buildstream11:14
*** alatiera has joined #buildstream11:15
*** alatiera has joined #buildstream11:16
valentindI think I figured out why the tests failed on aarch64. We do not make sure that cache/integration-cache exists. For x86_64, this comes from the gitlab cache. But there was no build that populated the cache yet on aarch64. Or the cache has been invalidated.11:27
*** nimish has quit IRC11:35
*** nimish has joined #buildstream11:36
gitlab-br-botvalentindavid opened MR !1092 (valentindavid/make_cache_dir->master: Make sure testing cache directory exists) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/109211:37
*** nimish has quit IRC11:41
*** nimish has joined #buildstream11:41
tlater[m]valentind: I noticed that while running tests offline as well, integration-cache must exist for integration tests to run successfully. It's a little annoying, we should probably change that.11:43
tpollardyep same, when running tests outside of my main bst checkout11:44
*** nimish has quit IRC11:46
*** nimish has joined #buildstream11:46
gitlab-br-botvalentindavid approved MR !1085 (richardmaw/centos-oldgit-test-fixes->master: Fix CentOS) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/108511:47
gitlab-br-botvalentindavid closed issue #833 (Tests are broken on CentOS) on buildstream https://gitlab.com/BuildStream/buildstream/issues/83311:47
gitlab-br-botvalentindavid merged MR !1085 (richardmaw/centos-oldgit-test-fixes->master: Fix CentOS) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/108511:47
*** nimish has quit IRC11:56
*** nimish has joined #buildstream11:57
*** nimish has quit IRC12:02
*** nimish has joined #buildstream12:02
juergbiraoul_: I meant consistency of the interaction between the pair ArtifactCache/Element and the pair SourceCache/Source12:07
jennisWould appreciate it if anyone could spare some time to review: https://gitlab.com/BuildStream/buildstream/merge_requests/108812:07
jennisIt involves adding two new profiling topics12:08
*** raoul_ is now known as raoul12:08
juergbiraoul: i.e., following the approach of ArtifactCache/Element, Source would call into SourceCache12:08
*** nimish has quit IRC12:12
*** nimish has joined #buildstream12:12
*** nimish has joined #buildstream12:13
*** nimish has joined #buildstream12:13
raoulAh right, I misunderstood then. I'll have a think about what this would require then, as I think implementation would require more changes12:14
*** nimish has quit IRC12:18
*** nimish has joined #buildstream12:19
*** nimish has joined #buildstream12:19
*** raoul has quit IRC12:22
*** nimish has quit IRC12:34
*** nimish has joined #buildstream12:35
*** nimish has quit IRC12:54
*** nimish has joined #buildstream12:55
*** nimish has quit IRC12:56
*** nimish has joined #buildstream12:56
*** nimish has quit IRC13:06
*** nimish has joined #buildstream13:06
*** nimish has joined #buildstream13:06
*** raoul has joined #buildstream13:13
raouljuergbi, haven't had a bit more of a look it looks like the implementation would be marginally more tricky, but I think it's probably worth it to have the source cache and artifact cache follow the same logic. There doesn't really seem to be a significant advantage to have it one way or the other13:32
juergbiok, sounds good to me13:34
*** nimish has quit IRC13:36
*** nimish has joined #buildstream13:37
*** solid_black has joined #buildstream13:38
*** nimish has quit IRC13:42
*** nimish has joined #buildstream13:42
*** nimish has quit IRC13:47
*** nimish has joined #buildstream13:48
*** nimish has quit IRC13:58
*** nimish has joined #buildstream13:58
*** raoul has quit IRC14:02
*** raoul has joined #buildstream14:24
gitlab-br-botjennis opened issue #876 (Profiling - topics 'load-loader' and 'show' are not actually profiled) on buildstream https://gitlab.com/BuildStream/buildstream/issues/87615:07
gitlab-br-botjennis opened issue #877 (Profiling - 'all' option does not produce complete profiles for all topics) on buildstream https://gitlab.com/BuildStream/buildstream/issues/87715:16
*** lachlan has quit IRC15:40
*** lachlan has joined #buildstream15:42
*** nimish has quit IRC15:48
*** nimish has joined #buildstream15:48
*** raoul has quit IRC15:52
*** raoul has joined #buildstream16:05
*** nimish has quit IRC16:08
*** nimish has joined #buildstream16:09
tpollardDoes anyone mind taking a look at https://gitlab.com/BuildStream/buildstream/merge_requests/1050 please?16:14
*** bochecha has quit IRC16:31
*** nimish has quit IRC16:34
*** nimish has joined #buildstream16:34
gitlab-br-botcs-shadow opened MR !1093 (chandan/unpin-sphinx->master: tox.ini: Unpin sphinx dependency) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/109316:44
*** nimish has quit IRC16:59
*** nimish has joined #buildstream17:00
raoulsomeone correct me if I'm wrong, but is tests/integration/cachedfail.py::test_push_cached_fail set up wrong? It's set up to assert that a failed build is pushed on both continue and error, though it should be only be pushed on continue. I think it has been passing because the artifact cache was actually kept for the whole test session17:00
raoulso now that I've changed how that artifact cache bit works for #870 it's failing17:01
gitlab-br-botIssue #870: Root cache directory https://gitlab.com/BuildStream/buildstream/issues/87017:01
juergbiraoul: hm, so it pushed the failed artifact to the second remote as well because it was already in the local cache from the first test and thus the push could be scheduled before on-error quit is triggered?17:10
juergbinot pushing with on-error quit indeed makes sense for me as well17:11
raouljuergbi, Yeah I believe so17:14
*** nimish has quit IRC17:15
*** nimish has joined #buildstream17:15
raoulThough I have now realised a lot of artifacts are actually shared between tests and this slows down the test suite significantly so I may have to rethink this a bit17:15
*** nimish has joined #buildstream17:15
raoulregardless, that test should probably ensure it's not in the cache to begin with and isn't pushed in the on-error=quit case17:16
*** nimish has quit IRC17:20
*** nimish has joined #buildstream17:21
*** nimish has quit IRC17:26
*** nimish has joined #buildstream17:26
tpollardraoul: I agree, if integration tests need a 'non standard' artifact state that's not useful to other states then it shouldn't inherit the same local cache17:42
tpollardimo17:42
tpollardsome tests interacting with pull-buildtrees do that17:44
*** toscalix has quit IRC17:44
*** raoul_ has joined #buildstream17:46
*** raoul has quit IRC17:47
*** nimish has quit IRC17:51
*** nimish has joined #buildstream17:52
*** raoul_ is now known as raoul17:58
raoulyeah we should probably figure a better way of dealing with the integration artifact cache, having each test start with a fresh artifact cache does double the test suite time so that's not really acceptable17:58
*** tristan has joined #buildstream17:59
*** nimish has quit IRC17:59
tlater[m]raoul: Maybe we could add better helper functions to assert cache state.18:02
tlater[m]Adding a function to easily check for an exact set of artifacts would go a long way, integration tests could then call that whenever they're run.18:03
tlater[m]Maybe integrate that into the integration fixture, so that it will only run if either no artifacts are in the cache, or some specific, acknowledged artifacts are?18:04
*** nimish has joined #buildstream18:05
*** ChanServ sets mode: +o tristan18:06
tristanraoul, tlater[m] ... I see you're talking about what I wanted to ask about... looking at !1013 it seems that some of the things at least from the bullet list at the top have already been merged (refactor of the _artifactcache/ -> _cas/ folder already happened)... how much of that branch is still active ?18:06
gitlab-br-botMR !1013: Refactor artifactcache https://gitlab.com/BuildStream/buildstream/merge_requests/101318:06
tristanI have a queue of tweaks and twists for artifact cache management here which I have to polish off on !109118:07
gitlab-br-botMR !1091: Cache management fixes https://gitlab.com/BuildStream/buildstream/merge_requests/109118:07
raoultristan, that branch isn't active atm, it's been postponed as the rest of the stuff in there isn't needed for the source cache18:08
*** nimish has quit IRC18:08
raoulI've been looking at the cache restructuring that you brought up during the review of it though, seems sensible to get that in before source cache18:08
tristanraoul, thanks for letting me know, I wonder if it makes sense to take down the associated MR in the meantime ?18:08
tristanI was a bit worried that I would cause trouble/conflicts with a large open MR18:09
tristanraoul, at least, set it WIP please :)18:09
raoulYeah I'll do that, I think the plan is to finish work on it and get it merged after source cache so it doesn't quite make sense to close it18:10
*** nimish has joined #buildstream18:10
tristanthanks18:10
*** nimish has quit IRC18:13
*** finn has quit IRC18:14
raouland tlater[m], I think perhaps putting it into integration fixture makes sense, maybe have an option to either allow an artifact to be present or to delete an artifact before starting the test?18:15
tlater[m]raoul: Hm, should think through a proper API for this, I think.18:22
tlater[m]I think flat out failing if an artifact is there that shouldn't be is better; it shows the error18:22
tlater[m]Only permit certain artifacts, and the user needs to be explicit about those. We could allow the base image artifact by default.18:23
juergbitristan, cs-shadow: do you have any input on the desired user semantics with regards to buildtree pushing? https://gitlab.com/BuildStream/buildstream/merge_requests/1024#note_13236070918:23
raoultlater[m], but that relies on the tests that happen before it which I don't think is a good idea.18:24
cs-shadowjuergbi: hi, I’m on the phone at present but I’ve added it to my list to have a look at the MR18:25
tlater[m]raoul: Nah, I don't think that's a good idea either.18:25
tlater[m]I'm suggesting allowing certain artifacts to exist when a test starts18:26
juergbita cs-shadow, no rush18:26
tristanjuergbi, I'll give it some more thought, I'm seeing the cache fill up dramatically with these build trees so I'm actually building a perspective as we speak18:29
juergbiok, ta18:30
* tristan now has a cache usage percentage indicator in his status bar and watches it rise while artifacts get cached18:30
raoultlater[m], as in a whitelist and not allow others? That doesn't seem ideal either, or am I misunderstanding?18:33
*** jonathanmaw has quit IRC18:34
tlater[m]That's what I mean. What would the downside be?18:34
*** nimish has joined #buildstream18:35
raoulIf a test is changed to create an extra artifact, a following test might fail18:36
raouleach test shouldn't have a chance of failing dependent on the state of the shared artifact cache18:37
tlater[m]Right, that's a point.18:37
tlater[m]Removing does seem more adequate then18:37
*** nimish has quit IRC18:38
raoulyeah it would ensure a specified state before a test, but wouldn't have a chance of failing18:39
*** tristan has quit IRC18:41
*** nimish has joined #buildstream18:41
*** raoul has quit IRC18:50
*** nimish has quit IRC18:55
*** finn has joined #buildstream18:58
*** tristan has joined #buildstream18:59
gitlab-br-botcs-shadow opened MR !1094 (chandan/fix-contrib-link->master: CONTRIBUTING.rst: Fix formatting of link to pip docs) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/109419:39
*** tristan has quit IRC19:45
gitlab-br-botcs-shadow merged MR !1094 (chandan/fix-contrib-link->master: CONTRIBUTING.rst: Fix formatting of link to pip docs) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/109420:16
*** jonathanmaw has joined #buildstream21:07
*** lachlan has quit IRC21:15
*** alatiera has quit IRC23:06
« Sunday, 2019-01-20 Index Tuesday, 2019-01-22 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!