IRC logs for #buildstream for Wednesday, 2020-01-22

*** juergbi has quit IRC02:55
*** juergbi has joined #buildstream03:15
*** delli3_ has joined #buildstream03:24
*** dellie__ has quit IRC03:26
*** narispo has joined #buildstream04:17
*** dellie__ has joined #buildstream06:24
*** delli3_ has quit IRC06:26
*** phildawson has joined #buildstream08:19
*** traveltissues has joined #buildstream08:33
*** tme5 has joined #buildstream09:41
*** dellie__ has quit IRC09:48
*** slaf has quit IRC09:56
*** jonathanmaw has joined #buildstream10:27
*** slaf has joined #buildstream10:36
*** slaf has joined #buildstream10:36
*** slaf has joined #buildstream10:36
*** slaf has joined #buildstream10:36
*** slaf has joined #buildstream10:37
*** phildawson-ct has joined #buildstream10:58
*** phildawson has quit IRC10:58
*** santi has joined #buildstream11:37
*** phildawson-ct has quit IRC11:50
*** pointswaves has joined #buildstream12:15
coldtomwould it be possible to get some review and hopefully a merge on !1806 please?13:13
gitlab-br-botMR !1806: Stop plugin tests using private API https://gitlab.com/BuildStream/buildstream/merge_requests/180613:13
*** pointswaves has quit IRC13:18
gitlab-br-bottpollard approved MR !1806 (coldtom/testing-api->master: Stop plugin tests using private API) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/180613:21
tme5is there any reason why a test using a git repo wouldn't be doing a full fetch?14:03
tme5tempfiles sticking around? would have thought they'd be cleared14:03
tme5or a reason why some status messages might not be printed out during a test14:23
coldtomtme5: i suspect the source cache may be your issue14:23
tme5i've tried rm -rf .tox/py37/tmp14:24
tme5is there somewhere else i should delete?14:26
tme5but tbf the "running host command" of the thing which checks if it even needs to fetch is not showing up14:26
coldtomintegration_cache is the offender i believe14:27
* coldtom just had a similar problem with https://gitlab.com/BuildStream/bst-plugins-experimental/merge_requests/7214:28
tme5integration_cache where?14:28
benschubertNormally, all caches and such are set to new temporary directories so they should always be empty at the start of a specific test14:29
coldtomshould be in the project root i think14:29
tme5n'existe pas14:29
tme5i think it might be that the logs are incomplete somehow14:30
coldtomi only experienced this issue when using buildstream.testing.runcli.cli_integration rather than buildstream.testing.cli14:31
tme5ah, looking in the full log file shows me everything14:31
*** phildawson has joined #buildstream14:32
coldtomunrelated, but https://gitlab.com/BuildStream/bst-plugins-experimental/merge_requests/72 could do with a review if anyone has time14:32
tme5benschubert, another issue with my branch currently is that a user will get the no-ref-in-track warning when opening a workspace on a shallow-fetched repo14:39
benschubertthat is expected if you get it when you fetch it the first time too14:40
benschubertor do you mean you can fetch correctly but get it at workspace open time?14:40
tme5the check happens in validate_cache14:41
tme5i'm seeing the warning when i open the workspace14:41
benschubertDo you see it too when you track/fetch ?14:42
tme5not when fetching14:46
benschubertwhen tracking?14:47
tme5nope14:50
benschubertIf it is not when tracking nor fetching, then there is something wrong with your local cache. Would something get modified between?14:51
*** xjuan has joined #buildstream15:39
tme5ah looks like I misread. it's actually after fetching that I get the error15:41
tme5what is the purpose of the warning?15:46
*** santi has quit IRC15:47
*** santi has joined #buildstream15:47
tme5i suppose to make sure the element is not in some unresolved configuration. but if a project makes this warning an error they will be unable to use the 'git' source with tag refs15:47
benschubertthis should probably be checked differently for tag refs then15:50
benschubertI really wonder whether we should not instead slim down the gitsourcebase, separate cleanly git/git_tag/git_lfs/git_submodule and try to have smaller plugins that do one thing well (sharing common parts and living under a single repo is a big plus :) )15:54
coldtombenschubert: i think that's a nice idea with some implementation details to be figured out15:55
coldtomfor example, we likely have a combinatorial explosion of plugins, such as git_lfs_tag_submodule, if i want to track tags on a git submodule that uses lfs, say15:55
tme5personally i don't think that solves any of our issues15:55
benschubertcoldtom: not necessarily. a git submodule is still a git repo. So you could have a first 'git_tag' source, then a 'git_submodule' source transform and finally a 'git_lfs' source transform in the sub directory15:56
benschuberttme5: could you elaborate? :)15:57
coldtomah, i see if we use source transforms for lfs then it makes sense15:57
coldtomat present we have git-lfs support as part of git_tag, but only in bst-external i think15:57
tme5what is a source transform?15:57
benschubertYeah, lfs cannot live anywhere else than in a git repo. So it seems to be a good fit for a source transform15:57
benschuberttme5: it's a source that cannot be the 'top level' source, and gets access to its parent sources15:57
benschubert(when you track, you already have your parents in the staging area)15:58
benschubertFor example, we have 'patch' and 'quilt' that are source transforms15:58
benschubertor pip_source15:58
benschubertand it is allowed to operate on the parent source15:58
benschubertcoldtom: I would be worried to move 'lfs' support in the 'git' plugin. It would end up doing many different things, and require 2h of documentation read to know how to implement correctly, and would be harder to know what's happening at a glance16:01
tme5I see. But I think that's beside the point right now. This fetch optimisation would have this issue regardless of which specific git plugin I might add it too16:04
benschubertoh, about only fetching with --depth=1 ?16:05
tme5yes16:05
benschubertgah, I misunderstood you, I thought that was when using the 'git_tag' functionallity moved to the 'git' plugin16:06
tme5oh sorry, I'm talking about my new branch/MR16:06
benschubertI jsut did:16:09
benschubertgit clone $my_repo --depth 116:09
benschubertcd my_repo16:09
benschubertgit branch --list master --contains $(git rev-parse EHAD)16:09
benschubertThis does give me the ref. Is there something else I'm missing?16:09
tme5yes, you cloned the master branch ref16:10
tme5if you fetch a tag you get no branch git refs16:10
benschubert                _, tag = self.check_output(16:10
benschubert                    [self.host_git, "tag", "--list", self.tracking, "--contains", self.mirror.ref],16:11
benschubert                    cwd=self.mirror.mirror,16:11
benschubert                )16:11
benschubertShould be taking care of that part no?16:11
tme5hmm, yeah it should16:13
tme5ah no it shouldn't16:14
tme5because it uses self.tracking16:14
tpollardI was literally talking about that snippet elsewhere16:15
tpollardhow weird16:15
benschuberttme5: not sure I follow around the 'self.tracking'16:28
tme5the command will run .e.g git tag --list master16:28
tme5but master will not exist in the local repo16:29
benschubertgah I see16:44
tme5i would suggest i depth=1 fetch all refs but i don't think that would solve anything, since the refs would just be floating and the --contains would probably not work16:47
tme5will try though16:48
*** traveltissues has quit IRC16:52
benschubertand that will potentially pull a lot of things x')16:58
benschubertcould we also fetch the branch in addition to the tag or something?16:58
tme5the whole branch?17:16
tme5that would work, but again that's a lot of things17:17
benschubertor everything between both commits? I don't know if that is feasible or not17:17
benschubertyeaha ctually the whole branch would remove the interest of the patch17:18
*** santi has quit IRC17:31
*** tme5 has quit IRC17:36
*** xjuan has quit IRC17:41
*** pointswaves has joined #buildstream17:41
*** xjuan has joined #buildstream17:42
*** xjuan has quit IRC17:46
*** santi has joined #buildstream17:48
*** pointswaves has quit IRC17:53
*** pointswaves has joined #buildstream17:57
*** pointswaves_ has joined #buildstream18:00
*** pointswaves has quit IRC18:00
*** pointswaves_ has quit IRC18:02
*** pointswaves_ has joined #buildstream18:03
*** xjuan has joined #buildstream18:03
*** xjuan has quit IRC18:06
*** pointswaves_ has quit IRC18:12
*** pointswaves has joined #buildstream18:16
*** pointswaves_ has joined #buildstream18:21
*** xjuan has joined #buildstream18:22
*** pointswaves_ has quit IRC18:22
*** pointswaves has quit IRC18:22
*** xjuan has quit IRC18:34
*** jonathanmaw has quit IRC18:49
*** santi has quit IRC19:40
*** santi has joined #buildstream20:01
*** pointswaves_ has joined #buildstream20:35
*** cs-shadow has quit IRC21:51
*** santi has quit IRC21:55
*** pointswaves_ has quit IRC22:36
*** cgmcintyre[m] has quit IRC23:00
*** verdre[m] has quit IRC23:00
*** reuben640[m] has quit IRC23:01
*** theawless[m] has quit IRC23:01
*** cgmcintyre[m] has joined #buildstream23:12
*** verdre[m] has joined #buildstream23:14

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!