IRC logs for #buildstream for Thursday, 2019-03-07

*** nimish has joined #buildstream00:54
*** nimish has quit IRC01:05
*** swick has quit IRC03:04
*** tristan has joined #buildstream03:46
*** phil has quit IRC03:47
*** tpollard has quit IRC04:07
*** tristan has quit IRC04:43
*** tristan has joined #buildstream04:49
*** tristan has quit IRC05:02
*** tristan has joined #buildstream05:30
*** mohan43u has joined #buildstream05:50
*** toscalix has joined #buildstream06:31
*** toscalix has quit IRC06:44
*** toscalix has joined #buildstream06:45
*** tpollard has joined #buildstream08:08
*** toscalix has quit IRC08:22
*** ChanServ sets mode: +o tristan08:45
tristanjuergbi, Think it would be reasonable to have BuildStream make an attempt to configure the terminal settings in early startup ?08:45
tristanjuergbi, i.e. see latest comment: https://gitlab.com/BuildStream/buildstream/issues/929#note_14813733908:46
juergbitristan: could make sense. maybe check whether ncurses apps typically do this and do the same?08:51
tristanYeah, I think it's either do that or error out early08:52
*** ikerperez has joined #buildstream09:06
*** jennis has joined #buildstream09:11
gitlab-br-botjennis opened MR !1209 (jennis/remove_node_chain_stuff->master: Rip out ChainMap(), node_chain_copy(), node_list_copy()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/120909:26
*** toscalix has joined #buildstream09:31
*** toscalix has quit IRC09:33
gitlab-br-botdanielsilverstone-ct approved MR !1209 (jennis/remove_node_chain_stuff->master: Rip out ChainMap(), node_chain_copy(), node_list_copy()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/120909:34
*** toscalix has joined #buildstream09:34
gitlab-br-botBenjaminSchubert opened (was WIP) MR !1070 (bschubert/pipeline->master: Refactor _update_state() to be called only when needed) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/107009:39
benschuberttristan: https://gitlab.com/BuildStream/buildstream/merge_requests/1070/ I've updated the PR to it would also take runtime deps of build deps in account. Could you have a last look ? I also added my latest benchmarking information.09:40
*** phildawson has joined #buildstream09:42
tristanbenschubert, of course, I'm waiting to backport that to 1.2 haha09:42
tristanbenschubert, I think you already squashed it as per jonathanmaw's comment (so that we don't have history of approaches which didnt land)09:44
tristanis that right ?09:45
benschuberttristan: I didn't yet no09:45
benschubertwas planning to do that after the review09:45
gitlab-br-botphildawson opened issue #949 (`integration` pytest mark we use in our test suite is a misnomer) on buildstream https://gitlab.com/BuildStream/buildstream/issues/94909:52
benschuberttristan: should I merge all commits together or would you keep some separate?09:52
tristanbenschubert, I like commits to be separate and granular, so long as: A) We keep the tests passing for every commit... B.) We don't include stuff that's not going in09:54
benschuberttristan: then let's keep it like that and just rewrite the one commit message you noticed?09:54
tristanI think so, your branch doesnt appear to contain fixes of previous commits or pivots in approaches, that's the kind of thing we try to avoid09:55
benschubertagreed09:55
benschuberttristan: one question to make sure about the recursive stuff. dependencies() with scope=Scope.BUILD, recurse=False, will give only direct build dependencies AND their runtime dependencies? IS that correct? or only the direct build dependencies?10:00
tristancrap did I discard the comment which took me a while to write ?10:01
benschubertOh the previosu one, no it's still there10:01
benschuberthttps://gitlab.com/BuildStream/buildstream/merge_requests/1070#note_148176406 you mean?10:01
benschubertor which one?10:02
tristanAh no, I didnt, https://gitlab.com/BuildStream/buildstream/merge_requests/1070/diffs#note_148176406 indeed showed up :)10:03
tristangitlab UI slow10:03
tristanbenschubert, to answer your question, recursive=False will only yield direct dependencies, and furthermore, it will never yield the element itself10:06
tristanit is a rather special case for examining only one level10:06
benschubertok! I will fix that, great, it will make performance even better :D10:07
tristanbenschubert, regarding https://gitlab.com/BuildStream/buildstream/merge_requests/1070/diffs#note_148173233...10:07
tristanI don't think making recurse=False in that call fixes things, or I doubt it10:07
tristanunconditionally building a list of reverse dependencies would, as I understand you want to be selective and reduce that list10:08
tristanif we do, then that list should probably be a set() ?10:08
benschubertYeah that's true, I don't need recurse :)10:10
benschubertand the set is not needed because the .dependencies function only returns every element once10:11
tristanUh oh10:12
tristanbenschubert, we have a bug I believe10:12
tristanbenschubert, you are doing this from _new_from_meta()10:13
benschubertyes10:13
tristanbenschubert, which has an early return for a good reason, elements will be hit multiple times from different dependency paths10:13
tristanI believe that you are only capturing the reverse dependencies from one of the paths, even though the element can be reached by many10:14
tristanI think that it will be much simpler to unconditionally collect all reverse dependencies, and avoid calling Element.dependencies() at all for this10:15
benschuberttristan: I am registering when I complete an element on all its dependencies10:15
benschuberttristan: "I think that it will be much simpler to unconditionally collect all reverse dependencies, and avoid calling Element.dependencies() at all for this" => we will be adding too much, the real thing we need is the build dependencies10:15
benschubertso I'm calling once the element is complete10:16
benschubertI don't see what the bug would be?10:16
tristanbenschubert, consider the case where something *else* depends on that element, and it is already constructed10:17
tristani.e. we iterate from top down, from a target into a bunch of orthogonal dependencies which all depend on the same base10:17
tristanThe first iteration constructs the base, and updates this reverse dependency list10:17
benschubertthe link is only created as we move up10:17
benschubertNo10:18
benschubertyes10:18
tristanThe next iterations (in parent stack frames), construct orthogonal reverse dependencies and then early return in the common, already constructed base dependency10:19
benschubertyes10:19
tristanSo those orthogonal elements which depend on the base, will not get added to the reverse dependency list, as this list is only constructed for the first time _new_from_meta() is called10:20
tristanbenschubert, understand ?10:20
benschubertNo, let me try:10:20
tristan "if meta in cls.__instantiated_elements:"10:20
benschubert1) You construct your base. Then go through all its dependencies, and register the base as one of their rdep. There is none.10:21
benschubert2) You construct all the elements (orthogonal) that have a dependency to your base, and for each element you register them in the base as a rdep10:21
tristanbenschubert, except you dont do (2)10:22
tristannot in your branch10:22
benschubertline 981 and 982 exactly do that10:22
benschubertnote that they are working on my dependencies, not on my element10:22
benschubertwhen an element is built, all its dependencies are already built10:23
benschubertso I can safely register my element on them10:23
benschubertas a rdep10:23
tristanHmmm10:23
benschubert(brb)10:23
* tristan will not however, have a dinner appointment10:24
tristanbenschubert, I'll take a closer look, but I think you are right10:24
*** jonathanmaw has joined #buildstream10:24
benschuberttristan: I'll address your comments already10:25
benschubertWould be good if we can merge this soon10:25
tristanyeah, it fixes a bad bug (I'm not really concerned much about the performance gains it might bring, and I expect a different round of refactoring and splitting up the activities of _update_state() to be much more effective for optimizing purposes)10:27
tristanit's also nice because it makes more sense to have reverse dependencies notified, the algorithm is more reliable10:28
*** lachlan_ has joined #buildstream10:38
*** swick has joined #buildstream10:46
benschuberttristan: I updated all your comments!10:51
benschubertSo waiting for the last greenlight before sending it to marge10:52
benschubertif anybody else want to have a look, now is the time10:52
* Kinnison doesn't think he has anything to add10:52
*** jonathanmaw_ has joined #buildstream10:53
*** jonathanmaw has quit IRC10:53
*** cs-shadow has quit IRC10:53
*** cs-shadow has joined #buildstream10:53
*** pro[m] has quit IRC10:53
*** cgmcintyre[m] has quit IRC10:53
*** mattiasb has quit IRC10:53
*** Demos[m] has quit IRC10:53
*** skullone[m] has quit IRC10:53
*** dbuch has quit IRC10:53
*** jjardon[m] has quit IRC10:53
benschubertjonathanmaw_: ?10:53
* jonathanmaw_ reads irclogs10:54
benschubertjonathanmaw_: ah I meant do you want a last look at !1070?10:59
gitlab-br-botMR !1070: Refactor _update_state() to be called only when needed https://gitlab.com/BuildStream/buildstream/merge_requests/107010:59
* jonathanmaw_ will have a look10:59
*** jonathanmaw_ is now known as jonathanmaw10:59
*** pro[m] has joined #buildstream11:01
*** cgmcintyre[m] has joined #buildstream11:01
*** mattiasb has joined #buildstream11:02
*** Demos[m] has joined #buildstream11:12
*** lachlan_ has quit IRC11:14
*** lachlan_ has joined #buildstream11:31
*** lachlan_ has quit IRC11:36
*** skullone[m] has joined #buildstream11:48
*** lachlan has joined #buildstream11:51
gitlab-br-botjennis opened MR !1211 (jennis/move_node_get_project_path->master: Cleanup: Move _yaml.node_get_project_path to Project._get_path_from_node()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/121112:00
*** raoul_ has joined #buildstream12:01
jennisjuergbi, regarding our recent discussion, I'd appreciate if you could spare a couple of minutes for ^ (!1211), no worries if not12:02
*** dbuch has joined #buildstream12:20
*** jjardon[m] has joined #buildstream12:22
*** raoul_ has quit IRC12:28
*** jonathanmaw has quit IRC12:43
*** jonathanmaw has joined #buildstream12:45
coldtomis anybody from bst-external around to take a look at https://gitlab.com/BuildStream/bst-external/merge_requests/70 ?12:50
*** raoul_ has joined #buildstream13:23
benschuberttristan: I don't see why we need "Scope.ALL" non recursive in !1070, I answered your comment13:24
*** nimish has joined #buildstream13:37
*** gokcennurlu has joined #buildstream13:53
*** toscalix has quit IRC14:04
*** mablanch_ is now known as mablanch14:16
*** toscalix has joined #buildstream14:36
gitlab-br-botgokcennurlu opened issue #950 (Things to think about when redesigning YAML) on buildstream https://gitlab.com/BuildStream/buildstream/issues/95015:47
gitlab-br-botmarge-bot123 merged MR !1022 (chandan/base-git-mirror->master: Expose _GitMirror as part of plugin author facing API) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/102215:48
*** nimish has quit IRC16:41
*** nimish has joined #buildstream16:45
*** phildawson has quit IRC16:59
*** toscalix has quit IRC17:00
*** nimish has quit IRC17:03
*** jonathanmaw has quit IRC17:05
*** nimish has joined #buildstream17:14
*** phildawson has joined #buildstream17:40
*** tpollard has quit IRC18:00
*** lachlan has quit IRC18:24
*** phildawson has quit IRC18:29
*** nimish has quit IRC18:54
tristanbenschubert, just getting home in a comfortably numb state... I dont see how you handle the case of indirect runtime only dependencies of build-only dependencies in reverse dependency invalidations while considereing only build-only dependencies19:16
tristanbut I will be happy to provide a test case to prove it tomorrow :)19:16
benschuberttristan: ok! I'll be awaiting the test case :D19:18
tristanbenschubert, it will be along the lines of the existing test case which you have cherry picked from #919 already19:19
gitlab-br-botIssue #919: 'bst build <elem>' does not assemble all required elements in some circumstances https://gitlab.com/BuildStream/buildstream/issues/91919:19
benschuberttristan: great, let's add more testcases!19:20
tristanexcept it will additionally contain a case where there is a runtime-only dependency of the build-only dependency19:20
tristanlets drink more soju !20:17
*** tristan has quit IRC21:34
*** cs-shadow has quit IRC22:22

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!