IRC logs for #buildstream for Monday, 2019-11-25

*** tristan has quit IRC01:16
*** narispo has quit IRC02:06
*** narispo has joined #buildstream02:08
*** narispo has quit IRC02:51
*** narispo has joined #buildstream02:51
*** tristan has joined #buildstream08:31
*** rdale has joined #buildstream09:30
*** benschubert has joined #buildstream09:40
*** phildawson has joined #buildstream09:41
*** tpollard has joined #buildstream09:53
*** jonathanmaw has joined #buildstream10:00
benschuberttpollard: hey, do you have the exact ocommit I need to benchmark and the exact command? :)10:03
tpollardbenschubert: https://gitlab.com/traveltissues/scraps/blob/buildsubprocess/results.md :)10:04
*** bochecha has joined #buildstream10:04
benschubertwait what? The code was profiled for your branch and not for master? Or was it not?10:05
tpollardthat profile can only be ran on my branch10:05
benschubertyup, but the benchmarks were run without right?10:06
tpollardI believe traveltissues ran the benchmarks with that profile turned on. It will have added a small overhead to the time, but I don't think it's too relevant right now10:06
tpollardI think getting an exact matching profile of the branch is the main issue10:07
tpollardtraveltissues is off today, I'm sure we can get another end to end time without the profiler on10:07
benschubertok10:08
juergbithe times I reported were without profiling10:10
juergbiftpmirror.gnu.org is down breaking CI due to the autotools example10:11
tpollardouch10:13
*** misterwhatever has joined #buildstream10:29
*** lachlan has joined #buildstream10:29
*** misterwhatever has quit IRC10:35
*** tristan has quit IRC10:50
*** lachlan has quit IRC10:50
*** lachlan has joined #buildstream10:53
*** akvilebirgelyte_ has quit IRC10:59
*** akvilebirgelyte_ has joined #buildstream10:59
*** lachlan has quit IRC11:11
*** lachlan has joined #buildstream11:20
*** mahmoud has joined #buildstream11:26
*** mahmoud has quit IRC11:27
gitlab-br-bottpollard approved MR !1723 (tlater/update-workspace-manpages->master: man/bst-workspace-reset.1: Update workspace re-open help message) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/172311:29
tlater[m]Hrm, wonder if marge is smart enough to handle that11:29
*** lachlan has quit IRC11:31
gitlab-br-botcs-shadow approved MR !1722 (valentindavid/compose-integration-commands->master: compose: Allow compose elements to run integration commands) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/172211:34
*** misterwhatever has joined #buildstream11:46
*** lachlan has joined #buildstream11:47
benschuberttpollard: terminate is broken on your branch btw :)11:50
tpollardthe signal handling on master for me on that build is broken for me too11:51
benschubertwait signal handling is broken on master?11:51
tpollardit behaves really intermittently for me on that build11:52
tpollardbut not in more realistic builds11:52
benschubertso, if you ctrl + c then terminate on master it just continues? Oo11:53
tpollardctrl+c quite often doesn't behave11:55
benschubertah yeah that's another problem11:55
benschubertI meant once ctrl + c has stopped, can you 'terminate' ?11:55
tpollardworks fine on a smaller build, or something like fdsdk11:56
tpollardthe child_handler for the child task in job start() is giving me all sorts of async/signal handling fun12:00
tpollardon that build, specifically12:01
benschubertwhat do you mean?12:01
tpollardpresumably because the pid life is so short12:01
*** lachlan has quit IRC12:02
tpollardbenschubert: basically, I'm trying to handle this failure (which only happens in CI for me) https://gitlab.com/BuildStream/buildstream/-/jobs/35713311812:03
tpollarda watcher is getting triggered that should not have even been attached in this instance (one in Stream, which only gets added if running build)12:04
benschubertbut that's only on your branch right?12:05
tpollardI got around this by explicitly setting the global event loop to None in streams cleanup, and explicitly add the childjob watchers to schedulers event loop (not the global one)12:06
tpollardhowever, on that debian build it seems to trip up adding the childjob handler explicitly to a given loop and not the global one because of pids being reaped12:06
benschubertWhy do you need to change this?12:06
benschubertthere should be _exactly_ one event loop on the whole program no?12:06
*** lachlan has joined #buildstream12:09
benschuberttpollard: how can https://gitlab.com/BuildStream/buildstream/merge_requests/1613/diffs#65c9089123965199008f5d09bc3e5d13b56777dd_189_209 ever be true?12:09
tpollardbenschubert: gitlab's not loading that link to a specific change, which exactly?12:10
benschubertload the page twice :)12:10
tpollardstill does nothing for me, just takes me to the change list12:11
benschubertscheduler.py line 20912:13
tpollardpotentially because it had chosen to auto collapse it for me12:15
tpollardbenschubert: the notify_front_queue should only exist if running bst build12:16
*** lachlan has quit IRC12:17
tpollards/running bst build/running a subproccessed stream method12:17
tpollardwhich in that branch for now is just build12:17
tpollardif there's no multiprocessing, it just notifies as a callback12:18
benschubertAh no sorry, I meant:12:20
benschubert```12:20
benschubert        if not subprocessed:12:20
benschubert            self._casd_process = casd_process12:20
benschubert            _watcher = asyncio.get_child_watcher()12:20
benschubert```12:20
benschubertWhy is there a difference there?12:20
tpollardin the current implementation, if subprocessing I have stream in the 'main process' watching casd & the interrupt signals. Due it being the parent process where casd was launched12:23
tpollardif we're not subproccessing, then the scheduler is executing in the main process, and as such can watch the casd child12:26
tpollardthis may have changed since the casdprocessmanager landed, I haven't rebased to support that yet12:28
tpollardside effect of having the frontend initialise casd before it's actually really needed12:31
*** narispo has quit IRC12:41
*** narispo has joined #buildstream12:42
*** narispo has quit IRC12:53
*** tristan has joined #buildstream12:54
*** narispo has joined #buildstream12:56
gitlab-br-botmarge-bot123 merged MR !1724 (juerg/umask->master: Respect umask for created file and directories) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/172413:07
*** narispo has quit IRC13:19
*** narispo has joined #buildstream13:19
*** misterwhatever has quit IRC13:29
*** narispo has quit IRC13:41
*** narispo has joined #buildstream13:42
*** akvilebirgelyte__ has joined #buildstream13:50
*** akvilebirgelyte_ has quit IRC13:50
*** akvilebirgelyte__ has quit IRC13:52
*** akvilebirgelyte__ has joined #buildstream13:52
*** akvilebirgelyte__ has quit IRC13:57
*** akvilebirgelyte__ has joined #buildstream13:57
*** akvilebirgelyte__ has quit IRC14:21
*** akvilebirgelyte__ has joined #buildstream14:24
*** lachlan has joined #buildstream14:42
*** lachlan has quit IRC14:53
*** bochecha_ has joined #buildstream15:09
gitlab-br-bottlater opened (was WIP) MR !1645 (tlater/artifactserver-casd->master: Refactor casserver.py: Stop relying on the buildstream-internal `CASCache` implementation) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/164515:15
tlater[m]All CI is failing, I guess15:17
tlater[m]Are we skirting around marge until the server issues are resolved or do we just accept no movement for a few days?15:18
benschuberttlater[m]: is it a planned outage? what's happening? :)15:18
tlater[m]benschubert: I haven't checked, I tend to assume not-planned15:19
benschubertBecause I seem to be able to access http://ftp.snt.utwente.nl/pub/software/gnu/15:20
tlater[m]Oh, no, the version of automake we're using looks to just have dropeed D:15:21
tlater[m]s/dropeed/dropped/15:21
tlater[m]Yeah, let me fix that15:21
benschubertah :D15:21
tlater[m]benschubert: I think it's a patch release and they've pulled the original15:21
tlater[m]Probably a security flaw?15:21
benschubertpossibly :) as long as we can fix easily, I don't mind :)15:22
tlater[m]benschubert: Eugh, I'm going to have to run the docs generation. But yeah, not too bad, 2 line fix outside of auto-generated stuff15:23
benschubertgreat, let me know when I should review the fix15:24
*** narispo has quit IRC15:41
*** narispo has joined #buildstream15:41
benschuberttpollard: where is best for me to add my times and profile? on the MR in a comment? I can't add to traveltissues' repo as I don't have access15:56
tpollardsounds perfectly fine to me15:57
benschuberthttps://gitlab.com/BuildStream/buildstream/merge_requests/1613#note_250083875 tpollard Kinnison juergbi :)16:00
Kinnisonare those timings without profiling?16:00
benschubertcorrect16:01
benschubertand minimal usage of my computer16:01
benschubertno other wsl thing running16:01
KinnisonUrgh16:01
tpollardat least it's constant behaviour.....16:01
benschubertI should have access to wsl2 in some time (hopefully before thursday) and will add new benchmarks with this16:02
* tlater[m] wonders why we have an in-tree dependency on fuse.py16:12
tlater[m]It's a little... annoying.16:13
juergbiwill go away with buildbox-run16:13
* tlater[m] is looking forward to that then :D16:17
tlater[m]It's currently impossible to run fuse.py without patching it on NixOS16:17
juergbitlater[m]: ooi, what's the issue?16:24
tlater[m]juergbi: It's not smart enough to find libfuse in /nix/var/nix/...16:25
*** lachlan has joined #buildstream16:25
* tlater[m] assumes it doesn't check some path variable that's supposed to be checked16:25
tlater[m]The nix package for fusepy is patched to simply use a static file path for that16:25
tlater[m]But that obviously doesn't apply to the in-tree version in BuildStream, so it takes manual patching16:26
*** lachlan has quit IRC16:28
*** bochecha_ has quit IRC16:46
*** bochecha has quit IRC16:48
*** dylan-m_ has joined #buildstream17:02
* tlater[m] wishes we didn't use ostree in the docs' examples17:21
benschubertjjardon: you added flawfinder-sast right? https://gitlab.com/BuildStream/buildstream/-/jobs/360710103 will never be running it seems or am I missing something?17:25
*** lachlan has joined #buildstream17:32
tlater[m]Would someone mind reviewing https://gitlab.com/BuildStream/bst-plugins-experimental/merge_requests/59?17:37
tlater[m]I sort of need that to fix our CI17:37
benschubertlink to docs is a 40417:38
benschubertthat's a change in the API, I'm not sure this would pass on a non-fedora syste17:38
tlater[m]benschubert: But then it doesn't pass on fedora systems17:39
benschubertwould you mind checking if one of the debian works with it?17:39
tlater[m]I'll give it a try17:39
*** lachlan has quit IRC17:40
* tlater[m] couldn't see any change in the API going through the gobject-introspection history, though17:40
tlater[m]That interface has been the same for as long as it existed17:40
benschubertthat's interesting x)17:46
benschubertOk then17:46
*** jonathanmaw has quit IRC17:47
tlater[m]benschubert: My exact thoughts, hehe17:49
benschuberttlater[m]: workspace_reset:17:49
benschubert@click.option("--track", "track_", is_flag=True, help="Track and fetch the latest source before resetting")17:49
benschubertIs this an oversite?17:49
benschubert(I can remove it)17:49
tlater[m]Huh, yes, very much so17:50
benschubertok, I'll add it to my PR :D17:51
tlater[m]benschubert: If you don't mind, there are a couple other references to --track17:51
tlater[m]i.e., one on the deprecated fetch17:51
tlater[m]And one in the completion code17:51
* tlater[m] must have made a mistake grepping for those17:52
gitlab-br-botBenjaminSchubert opened MR !1733 (bschubert/track-cleanup->master: _stream.py: Remove dead code and comments about tracking) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/173317:52
benschuberttlater[m]: ^  or am I missing more?17:52
benschubertah17:52
benschubertok, let me remove those too17:52
tlater[m]Hrm, sorry about that17:53
tlater[m]I figure some of that may have snuck back in after the black reformatting17:53
benschubertno worries17:53
benschubertPR should be ready for review now if you want :)17:53
tlater[m]benschubert: 4 more references from a grep17:57
benschuberttlater[m]: ah, can you give me the files or the grep and I'll gix them :)17:57
benschubert*fix17:57
benschubertah nice17:58
tlater[m]benschubert: https://gitlab.com/BuildStream/buildstream/merge_requests/1733#note_25014683917:58
tlater[m]Also, the gobject stuff works on debian-9: https://gitlab.com/BuildStream/buildstream/-/jobs/36072778117:59
benschubertomg there's actually so much more of the track :)17:59
tlater[m]Gah!18:00
tlater[m]benschubert: I assume you're backtracking from `workspace_open`?18:00
benschubertyep18:01
tlater[m]I figure that's just the tip of the iceberg, because that one happens to have something searchable18:01
* tlater[m] should dig through the files manually18:02
benschubertwe should just ensure we cleanup when we see something18:03
* tlater[m] is very annoyed that he can't get `bst2html` to do what it should :|18:04
tlater[m]`junctions-build.html` just doesn't want to be updated18:04
benschuberttlater[m]: updated my PR :)18:05
tlater[m]benschubert: Does `bst track` implicitly track junctions?18:06
tlater[m]Doesn't look like it18:07
benschubertI mean, it's like any other element, you need to run 'bst track my_element'18:07
benschubertno?18:07
tlater[m]benschubert: Regarding the comment there, you need to run `bst track --track-cross-junction`18:08
tlater[m]To be able to track elements inside a junction18:08
benschubertAh right, the last part of the comment is still true18:09
tlater[m]Actually, `--cross-junctions`18:09
benschubertbut where you added your comment I don't think it is18:09
tlater[m]Ah, yes, that's right18:09
benschubertAnyways, I don't believe the trackin cross junction should be there, it should be on the cli no?18:10
tlater[m]Hm, that's debatable18:12
tlater[m]It's in both places currently18:12
tlater[m]Here it just explains it in a bit more detail18:12
* tlater[m] thinks it's ok here because it explains how to work with junctions18:12
cs-shadowit's not in both places currently18:12
benschubertok, I'll add it back :)18:12
cs-shadowthe CLI doesn't talk about this pecularity at all18:12
tlater[m]cs-shadow: `bst source track` has docs for it18:13
cs-shadowtlater[m]: the docs document this option, but not in this prose form18:13
tlater[m]Yeah, that's fair18:13
cs-shadowi meant to say that this text is useful, and we should ensure that we don't throw it away18:13
* tlater[m] thinks it makes sense for it to be explained in detail in both locations18:13
tlater[m]Since I'd be in junctions.html when I first figure out how to use junctions18:14
tlater[m]But then read `bst source track --help` to figure out why my build doesn't work again18:14
benschubertPr updated with the text back18:14
tlater[m]benschubert: You did get me curious; how did you manage to duplicate that line in an entirely unrelated file? :D18:19
benschubertI didn't know which junction file you were talking about18:19
benschubertso was looking at them18:19
benschubertand must have ctrl + y instead of z18:20
benschubertstill getting that quite often, due to CH keyboard <-> UK keyboard18:20
tlater[m]Ahhh18:22
tlater[m]Yeah, qwertz took me a while to forget as well :D18:22
* tlater[m] thinks he'll give up on fixing CI today18:24
tlater[m]Early tomorrow morning...18:24
tlater[m]It's just really hard to run that doc regeneration script and actually get something out of it.18:25
benschubertWe seem to have security scans on CI now? Is it expected that they'd be failing all the time?18:25
benschubertwe probably should rework it to have it work correctly all the time :)18:25
tlater[m]benschubert: The flawfinder stuff?18:26
* tlater[m] thinks it's a broken filepath, yes18:26
benschubertah x')18:26
benschubertHow did it go through review while failing all along?18:27
tlater[m]Oh, yeah, typo'd copy-paste18:27
tlater[m]Should be something like `flawfinder-sast.json`, I assume, just reusing another jobs'18:27
tlater[m]Huh, strange, we don't configure that18:29
tlater[m]Must be a gitlab bug18:29
benschubertare we missing a file that we should have created?18:29
* tlater[m] just wonders where those templates are18:30
tlater[m]Not much we can do without seeing the actual config18:30
*** lachlan has joined #buildstream18:43
*** lachlan has quit IRC18:53
*** lachlan has joined #buildstream19:08
*** bochecha has joined #buildstream19:35
*** bochecha has quit IRC19:40
*** bochecha has joined #buildstream19:40
*** bochecha has quit IRC19:44
*** lachlan has quit IRC19:48
*** lachlan has joined #buildstream19:50
*** bochecha has joined #buildstream20:06
*** misterwhatever has joined #buildstream20:08
*** lachlan has quit IRC20:13
*** lachlan has joined #buildstream20:16
*** kapip has quit IRC20:29
*** lachlan has quit IRC20:58
*** rdale has quit IRC22:13
*** cs-shadow has quit IRC22:32
*** bochecha has quit IRC22:48
*** dylan-m_ has quit IRC22:59
*** dylan-m_ has joined #buildstream23:25
*** slaf has quit IRC23:45
*** slaf has joined #buildstream23:58

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!