| *** traveltissues has joined #buildstream | 07:42 | |
| *** tpollard has joined #buildstream | 09:17 | |
| gitlab-br-bot | jjardon reopened MR !894 (valentindavid/sysroot_dependencies->master: Add support for importing build dependencies in sysroots.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/894 | 09:23 |
|---|---|---|
| *** santi has joined #buildstream | 09:40 | |
| gitlab-br-bot | traveltissues merged MR !1753 (traveltissues/remove-unused-functions->master: remove unused functions 1/2) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1753 | 09:43 |
| *** tiagogomes has joined #buildstream | 09:45 | |
| *** ChanServ sets mode: +o jjardon | 09:48 | |
| * jjardon benschubert: Hey, what is the relation of you patch for source plugins and https://gitlab.com/BuildStream/buildstream/merge_requests/1607? Is it you MR maen to replace it? | 09:48 | |
| benschubert | jjardon: that's what I am hoping yes, if we get consensus on the ML thread about it. Otherwise I'll fallback to !1607, but I believe the 'new' proposal is cleaner and easier to extend | 09:49 |
| gitlab-br-bot | MR !1607: WIP: Run standardised source tests from external plugins https://gitlab.com/BuildStream/buildstream/merge_requests/1607 | 09:49 |
| * coldtom reads the ML thread | 09:49 | |
| benschubert | Thanks :) | 09:50 |
| *** bochecha has joined #buildstream | 09:50 | |
| jjardon | benschubert: ok, thanks for clarify! | 09:50 |
| *** tiagogomes has quit IRC | 09:51 | |
| *** tiagogomes has joined #buildstream | 10:01 | |
| tpollard | nice improvements on !1755 benschubert :) | 10:07 |
| benschubert | tpollard: I can now scale, 64 workers+ and no slow down :) | 10:08 |
| benschubert | However, there is still one bottleneck around forking the process that I want to investigate | 10:08 |
| benschubert | but I'm now able to process 3400 elements in <7 minutes | 10:09 |
| juergbi | benschubert: that's on WSL? | 10:10 |
| benschubert | juergbi: no that was on LInux | 10:10 |
| benschubert | basically, it takes over a second to spawn 32 processes | 10:11 |
| juergbi | on native Linux that seems odd. at least for the raw fork() part | 10:11 |
| benschubert | Yep, I need to have a deeper look into it :) | 10:12 |
| benschubert | I wonder if it might just be an artifact from the profiler at that point | 10:12 |
| benschubert | I'll need to go back to other methods | 10:12 |
| *** jonathanmaw has joined #buildstream | 10:13 | |
| *** theawless[m] has quit IRC | 10:16 | |
| *** jjardon[m] has quit IRC | 10:16 | |
| *** kailueke[m] has quit IRC | 10:16 | |
| *** doras has quit IRC | 10:16 | |
| *** reuben640[m] has quit IRC | 10:16 | |
| *** gimpnet-irc[m] has quit IRC | 10:16 | |
| *** tlater[m] has quit IRC | 10:16 | |
| *** segfault3[m] has quit IRC | 10:16 | |
| *** Demos[m] has quit IRC | 10:16 | |
| *** nielsdg has quit IRC | 10:16 | |
| *** Trevinho[m] has quit IRC | 10:16 | |
| *** skullone[m] has quit IRC | 10:16 | |
| *** verdre[m] has quit IRC | 10:16 | |
| *** tchaik[m] has quit IRC | 10:16 | |
| *** ssssam[m] has quit IRC | 10:16 | |
| *** waltervargas[m] has quit IRC | 10:16 | |
| *** abderrahim[m] has quit IRC | 10:16 | |
| *** m_22[m] has quit IRC | 10:16 | |
| *** pro[m] has quit IRC | 10:16 | |
| *** krichter[m] has quit IRC | 10:16 | |
| *** awacheux[m] has quit IRC | 10:16 | |
| *** mrmcq2u[m] has quit IRC | 10:16 | |
| *** albfan[m] has quit IRC | 10:16 | |
| *** cgmcintyre[m] has quit IRC | 10:16 | |
| *** rafaelff[m] has quit IRC | 10:16 | |
| *** asingh_[m] has quit IRC | 10:16 | |
| *** mattiasb has quit IRC | 10:16 | |
| *** dbuch has quit IRC | 10:16 | |
| gitlab-br-bot | juergbi closed issue #336 (Internal python errors printed when the scheduler has too many jobs to process) on buildstream https://gitlab.com/BuildStream/buildstream/issues/336 | 10:18 |
| *** rdale has joined #buildstream | 10:22 | |
| *** juergbi has quit IRC | 10:30 | |
| *** juergbi has joined #buildstream | 10:30 | |
| *** lachlan has joined #buildstream | 10:36 | |
| *** Trevinho[m] has joined #buildstream | 10:39 | |
| gitlab-br-bot | BenjaminSchubert opened MR !1760 (bschubert/add-randomized-order-tests->master: tox.ini: Add ability to run tests in a randomized order) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1760 | 11:03 |
| *** lachlan has quit IRC | 11:05 | |
| tpollard | is there any reason we don't treat give app.initialized() a session_name for some commands, for instance source_checkout? | 11:23 |
| benschubert | I believe we don't start a scheduler for this? | 11:26 |
| *** lachlan has joined #buildstream | 11:26 | |
| tpollard | benschubert: it will attempt to fetch source if they're not cached | 11:28 |
| benschubert | oh, then it's probably an oversight :) | 11:28 |
| tpollard | it's a weird mixture, because in the case we call the session start callback some get some UI for that, but then don't get a print summary because we didn't give it a name | 11:28 |
| tpollard | although at cli time we don't check if the source is cached, so I guess it's valid to not know if it will start a scheduler | 11:29 |
| benschubert | then it might be because we don't want the summary? But that seems an ugly way of disablign it | 11:29 |
| benschubert | tlater[m] might know more | 11:30 |
| *** lachlan has quit IRC | 11:30 | |
| *** tlater[m] has joined #buildstream | 11:30 | |
| tlater[m] | Is this in regards to that MR? | 11:30 |
| * tlater[m] takes a peek | 11:30 | |
| tpollard | nope | 11:31 |
| tpollard | not related | 11:31 |
| tpollard | I've got an MR that changes how we handle messages | 11:31 |
| tpollard | and there's a few edge cases to my implementation | 11:32 |
| tpollard | one of which is when a scheduler is started but we haven't declared it as a named session | 11:32 |
| tlater[m] | Hm, no, I'm not sure | 11:33 |
| tlater[m] | I think that's entirely happenstance | 11:33 |
| tlater[m] | If we want to avoid summaries we should probably do that slightly more explicitly | 11:34 |
| benschubert | ^ +1 | 11:34 |
| * tlater[m] expects it's just one of those things that get accidentally copy-pasted around after someone missed a little detail somewhere | 11:34 | |
| benschubert | Are our plugins expected/allowed to show warnings while run as a job? | 11:35 |
| tlater[m] | benschubert: yes, afaik - there were plans to add a nice messaging API and everything, but it's always been possible to send warning messages | 11:36 |
| benschubert | Good, then we don't seem to be catching that in _track_ :D | 11:36 |
| benschubert | I'll fix this | 11:36 |
| *** lachlan has joined #buildstream | 12:00 | |
| *** bochecha_ has joined #buildstream | 12:34 | |
| *** bochecha has quit IRC | 12:36 | |
| *** bochecha_ is now known as bochecha | 12:36 | |
| *** bochecha_ has joined #buildstream | 12:55 | |
| *** bochecha has quit IRC | 12:57 | |
| *** bochecha_ is now known as bochecha | 12:57 | |
| *** santi has quit IRC | 13:25 | |
| *** santi has joined #buildstream | 13:34 | |
| gitlab-br-bot | BenjaminSchubert approved MR !1758 (chandan/remote-setuppy-test->master: Drop support for `setup.py test`) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1758 | 13:48 |
| *** Trevinho[m] has quit IRC | 13:58 | |
| *** abderrahim[m] has joined #buildstream | 14:56 | |
| *** albfan[m] has joined #buildstream | 14:56 | |
| *** asingh_[m] has joined #buildstream | 14:56 | |
| *** awacheux[m] has joined #buildstream | 14:56 | |
| *** cgmcintyre[m] has joined #buildstream | 14:56 | |
| *** connorshea[m] has joined #buildstream | 14:56 | |
| *** dbuch has joined #buildstream | 14:56 | |
| *** Demos[m] has joined #buildstream | 14:56 | |
| *** dineshdb[m] has joined #buildstream | 14:56 | |
| *** doras has joined #buildstream | 14:56 | |
| *** jjardon[m] has joined #buildstream | 14:56 | |
| *** gimpnet-irc[m] has joined #buildstream | 14:56 | |
| *** krichter[m] has joined #buildstream | 14:56 | |
| *** m_22[m] has joined #buildstream | 14:56 | |
| *** mattiasb has joined #buildstream | 14:56 | |
| *** mrmcq2u[m] has joined #buildstream | 14:56 | |
| *** nielsdg has joined #buildstream | 14:56 | |
| *** kailueke[m] has joined #buildstream | 14:56 | |
| *** pro[m] has joined #buildstream | 14:56 | |
| *** rafaelff[m] has joined #buildstream | 14:57 | |
| *** reuben640[m] has joined #buildstream | 14:57 | |
| *** segfault3[m] has joined #buildstream | 14:57 | |
| *** ssssam[m] has joined #buildstream | 14:57 | |
| *** tchaik[m] has joined #buildstream | 14:57 | |
| *** theawless[m] has joined #buildstream | 14:57 | |
| *** verdre[m] has joined #buildstream | 14:57 | |
| *** waltervargas[m] has joined #buildstream | 14:57 | |
| *** skullone[m] has joined #buildstream | 14:57 | |
| gitlab-br-bot | tlater merged MR !1723 (tlater/update-workspace-manpages->master: man/bst-workspace-reset.1: Update workspace re-open help message) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1723 | 15:01 |
| tlater[m] | Hey, it worked! | 15:01 |
| *** dbuch has quit IRC | 15:15 | |
| benschubert | Hey, question: the source_checkout would checkout the sources as we find them in the source cache (no .git dir for example) that's the expected behavior right? | 15:30 |
| tlater[m] | I personally don't expect that | 15:32 |
| tlater[m] | But I'm not sure if this was discussed during the original proposal | 15:33 |
| tlater[m] | I feel like we took care for that not to happen | 15:33 |
| benschubert | > for that not to happen | 15:33 |
| benschubert | What do you mean? | 15:33 |
| tlater[m] | benschubert: I'd expect to be able to run `git status` after checking out a git source | 15:34 |
| tlater[m] | And get reasonable output ;) | 15:34 |
| benschubert | Yep | 15:34 |
| benschubert | me too | 15:34 |
| benschubert | and that also means that when you do a 'source fetch' you expect to be able to open a workspace or a source_checkout after right? | 15:34 |
| tlater[m] | But that would mean we need the `.git` directory, no? | 15:34 |
| tlater[m] | Yeah | 15:34 |
| benschubert | which is not the case | 15:35 |
| tlater[m] | Why else would I be fetching them? | 15:35 |
| benschubert | good, I'll write a ML post then :'D | 15:35 |
| tlater[m] | benschubert: You're on a bug hunt today, hehe | 15:37 |
| benschubert | I'm fighting on the fact that I can't get my consistency tests to pass because we are not consistent enough :'D | 15:38 |
| benschubert | BUT on an internal project my 'bst show' is now 15s vs 1m30 on master | 15:38 |
| *** lachlan has quit IRC | 15:40 | |
| tlater[m] | Oh, that's awesome | 15:42 |
| tlater[m] | Any idea if there was a specific change that did that or is it just all the improvements lately? | 15:42 |
| tlater[m] | Or is there a major bug and we're only printing like 10% of the elements? ;p | 15:42 |
| tpollard | benschubert: I've put some times up for the message rate MR after rebasing (so much nicer without the signal warning...) and fixing a few corner cases. The times on the face of it look encouraging | 15:44 |
| benschubert | oh? :) | 15:44 |
| tpollard | from 5m11 down to 3m15 on my system | 15:44 |
| benschubert | Wow | 15:45 |
| benschubert | and is everything printed still? | 15:45 |
| tpollard | *should* be | 15:45 |
| gitlab-br-bot | tlater approved MR !1752 (juerg/buildbox-run-bubblewrap->master: Add CI job to test buildbox-run-bubblewrap and buildbox-fuse) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1752 | 15:45 |
| tpollard | I'd like to create a test to confirm it though | 15:45 |
| *** lachlan has joined #buildstream | 15:45 | |
| benschubert | That's awesome. let me know when I should review it attentively :) | 15:45 |
| tpollard | all current tests that parse the output pass though, so I'm pretty confident | 15:49 |
| tpollard | and I've not changed when we write to log, that's still message driven | 15:54 |
| gitlab-br-bot | tlater closed issue #1211 (Stop using non-POSIX features in small test suite tests to avoid fake test failures) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1211 | 16:00 |
| gitlab-br-bot | tlater merged MR !1725 (1211-stop-using-non-posix-features-in-small-test-suite-tests-to-avoid-fake-test-failures->master: Resolve "Stop using non-POSIX features in small test suite tests to avoid fake test failures") on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1725 | 16:00 |
| benschubert | tpollard: oh ok then that should be fine :) | 16:07 |
| tpollard | my main concern is if the UI behaviour change is suitable to be default | 16:08 |
| benschubert | between a stroboscopic outptu and a readable one, I think we can make a case :) | 16:09 |
| tpollard | depends if people are also avert to a 1Hz strobe... | 16:11 |
| benschubert | then a ML post :) | 16:11 |
| tpollard | on builds that are way less output intensive the the tick is barely different to unrestricted | 16:12 |
| benschubert | Yep | 16:12 |
| *** lachlan has quit IRC | 16:15 | |
| benschubert | tlater[m]: the perf optimizations I made are https://gitlab.com/BuildStream/buildstream/merge_requests/1739 :) It's not ready and I think I will need a ML post for this | 16:21 |
| tlater[m] | Right | 16:21 |
| * tlater[m] will take a look anyway :) | 16:22 | |
| benschubert | let me know if anythign seems weird :) | 16:24 |
| benschubert | I still have a few things to do to completely remove the consistency check | 16:24 |
| *** lachlan has joined #buildstream | 16:26 | |
| *** traveltissues has quit IRC | 16:31 | |
| *** lachlan has quit IRC | 16:34 | |
| benschubert | tlater[m]: on the benchmarks, this branch is 7-10% faster :) | 16:34 |
| gitlab-br-bot | jjardon opened MR !1762 (jjardon/fdsdk-190805->master: .gitlab-ci.yml: Use latest freedesktop-sdk 19.08.5) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1762 | 16:38 |
| gitlab-br-bot | BenjaminSchubert approved MR !1762 (jjardon/fdsdk-190805->master: .gitlab-ci.yml: Use latest freedesktop-sdk 19.08.5) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1762 | 16:39 |
| *** santi has quit IRC | 16:43 | |
| *** lachlan has joined #buildstream | 16:44 | |
| *** santi has joined #buildstream | 16:58 | |
| tpollard | heh, an asciinema of master on the debian target is over 300MB, it's 34MB on my branch. highlights how much we have to redraw | 17:16 |
| benschubert | wow :) | 17:16 |
| benschubert | even with my patches in? that reduced already quite a bit things :) | 17:16 |
| tpollard | should be, it's 4 commits behind but those aren't related afaict | 17:18 |
| tpollard | it wouldn't be feasible without the scheduling changes though, the signal warning was being masking heavily | 17:20 |
| tpollard | s/masking/masked | 17:20 |
| *** bochecha has quit IRC | 17:20 | |
| benschubert | ah right | 17:20 |
| tlater[m] | benschubert: Ah, I see why you were asking about warnings earlier :D | 17:29 |
| tlater[m] | Yeah, that might be leftover from ages ago | 17:30 |
| benschubert | Yeah I'm not sure what to do :'D | 17:30 |
| tlater[m] | I'd say ping tristan at some point to see if he has an opinion | 17:31 |
| tlater[m] | I think I'm with you, but he'll probably have some more background info on that | 17:32 |
| benschubert | I think we have a few confusions on source cache vs 'plugin cache', which make a lot of things awkwarf | 17:33 |
| benschubert | *awkward | 17:33 |
| tpollard | I've put a quick clip of the side by side output. TIL gitlab has an embedded video player | 17:39 |
| benschubert | Oo | 17:40 |
| tpollard | only 10MB file limit though | 17:40 |
| tlater[m] | Hah, that's adorable | 17:42 |
| tlater[m] | GitLab's really fun when it works :) | 17:43 |
| benschubert | tlater[m]: my changes allow me to remove a '__update_source_state' :D -> more fine grained changes! | 17:43 |
| tlater[m] | benschubert: Yeah, it's a really good change | 17:44 |
| * tlater[m] also likes how much more obvious it makes "Consistency" | 17:44 | |
| benschubert | but gosh I'll be happy once it will be polished, it's been roughly as hard as the yaml rework | 17:44 |
| benschubert | (actually harder) | 17:44 |
| tlater[m] | benschubert: Are you sure about that? | 17:45 |
| tlater[m] | Maybe all the update_state things together | 17:45 |
| tlater[m] | But I just remember how many iterations of the yaml rework we had | 17:45 |
| tlater[m] | And how you actually went out and redid it all in cython | 17:46 |
| tlater[m] | I don't think it quite compares ;) | 17:46 |
| gitlab-br-bot | tlater opened MR !1763 (tlater/tar-test-security->master: tests/sources/tar.py: chown+chmod instead of giving 777 permissions) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1763 | 17:49 |
| benschubert | fair point :) | 17:49 |
| *** tpollard has quit IRC | 17:50 | |
| *** tpollard has joined #buildstream | 17:50 | |
| *** santi has quit IRC | 17:59 | |
| *** jonathanmaw has quit IRC | 18:31 | |
| *** lachlan has quit IRC | 18:50 | |
| *** lachlan has joined #buildstream | 19:00 | |
| *** lachlan has quit IRC | 19:45 | |
| *** lachlan has joined #buildstream | 19:45 | |
| *** lachlan has quit IRC | 19:56 | |
| *** lachlan has joined #buildstream | 19:58 | |
| *** lachlan has quit IRC | 20:04 | |
| *** lachlan has joined #buildstream | 20:49 | |
| *** lachlan has quit IRC | 21:19 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!