IRC logs for #buildstream for Wednesday, 2020-05-13

*** rdale has quit IRC00:50
*** tristan has quit IRC01:17
*** aminbegood has joined #buildstream04:15
*** aminbegood has quit IRC05:15
*** narispo has joined #buildstream05:35
*** aminbegood has joined #buildstream06:06
*** aminbegood has quit IRC06:21
WSalmonchipb, because freedestop build everything on top of the tar ball as a build only dependency, if you work on top of it (or your version) then your devs never tuch the base tar ball06:44
*** pointswaves has joined #buildstream06:52
WSalmona nice developer experince thing is if you build your toolchain in to a element in ci and push it to a shared cache then so long as your devs want a cached version then they dont need to do anything but have bst installed to build on top of it07:16
WSalmonusing somethink like buildbarn or buildgrid the cache servers can scale to meet the needs of hundreds of deves + CI + remote build etc07:17
*** pointswaves has quit IRC07:18
*** benschubert has joined #buildstream07:20
juergbibenschubert: thanks for the quick merge. I was confused why my benchmark builds were suddenly taking 3-4 times as long07:30
juergbiI had a global artifact server configured in my buildstream config07:31
benschubertjuergbi: no problem, we'd also need to update the debian project to have a 'min-version' sometimes07:31
juergbiyes, I already pushed this to juerg/bst2 there07:31
benschubertah awesome, should we merge that into master?07:32
juergbiwondering about the current branches we have there07:32
juergbiwe currently don't use master in the benchmark07:32
juergbistill jennis/add_all_files07:32
benschubertah right, I wonder, should we move your branch as master and delete all the others?07:32
juergbiyes, or was this intentional to separate the small branch with only the script from the generated files?07:33
benschubertI don't know07:33
benschubertah maybe the automated benchmarks use master?07:34
juergbithere is also a local (jennis) path in the generated/modified project.conf07:34
juergbibtw: shall we also move bst-benchmarks from your personal gitlab area to the BuildStream/benchmarking group?07:35
benschubertSure, that sounds good07:36
benschubertlet me do that now07:36
juergbita07:36
benschubertit's probably official enough now :D07:36
benschubertuh ? I don't seem to have the rights to transfer to BuildStream07:37
benschubertah I don't manage BuildStream07:37
benschubertjuergbi: can you? https://gitlab.com/BenjaminSchubert/bst-benchmarks/edit07:38
WSalmonbenschubert, you will need to make juergbi a owner of that if he is not already07:38
juergbiah, right, moving is tricky privilege-wise07:38
WSalmonIIRC07:38
juergbibenschubert: I obviously don't have the rights on your side07:38
juergbiI can make you a benchmarking group owner07:38
benschubertI just added you as a maintainer07:39
benschubertBut that would work too yep07:39
juergbibenschubert: maintainer is not enough for move. I've set you as owner for the benchmarking subgroup07:41
benschubertdone :) thanks07:41
benschubertany idea for a better name though?07:41
juergbinot sure. the benchmarking repo list is already somewhat confusing07:42
juergbinaming is hard ;)07:42
*** devcurmudgeon has joined #buildstream07:44
benschubertjuergbi: btw if you have time: https://gitlab.com/BuildStream/buildstream/-/merge_requests/1840 :)07:44
juergbibenschubert: right, I took a quick look already this morning. the changes look fine to me. I'm not sure whether that part is ever slow, especially if/when dynamic plan is working again07:46
benschubertI can see only one time: when your source cache doesn't have it, but they are in the plugin-specific cache07:47
benschubertwhich is sadly not that uncommon (disk filled with those sources)07:48
juergbiwith dynamic plan it doesn't even call the status() methods07:48
benschuberttrue07:48
juergbibut right now this might indeed be the case07:48
benschubertok then let's park this one until we have dynamic plans again?07:48
benschubertbecause if that's not needed, it's all the less handling in the scheduler07:48
juergbiok. benchmark for the dynamic plan reverts should be finished soon07:49
juergbibut there is no test case yet07:49
traveltissuescan i get some reviews please: https://gitlab.com/BuildGrid/buildbox/buildbox-common/-/merge_requests/267 https://gitlab.com/BuildGrid/buildbox/buildbox-common/-/merge_requests/26807:54
juergbibenschubert: added benchmark result in comment of !1926. the timing differences seem minor to me. source fetch seems to be a tad slower07:56
traveltissuesbenschubert: i'm increasingly confused by the benchmarking subgroup, i know we were using the benchmark_debian project which used the debian-stretch-bst project at some point. I made the local benchmarks helper as a way for people to run that locally (which was just an adaptation of the jobs that published the results which were being sent to the mailing list)07:58
traveltissuesthere's also a bot and an 'official' tool in there which i have never used07:58
traveltissueshas there been some discussion about cleaning this up and deciding what is the common group of benchmarks the buildstream world recognizes07:59
*** seanborg_ has joined #buildstream07:59
benschubertjuergbi: Ok, sad to see this one back in, but I guess we'll need a bigger revamp at some point. Let's add tests this time08:01
benschuberttraveltissues: good question, I don't think so and I know efforts have been rather scattered in that place08:01
juergbibenschubert: yes, definitely want to clean this up. but this shouldn't have been removed in the track-related MR. and it degraded UX08:02
juergbitraveltissues: !267 looks fine to me, approved08:02
benschubertcorrect08:02
traveltissuesthanks juergbi08:06
traveltissuesbenschubert: we might like to consider benchmarks that also hit the caching at larger scales. i know that WSalmon and co have been having some issues recently08:07
benschuberttraveltissues: that's a resource problem, we can't even have a fsdk build reliably at night, and that's a small project08:08
benschubertif we want to scale our benchmarks we need more resources there and people wanting to work more on benchmarks08:08
traveltissuesyeah, good point08:09
traveltissuesthat we can't reliably build the overnights still troubles me08:09
juergbimaybe we should get some relatively cheap but powerful dedicated servers instead of flexible but expensive cloud instances08:11
traveltissuesimo if the overnights are mostly failing then it's not really worth running them. i generally see they've failed and we probably all just ignore that08:11
traveltissuesso it's a bit of a waste of time/resources unless we redefine them in some way to minimise the failure rate from resources08:12
benschuberttraveltissues: yeah, same for the out of order tests, they've been failing for a while. I try to fix those when I have time, but hey08:12
*** phildawson has joined #buildstream08:12
benschubertjuergbi: not sure it's worth it right now but yeah could be a solution08:13
traveltissuesjuergbi: i don't have permissions to hit the merge button on that mr, could you do that please or could i have the permissions08:13
juergbitraveltissues: I've merged it. I don't have an objection to granting you merge permission but you should ask that on slack in #buildbox to reach the right group08:16
traveltissuesthanks08:17
traveltissuesdone08:24
*** jude has joined #buildstream08:25
*** ikerperez has joined #buildstream08:37
*** tpollard has joined #buildstream08:43
*** santi has joined #buildstream08:44
*** seanborg_ has quit IRC08:46
*** seanborg_ has joined #buildstream08:46
*** tristan has joined #buildstream09:19
*** phoenix has joined #buildstream10:26
*** jward has quit IRC10:31
*** jswagner has quit IRC10:31
*** jward has joined #buildstream10:32
*** jswagner has joined #buildstream10:32
*** jswagner has joined #buildstream10:33
*** jswagner has joined #buildstream10:35
*** jswagner has joined #buildstream10:35
*** jswagner has joined #buildstream10:37
*** jswagner has joined #buildstream10:37
*** jswagner has joined #buildstream10:39
*** ChanServ sets mode: +o tristan10:42
*** tristan changes topic to "BuildStream 1.4.3 is out ! | https://gitlab.com/BuildStream/buildstream | Docs: https://docs.buildstream.build/ | IRC logs: https://irclogs.baserock.org/buildstream | Mailing List: https://mail.gnome.org/mailman/listinfo/buildstream-list | Roadmap: https://wiki.gnome.org/Projects/BuildStream/Roadmaps"10:42
jjardonthanks tristan ! :) can you please upload it to pip as well?11:25
jjardontristan: seems the ostree fixes are different in the bst-1 and bst-1.4 branches; is that intended?11:34
tristanjjardon, you got the password ?11:37
jjardontristan: nope, I've never had aceess to that11:37
tristanjjardon, are they actually different ?11:37
tristanor just the commit shas are different so you think they are different ?11:38
jjardontristan: seems the branch point is commit 62eee7be681c74c6b6aa679acac9d27bf1b871e911:38
jjardontristan: the commit is different11:38
tristanjust shas11:38
tristanthe commit ? or the sha ?11:38
jjardonsha, sorry11:38
tristanThe content of the commit should be the same11:38
tristanit's cherry picked back to the 1.4 branch11:38
jjardonah rigth11:38
tristanjjardon, as I mentioned the other day, I branched 1.4 off so I could land the includes thing... and *then* I crashed face first into the CI brick wall11:39
tristanand had to fix ostree11:39
jjardonok, sure ,I was only double checking11:39
* persia finds comparison of tree SHAs useful as a way of avoiding confusion from different commit SHAs for the same content11:40
* jjardon tags "bst-1.4-branchpoint" for future reference11:41
jjardontristan: did you add all the checks to the bst-1.4 branch? (daily build, protections ..etc) (I can do it now if you are busy)11:42
tristanjjardon, what are those, I have no idea about those11:43
tristanhonestly, there used to be this information in the CONTRIBUTING.rst, now it's been broken out into a page which our docs don't even link to11:43
tristanand I'll bet that this information of what you're supposed to do at this point didn't end up in that forgotten unreachable docs page :-S11:44
*** seanborg_ has quit IRC11:52
*** seanborg_ has joined #buildstream11:52
jjardontristan: overnigth test is a simple check to verify the branch keep building a big bst project even without the cache enabled11:54
jjardonI have added the job now11:54
tristanjjardon, What are the "protections ..etc" part ?11:56
jjardontristan: basically no one can directly force push, typical gitlab config for official branches11:56
tristanAh that, nothing ever needs to be done for that11:57
tristanThat would be a real pain if we had to do that every minor point11:58
jjardonah, we have a wildcard bst-*, so all done automatically :)11:58
jjardontristan: not minor point, every time we branch11:58
tristanWe branch on minor points though11:58
jjardonwhich should not happen often11:58
jjardontristan: only if needed11:58
tristanWell, happens every 6 months or so11:58
tristannot lately but probably again after 2.011:59
tristanAnyway as I recall, it's a huge list of manually entered committers in that thing11:59
tristantakes an hour to setup11:59
jjardontristan: as I said, automatically done; we have a wildcard12:00
* tristan 's hardware is getting all flaky :'(12:00
* tristan still has to take in his new laptop to the service center and maybe get it replaced (still a long time on warrenty, don't feel like going there and doing that chore)12:01
tristannow my 4 year old laptop's screen is starting to flicker, which it only previously did when reaching low power :-/12:01
*** tristan has quit IRC12:06
*** tristan has joined #buildstream12:22
*** cs-shadow has joined #buildstream12:23
*** ChanServ sets mode: +o tristan12:27
tristanReboot to the rescue12:27
tristanWow [00:00:35][][] SUCCESS Loading elements12:44
tristanoops12:44
tristanbenschubert, tiny MR look see ? https://gitlab.com/BuildStream/buildstream/-/merge_requests/192812:58
tristanAlso, not sure what to look at tomorrow... I think I'll leave the junction conflicts story fester a bit longer12:59
tristanbenschubert, are you planning to pickup cross-junction plugin loading ?12:59
* tristan thinks benschubert wanted to do that one13:00
tristanOh wait... shouldnt deprecation warnings be allowed to be fatal ?13:01
benschuberttristan: go for it if you want, my plate is currently pretty full :)13:01
tristanI think they should... however... it poses an interesting problem for cache keys13:01
tristanI think also there was a request to (optionally) consider plugin exact versions in the cache key, which seems worth visiting13:02
tristanSeems these low hanging fruit do still require a bit of thought though13:03
tristanbenschubert, I don't think I'll have time to get started on that tomorrow, but if cross junction loading is still on the table I may start that on friday and aim to have a ready MR by early next week13:04
benschuberttristan: I don't htink I'll be able to focus on BuildStream until next week anyways, at least not for such a thing13:04
tristanOk13:05
tristanHopefully by next week we make more progress on what to do with double-loaded junctions13:05
benschuberthappy to help if you want reviews/anything else then13:05
* tristan wonders if we're just making too much fuss about that13:05
tristan!1928, but I think that's really a nobrainer ;-)13:06
tristanI'll just land it if nobody reviews it13:06
benschubertreviewed :) https://gitlab.com/BuildStream/buildstream/-/merge_requests/1925 also if you have time, it's also a simple one13:10
tristangitlab's a bit sluggish today13:19
tristanbenschubert, while the page loads... does your patch ensure that we still test the pip source plugin in buildstream, possibly by requiring bst-plugins-experimental and testing it at a specific commit sha in our tests ?13:20
tristanor is that already there ?13:20
tristanAs I recall, `pip` and `cargo` are the only source plugins which exercise the feature of requiring previous sources at fetch and track time13:21
benschuberttristan: this removes all the tests for the pip source in buildstream, as they were only 5-6 and they are duplicated in experimental already13:21
tristanRight, but do we run those tests from bst-plugins-experimental in the context of a BuildStream CI session ?13:21
tristanI think that's a hard requirement, and I think I saw we do something like that in tox.ini (we use a specific version of bst-plugins-experimental right ?)13:22
* tristan thinks the `tox -e py{version}-plugins` cases cover those13:24
tristanbut worth verifying13:24
*** aminbegood has joined #buildstream13:27
benschuberttristan: no we only run the standardized source tests for this13:33
benschubertso all the specific ones are not run (basically, hte ones removed by the PR are not run)13:33
benschubertah so track and fetch would still be run I think13:34
benschubertthe only real change is the tests removed there13:34
tristanbenschubert, Do we know that the tests which we do run in BuildStream CI actually test this functionality ?13:35
tristanI think the "standardized tests" are only source plugins which provide repo implementations13:35
tristanThis is really an important and delicate part of plugin separation, we went though a lot of headache getting it "maybe right" a very long time ago :-S13:36
tristanPoint is, we need to ensure we keep testing the aspects of the core (of course you get the point, I know)13:37
*** aminbegood has quit IRC13:37
benschuberttristan: if pip doesn't implement the 'repo', then we never tested that part13:37
benschubertaaaand https://gitlab.com/BuildStream/bst-plugins-experimental/-/tree/master/src/bst_plugins_experimental/testutils/repo it doesn't13:38
* tristan feels like a lot of attention was spent on making something plug-and-play, but less attention to the actual requirements13:38
tristanRight13:38
tristanbenschubert, then probably we still have the `pip` source in BuildStream *because* this aspect was not considered yet ?13:38
tristanWe probably need to have some custom entry points for testing from external plugin repos, this will also be important for git and maybe others13:39
tristanfor instance git implements the mirrors13:39
tristanwe need to keep all of this covered13:39
benschuberttristan: I'd rather have testing plugins that do test this part eexplicitely13:40
benschubertusing other plugins to test it as a side effect is not great13:40
tristanThat works too, but there is a danger with that: it may allow us to shift goal posts unfairly13:41
tristanfrom the core perspective I mean13:41
benschubertwhich is true for anything we would not be testing with external plugins13:41
tristanHonestly I don't mind, but I think moving plugins out of the core is not such a hurry, and we should ensure that while removing plugins, we don't lose coverage of features in the core by doing so13:42
tristanif that's the case, we can just drop all the work which was already done to test bst-plugins-experimental in BuildStream test runs itself too13:42
tristanWe gotta choose a way and stick to it I guess13:42
benschuberttristan: agreed on that, my problem is the botched job that was done ending in having plugins duplicated, now we need to keep those in sync and it's a mess13:42
* persia is a bit concerned by "moving plugins out of the core is not such a hurry", as that sort of phrasing has led to a very long transition so far13:43
benschubertpersia: agreed, however I think the problem was more of trying to move things out of core in a unsynchronized way without ensuring things could be removed cleanly, and thus leading to duplication13:43
tristanpersia, I think there was some kind of misconception that they had to move fast, which may be affecting the optics13:43
persiabenschubert: Absolutely agreed.  I think the current plan is more likely to succeed than the last.13:44
tristanTrue, we would be better off if nobody had started pushing to move plugins in a hurry13:44
tristanAnyway here we are13:44
persiatristan: I think there was just confusion about what was "in" and what was "out" and when what moved, which blurred optics :)13:45
tristanI think we really should resolve coverage in advance of moving things out which remove important coverage - even if it looks messy in the meantime13:45
benschuberttristan: ok, I'll postpone this MR, it's the last of the duplicated plugins anyway, pip element was removed from core a few days ago and tar should be removed from experimental not core13:45
persiaI think here is a good place to be, and we have a good plan.  I just think we need to be careful not to let the plan get desynchronised by deferring some parts of it.  Doing it synchronised later is better than doing it desynchronised *now*, but doing part later without defining "later" or the part to be done risks what we do.13:46
* tristan gotta leave the closing coffee shop13:46
tristanpersia, I think we're on a roll for now anyway13:46
tristanthere is a lot of things to do, but they're getting done13:47
persiaYes :)13:49
*** tristan has quit IRC13:51
traveltissuespersia: for example: https://gitlab.com/BuildStream/bst-plugins-experimental/-/issues/29 so it seems like this has moved a few times and there are WIP for moving things again14:11
*** cs-shadow has quit IRC14:43
*** cs-shadow has joined #buildstream14:53
*** hasebastian has joined #buildstream15:31
WSalmonlooks unlucky as a restart fixed it but has any one ever seen bst go bang like https://gitlab.com/celduin/bsps/example-app/-/jobs/55082171715:32
juergbiWSalmon: eh, no. might be an issue with gRPC. pure Python shouldn't be able to trigger this and I don't think we use any mutexes in Cython code given our code is multi-process but single-threaded (GIL)15:49
WSalmonjuergbi, im not convinced about your reverting patch https://gitlab.com/celduin/bsps/example-app/-/jobs/550859879 this seems to start builds15:51
juergbiWSalmon: the first thing it tries is pull deploy/image.bst15:51
juergbibut it seems that's not cached15:51
juergbimaybe you're hit by the cache key instability in master. there was a very recent cache key calculation fix15:52
WSalmonthat may well be it15:53
WSalmonthanks juergbi loads of things changed recently and have had to fix loads of things15:54
WSalmonsorry i missed that15:54
WSalmonill retry it with a clean cache once its all rebuilt15:54
juergbita15:55
WSalmonbut the fact its pulling in the right order is a massive pluse!15:56
WSalmonthanks15:56
WSalmonjuergbi, why dose it not try to pull deploy/image.bst's build deps next? it gose straight to the back of the que? `bst show --format "%{name} %{deps}" deploy/image.bst` gives `deploy/image.bst [components/genimage.bst, deploy/filesystem.bst, integration/mtab.bst, vm/prepare-image.bst]`16:06
juergbiWSalmon: at least some of these runtime-depend on freedesktop bootstrap-import.bst16:09
juergbiand that's a stack that indirectly depends on these bootstrap elements16:10
juergbii.e., to build deploy/image.bst these bootstrap elements are all needed16:10
juergbieven if the direct build dependencies are already built16:10
juergbiBuildStream thus marks all of them as 'required' at the same time, and then attempts to pull them in parallel16:12
juergbiand it generally tries to pull elements at the bottom of the stack first (if they are required) as they are often needed by multiple jobs => improving parallelism16:13
*** santi has quit IRC16:22
*** santi has joined #buildstream16:22
WSalmonthanks16:32
WSalmoncould we have `requires` as a format option? would that be tricky, im finding all this really tricky to be sure of16:33
WSalmonhence all my silly questions16:34
WSalmonsomething in buildstream ( juergbi's build order patch ) or bst-plugins-experimental, master in the last few days cause this https://gitlab.com/celduin/bsps/example-app/-/jobs/550906009 ?16:40
*** pointswaves has joined #buildstream16:41
juergbiWSalmon: `bst show --deps build` should show you build dependencies and indirect runtime dependencies, i.e., what's required for a build16:46
*** narispo has quit IRC16:48
*** narispo has joined #buildstream16:48
juergbiWSalmon: I can't think of anything that would affect staging/overlaps16:49
*** santi has quit IRC17:07
*** santi has joined #buildstream17:08
*** jude has quit IRC17:09
*** tristan has joined #buildstream17:15
juergbiWSalmon: I can reproduce the issue locally, also seeing it without my changes. hm17:33
*** tpollard has quit IRC17:43
*** santi has quit IRC17:44
*** hasebastian has quit IRC17:50
juergbie5ed02da0b99c16481eac58963f4ebbdf0481590 is the first bad commit17:53
juergbi    element.py: Always expand all variables at element creation17:53
juergbibenschubert: this appears to break freedesktop-sdk build17:53
juergbinot sure yet what exactly is happening17:53
benschubertouch what?17:54
benschuberthow can I reproduce? (which element in fsdk?)17:54
*** pointswaves has quit IRC17:56
juergbipossibly related to variables from includes17:56
juergbii.e., maybe variables resolved before the includes have been processed17:56
juergbiI see build error in bootstrap/build/glibc-stage1.bst17:56
juergbibut have to recheck whether it happens when building that element alone or only as part of the full build17:57
juergbiin the failure case %{builddir} is resolved to empty string instead of bst_build_dir as set in the include17:57
*** pointswaves has joined #buildstream17:59
benschubertvariable expansion is done after the compose18:00
juergbibenschubert: I can't reproduce it if I only build glibc, however, it seems reproducible as part of a full "make build" (happens after a couple of minutes here)18:01
benschubertI'm resetting my build and will retry18:01
juergbibootstrap/build/debugedit-build.bst fails with overlap errors that shouldn't happen18:02
juergbi(presumably due to incorrect builds of dependencies due to the variable issue)18:02
benschubertjuergbi: weird. Which branch of fsdk? My previous MR fails with ConfigParser errors?18:04
juergbiah, I'm on juerg/bst2 + pip patch from WSalmon18:06
juergbiand without artifact cache18:06
benschubertstill configparser issue? what18:06
benschubertthere is an element that fails that is not in the sdk-image.bst cone18:07
benschubert(trying to build sdk-image.bst)18:07
juergbiI'm using 'make build'18:07
benschubertjuergbi: https://gitlab.com/BuildStream/bst-plugins-experimental/-/blob/master/src/bst_plugins_experimental/elements/flatpak_image.py#L51 that can never work. are you not using bst-plugins-experimental master?18:11
benschubert(interesting bug found x')18:11
juergbiah, no, I haven't updated bst-plugins-experimental for a few days18:11
juergbion f77f5c4dfbe9e8d147fcd0500d76a68634e7f92e right now18:11
benschubertthis has been here for two years?18:12
benschubertwait what18:12
juergbiwhat configparser error do you get exactly?18:13
benschubertthat the configparser can't be converted into json18:14
benschubertwhen trying to build the cache key18:14
*** seanborg_ has quit IRC18:14
juergbimaybe different dependency versions, I haven't reinstalled my venv recently18:14
benschubertmine is latest18:15
benschubertok make build is running18:17
benschuberthowever, that doens't happen with the sdk-image sub-cone, interesting18:19
benschuberthttps://paste.gnome.org/pehfmzf1x uh?18:28
cs-shadowbenschubert: the paste.gnome link gives me a 404 :/ is there a typo somewhere?19:14
benschubertah no, outdated19:14
cs-shadowor perhaps I need to authenticate ?19:14
cs-shadowah ok19:15
benschubertit as a "proc is none" in the scheduler somewhere because of a problem with a remote cache19:15
benschubertcs-shadow: https://paste.gnome.org/plu3ebghu here it is again :'D19:21
cs-shadowthanks!19:21
cs-shadowinteresting ..19:21
*** phoenix has quit IRC20:38
pointswavesfyi https://gitlab.com/BuildStream/buildstream/-/issues/1309 this completely killed my terminal and had to kill off the hole terminal22:57
*** narispo has quit IRC23:17
*** narispo has joined #buildstream23:17
*** narispo has quit IRC23:29
*** narispo has joined #buildstream23:29
*** pointswaves has quit IRC23:36

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!