IRC logs for #buildstream for Friday, 2019-08-30

*** narispo has quit IRC03:06
*** narispo has joined #buildstream03:06
*** narispo has quit IRC04:53
*** narispo has joined #buildstream04:53
gitlab-br-botjuergbi opened (was WIP) MR !1571 (juerg/tox-home->master: tests: Set HOME environment variable and work around bzr race condition) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/157105:34
juergbibenschubert: does !1571 look good to you? I *think* this fixes the issue05:57
juergbisetting HOME wasn't sufficient as {envtmpdir} is per environment, not per test, so the race condition was still an issue with parallel tests05:57
juergbithe issue was actually triggered by bzr creating mock repositories, not by the bzr source plugin itself05:58
juergbiso I also added ~/.bazaar directory creation there to work around the bzr race condition05:59
*** rdale has joined #buildstream08:01
juergbithere is also an issue with hanging tests again. I think I know why that's happening but a complete solution is not trivial08:09
juergbihopefully not too much trouble in CI until that's properly fixed08:09
benschubertjuergbi: Oo is it me or all tests share the same config directory? -_-'08:11
benschubertYeah, seems an acceptable patch, haven't realized those things where hardcoded like that08:11
juergbiyes, HOME/XDG_CONFIG_HOME are (still) shared among tests08:13
juergbiat least it avoids host HOME contamination now, and works around the bzr issue08:13
benschubertcorrect08:13
benschubertThat also explains for some reasons I had tests failing when mounting my host in docker... -_-'08:14
gitlab-br-botBenjaminSchubert approved MR !1571 (juerg/tox-home->master: tests: Set HOME environment variable and work around bzr race condition) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/157108:15
gitlab-br-botmarge-bot123 merged MR !1571 (juerg/tox-home->master: tests: Set HOME environment variable and work around bzr race condition) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/157108:15
jennisnice catch08:26
*** tme5 has joined #buildstream08:50
*** jonathanmaw has joined #buildstream08:58
tme5do I have the OK to merge !1557 ?09:01
gitlab-br-botMR !1557: Add in_subprocess pytest mark and modify tests which run in another process to use it https://gitlab.com/BuildStream/buildstream/merge_requests/155709:01
jennistme5, I've given it to marge09:05
jennisI see that tlater[m] and benschubert approved it09:05
jennisjuergbi, do you think your cache quota MR can close https://gitlab.com/BuildStream/buildstream/issues/996 ?09:05
benschuberttme5: thanks a lot for this MR, it will help a lot when debugging those tests09:06
jennis+109:06
tme5thank you, hope so :)09:07
juergbijennis: I don't think that MR solves #996. however, it might actually already have been solved by the casd MR09:07
gitlab-br-botIssue #996: Cache quota shouldn't be computed until we know we need to write to the cache https://gitlab.com/BuildStream/buildstream/issues/99609:07
juergbicasd always calculates disk usage on startup09:08
juergbihowever, it does this in a separate process and is terminated when bst terminates09:08
juergbiso if I'm not missing anything, it's essentially solved from the user point of view09:08
juergbibut we'll have to verify it09:08
jennisCool, perhaps benschubert could confirm?09:08
benschubertjennis, juergbi I can't try this now, on my work machine, however, I think we can indeed close it since it's done out of band09:10
gitlab-br-botjennis opened (was WIP) MR !1565 (jennis/update_checkout->master: Update artifact checkout to handle artifact refs) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/156509:18
gitlab-br-botmarge-bot123 merged MR !1566 (juerg/cache-quota->master: Cache quota configuration fixes) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/156609:27
*** lachlan has joined #buildstream09:30
*** tpollard has joined #buildstream09:47
*** lachlan has quit IRC09:48
*** tpollard has quit IRC09:51
*** lachlan has joined #buildstream10:06
*** cs-shadow has joined #buildstream10:10
gitlab-br-botQinusty opened issue #1114 (Improve compatibility of Buildstream with Buildbarn and Buildfarm) on buildstream https://gitlab.com/BuildStream/buildstream/issues/111410:13
*** lachlan has quit IRC10:29
*** lachlan has joined #buildstream10:31
gitlab-br-botmarge-bot123 closed issue #1108 (Consolidate helpers to run tests in subprocess) on buildstream https://gitlab.com/BuildStream/buildstream/issues/110810:51
gitlab-br-botmarge-bot123 merged MR !1557 (tmewett/test-in-subprocess->master: Add in_subprocess pytest mark and modify tests which run in another process to use it) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/155710:51
*** tpollard has joined #buildstream10:54
*** tpollard has quit IRC11:03
*** lachlan has quit IRC11:10
qinustyIs ReferenceStorage required for remote execution? Buildgrid configures it https://buildgrid.gitlab.io/buildgrid/reference_server_config.html#reference-configuration and it is defined in buildstream protos https://gitlab.com/BuildStream/buildstream/blob/master/src/buildstream/_protos/buildstream/v2/buildstream.proto#L2211:11
juergbiqinusty: no, ReferenceStorage is obsolete for bst master/2.x11:15
juergbirelated to this, I wanted to clarify on the issue you filed that also bst-artifact-server is not a requirement for remote execution11:15
juergbiit's required to store artifacts but for remote execution itself it's not strictly required11:16
jjardonabderrahim[m]: tristan any idea when buildstream 1.4 is going to be released? gnome freeze is on Monday and it would be nice to have it by then11:51
*** phoenix has joined #buildstream12:00
abderrahim[m]jjardon: Sorry I didn't have much time to work on it. tristan did a good job of packporting the uncontroversial things. I'm using the bst-1 branch exclusively now, and it's stable. I think it's releasable, although I'd like to have some more features there12:59
*** phoenix has quit IRC13:02
*** ChanServ sets mode: +o tristan13:06
tristanabderrahim[m], jjardon Feature wise, I had picked up !1542 for master because it's been long unsolved and I thought it would be a good thing to have in 1.413:06
gitlab-br-botMR !1542: Support strict build dependencies https://gitlab.com/BuildStream/buildstream/merge_requests/154213:06
tristanOther than this though, jjardon has found something which needs a bisection, that is the tougher part I suppose13:07
jjardonabderrahim[m]: have you seen https://gitlab.com/BuildStream/buildstream/issues/1098 I'm  bit worried that only happen on the bst-1 branch but not in the bst-1.2 one13:07
tristanthat's the one13:07
tristanjjardon, there are not many patches really, I think it would be sane to just (A) Create a patch which makes the overnight tests run as regular pipelines... (B) Apply it at the tip of a new branch for every merged branch on bst-113:08
tristanjust run it all at once as a bisection via CI ?13:08
* tristan could try to cook that up13:08
jjardonthat would work indeed13:09
tristanjjardon, as I'll be going on vacation after tomorrows flight, probably you or juergbi will have to do the *actual* release dance13:11
tristanalthough that is mostly a mindless ritual :)13:12
tristanjjardon, Why do we have this "Test" and "Revert test" noise sitting directly on the bst-1 branch ?13:15
tristansince we've not tagged anything since, shall we just remove that ?13:15
*** julien has quit IRC13:35
jjardontristan: that was a mistake. I tend to not modify history on public branches even if they are mistakes but your choice13:37
*** lachlan has joined #buildstream13:37
tristanjjardon, yeah I got it... I mean you are totally right in general, this just happens to be a lucky case where nothing was ever published from the branch so, could be done13:52
gitlab-br-bottmewett opened MR !1572 (tmewett/cli-defaults->master: cli.py: Use Click's show_default for defaults in help text) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/157213:52
tristanbut I'm ambivalent, just noticed that when observing the history to bisect hehe13:52
*** lachlan has quit IRC14:05
*** jonathanmaw has quit IRC14:09
tme5hi tristan, I wondered if you had any more thought on my reply to your email about DownloadableFileSource and local files14:12
tristanjjardon, Whats with https://gitlab.com/BuildStream/buildstream/-/jobs/284284407 ? Did a valid commit of bst-external really disappear ?14:17
tristanjjardon, I ask because some of your commits from august 5th seem to update this... I didn't backport *that* for the bisect as it should work still how it used to right ?14:18
tristantme5, ummm, yeah I guess I do14:20
tristantme5, You say "Without abstracting track/fetch, the behaviour in (2) would have to be implemented manually again", I'm not sure what you mean by that14:21
jjardontristan: seems something is missing in https://files.pythonhosted.org ?14:21
tristantme5, Looking back at your original mail, (2) seems to indicate that you want an implicit behavioural change to occur, depending on whether someone specified a URL which begins with the text "file://"14:21
*** lachlan has joined #buildstream14:21
qinustyjuergbi, RE artifact cache not being required. How would user experience and performance differ if we weren't storing artifacts during remote execution? And what about running RE services with buildstream 1.3+x? Is a reference store required?14:21
tristantme5, I think that would be a really bad idea, I originally read the mail (as you noticed), to mean that "file:" was going to be a separate key right ?14:22
tristanWell14:22
jjardontristan: we only know bst-1 works when it was == to bst-1.2, after that it never have CI so not sure if it ever work14:22
tristantme5, I don't know honestly it's a bit far and I'm foggy on this... lemme try to decrypt this whole subject a bit further14:23
tristanjjardon, It had CI sure... nightlies we dont know if they passed14:23
jjardontristan: yeah sorry, I meant that14:24
tristanjjardon, but we never changed the ref we were using to bst-external, there's no excuse for it to break with the same ref we were using right ?14:24
jjardontristan: only if you change the ref of freedesktop-sdk you use14:24
tristanOr do we have a change in there which changes the gitlab-ci to use a non-existing ref ?14:24
tristanRight but why would we change the ref of freedesktop-sdk we're using ?14:25
tristanThis job is failing when cloning bst-external afaics14:25
jjardontristan: because some upstream of older freedesktop-sdk dissapear, for example14:26
tristanjjardon, What does that mean, how can something disappear ?14:26
jjardontristan: some refs can go away14:27
tristanjjardon, this is doing `pip3 install --user -e ${bst_ext_url}@${bst_ext_ref}#egg=bst_ext`14:27
tristanand it fails14:27
*** lachlan has quit IRC14:27
jjardonyeah, I mean some upstream refs from freedesktop-sdk14:27
jjardonnot sure why that is failing14:28
tristanok but this is besides the point right ?14:28
tristanjjardon, How can bst-1.2 CI possibly work then ?14:28
tristanOr, I see you updated bst-1.2 to make that work too ?14:28
tristanthat is not very nice :-/14:28
jjardonwhat is not very nice?14:28
tristanjjardon, I mean that the ref disappear at all14:29
abderrahim[m]Hi14:29
tristannot nice at all, we have to trust that the refs we make to external components in CI can be relied upon14:29
jjardonyeah :(14:29
* abderrahim[m] is back14:29
tristanWe use refs to branches sometimes, but only in side branches14:29
tristanjjardon, Technically this means I cannot truly conduct this bisection14:30
abderrahim[m]Oh, yeah I noticed #1098 once14:30
gitlab-br-botIssue #1098: bst 1.x overnigth tests are failing: Message handling out of sync https://gitlab.com/BuildStream/buildstream/issues/109814:30
tristanwell, I can do it but there is this vector of uncertainty that is impossible to trace :-/14:30
tristanabderrahim[m], apparently that race condition is happening consistently with bst-114:30
*** phoenix has joined #buildstream14:31
tristanok well, I'll backport the change to all the branches and run it all again14:31
jjardontristan: upgrade bst-external to the latest, then start to bisect?14:31
tristanjjardon, Right, but then we are based on a changed bst-external, so we don't know if a change in that ref has an effect on why this happens14:31
tristananyway14:31
tristanit should still be helpful, but it's not the test I intended to run :-/14:31
abderrahim[m]Not really consistently, I've only seen it once14:31
tristanabderrahim[m], I believe the issue is about consistent failure... yeah I've seen that message rarely too :)14:32
abderrahim[m]Using e48a03b2b226f1748edf5c2d5e28093da7ca6a1a, I don't know more things were added later14:32
jjardonabderrahim[m]: It's 100% reproducible in CI, but only in the bst-1 branch, bst-1.2 is fine14:32
abderrahim[m]I'm using bst-114:33
jjardonabderrahim[m]: note it fails when downloading the first image, probably you already have that cached14:33
tristantme5, Ok so right... looking back at this email thread, remembering a bit more clearly.... my thoughts are indeed still that the value of a specified "url" should *definitely* not change how the file is handled14:34
abderrahim[m]The failure I had was with building webkit, so I guess it has to do with long operations14:34
tristantme5, This means that I would expect that if you used a "file://" as a url, and automated a relative path expansion... BuildStream would *STILL* fetch it and cache it into it's sources directory, and calculate the refs and everything as normal14:34
tristantme5, The user did not "specify a project path", the user has specified a URL, and the magic in Source.translate_url() makes it magically work for everyone14:35
abderrahim[m]tristan: tme5 I'd say it's better to have either url or path14:35
abderrahim[m]rather than the file url14:35
tristanabderrahim[m], I rather think so, but if there is a {project_dir} kind of automatic expansion in URLs automatically, it would allow relocation of data to a local disk easily for any kind of Source14:36
tristanOne could even define a local "mirror" for it14:36
tristanAnd since the refs are all in place, they'd still pretty much work the same way14:37
tristan(you'd still perform a checksum check on a file downloaded from "file://..." that you would on a file downloaded from "https://...", so I think it would just be an overall convenience)14:38
tme5tristan, I see your point. I too would expect file: URLs to be handled no different to network schemes14:47
tme5what about changing the different behaviour to not be on file: URLs but instead only when project-relative paths are given? I think then a user would expect different behaviour14:49
tristantme5, I think it's important that this decision is made depending on the *key* which the user provided, and not depending on the *value* provided to an existing key14:49
tristantme5, especially considering source aliases exist, and if we have different behaviours for the same source depending on which alias is in use (depending on mirroring and fallbacks), very strange things could happen14:50
tristanif we behaved differently depending on the *value*14:51
tme5so, for example, one of either a) 'url' and 'ref', or b) 'path' keys?14:51
tme5yes that's understandable14:52
tristantme5, yeah exactly that14:53
tristanlets see what the pipelines are doing14:53
tme5I was clearly idealising some amazing unification of specifying location when I wrote the proposal :P14:53
tlater[m]:o I just discovered `tox -e py37-nocover`14:55
tlater[m]And here I was just running `tox -e py37 -- --no-cov` and accepting that my tests will always fail locally.14:56
* tlater[m] wonders why CONTRIBUTING.rst doesn't mention tox factorization14:56
*** phoenix has quit IRC14:59
gitlab-br-botjennis opened MR !1573 (jennis/tasks->master: Improve long-running task reporting) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/157315:02
tristanjjardon, Have you noticed an issue installing recently in the overnight tests ? After backporting your new refs and stuff to all the branches, there seems to be an ignored error installing PyGObject (it fails to install cairo, no idea why we'd want cairo but PyGObject somehow complains at install time ?)15:03
*** phoenix has joined #buildstream15:04
coldtom PyGObject has a dependency on pycairo iirc15:05
gitlab-br-botjennis approved MR !1572 (tmewett/cli-defaults->master: cli.py: Use Click's show_default for defaults in help text) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/157215:09
jennisah, nice tme515:09
tristancoldtom, yeah that rings a bell indeed15:09
tristancoldtom, jennis ... careful with "show_default"15:12
tristanoops15:12
tristanI mean jennis, tme5 :)15:12
jennishow comes?15:12
tristanSo... a huge portion of the `bst` main options are all explicitly None by default as far as click knows15:13
tristanjennis, This would mean that we would be showing None explicitly as a default value wherever the default value is derived from "BuildStream defaults + user config file" ?15:13
gitlab-br-botQinusty approvaled MR !1540 (tlater/cache-endpoints->master: Support separate end points for artifact caches) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/154015:13
tristanIf so, that is a bit weird15:13
juergbiqinusty: sorry, missed your message before. if an action cache is used, reusing a previously built element that is not in the local cache will be a bit slower (dependencies have to be virtually staged/integrated) but reuse should still be possible15:13
gitlab-br-botQinusty unapprovaled MR !1540 (tlater/cache-endpoints->master: Support separate end points for artifact caches) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/154015:13
tme5of course, i haven't changed any of those options tristan15:14
jennisI see this has been changed for checkout: https://gitlab.com/BuildStream/buildstream/merge_requests/1572/diffs#48a5425791d6d9d502dfdb9fad1e14e38ec05629_1038_103815:14
juergbiqinusty: without action cache, remote execution will still work but it will never reuse a previously built artifact, unless it's in the local cache15:14
tristantme5, Sure... I'm just saying "careful" :)15:14
jennisbut I have an MR that also changes this, I don't see why that should be None15:14
tme5ah ok, thanks haha15:14
tristantme5, if you've verified this... I just wanted to point it out :)15:14
tme5appreciated15:15
tlater[m]tme5: Have your test suite changes landed btw?15:16
tme5they have15:16
tlater[m]\o/15:17
tme5a week later than i told you... sorry lol15:17
qinustyAh okay, so performance will be slightly hindered whilst using an action cache but in bst master/2.x we shouldn't see these slow downs? (Or is the slow down just accepted with the deprecation of ReferenceStorage?) juergbi15:17
juergbiqinusty: RE support is only in master/2.x15:18
juergbiReferenceStorage was replaced by the Artifact/Source service but is equivalent in this regard15:19
qinustyOkay, so maybe I am misunderstanding the whole master/2.x versioning. Is buildstream:nightly 2.x? --version provides 1.3.0+<ref>15:21
* tristan finds it a little bit unfortunate that ReferenceStorage sounds a lot like ref-storage (assuming that the former is completely unrelated to the latter)15:21
tristanjjardon, Still around ? I have an interesting observation...15:23
jjardontristan: sorry I was in a meeting; I do not see any issues with the latest nigtlhy from yesterday on the bst-1.2 branch15:23
tristanjjardon, My question to you is... have you ever seen this out-of-sync message bug happen with a source other than the OSTree source ?15:23
tristanjjardon, I think it is related to this failure to get the PyGObject deps installed (cairo) which seems to be consistently ignored15:24
jjardontristan: I actually think that out-of-sync is not the cause of the problem; I have seen that error in succesful builds15:24
tristanjjardon, I.e. OSTree is borked if PyGObject doesnt install, and we only really find out at fetch time, is a new theory15:24
jjardontristan: and to answer your question: no, I think only see that when importing ostree repos15:25
jjardontristan: interesting15:25
qinustyI'm working from the buildgrid docs really https://buildgrid.gitlab.io/buildgrid/reference_server_config.html#reference-configuration. Would I be correct in assuming that as of master/2.x, buildstream is REAPI compatible aside from the bst-artifact-server requirement for remote caching? (Obviously we have issues with specific server implementations15:25
qinusty)15:25
tristanjjardon, I have a new failure anyway which supports this theory... I'll see if I can fix the install to both install properly and fail the CI properly if it fails, and hope that fixes it15:26
ironfootI keep having this issue, does anybody know what can be causing it? https://paste.gnome.org/papsyni5s15:28
ironfootthe element I'm trying to build (compose)  includes a stack element. This stack element has 2 elements inside.15:29
ironfootThe failure has happened when adding the second element in the stack, wich literally copies a file from build-root to install-root/lib/firmware/15:29
ironfootBuilding and checking out the stack element works fine and looks correct15:32
tristanironfoot, looks suspicious, trying to link / -> /install/lib/firmware ?15:35
tristanor rather to /buildstream/install15:36
ironfootI've looked into the cache/buildstream/build dir, and at that point /lib is a broken symlink15:37
ironfootbut yes, it's odd15:37
tristana test case would be very helpful of course...15:37
ironfootmaybe I shouldn't be doing this? https://paste.gnome.org/pmjek4ndh15:38
tristanthat doesnt look wrong to me15:38
tristanwell, weird maybe a bit15:38
tristanironfoot, remember this is a usr merge system15:39
tristanironfoot, so you are essentially calling `cp -r` on a symlink to lib -> usr/lib ?15:39
ironfootwill try changing that15:40
*** lachlan has joined #buildstream15:41
ironfootcreating the usr/lib/firmware and copying the files inside worked16:00
ironfootprobably it unpacks this in the staging area before usr/lib is present16:01
ironfootnvm, it's working now16:01
*** raoul has quit IRC16:08
tristanironfoot, conceptually there should be more efficient ways to do this but we lack features to tell elements like compose/filter to *relocate* certain files16:10
tristanironfoot, just pointing out that we shouldnt really need a real disk copy to happen in order to say "Take this set of files from the input, and include it in the output under this new location"16:11
tristanjjardon, :)16:13
tristanjjardon, it's fixed by reverting 08ead05bbe24a6de94e26b1121e5aa8dd341f691 ... and it appears that reverting that move to a fedora 30 docker does not reintroduce #109716:14
gitlab-br-botIssue #1097: bst 1.x overnigth tests are failing: SyntaxError: invalid syntax https://gitlab.com/BuildStream/buildstream/issues/109716:14
tristanoh not even ? this is very strange16:14
tristanhttps://gitlab.com/BuildStream/buildstream/-/jobs/28439883016:15
tristanjjardon, With the revert of the docker image change... we end up successfully pulling the base image with ostree... BUT we still have the stack trace while installing PyGObject (same problem error with cairo)16:16
tristanWho knows how long that stack trace has been silently raising without actually causing anything to break16:16
*** lachlan has quit IRC16:33
*** tiagogomes has quit IRC16:36
*** tme5 has quit IRC16:38
gitlab-br-bottristanvb closed issue #1098 (bst 1.x overnigth tests are failing: Message handling out of sync) on buildstream https://gitlab.com/BuildStream/buildstream/issues/109816:40
*** lachlan has joined #buildstream16:55
*** rdale has quit IRC19:09
*** lachlan has quit IRC19:16
*** lachlan has joined #buildstream19:23
*** lachlan has quit IRC19:34
*** lachlan has joined #buildstream19:46
*** lachlan has quit IRC19:53
*** bochecha has joined #buildstream19:58
*** cs-shadow has quit IRC20:09
*** phoenix has quit IRC22:31
*** bochecha has quit IRC22:51

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!