IRC logs for #buildstream for Wednesday, 2019-08-14

*** tristan_ has joined #buildstream01:17
*** tristan_ has quit IRC01:38
*** tristan_ has joined #buildstream02:31
*** tristan_ has quit IRC02:38
*** admin has quit IRC03:07
*** swick has joined #buildstream03:08
*** tristan_ has joined #buildstream03:34
*** tristan_ has quit IRC03:39
*** tristan_ has joined #buildstream03:58
*** tristan_ has quit IRC03:59
*** tristan_ has joined #buildstream04:00
*** tristan_ has quit IRC04:02
*** tristan_ has joined #buildstream04:05
*** tristan_ has joined #buildstream04:30
*** tristan_ has quit IRC04:31
*** tristan_ has joined #buildstream04:37
*** tristan_ has quit IRC04:39
*** tristan_ has joined #buildstream04:44
*** tristan_ has quit IRC04:49
*** tristan_ has joined #buildstream04:49
*** tristan_ has quit IRC04:50
*** tristan_ has joined #buildstream04:51
*** tristan_ has quit IRC04:53
*** tristan_ has joined #buildstream05:08
*** tristan_ has quit IRC05:09
*** tristan_ has joined #buildstream05:10
*** tristan_ has quit IRC05:11
*** tristan_ has joined #buildstream05:12
*** tristan_ has quit IRC05:13
*** tristan_ has joined #buildstream05:59
*** tristan_ has quit IRC06:01
*** tristan_ has joined #buildstream06:01
*** tristan_ has quit IRC06:04
*** tristan_ has joined #buildstream06:05
*** tristan_ has quit IRC06:07
*** tristan_ has joined #buildstream06:12
*** tristan_ has quit IRC06:14
*** tristan_ has joined #buildstream06:19
*** tristan_ has joined #buildstream06:21
*** tristan_ has quit IRC06:23
*** tristan_ has joined #buildstream06:34
*** tristan_ has quit IRC06:36
*** tristan_ has joined #buildstream06:57
gitlab-br-botdanielsilverstone-ct opened MR !1543 (danielsilverstone-ct/missing-sh-fix->master: Cache failed builds caused by missing-command) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/154307:15
*** tristan_ has joined #buildstream07:22
*** tristan_ has quit IRC07:25
*** narispo has quit IRC07:31
*** narispo has joined #buildstream07:31
*** narispo has quit IRC07:36
KinnisonHave I recently been granted more access than I previously had?  I now seem to have "merge when pipeline succeeds" buttons where I didn't have them before07:40
qinustyDoesn't buildstream require atleast 1 approval?07:40
* qinusty shrugs07:41
qinustyapparently it's changed, who knows07:41
juergbiis there a sensible place where I can upload a ~2MB static buildbox casd binary (maybe digitalocean server)? it will be used by WSL CI (as it doesn't use docker images) and by users who don't want to build it on their own07:44
*** narispo has joined #buildstream07:44
juergbiwould initially only be linux x86-64, but we could add binaries for other platforms07:45
juergbibuilt using buildstream but not yet [buildbox] CI07:46
jennisjuergbi, on digital ocean we have the bastion and the tutorial cache server which are permanent08:04
jennisI don't think the tutorial cache server is being used though, perhaps we can take that down and spin up another one for this?08:05
juergbiwell, I don't think we need an extra VM for this08:06
juergbidon't we already store e.g. the alpine base image on digital ocean?08:07
juergbican we easily store additional files like that?08:07
alexandrufazakas+08:07
alexandrufazakasoops08:07
jennismhmm yeah, let me see if I can find it (assuming it's on the BuildStream account...)08:08
juergbithanks08:08
jennisjuergbi, it says you're on the digital ocean account, can you log on?08:17
jennisI've created a "space" called buildbox-casd-linux-x86-64, you can upload the file here08:17
jennisJust go to spaces on the LHS08:18
jennisIn fact, if the spaces will take additional binaries, I'll change the name to buildbox-casd-binaries08:20
juergbiyes, I think that works. didn't realize I already had access08:22
juergbithanks08:22
jennisjuergbi, looks like your branch is somehow appending a new line to the summary each time we write output08:29
*** rdale has joined #buildstream08:41
gitlab-br-botcoldtom opened issue #1104 (Filter elements don't work on stacks) on buildstream https://gitlab.com/BuildStream/buildstream/issues/110408:43
*** phildawson has joined #buildstream09:04
*** jonathanmaw has joined #buildstream09:07
*** tme5 has joined #buildstream09:17
gitlab-br-bottlater approved MR !1524 (jennis/push_unbuilt_artifact->master: Ensure push fails when trying to push an unbuilt element) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/152409:19
*** lachlan has joined #buildstream09:28
*** phildawson_ has joined #buildstream09:36
*** phildawson has quit IRC09:36
gitlab-br-bottlater approved MR !1543 (danielsilverstone-ct/missing-sh-fix->master: Cache failed builds caused by missing-command) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/154309:41
*** traveltissues has joined #buildstream09:42
jennisKinnison, do you mind if I cherry-pick Ben's commit and repush your branch?10:11
jennisThis commit was the test Ben added which made me realise this error in the first place10:11
KinnisonI do not object10:11
jennisOk, thanks10:11
gitlab-br-botjennis approved MR !1543 (danielsilverstone-ct/missing-sh-fix->master: Cache failed builds caused by missing-command) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/154310:15
jennisKinnison, done, thanks for fixing that10:17
Kinnisonno probs, took me a bit of time to track down, and then nearly no time to fix :D10:17
tpollard\o/10:17
jennisgitlab is being slow10:18
tme5my MR is ready for a look also :) https://gitlab.com/BuildStream/buildstream/merge_requests/154110:18
tme5it's sooo slow isn't it10:18
jennisAlso the pipeline are "pending" for quite a while before they begin, but I've checked digital ocean, we have space10:19
jennisSo that might be related to the general gitlab slowness10:19
Kinnisonthey're claiming that they *had* an incident earlier today but that it's now resolved10:19
Kinnisonthis might just be residual slowdown as it recovers10:19
gitlab-br-bottmewett opened issue #1105 (Dependency selection is inconsistent across commands) on buildstream https://gitlab.com/BuildStream/buildstream/issues/110510:44
tme5RFC on above10:46
tlater[m]tme5: ooi, are you sure PLAN includes the runtime dependencies?10:57
tlater[m]iirc it only includes runtime dependencies of build dependencies10:58
jennistlater[m], why did you unassign marge to Kinnison's MR?10:58
jenniss/to/from10:58
tlater[m]jennis: Did I?10:58
* tlater[m] thought he assigned her10:58
jennis Tristan Maat @tlater unassigned @marge-bot123 21 minutes ago10:58
tlater[m]:|10:58
tlater[m]gitlab!10:58
jennishaha10:58
tlater[m]tme5: coming back to that point...10:59
tlater[m]We obviously need runtime dependencies of build dependencies because, well, we need to run build dependencies to build our element.10:59
tlater[m]So it's not as arbitrary as you might think :)10:59
tlater[m]Provided that it doesn't include runtime dependencies of our element, but I suspect you mis-wrote/read that.11:00
tme5tlater[m], I am sure11:00
tme5look in Planner.plan_element11:00
* tme5 really hopes his python reading was correct11:00
tlater[m]tme5: I think it might not have been11:02
tlater[m]Oh, hang on11:03
tlater[m]No, you're right11:03
* tlater[m] sent a long message: < https://gnome.modular.im/_matrix/media/v1/download/gnome.org/rNIQggUVhyNgHeUgssfutfet >11:03
tme5jennis, Kinnison and I looked into this yesterday, and I'm fairly sure that's what we found empirically too11:04
tlater[m]Ok, cool, I said nothing :)11:05
*** lachlan has quit IRC11:34
*** lachlan has joined #buildstream11:47
tlater[m]jennis: Could you put marge back onto that MR?12:01
tlater[m]She's not appearing in my assignment list12:01
*** lachlan has quit IRC12:04
tpollardcan't until the pipeline is fixed or she can push new commits12:04
jennistlater[m], ahh workspaces with buildbox need to be marked with xfail12:04
jennisI'll sort it12:04
tlater[m]Can you not assign marge to anything that has a failing pipeline?12:05
Kinnisonshe'll just hand it back because it fails12:06
jennismhmm failures on both MRs for the tests-buildbox12:06
tlater[m]Kinnison: Well, she doesn't show up in the list of possible assignees in the first place for me, I wanted to try if she'd restart the pipeline for me :)12:16
KinnisonI don't think she'll rerun pipelines, no12:16
adds68thanks for the review cs-shadow12:29
*** lachlan has joined #buildstream12:30
*** alexandrufazakas has quit IRC12:35
gitlab-br-botmarge-bot123 closed issue #1101 (Sometimes failed builds are not cached as failed) on buildstream https://gitlab.com/BuildStream/buildstream/issues/110112:43
gitlab-br-botmarge-bot123 merged MR !1543 (danielsilverstone-ct/missing-sh-fix->master: Cache failed builds caused by missing-command) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/154312:43
* tlater[m] is annoyed there is no `bst source delete`13:18
tlater[m]I think we need to do something to make artifacts and sources equivalent to the front-end13:19
tlater[m]Because under the hood they're all just CAS keys anyway13:19
tlater[m]Would keep us from duplicating so damn much for the two types of CAS content.13:20
qinustyWithout the support of seperate endpoints for CAS/Artifact server does buildstream currently work with buildgrid (with remote caching support)? Perhaps tlater[m]?13:38
tlater[m]qinusty: It should do13:51
tlater[m]To my knowledge13:51
juergbitraveltissues: thinking about it more, I expect it will be somewhat cumbersome and also not very efficient to maintain a separate file for timestamps of the whole tree14:17
traveltissuesi can't immediately think of an alternative if we're ruling out metadata changes14:18
juergbiextending the Directory proto would likely be much simpler and more efficient. the main concern there is compatibility with other servers14:19
Kinnisontlater[m]: sources and artifacts are different in terms of where they live and what their semantics are in terms of sharing14:19
Kinnisontlater[m]: It'd not be complex to add `bst source {list,delete}`14:19
juergbitraveltissues: however, it's also not trivial to achieve compatibility with a separate file (would have to apply timestamps at the beginning via shell script and then create that timestamp file via shell script at the end)14:19
tlater[m]No, but the underlying code is extremely similar14:19
tlater[m]Kinnison: I'm worried that we're just writing the same bits of boilerplate for the two over and over again.14:20
traveltissuesjuergbi, yes, aiui this compatibility issue was a blocker14:20
tlater[m](Having gone through the effort of implementing source {push, pull})14:20
juergbitraveltissues: so I'm wondering whether we should rather say that (incremental) workspace builds are restricted to REAPI servers that support that extension (+ local builds, of course).14:20
juergbimight be acceptable as long as we don't break compatibility for non-workspace builds14:21
Kinnisontlater[m]: I suppose the underlying stuff might be unifiable, though I wouldn't be sad if it weren't short-term14:21
tlater[m]Yeah, it's a long-term thought14:21
tlater[m]Since I'm sitting here getting annoyed over how many Queues I need to implement that check the same 3 variables with slightly different names ;)14:22
juergbitraveltissues: this might need some further/broader discussion14:22
Kinnisonhehe14:23
juergbi#38 is also somewhat related to this14:23
gitlab-br-botIssue #38: Lost file metadata in artifacts and images https://gitlab.com/BuildStream/buildstream/issues/3814:23
traveltissuesjuergbi, we can definitely/should support that locally. but this would mean having to maintain a separate proto. I can't really say what the performance hit from a data file would be atm14:24
juergbimaybe we could standardize an extension mechanism14:25
traveltissuesyes, that's related at some point in this effort. bb would probably need some work to support this aiui14:25
juergbipossibly key-value pairs for extended attributes14:25
traveltissuesin bb fuse?14:26
juergbiI mean standardize in REAPI upstream14:26
juergbiand then support it in bb, of course14:26
juergbiafaik, other REAPI users are not currently interested in this, but maybe we could still get agreement on at least reserving a field that would be suitable for extensions14:27
juergbiI think we should at least going down that route as it would be much nicer in many aspects14:28
juergbi*at least try14:28
juergbior do you see this differently?14:28
traveltissuesno, i'd prefer to avoid using some sort of index file if possible14:31
juergbiwe should send a mail to the remote-execution-apis list. or possibly first discuss it more broadly among BuildStream/BuildGrid/BuildBox devs and then send a mail there14:33
gitlab-br-botmarge-bot123 merged MR !1539 (coldtom/move-modulebuild->master: Move modulebuild plugin to bst-plugins-experimental) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/153914:35
gitlab-br-botcoldtom opened MR !1545 (coldtom/move-qmake->master: Move qmake plugin to bst-plugins-experimental) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/154514:44
*** bochecha_ has joined #buildstream15:01
*** lachlan has quit IRC15:03
*** bochecha has quit IRC15:03
*** bochecha_ is now known as bochecha15:03
gitlab-br-botmarge-bot123 closed issue #1095 (BuildStream crashes when trying to push a non-build artifact) on buildstream https://gitlab.com/BuildStream/buildstream/issues/109515:18
gitlab-br-botmarge-bot123 merged MR !1524 (jennis/push_unbuilt_artifact->master: Ensure push fails when trying to push an unbuilt element) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/152415:18
Kinnisontme5: I'm not sure about !154115:30
gitlab-br-botMR !1541: Clarify bst artifact subcommand help text https://gitlab.com/BuildStream/buildstream/merge_requests/154115:30
* tlater[m] takes a peak while waiting for his test suite15:40
tlater[m]Grmbl grmbl pytest15:40
adds68cs-shadow, hey sorry for the delay: https://gitlab.com/BuildStream/buildstream-docker-images/merge_requests/129 is that ok?15:42
tlater[m]Kinnison: isn't that what the doc says?15:42
adds68Gitlab will squash those commits also15:42
tlater[m]Kinnison: > Element15:43
tlater[m]paths may be given instead, in which case the artifact ref is calculated15:43
tlater[m]from the element.15:43
tlater[m]Or are you unsure whether we can mix them?15:43
*** bochecha_ has joined #buildstream15:53
tpollardis the buildbox pipeline known to be broken/15:55
tpollard?15:55
*** bochecha has quit IRC15:56
*** bochecha_ is now known as bochecha15:56
tme5as far as I understand (which may not be far at all), elements give rise to artifact refs15:59
tme5so i wrote it in the way I did to clarify that; the commands operate on artifacts, but you can give element paths to use their innate artifact ref16:00
tme5tlater[m], on the subject of mixing them, i've seen jennis write in multiple places that bst artifact checkout/pull/push are not actually supposed to take refs yet. So this documents current behaviour, maybe not intended :)16:03
*** phil has joined #buildstream16:40
*** phildawson_ has quit IRC16:41
*** toscalix has joined #buildstream16:41
*** lachlan has joined #buildstream16:46
*** toscalix has quit IRC16:51
*** lachlan has quit IRC16:53
*** jonathanmaw has quit IRC17:03
*** lachlan has joined #buildstream17:09
*** lachlan has quit IRC17:15
*** tme5 has quit IRC17:17
*** lachlan has joined #buildstream17:30
*** phil has quit IRC17:43
*** bochecha has quit IRC17:49
Kinnisontlater[m]: when I read it, it felt like the emphasis had shifted enough to confuse me, I'll re read it tomorrow17:49
*** traveltissues has quit IRC17:55
*** lachlan has quit IRC17:56
*** lachlan has joined #buildstream18:17
*** tristan_ has joined #buildstream18:22
*** lachlan has quit IRC18:22
*** lachlan has joined #buildstream18:23
*** tristan_ has quit IRC18:38
*** tristan_ has joined #buildstream18:39
*** tristan_ has quit IRC18:41
*** tristan_ has joined #buildstream18:51
*** tristan_ has quit IRC18:53
*** tristan_ has joined #buildstream18:59
*** tristan_ has quit IRC19:02
*** tristan_ has joined #buildstream19:02
*** tristan_ has joined #buildstream19:04
*** tristan_ has quit IRC19:05
*** tristan_ has joined #buildstream19:05
*** tristan_ has quit IRC19:10
*** tristan_ has joined #buildstream19:11
*** tristan_ has quit IRC19:13
*** narispo has quit IRC19:16
*** narispo has joined #buildstream19:18
*** tristan_ has joined #buildstream19:20
*** tristan_ has quit IRC19:24
*** lachlan has quit IRC20:04
*** cs-shadow has quit IRC20:57
*** narispo has quit IRC22:01
*** narispo has joined #buildstream22:01

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!