IRC logs for #buildstream for Tuesday, 2018-11-27

*** cs-shadow has quit IRC01:27
*** nimish has joined #buildstream02:29
*** abderrah1 has joined #buildstream03:15
*** abderrahim has quit IRC03:17
*** tristan has joined #buildstream05:17
*** nimish has quit IRC06:28
*** nimish has joined #buildstream06:58
*** ChanServ sets mode: +o tristan07:47
tristanHrrrrrm :-/07:47
tristanHow come when I run `bst build --track-all sdk/gtk+-3.bst`... BuildStream master is pulling stuff ?07:47
tristanIt cannot possibly know the cache keys until the elements get tracked07:47
tristanahh, answer my own question07:48
tristanIt's pulling freedesktop-sdk stuff07:48
*** toscalix has joined #buildstream08:22
*** tristan has quit IRC08:26
*** alatiera has joined #buildstream08:36
*** finn has joined #buildstream08:41
Kinnison:-)09:03
*** nimish has quit IRC09:07
*** nimish has joined #buildstream09:08
*** solid_black has joined #buildstream09:14
*** toscalix has quit IRC09:17
jjardonHi buildstreamers, ok to merge https://gitlab.com/BuildStream/buildstream/merge_requests/972 for 1.4 then? The MR already has some reviews09:21
*** tristan has joined #buildstream09:26
jmacHmm "remote: GitLab: API is not accessible"09:32
jmacWebsite seems fine but I can't push with git09:32
Kinnisonpushing over https or ssh ?09:33
jmacssh09:34
* Kinnison nods, I just tested over https and have the same issue09:34
KinnisonIt seems to be that the hooks inside gitlab aren't running09:34
* Kinnison imagines they'll be on it09:34
jmacLooks like it's working again09:49
*** jonathanmaw has joined #buildstream09:58
jmacgitlab-br-bot doesn't seem to have returned after the disconnection on Monday10:00
*** tpollard has joined #buildstream10:10
*** nimish has quit IRC10:33
valentindtristan, I think the issue with ruamel was maybe it was missing on some older docker images. But I forgot to write down on the MR why.10:33
*** nimish has joined #buildstream10:33
*** lachlan has joined #buildstream10:34
*** nimish has quit IRC10:38
*** nimish has joined #buildstream10:38
*** nimish has quit IRC10:48
*** nimish has joined #buildstream10:49
*** raoul_ is now known as raoul10:50
raoulI've taken MR !969 out of WIP if anyone wants to take a look. Probably most relevant to juergbi and tristan who discussed it on the mailing list (and sander but I'm not sure if he's here or what his irc name is)10:52
*** nimish_ has joined #buildstream10:54
*** nimish has quit IRC10:56
*** nimish_ has quit IRC10:57
*** lachlan has quit IRC10:59
*** ChanServ sets mode: +o tristan11:03
tristanraoul, not diving into a full review right now... looks like a big one, just added one comment for juergbi there11:03
tristanvalentind, Do we know that BuildStream works for the selected range of ruamel.yaml versions ?11:03
tristanjust curious, in any case nailing it down to something a bit more specific will be good, and will hopefully resolve the new crasher11:04
raoultristan, nw I'll keep an eye out for future comments11:04
valentindtristan, I remember I tested python 3 and found it works between 41 and 51.11:04
valentindBut it did not work on .4011:05
tristanMkay, it is strange that I have had .30 for a long time and it has been "working" for me11:05
*** lachlan has joined #buildstream11:05
tristanvalentind, Do you recall when python3 support was added ?11:05
valentindtristan, this I do not know.11:06
tristanvalentind, Ummmmm11:07
tristanOkay but, it was clearly >= 0.15 ?11:07
valentindPython 3.5 was on 0.10.10 according to the changes.11:07
valentindPython 3.6 worked on 0.14. This I remember.11:07
tristanvalentind, the "right thing" to do would be to follow the maintainers instructions, and say <= 0.1511:07
valentindIt was python 3.7 that needed .41.11:07
valentindtristan, that does not work with python 3.7.11:07
tristanOhhhhh, there is a 3.7 already, and no ruamel.yaml ?11:08
tristanDo we support 3.7 ?11:08
valentindWe need at least 0.15.41.11:08
valentindBuildStream works on Python 3.7.11:08
tristanIn the above, s/and no ruamel.yaml/and no stable ruamel.yaml version which supports 3.7/11:08
valentindI know juergbi always uses 3.7.11:08
tristanSo, for BuildStream to support 3.7, we *must* use an unstable version of ruamel.yaml, that is not very ideal :-S11:09
tristanProbably worth a comment in setup.py beside the ruamel.yaml requirement, mentioning how dangerous and irresponsible this is :-S11:09
valentindtristan, I think it is irresponsible for ruamel not to maintain 0.14.11:11
tristanWell11:12
tristanvalentind, That doesnt make us more responsible for depending on a library which does not11:12
tristanThat reflects badly on ruamel.yaml maintainers, and on Python in general for bumping minor versions and introducing API breakages11:13
tristanThis reflects badly on us :)11:13
tristanSo Python maintainers, ruamel.yaml maintainers, and us... all irresponsible :)11:13
tristanwe really need a C library for round tripping YAML anyway :-S11:14
valentindI can look at doing that.11:15
tristanI know !11:18
*** lachlan has quit IRC11:18
tristanvalentind, But it might take a while :)11:18
tristanhehe11:18
tristanI mean, we *also* need a python binding to it, which doesnt need to match `ruamel.yaml` exactly...11:19
tristanI guess not the hardest part; it needs to support round tripping, which I dare say you might have a better chance at achieving when writing from scratch11:20
tristanbecause ruamel.yaml is buggy in some cases, loses some existing whitespace when round tripping11:20
tristanand needs to remember the line/column for every loaded node, that aint hard11:20
tristanand then the whole parser11:20
tristanseems like... a lot ?11:20
valentindtristan, It is a difficult problem to keep indentation.11:21
tristanWell, I suppose it's much less of a difficult problem going off the assumption that anyway you *must* have the line/column information for every key/value/listelement/etc11:21
valentindThe parser, we can probably use an LL(k) parser generator.11:21
tristanSo that intendation *has* to be there anyway11:21
*** lachlan has joined #buildstream11:25
*** lachlan has quit IRC11:30
*** lachlan has joined #buildstream11:31
*** solid_black has quit IRC11:32
*** solid_black has joined #buildstream11:32
*** lachlan has quit IRC11:35
laurenceJuerg is probably not able to be online much today, but i agreed with him yesterday that I'd chase people for reviews on this optimization for local junctions - https://gitlab.com/BuildStream/buildstream/merge_requests/29011:40
laurenceit's a relatively small patch if anyone has the brainspace11:40
tristanlaurence, reviewed, was a no brainer really11:43
*** nimish has joined #buildstream11:43
tristanjuergbi, has been very busy and the MR got lost under a mountain of other things I suspect, it was just a very easy change to his patch which he now fixed11:44
jmacYep, I've just taken a look and it looks good to me too11:45
laurencetristan, that's exactly it, cheers for the review11:45
*** lachlan has joined #buildstream11:50
*** nimish has quit IRC12:03
*** nimish has joined #buildstream12:04
juergbita, will merge12:11
juergbitristan: full build of freedesktop-sdk/gnome-meta via remote execution requires !915 would be great to merge this ;)12:17
tristanAha !12:17
tristanor, it will be slow without command batching12:18
juergbino, that's not just a performance issue, it will fail12:18
*** alatiera has quit IRC12:18
juergbibecause at least cairo has peculiar mtime requirements between configure and make steps12:18
tristanAh that issue12:18
tristanRight12:18
tristanAh I didnt know about contextlib.suppress()12:21
tristanI usually use ExitStack() for conditional context managers12:21
*** alatiera has joined #buildstream12:21
tristanjuergbi, see: https://gitlab.com/BuildStream/buildstream/merge_requests/915#note_12039202812:27
*** lachlan has quit IRC12:28
tristanjmac, since you spoke up in the unit test debate on the ML, you might be interested to comment on !973, either on the MR or ML... maybe particularly this: https://gitlab.com/BuildStream/buildstream/merge_requests/973#note_120329379 which I wrote up (amongst other comments this morning)12:41
tristanjmac, I totally accept that your opinion might not reinforce my own, and am willing to bend my own obstinate viewpoints... but for that I need other people to voice opinions :)12:42
tristanKinnison, maybe you want to weigh in on this as well ?12:42
tristanit's not great when the whole argument is just my opinion vs Angelos's12:43
jmacI'll try and offer some comments on it.12:46
jmacTo be honest, the volume of traffic on the mailing list and gitlab is too much for me to keep track of and make informed comments on.12:46
*** nimish has quit IRC12:52
*** lachlan has joined #buildstream12:53
tristanjmac, Indeed, do you follow the project wide notifications ?12:58
* paulsherwood notices that folks on the ml don't use 'snip', which makes it seem as though there's way more to read12:59
paulsherwood(and some folks just write very long emails :-)12:59
tristanjmac, I'm slowly planning on reducing spam there so that developers can subscribe the notifications stream and be included in the main feedback loop, one of the big steps towards this is to stop with all the premature WIP merge requests which bombard the stream with tons of irrelevant, premature information12:59
*** lachlan has quit IRC13:00
*** lachlan has joined #buildstream13:01
Kinnisontristan: I'll have a look at that MR, I'm broadly in favour of testing the right surfaces, which last we spoke matched both your ideas and Angelos', so I need to try and work out where the disconnect is13:19
jmactristan: My GitLab notifications come through a completely separate account; they're not a problem13:21
jmacbuildstream-list on its own is too high volume13:21
*** lachlan has quit IRC13:22
*** tristan has quit IRC13:29
*** solid_black has quit IRC13:37
jmacWow, you really ran with the rivet analogy, huh13:39
*** nimish has joined #buildstream13:59
*** tristan has joined #buildstream14:02
jjardontristan: any thoughts about https://gitlab.com/BuildStream/buildstream/merge_requests/972 ? I think is pretty important14:07
*** nimish has quit IRC14:09
*** nimish has joined #buildstream14:09
tpollardjjardon: interesting14:10
*** cs-shadow has joined #buildstream14:12
juergbitristan: as mentioned in a mail last week, do you agree that #631 should be a 1.4 blocker? I think #780 should be a blocker as well14:20
juergbi#780 should be pretty easy to fix, if I'm not missing anything14:21
*** lachlan has joined #buildstream14:22
*** ChanServ sets mode: +o tristan14:37
tristanjuergbi, I thought it was pretty ridiculous of a question and didnt answer14:38
tristanjuergbi, i.e. it looks like "we've implemented remote execution, there is just no way to enable it !"14:38
tristanIt is not good enough to say that we want to include the remote execution feature ? Or do we need to spell out every little detail in little issue reports ?14:38
tristanjuergbi, regarding #780, I think that the question revolves mostly around "Will adding the encryption keys to the configuration cause a user configuration API break ?" if yes, then blocker14:40
tristanOtherwise, I'd rather lay off the blocker buzzword and expect that that might make sense to land in 1.4 anyway14:41
tristanWithout adding undue stress to people14:41
juergbiI don't think it's an issue compatibility wise. I just think it's pretty bad to release 1.4 without those fixes. and marking them as blocker would show that14:49
tristanjuergbi, How can remote execution possibly work at all if you cannot configure the URL for it ?14:54
tristanSince you are saying "fixes"14:55
tristanIs there some project data ? That would be wrong, a remote execution service is certainly a user configuration for which the project can make a recommendation of a default14:55
tristanbut it is not "project data", it lives on Context for sure14:56
tristanjuergbi, if that's the case -> please blocker14:56
tristanjuergbi, Please blocker anyway; the only reason I didnt respond is because it sounded like such an absurd question :)14:56
tristanLike duh... of course14:56
tristanjuergbi, take this comment I wrote today for context: https://gitlab.com/BuildStream/buildstream/issues/566#note_12040014114:58
tristanThere is a pattern already for what configurations are in what domain; remote execution should not just break this pattern14:58
juergbitristan: yes, that's the case, which is why I think it's extremely important15:00
juergbiwill mark it as blocker. wasn't sure about whether I can just go ahead :)15:00
tristanjuergbi, yes then, please -> blocker for #631 at least15:00
tristanjuergbi, for the other, I just feel that "blocker" is a big word, there is a difference between "really really should" and "must"15:01
tristanLet's try to be gentle with the "must" messaging, and presume that since it is very important, it will very highly likely get into 1.4 anyway15:01
tpollardtristan: a quick question on the recommendations thing, if a user has config that explicitly sets something to the default value, then the project cannot override it, even if it is set to the default value?15:01
*** nimish has quit IRC15:01
tristantpollard, Correct, for user preferences, the project can never override the user's choice15:02
juergbiok, marked the other as urgent instead (our priority level 3/4). I think that's warranted15:02
tristantpollard, There is a way and pattern in which this is implemented which makes that always work15:02
tristanI.e. the composition order of Context dictionaries15:03
* tpollard nods15:03
tpollardjust wanted to confirm15:03
tristanadmittedly, I believe the way that the project recommends artifact servers is totally a mess and is an exception to that rule15:04
tristanas it goes ahead and parses things in the wrong places (ArtifactCache and ArtifactCacheSpec etc, goes and takes ownership of config parsing which it really shouldnt, imnsho)15:04
juergbiwe also still have the outstanding issue about the current combination of artifact servers across junctions may not always be what the user wants15:07
tpollardyep15:08
juergbibut I don't think we have a full proposal yet how to better handle this15:08
*** bochecha has joined #buildstream15:12
jonathanmawhmm, it seems `bst source-bundle` doesn't work properly if there's an open workspace15:17
jonathanmawgiven there are discussions on retiring it (https://gitlab.com/BuildStream/buildstream/issues/672), I gather I should stop making source-bundle specific changes15:18
*** toscalix has joined #buildstream15:28
*** ChanServ sets mode: +o ironfoot15:48
*** lachlan has quit IRC16:09
KinnisonNexus: looks like you have some lint failures on your MR, but otherwise the tests seem to be passing now16:18
KinnisonOh no, one test failure too16:18
Kinnison:(16:18
*** nimish has joined #buildstream16:18
KinnisonMay need to tweak error domains16:18
*** xjuan has joined #buildstream16:30
*** lachlan has joined #buildstream16:41
jmacjuergbi: Thanks for the re-review on !946. I've fixed the bugs you found.16:46
*** gitlab-br-bot has quit IRC16:51
*** gitlab-br-bot has joined #buildstream16:51
ironfoot!94616:53
gitlab-br-botMR !946: Split remote execution from artifact cache https://gitlab.com/BuildStream/buildstream/merge_requests/94616:53
*** nimish has quit IRC17:03
*** nimish has joined #buildstream17:03
*** finn has quit IRC17:08
*** finn has joined #buildstream17:09
NexusKinnison: yay it passed \o/17:24
KinnisonNexus: wonderful, have you dealt with all the discussions do you think?17:25
NexusKinnison: other than the one i opened for juergbi and tristan to reply to, i believe so.17:26
Nexusi've split it into more commits now, i'd apprecite your feedback on the structure17:26
KinnisonOkay, I'll have a look, though it might have to be in the morning since I need to skedaddle in a few mins17:27
Nexusno problem :)17:27
* Kinnison skims the commit list now17:27
KinnisonThe only comment I have from the skim is that the first commit might be better worded as: "_context.py: Move `warn()` here from _plugin.py"17:28
Nexusyeah i wasn't certain about that one, thanks17:28
*** nimish has quit IRC17:28
*** nimish has joined #buildstream17:29
NexusDone17:29
KinnisonI'll look deeper tomorrow, modulo any feedback from juergbi or tristan :-)17:29
Nexuskk c:17:30
*** finn has quit IRC17:32
*** toscalix has quit IRC17:43
*** tpollard has quit IRC17:46
*** jonathanmaw has quit IRC17:58
*** nimish has quit IRC17:59
*** nimish has joined #buildstream18:00
*** xjuan has quit IRC18:04
gitlab-br-botjuergbi approved MR !946 (jmac/remote_execution_split->master: Split remote execution from artifact cache) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94618:15
gitlab-br-botjmacarthur closed issue #750 (Alter SandboxRemote to use separate CAS store) on buildstream https://gitlab.com/BuildStream/buildstream/issues/75018:17
gitlab-br-botjmacarthur merged MR !946 (jmac/remote_execution_split->master: Split remote execution from artifact cache) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/94618:17
*** bethw has quit IRC18:17
*** valentind has quit IRC18:17
*** johnward has quit IRC18:17
*** WSalmon has quit IRC18:17
*** laurence has quit IRC18:17
*** mablanch has quit IRC18:17
*** aiden has quit IRC18:17
*** aiden has joined #buildstream18:17
*** tiagogomes_ has quit IRC18:18
*** johnward has joined #buildstream18:18
*** nimish has quit IRC18:19
*** valentind has joined #buildstream18:19
*** nimish has joined #buildstream18:20
*** bethw has joined #buildstream18:20
*** raoul has quit IRC18:20
*** laurence has joined #buildstream18:21
*** WSalmon has joined #buildstream18:21
*** finn has joined #buildstream18:23
*** mablanch has joined #buildstream18:23
*** nimish has quit IRC18:30
*** nimish has joined #buildstream18:30
*** alatiera has quit IRC18:55
*** nimish_ has joined #buildstream19:00
*** nimish has quit IRC19:02
*** nimish_ is now known as nimish19:02
*** lachlan has quit IRC19:06
*** nimish has quit IRC19:22
*** nimish has joined #buildstream19:24
*** hergertme has joined #buildstream19:31
*** finn has quit IRC19:33
*** nimish has quit IRC19:48
*** nimish has joined #buildstream19:49
*** finn has joined #buildstream19:59
*** finn has joined #buildstream20:57
*** bochecha has quit IRC21:20
*** finn has joined #buildstream21:28
*** finn has quit IRC21:35
*** finn has joined #buildstream21:40
*** nimish has quit IRC22:17
*** tristan has quit IRC22:30

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!