IRC logs for #buildstream for Friday, 2018-06-29

*** Prince781 has joined #buildstream05:17
*** Prince781 has quit IRC05:27
*** Prince781 has joined #buildstream05:28
*** Prince781 has quit IRC06:11
*** coldtom has joined #buildstream07:49
gitlab-br-botbuildstream: merge request (valentindavid/331_include->master: WIP: Add support for include in project.conf) #471 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/47108:22
*** Phil has joined #buildstream08:34
*** jonathanmaw has joined #buildstream08:37
gitlab-br-botbuildstream: merge request (jennis/output_useful_error_when_trying_to_load_directory->master: Output a useful error message when trying to load a directory) #522 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/52208:46
noisecelltristan, ok, I will try to do what you suggest but I think we should think about the possibility to be able to configure the image as we would like after the compose happens without having to have the tools which modify your image inside of it.08:52
*** bochecha has joined #buildstream08:59
*** brejoc has joined #buildstream09:20
*** brejoc has left #buildstream09:20
gitlab-br-botbuildstream: merge request (valentindavid/331_include->master: WIP: Add support for include in project.conf) #471 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/47109:27
*** dominic has joined #buildstream10:31
gitlab-br-botbuildstream: merge request (phil/437-workspaces-tutorial->master: Phil/437 workspaces tutorial) #519 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/51910:34
* noisecell wonders if there is a way to set up ENV variables inside of the sandbox using script element and variables?10:59
jmacYes, you should be able to pass them into a sandbox11:00
jmacThere's an environment: option in project.conf11:01
jmacJust looking for per-element11:01
noisecellper element11:01
jmacYeah, same option in an element according to the documentation11:01
noisecelljmac, thanks!11:02
jmacThere's an example using LD_LIBRARY_PATH on https://buildstream.gitlab.io/buildstream/format_declaring.html11:03
jmacI haven't tried using variables, but I'd assume they would work11:03
noisecelljmac, thank you very much11:14
noisecellthat example is very helpful11:15
*** bochecha has quit IRC12:03
gitlab-br-botbuildstream: issue #451 ("Run CI on more distributions") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/45112:24
gitlab-br-botbuildstream: merge request (phil/add-ubuntu-ci-job->master: .gitlab-ci-yml: Add ubuntu 18 test) #523 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/52312:47
*** solid_black has joined #buildstream13:01
gitlab-br-botbuildstream: merge request (phil/437-workspaces-tutorial->master: Phil/437 workspaces tutorial) #519 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/51913:45
*** solid_black has quit IRC14:25
*** bochecha has joined #buildstream14:27
*** dominic has quit IRC14:31
*** Phil has quit IRC14:31
*** jmac has quit IRC14:31
*** benbrown has quit IRC14:31
*** tlater has quit IRC14:31
*** aiden has quit IRC14:31
*** mohan43u has quit IRC14:31
*** jennis has quit IRC14:31
*** skullman has quit IRC14:31
*** juergbi has quit IRC14:31
*** gitlab-br-bot has quit IRC14:31
*** pro[m] has quit IRC14:31
*** krichter[m] has quit IRC14:31
*** finn has quit IRC14:31
*** laurence has quit IRC14:31
*** awacheux[m] has quit IRC14:31
*** ernestask[m] has quit IRC14:31
*** inigomartinez has quit IRC14:31
*** m_22[m] has quit IRC14:31
*** paulsherwood has quit IRC14:31
*** valentind has quit IRC14:31
*** Demos[m] has quit IRC14:31
*** segfault3[m] has quit IRC14:31
*** asingh_[m] has quit IRC14:31
*** kailueke[m] has quit IRC14:31
*** ssssam[m] has quit IRC14:31
*** cgmcintyre[m] has quit IRC14:31
*** rafaelff[m] has quit IRC14:31
*** jjardon[m] has quit IRC14:31
*** mattiasb has quit IRC14:31
*** alatiera has quit IRC14:31
*** abderrahim[m] has quit IRC14:31
*** ptomato[m] has quit IRC14:31
*** oknf[m] has quit IRC14:31
*** theawless[m] has quit IRC14:31
*** connorshea[m] has quit IRC14:31
*** albfan[m] has quit IRC14:31
*** waltervargas[m] has quit IRC14:31
*** slaf has quit IRC14:31
*** cs_shadow has quit IRC14:31
*** jjardon has quit IRC14:31
*** csoriano has quit IRC14:31
*** persia has quit IRC14:31
*** persia has joined #buildstream14:32
*** csoriano has joined #buildstream14:32
*** jjardon has joined #buildstream14:32
*** cs_shadow has joined #buildstream14:32
*** slaf has joined #buildstream14:32
*** valentind has joined #buildstream14:32
*** ptomato[m] has joined #buildstream14:32
*** theawless[m] has joined #buildstream14:32
*** waltervargas[m] has joined #buildstream14:32
*** rafaelff[m] has joined #buildstream14:32
*** abderrahim[m] has joined #buildstream14:32
*** asingh_[m] has joined #buildstream14:32
*** connorshea[m] has joined #buildstream14:32
*** mattiasb has joined #buildstream14:32
*** kailueke[m] has joined #buildstream14:32
*** cgmcintyre[m] has joined #buildstream14:32
*** jjardon[m] has joined #buildstream14:32
*** alatiera has joined #buildstream14:32
*** segfault3[m] has joined #buildstream14:32
*** ssssam[m] has joined #buildstream14:32
*** albfan[m] has joined #buildstream14:32
*** Demos[m] has joined #buildstream14:32
*** oknf[m] has joined #buildstream14:32
*** paulsherwood has joined #buildstream14:32
*** phildawson has joined #buildstream14:32
*** jennis_ has joined #buildstream14:32
*** pro[m] has joined #buildstream14:32
*** krichter[m] has joined #buildstream14:32
*** finn has joined #buildstream14:32
*** m_22[m] has joined #buildstream14:32
*** ernestask[m] has joined #buildstream14:32
*** laurence has joined #buildstream14:32
*** awacheux[m] has joined #buildstream14:32
*** alatiera has quit IRC14:32
*** alatiera has joined #buildstream14:32
*** inigomartinez has joined #buildstream14:33
*** skullman has joined #buildstream14:33
*** mohan43u has joined #buildstream14:34
*** aiden has joined #buildstream14:37
*** tlater has joined #buildstream14:37
*** juergbi has joined #buildstream14:38
*** jmac has joined #buildstream14:40
*** benbrown has joined #buildstream14:40
*** dominic has joined #buildstream14:44
*** toscalix has joined #buildstream14:57
*** noisecell has quit IRC15:04
*** noisecell has joined #buildstream15:14
*** Prince781 has joined #buildstream15:20
*** toscalix has quit IRC15:21
*** Prince781 has quit IRC15:21
*** Prince781 has joined #buildstream15:24
*** tristan has joined #buildstream15:25
*** Prince781 has quit IRC15:32
*** Prince781 has joined #buildstream15:33
*** Prince781 has quit IRC15:36
*** Prince781 has joined #buildstream15:38
*** gitlab-br-bot has joined #buildstream15:38
*** Prince781 has quit IRC15:45
*** Prince781 has joined #buildstream15:47
*** Prince781 has quit IRC15:48
*** Prince781 has joined #buildstream15:51
jjardonHi, question about buildstream artifact management; if something is build, is the artifact pushed to the cache server immediately? or when the whole build of the project ends?15:51
jmacEach artifact is pushed when it is built15:55
*** dominic has quit IRC15:58
*** Prince781 has quit IRC15:58
gitlab-br-botbuildstream: merge request (jennis/output_useful_error_when_trying_to_load_directory->master: Output a useful error message when trying to load a directory) #522 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/52215:58
*** Prince781 has joined #buildstream16:01
*** phildawson has quit IRC16:32
gitlab-br-botbuildstream: issue #446 ("Stack trace shown if you accidentially `bst COMMAND <directory>`") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/44616:37
gitlab-br-botbuildstream: merge request (jennis/output_useful_error_when_trying_to_load_directory->master: Output a useful error message when trying to load a directory) #522 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/52216:37
valentindtristan, one thing I need to do is make tracking automatic when needed on junctions. Otherwise we get stuck in case where we cannot even call "bst track" because it would fail before.16:39
valentindI am making an example of flaptak application using the include feature. And I found that problem.16:40
tristanvalentind, hrrrrrmmmm16:41
*** jcampbell has quit IRC16:41
valentindtristan, I think the problem is junction being elements. They should have not been elements.16:42
tristanvalentind, I think this is the current situation anyway16:42
valentindWe already automatically fetch. What I do is automatically track as well.16:42
tristanUhhh16:42
*** jcampbell has joined #buildstream16:43
tristanexcept that automatically fetching does not effect the input16:43
gitlab-br-botbuildstream: merge request (adamjones/systemd-cas->juerg/googlecas: Include systemd file examples for google-cas cache) #524 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/52416:43
tristanvalentind, really... projects should be revisioning their junction refs in junctions.refs at least16:43
tristanhaving the junctions.refs is what makes building gnome-build-meta a bit less annoying for the first time in that regard16:44
valentindShould I make this automatic tracking only when configure not inlined?16:44
tristanI dont think we really need to have automatic tracking16:45
valentindI mean when it is using with project.refs and junctions.ref?16:45
valentindHow do you want to do then? Manually write the junction.refs file?16:45
tristanrather a good error informing the user that the junction references have not been resolved yet and need to be16:45
gitlab-br-botbuildstream: merge request (tiagogomes/issue-289->master: Tiagogomes/issue 289) #521 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/52116:46
tristanThe user should be able to get out of that undefined project situation by doing `bst track my-junction.bst`16:46
valentindtristan, this is the problem.16:46
tristanif the upstream maintainer of that BuildStream project failed to commit the reference16:46
gitlab-br-botbuildstream: merge request (adamjones/systemd-cas->juerg/googlecas: Include systemd file examples for google-cas cache) #524 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/52416:47
valentindIf I call "bst track junction.bst", it fails and tells me to call "bst track junction.bst".16:47
tristanvalentind, tracking `my-junction.bst` should still work, though; it does not have any dependencies16:47
valentindtristan, sure, but then I need to disable full project loading.16:47
tristanThat should be able to change, currently (without the includes branch), that works - if not it's pretty alarming and broke recently16:47
valentindAnd the problem is to know when I do not need to fully load it.16:48
tristanvalentind, I thought that junctions are not allowed to have includes anyway ?16:48
valentindtristan, yes, but the project still needs to be fully loaded.16:48
valentindtristan, I could make the loading of the project lazy.16:48
tristanright, this dual pass thing annoys me a bit, it's certainly hard to reason about16:49
valentindtristan, let's say you call "bst track somejunction.bst someelement.bst".16:54
*** coldtom has quit IRC16:55
valentindWe need to first do tracking of somejunction.bst. In one pipeline.16:55
valentindAnd then later we can do tracking on someelement.bst.16:55
valentindBecause someelement.bst might be using somejunction.bst for including some file.16:56
tristanvalentind, in that case; it's better to force the user to track only the junctions independently: Drill it into their skulls that these are junction elements are very special and different17:02
tristanthan to automatically track, allowing the user to build anything that is not declared explicitly (like an arbitrary unspecified version of something)17:03
valentindtristan, could we make a special command for that? It would make it more obvious for the user. And for us, easier to know when to partially load the project.17:03
valentindbst junction-track junction.bst17:04
valentindor track-junction17:04
* tristan feels he has to redigest and properly understand the justifications behind the partial load again17:04
valentindtristan, I can explain it easily.17:04
tristanIt has to do with options right ?17:05
tristanwhat was it again ?17:05
valentindWe do not have any information on which order to load junctions.17:05
tristanthe "order to load junctions"17:05
*** jcampbell has quit IRC17:05
tristanSo what is that order exactly, and how is it significant ?17:05
valentindIf we have several junctions, and we load some includes from one, then load the second junction. Or we do the opposite, we get a different behavior.17:05
tristani.e. in normal cases, we recurse and load on demand17:05
tristanHmmmmmmmm17:06
tristanI'm still not catching on17:06
valentindLet's say in project.conf, you include a.bst:a.conf and b.bst:b.conf.17:06
tristanRight17:06
tristanfollowing17:07
valentindLet's say a.bst:a.conf modifies the behavior of b.bst.17:07
tristanhold on there17:07
valentindAnd b.bst:b.conf modifies the behavior of a.bst.17:07
tristanslow down17:07
tristan"modifies the behavior"17:07
valentindFor example options.17:07
tristanWhat is modifies the behavior ?17:07
valentindWait, I have maybe better. Let me think.17:07
valentindYEs.17:08
valentindYou can override source plugin configuration.17:08
valentindAnd junctions depends on source plugins.17:08
tristanAha!17:08
valentindsource override something.17:08
tristannot sure that is the final Aha!,... lets see where it goes...17:08
tristanSo a project.conf can globally modify sources or elements by their "kind", yes17:09
tristanWhere do things go wrong, we need a scenario17:09
*** jcampbell has joined #buildstream17:09
tristanproject.conf includes a.bst:a.conf... this causes a junction load of a.bst junction... a.conf from a.bst modifies source behaviors in the calling project.conf... invalidating how a.bst was originally loaded.17:10
tristanvalentind, basically ?17:11
valentindWell, this would be OK to just say we keep it loaded like that.17:12
valentindThe problem comes when you have several junctions that can influence each others.17:12
valentindThen you cannot really decide because you need ordering.17:12
tristanNot necessarily right ? I mean it's conceivable that a project.conf setting can influence the source you used to download the project overriding that setting17:13
tristanI dont follow now17:13
valentindSo basically at the end you want to say all junctions behaves like nothing was included.17:13
tristanNot anymore17:13
valentindOK, so in theory, it breaks like you say. Junctions need the full configuration. Full configuration needs the junctions.17:15
tristanOk so... first problem is, we cannot use project.conf overridden variables in junction declarations; which is pretty annoying because we quite recommend using variable expansion in junction declarations17:15
valentindtristan, for sure you cannot use it. And yes, it is annoying.17:16
tristanThere is something at least possibly circular about this to be sure17:16
tristanHowever, I dont feel like I've explored the problem enough17:16
tristanIntuition says that while circular chains can obviously form; the nature of a buildstream project is top-down dependency wise17:17
tristanSuch that, it might be that there is a way to crunch this top down one by one, so long as there isn't something circular found17:17
tristanbut then, not sure17:18
tristanthe loading is top down however include asks for the opposite17:18
tristanvalentind, it would seem to me that... lets imagine first that you *know* you have all the elements of all the projects on hand17:19
tristanvalentind, it would seem to me that we might want some kind of double phase where we first collect includes bottom up and then start processing the load top-down17:21
tristanof course that ignores the problem of junctions17:21
valentindBut you go from project.conf. You can only go top-down.17:22
*** tiago has quit IRC17:23
*** jcampbell has quit IRC17:25
*** tristan has quit IRC17:26
*** jcampbell has joined #buildstream17:26
valentindtristan, I am not sure I understood here what you mean by collecting includes bottom up.17:26
*** tristan has joined #buildstream17:35
tristanvalentind, oops... what was my last ?17:35
tristanah, lost all of it according to the log... one sec...17:36
tristan<tristan> I'm still pretty lost I think17:36
tristan<tristan> valentind, "The problem comes when you have several junctions that can influence each others."17:36
tristan<tristan> I think I have to understand still how that is possible17:36
tristan<tristan> junctions which can "influence each other"17:36
tristan<tristan> So, lets define what causes that to arise, and inspect whether that should be an error ?17:36
tristan<tristan> project A depends on projects B and C.... fine... the order in which files included from C and B are applied in project A is dictated by the order of the (@) include directives17:36
tristan<tristan> Then, project B depends also on project C... in *any* case project A cannot apply the fragment included from project B... until that fragment from project B has completely composited *it's* include from project C17:36
tristan<tristan> ...Along the way... we are loading junctions... and that somehow changes something... but I'm not entirely sure how17:36
tristan<tristan> Concretely, the only problem I'm really seeing, is that the definition of a junction may have changed as a result of it's being loaded17:36
tristan<tristan> valentind, and in *that* case, it is not enough to just say "junctions cannot use (@) directives"17:36
tristan<tristan> because variables can be used to declare the junction, and can also be overridden in the project.conf by the include file in the loaded junction17:36
*** jonathanmaw has quit IRC17:40
valentindtristan, Yes. the main problem is that a junction can change after a file from there has been loaded in the top level project.conf. But if you try to work around that, you will find that we cannot work around the problem when there are multiple junctions. The point with the multiple junction case is that it is not easy to try to work around it.17:47
valentindBut it is of course already a problem with the simple case.17:48
*** bochecha has quit IRC17:50
*** bochecha has joined #buildstream17:50
*** bochecha has quit IRC17:51
*** bochecha has joined #buildstream17:51
*** bochecha has joined #buildstream17:51
valentindtristan, you are correct in what you say.17:52
*** bochecha has quit IRC17:52
*** bochecha has joined #buildstream17:52
*** bochecha has quit IRC17:52
*** bochecha has joined #buildstream17:53
valentindtristan, so to solve the problem, I just said, let's interpret twice project.conf. Once without includes, once with includes. Then we give the one without includes to junctions.17:53
*** bochecha has quit IRC17:53
valentindAnd by without include, I actually fixed it now to strip all (@) directive completely for this pass.17:54
*** bochecha has joined #buildstream17:54
*** bochecha has quit IRC17:54
*** bochecha has joined #buildstream17:54
*** bochecha has quit IRC17:55
*** bochecha has joined #buildstream17:55
*** bochecha has joined #buildstream17:55
*** bochecha has quit IRC17:56
*** bochecha has joined #buildstream17:57
*** bochecha has joined #buildstream17:57
*** bochecha has joined #buildstream17:58
*** bochecha has quit IRC17:58
*** bochecha has joined #buildstream17:59
*** bochecha has quit IRC18:02
tristanvalentind, got stuck in an email window :-S18:02
tristanvalentind, not an easy problem :-S18:04
tristanthe problem is at least much simpler to understand now18:05
tristanvalentind, so what happens when you reload that same junction the second time ?18:05
tristanvalentind, when you find out that it's been modified by affected variables from the post-include phase, and you discover that you must re-load that junctioned project entirely a second time from scratch ?18:06
tristanhehe18:06
tristanI guess you don't do that but... there is still no answer18:06
valentindNo. What I have is Project has twice some of the configuration. And junctions only use configuration prior to includes.18:16
valentindtristan, so now on my branch, junctions will always behave the same, it will ignore includes in project.conf.18:16
tristanright, junctions wont take project.conf includes into account18:17
tristanthat sounds reasonable18:17
valentindSo it m18:17
valentindSo it makes it possible.18:17
valentindBut now I just have to deal with tracking junctions.18:17
valentindtristan, however, I could say that local includes are always included.18:18
tristanOk so, "junctions influencing other projects" and "order in which junctions are loaded"... are things which I can safely forget about right ?18:18
valentindtristan, I have not done that, but this behavior would not be problematic.18:18
valentindtristan, Yes. You can forget about. It is the same problem. It just shows you cannot hack around it.18:18
valentindAnd this situation works with my solution.18:19
tristanSo the only thing left is to raise a more decent error in the case that you are loading a project which depends on junctions but fails to specify a ref ? (the "Inconsistent junction source error" ?)18:20
tristanvalentind, I dont want to jump too quickly into making separate API surfaces in the CLI for junctions18:21
valentindtristan, I removed that.18:21
valentindI kept the "if"s. But I just check that kind is "junction".18:22
tristanif we must, let's discuss it on the ML, there are reasons that junctions are very much like elements, and reasons why users have to consider junctions differently18:22
valentindtristan, Ah you mean in the client. Yes. OK.18:23
valentindI can try to make the second pass load lazy. We can because there now Project.load_elements that could potentially load the configuration if needed.18:25
valentindI am not completely sure though. I will see where this goes.18:25
tristanright, I think this all started because you wanted to autotrack junction elements, and I want that to remain explicit18:26
tristanbut the whole walk through the park helped me understand the problem better :)18:27
valentindGood. I am happy that you understand what I have been struggling with.18:27
tristanhaha18:27
tristanYeah I still dont follow the junctions influence other junctions hehehe18:28
tristanbut that only happens when you tackle things a certain way18:28
*** Prince781 has quit IRC19:54
*** cs_shadow has quit IRC20:36
*** tristan has quit IRC21:13

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!