IRC logs for #buildstream for Monday, 2017-11-13

*** anjii has joined #buildstream00:58
*** anjii has quit IRC01:00
*** bochecha has joined #buildstream02:40
*** bochecha has quit IRC02:57
*** noah has quit IRC02:58
*** bochecha has joined #buildstream03:36
*** bochecha has quit IRC03:38
*** bochecha has joined #buildstream03:38
*** bochecha has quit IRC03:43
*** bochecha has joined #buildstream04:25
*** tristan has joined #buildstream06:57
*** WSalmon has joined #buildstream07:56
*** WSalmon has quit IRC07:58
*** adds68 has joined #buildstream08:35
*** jude has joined #buildstream09:08
gitlab-br-botbuildstream: issue #151 ("Link from https://people.gnome.org/~tvb/buildstreamdocs/ to https://buildstream.gitlab.io/buildstream/") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/15109:29
*** tiagogomes has joined #buildstream09:41
*** valentind has joined #buildstream09:42
*** jonathanmaw has joined #buildstream09:47
*** honza has joined #buildstream09:51
*** tiagogomes has quit IRC09:53
*** tiagogomes has joined #buildstream09:55
*** tpollard has joined #buildstream10:00
*** honza has quit IRC10:03
*** ssam2 has joined #buildstream10:08
gitlab-br-botbuildstream: issue #153 ("Unhandled exception in utils._force_rmtree()") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/15310:24
*** bochecha has quit IRC10:25
*** jude has quit IRC10:32
*** semanticdesign has joined #buildstream10:34
*** inigomartinez has joined #buildstream10:41
*** jude has joined #buildstream10:48
*** WSalmon has joined #buildstream11:15
*** valentind has quit IRC11:32
tristanjonathanmaw, what do you mean "args that switch between different behaviours in functions" ?11:33
jonathanmawtristan: functions like Element.dependencies, and ts "recurse" arg11:34
jonathanmaw*its11:34
*** bethw has joined #buildstream11:39
tristanjonathanmaw, I'm so confused, what does "switch between different behaviours" mean here11:42
tristanyou mean, the behavior changes depending on the argument value ?11:43
tristanrecurse is going to be a mandatory keyword argument right ?11:43
tristanjonathanmaw, anyway it's not immensely important, as I mentioned in the comment, *how* the function is invoked; but if we're making something explicitly an optional argument; it seems nice to leverage that and use it as such (especially if we're going and changing the callers anyway)11:45
tristanthe `default` parameter of Plugin.node_get_member() and Element.node_subst_member() is... very much like python's dict `.get()` method second argument11:46
tristani.e. instead of raising hell of the member doesnt exist, return this instead11:46
tristans/of/if11:46
jonathanmawokay. I'll make the changes to node_{get,subst}_member, and see if I can see any other cases where that logic requires me to make changes.11:47
tristanI think you got Plugin.node_get_member() correctly, but for some reason didnt do Element.node_subst_member() in the same way11:48
*** WSalmon_ has joined #buildstream11:58
*** WSalmon has quit IRC11:58
*** tristan has quit IRC12:02
*** bethw has quit IRC12:10
*** tristan has joined #buildstream12:20
tlaterIt looks like our CI currently downloads two different versions of the gnome-sdk12:31
tlaterThat might explain why it takes so long12:31
tlaterPerhaps we should run a `bst track` on the integration tests at some point?12:31
tristantlater, there was a change which happened for some reason12:34
tristantlater, and it does deeper than that12:34
tristanit installs the SDK twice (one runtime one sdk)12:34
tristanbecause we still didnt teach it how to properly do the symlink thin12:34
tristang12:34
tristananyway we have 2 different refs I think it's since the pip tests but dont recall, one of the tests needed a fresher version12:35
tlaterIs that deliberate? I feel saving some time on the CI might be worth the slight inconsistency between what we tested on two months ago and now.12:36
tristanwat ?!12:39
tristantlater, ah you mean separate versions ?12:39
tlatertristan: Yes12:39
tristantlater, no, the person (I dont recall) who updated the new ref for the new reason, was not as anal as I am12:39
tristanand did not go ahead and update the ref for all the other tests, unfortunately12:40
tristanI noticed this while moving integration-tests in from buildstream-tests repo12:40
tristanWhat should be done: A.) Use the new ref all around, B.) dont stage *both* gnome-platform and gnome-sdk, only the sdk with the proper symlinks trick12:40
tristanwhich can be found in gnome-modulesets-base repo, there is the correct way to do it12:41
tristanAlso, possibly updating the base causes binary output to be different, in which case, integration tests should be refactored as mentioned in a bug report I filed12:41
* tlater Checks the issues for now, and adds one if we don't have one yet :)12:41
tristanso that it doesnt test for bit for bit sameness causing tests upgrades to be a royal pain in the arse12:42
*** cs_shadow has joined #buildstream12:45
*** bethw has joined #buildstream13:01
*** WSalmon_ has quit IRC13:03
gitlab-br-botpush on buildstream@97-apply-pep-3102-to-all-public-api-surfaces (by Jonathan Maw): 18 commits (last: load.py: Migrate to new test style) https://gitlab.com/BuildStream/buildstream/commit/25f6bec66a742cf96e45ed4f2e16bbcde972174b13:07
gitlab-br-botbuildstream: merge request (97-apply-pep-3102-to-all-public-api-surfaces->master: WIP: Resolve "Apply pep 3102 to all public API surfaces") #127 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/12713:07
gitlab-br-botbuildstream: merge request (fixupimports->master: Refactor, remove some unused imports) #149 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/14913:15
gitlab-br-botbuildstream: merge request (fixup_internal->master: _variables: refactor, private 'subst_internal') #150 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/15013:28
*** tiagogomes has quit IRC13:31
*** tiagogomes has joined #buildstream13:37
*** adds68 has quit IRC13:48
*** adds68 has joined #buildstream13:48
*** WSalmon_ has joined #buildstream14:03
*** givascu has quit IRC14:16
gitlab-br-botbuildstream: merge request (fixup_internal->master: _variables: refactor, private 'subst_internal') #150 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/15015:15
*** bochecha has joined #buildstream15:18
ssam2I think 'elements' wasn't a great name for the package that bst-external.git installs15:19
ssam2as it will conflict with any other python project that installs a package named 'elements'15:19
ssam2such as, another buildstream plugins repo15:19
ssam2or https://pypi.python.org/pypi/Elements/0.2.015:20
tlaterOh, that's probably my fault, I just used 'elements' temporarily... It could technically be anything, it's not used in the actual code iirc.15:21
ssam2makes sense15:21
ssam2we should probably rename to bst_external15:21
tlaterHmm... I still can't reproduce what gitlab does here at all: https://gitlab.com/BuildStream/buildstream/-/jobs/4008392015:24
tlaterThe test just succeeds fine for me15:24
tlaterThat is, after replicating the exact environment with docker. There is no way outside of running gitlab-runner locally to get that container, is there?15:26
gitlab-br-botpush on buildstream@tracking-changes (by Tristan Maat): 14 commits (last: doc/source/docker.rst: Fix formatting) https://gitlab.com/BuildStream/buildstream/commit/5530bab052c25fe2c16f8f2116ca468b6a89517415:29
gitlab-br-botbuildstream: merge request (tracking-changes->master: WIP: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/11915:29
gitlab-br-botpush on buildstream@except_intersections (by Tristan Maat): 8 commits (last: doc/source/docker.rst: Fix formatting) https://gitlab.com/BuildStream/buildstream/commit/5530bab052c25fe2c16f8f2116ca468b6a89517415:29
gitlab-br-botpush on buildstream@131-behavior-of-except-argument-is-frustrating-and-confusing (by Tristan Maat): 8 commits (last: doc/source/docker.rst: Fix formatting) https://gitlab.com/BuildStream/buildstream/commit/5530bab052c25fe2c16f8f2116ca468b6a89517415:30
gitlab-br-botbuildstream: merge request (131-behavior-of-except-argument-is-frustrating-and-confusing->master: WIP: Resolve "Behavior of --except argument is frustrating and confusing") #147 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/14715:30
gitlab-br-botbuildstream: merge request (fixupimports->master: Refactor, remove some unused imports) #149 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/14915:38
gitlab-br-botpush on buildstream@master (by Tristan Van Berkom): 1 commit (last: Refactor, remove some unused imports) https://gitlab.com/BuildStream/buildstream/commit/cc6e09ce63c7576bd3f258515946e38a945903f415:38
gitlab-br-botbuildstream: Tristan Maat created branch 138-aborting-bst-push-command-causes-stack-trace15:53
*** jude has quit IRC16:17
*** jude_ has joined #buildstream16:17
*** jude_ has quit IRC16:24
tlaterDo we host a docker container set up specifically to be an artifact share yet? I think that could be handy.16:38
ssam2yes that would be handy16:42
ssam2i do have ansible scripts that set one up16:42
ssam2https://gitlab.com/baserock/infrastructure/tree/master/baserock_ostree16:42
ssam2(first run image-config, then instance-config)16:43
ssam2that assumes Fedora VM, and would need some modification for Docker, but it's a useful reference16:43
ssam2also such a thing could reuse the existing CI infrastructure in our docker-images repo16:44
* tlater is sort of building one for testing things right now, may set up a MR for that if I can work out how to make it configurable 16:47
tlaterMain issue is https16:47
tlaterI think it would be a poor idea to publish an image that makes it easy to serve with http16:47
ssam2tricky though, the best we could do is generate a self-signed certificate on startup16:50
ssam2getting a real cert requires a domain16:50
tlaterssam2: Yeah, that however makes the image only useful for testing.16:51
ssam2or usage inside trusted networks16:52
tlaterI have seen docker image setups that allow including certificates before.16:52
tlaterMaybe it's worth putting in the effort to see how that works16:52
tlaterThat said, I've figured out how to reproduce the segfault \o/ It doesn't look to be a segfault though, and is a bit harder to debug because it's in a container :/16:54
ssam2good that you can reproduce it16:57
tlaterGah, buildstream really hates being run inside a docker volume17:08
ssam2?17:09
tlater'WARNING Unable to create user namespaces with bubblewrap, resorting to fallback'17:10
ssam2is that with --privileged  ?17:12
tlaterssam2: Yup17:12
ssam2hmm, maybe there's another thing needed17:12
tlaterOh, nevermind, looks like I forgot that17:12
tlater-.-17:12
ssam2see https://gitlab.com/BuildStream/buildstream/blob/master/contrib/bst-here17:12
ssam2we also pass `--security-opt seccomp=undefined`17:12
*** xjuan has joined #buildstream17:16
*** valentind has joined #buildstream17:23
*** csoriano has joined #buildstream17:29
ssam2hi csoriano  :-)17:29
csorianohello, I was trying buildstream for building gtk+-317:29
csorianonow I did that because I'm developing a patch for gtk+3 that includes a new library17:30
csorianoI need two things: one is to include this library wherever gtk+ is built17:30
csorianotwo, pass a parameter to meson configure17:30
csorianoI got stuck in the former, I didn't know where to put this library, since all is opaque17:30
csorianossam2: hi :)17:31
tristancsoriano, I need to understand "include this library wherever gtk+ is built"17:34
tristanis "depend on it" the same thing ?17:35
csorianotristan: yes, it's an optional dependency17:35
csorianoyeah I used a bad wording :/17:35
tristanokay, so the current GNOME modulesets is currently driven by jhbuild conversions, and builds against a debootstraped base runtime17:36
tristanThe debootstrap base runtime will probably go away after freedesktop-sdk project materializes something we can use for both flatpak runtimes & GNOME system builds17:37
tristanso there is a bit of flux17:37
tristancsoriano, would you consider this optional dependency a "sysdep" basically ?17:37
tristancsoriano, sysdeps currently is just a list of packages coming from debian testing: https://gitlab.com/BuildStream/debootstrap-ostree/blob/master/data/multistrap.conf.in17:38
csorianotristan: no, it probably needs to be like glib17:38
tristana patch to that file, will make the dependency available17:38
tristanokay, so something you need to add to like, core-deps basically ?17:38
csorianotristan: I think so, yeah17:39
tristancsoriano, until we make a hard switch; adding it to core-deps in the 3.28 modulesets will cause it to automatically get converted and magically appear17:39
csorianotristan: it's not in Jhbuild yet. Can I use buildstream for that? And in an ideal world, how would it be the workflow for this?17:40
tristanI have been making changes to the jhbuild modulesets so that they effect the buildstream based builds17:40
tristanSo in a world after doing the hard switch and not converting the modulesets anymore, you would add a new element in elements/core-deps...17:41
* tristan gets url17:41
tristancsoriano, http://gnome7.codethink.co.uk/cgit/gnome-modulesets/tree/17:41
csorianotristan: that means I would need to build Buildstream? No right?17:41
tristane.g. here: http://gnome7.codethink.co.uk/cgit/gnome-modulesets/tree/elements/core-deps/glib.bst17:41
tristanBuildStream is python and doesnt really need to be "built"17:42
csorianotristan: right17:42
csorianotristan: reading that, I guess there is some "kind: local" then?17:42
tristanso BuildStream is the only thing you really need on your "host"17:42
csorianobecause I'm modifying the library and gtk+ at the same time17:42
tristannow I'm not following....17:42
tristanah17:43
tristanok so, no17:43
csorianotristan: I'm creating a library that gtk+ depends on17:43
csorianoimagine gtksourceview17:43
tristancsoriano, for that you want a workspace17:43
csorianothat's probably a better example17:43
tristanhttps://wiki.gnome.org/Newcomers/BuildSystemComponentBst#Developing17:43
csorianoyeah I did that for gtk+17:43
csorianonow how do I for putting my library inside this?17:43
tristancsoriano, so is there any upstream git yet ? it would help if it existed like, on github or anywhere17:44
csorianoyeah, but I need to test locally17:44
csorianolike what would you do with gtksourceview when testing from gtk+ right?17:44
tristanSo, create an element in core-deps/myfancylib.bst17:44
tristanwhich points to your upstream, and create a workspace for it17:45
tristanonce you do `bst track core-deps/myfancylib.bst`, then you can create a workspace for it17:45
csorianoand the changes in that workspace will be reflected in the gtk+ workspace?17:45
tristanIf I was hacking on gtksourceview and gtk+, I'd open two workspaces17:46
tristancsoriano, opening a workspace will cause the workspace sources to be used in builds of that project, as long as the workspace is open17:46
tristancsoriano, so yes it will17:46
tristancsoriano, i.e. lets pretend you are only working on GTK+17:47
tristanand you want to test epiphany17:47
tristanso you built everything once, you have your webkit and everything17:47
tristanyou are setup with --no-strict17:47
tristan(there is a user config for that on the wiki page)17:47
*** WSalmon_ has quit IRC17:48
tristancsoriano, then, the changes you make to GTK+, after a build... will be effective in the sandbox of epiphany17:48
tristansame sort of thing17:48
* tristan not sure if he explained that very clearly17:49
tristancsoriano, maybe if I just explain the meaning if "strict", i.e. the config option I recommend for hacking, here: https://wiki.gnome.org/Newcomers/BuildSystemComponentBst#Configuring_BuildStream17:49
csorianotristan: yeah I'm using no-strict, I'm just following that guide17:49
csoriano(btw, very bad there is no instructions for Fedora, cmon :P)17:50
tristanSo "strict" means, rebuild reverse dependencies whenever something changes17:50
csorianoI think you explained well, let me chew...17:50
tristancsoriano, yeah you mean in BuildStream itself ? I just need a package list...17:50
tristanand I can add a section to the docs17:50
tristanI think jjardon[m] is the guy who uses arch, and he sent us a patch for the docs for arch (iirc)17:51
csorianotristan: ok, so I have two different workspaces, whatever I build in there they are reflected in both17:52
csorianoso it's kinda like jhbuild17:52
csorianothey share the same "prefix"17:52
csorianoright?17:52
tristanThere is no "prefix" in that sense17:52
tristanevery time a build or a shell occurs, the output of every element is staged on demand using hardlinks17:53
csorianoyeah I imagined, that's what makes me a little confused17:53
tristanso prefix is actually /usr, in a chroot-like (bubblewrap actually) sandbox17:53
*** ssam2 has quit IRC17:54
tristancsoriano, so basically you need to *also* make the GTK+ depend on your new element17:54
tristanwhen you build GTK+, or anything that depends on GTK+, your new element's output will also be "staged" into the sandbox17:54
*** bochecha has quit IRC17:54
tristan(likewise, if you `bst shell` on any element that depends on GTK+, same thing)17:55
csorianoaha17:55
csorianoso.. when I open a shell, it will realize there are changes (even if no commited) and will pu that in a ostree fashion to be part of the build of gtk+, kinda right?17:56
tristanhehe kinda :)17:57
csorianoso if I don't want gtk+ to take into account my changes in this other library, I just need to close the shell/workspace of rhtat library right?17:57
tristancsoriano, anyway if you try to open a shell, it will complain if it's not already built17:57
tristanexactly yes17:57
csorianoperfect, I got it17:57
tristanyou can close the workspace and then the original upstream will be used instead17:57
csorianotristan: I don't we can expect much to understand the underlaying. Would be good to have these examples in a really easy way17:58
csorianolike how you said them17:58
tristanmyeah... docs are tricky things... I am quite verbose in general, but trying to keep things bite sized17:59
csorianoand kudos for the work, it's definitely a big improvement versus jhbuild. I was happy to have latest gtk+ so fast17:59
tristanotherwise you look at the wiki page and it's huge :-S17:59
csorianotristan: imho, you did really well in /Newcomers wiki18:00
csorianoso keep that spirit I guess :D18:00
tristanthanks :) ... we do have an ongoing project for documenting things in a howto guide kind of form18:00
tristanasides from the reference, which, I think the reference is really handy actually, but it's a bit terse18:01
tristancsoriano, especially this part can be handy once you get the feel for the basics: http://buildstream.gitlab.io/buildstream/pluginindex.html#plugins18:01
tristanshows how each plugin can be used whenever you need to know18:02
tristan(each element or source)18:02
*** bethw has quit IRC18:03
csorianoI see18:03
tristanAnyway I think jonathanmaw got started on that writing of a guide, but it was interrupted by blockers :-/18:04
csorianojust some feedback for the wiki in Newcomers, I didn't get much the difference between shell and track , etc. At some point it says "now downloads the sources" but later on also downloads the sources since I can see in the output a regular git clone18:04
tristanI'll see if we can try to prioritize that again :)18:04
csorianomaybe it's a bug rather than bad docs though18:05
csorianoI think providing the basics for the GNOME devs it's enough18:05
tristancsoriano, that is intentional but I admit a little confusing18:05
csorianofor that we don't really need to know how it works or the plugins and what does it support18:05
csorianobut just like "do this to build two sources" and that's it18:06
csorianoah ok18:06
tristanthe point of `bst track` is to find the latest reference for a given branch, and the point of `bst fetch` (automatically part of `bst build`) is to actually obtain the sources18:06
tristanCurrently though, most implementations do `fetch` as a side effect of `track`18:06
tristanWe hope to improve that so that for instance, you dont need to checkout *all* the linux kernel history, if you're only interested in building a given stable branch18:07
csorianotristan: then the docs saying "track downloads the sources" is wrong18:07
csorianoin the wiki18:07
csorianoI wouldn't worry much about it, seems it's fast and not requiring lot of space so far, compared to jhbuild18:07
tristanIndeed, however the effect is that, ok I'll try to clarify that18:07
*** mcatanzaro has joined #buildstream18:08
tristancsoriano, I was writing that with a perspective that, people will use this recommended workflow, and then learn the tooling later...18:08
tristancsoriano, however indeed, it's misleading in the sense that; people will take home the wrong thing from that sentence18:08
tristancsoriano, I'm surprised you say that it's not requiring a lot of space... technically; it should require *more* disk space haha18:09
tristanbecause no host tools18:09
tristanbut if it's going well, I'm happy for that :D18:09
csorianotristan: ah yeah. Yeah it's just a detail, it was just slightly confusing seeing things happenings multiple times (or that was the impression)18:10
csorianoidk, 4'1 GB here18:11
csorianofor jhbuild build gtk+ was definitely more18:11
tristanI'm trying to get at least the base always built on the artifact server, so that you dont need to have the base system *twice* on a developers machine, and then it only changes when sysdeps change18:11
*** jonathanmaw has quit IRC18:11
csorianoyeah, that's really helpfukl18:11
tristanyeah GTK+ is pretty low in the stack, when you run `bst build core/meta-gnome-core.bst apps/meta-gnome-apps-tested.bst` ... it takes a bunch more space18:12
* tristan guesses sources cost more than build outputs in general, too18:13
tristanor "artifacts" as we like to call them18:13
*** givascu has joined #buildstream18:14
csorianoyeah18:17
*** valentind has quit IRC18:43
*** valentind has joined #buildstream18:45
*** bethw has joined #buildstream18:46
*** semanticdesign has quit IRC18:54
*** semanticdesign has joined #buildstream18:54
*** semanticdesign has quit IRC19:15
*** semanticdesign has joined #buildstream19:15
*** bethw has quit IRC19:23
*** bethw has joined #buildstream19:53
*** givascu has quit IRC20:27
*** tristan has quit IRC20:28
*** semanticdesign has quit IRC20:46
*** semanticdesign has joined #buildstream20:48
*** semanticdesign has quit IRC20:56
*** semanticdesign has joined #buildstream20:57
*** semanticdesign has quit IRC21:02
*** tristan has joined #buildstream21:02
*** jude has joined #buildstream21:15
*** jude has quit IRC21:27
*** noah has joined #buildstream21:38
*** noah has quit IRC21:48
*** noah has joined #buildstream21:54
*** noah has quit IRC22:22
*** xjuan has quit IRC22:33
*** tristan has quit IRC22:56
*** bethw has quit IRC23:18
*** valentind has quit IRC23:21

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!