IRC logs for #buildstream for Monday, 2018-01-08

*** saxa has joined #buildstream00:16
*** tristan has quit IRC04:10
*** tristan has joined #buildstream04:38
ltuis there a difference between 'creating a new project' and 'authoring a project'?05:01
tristanSlight one I would say there is05:14
tristancreating a new project is a one time thing, while I would say authoring a project may imply maintenance, and anything to do with editing bst files and project.conf05:15
tristanI'm admittedly guessing into the context and thinking the motivation hiding behind the question has to do with documentation05:16
ltucorrect05:16
ltuok that makes sense then, thanks05:17
ltutristan, I'm making a list of what's done and what is left to do, to see where any remaining gaps will be after the current branch from nexus is merged05:24
ltuwe can review it later this week :)05:24
gitlab-br-botbuildstream: issue #183 ("Plugin loading documentation is confusing") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/18307:07
tristanjuergbi, I havent *really* got deep into reviewing junctions yet07:50
tristanjuergbi, as it's been in the back of my mind since last week, I just wanted to clarify that the options thing is not really a blocker to me, it's really just the first thing that jumped out at me while looking at the changes07:51
juergbiok. as i mentioned, if it wasn't for flags, i would have implemented it differently07:52
juergbii hope i can update the branch today with regards to the multi-cache changes07:53
tristanYeah I went through a review of that today...07:54
tristanI hope to land it asap too07:54
juergbigreat07:55
tristanfor multi-cache, this comment is the tricky part for me: https://gitlab.com/BuildStream/buildstream/merge_requests/166#note_5363856307:57
tristanor two comments rather07:57
tristan"When an artifact is built locally, BuildStream will try to push it to the first writable cache in the list"07:58
tristanmaybe if we only want to push to one artifact server, the configuration should identify which one(s) to push to08:00
*** noisecell has joined #buildstream08:30
*** jude has joined #buildstream08:45
*** valentind has joined #buildstream09:18
*** harshitaneja has joined #buildstream09:43
*** jonathanmaw has joined #buildstream09:48
*** ernestask[m] has joined #buildstream09:49
*** ssam2 has joined #buildstream10:04
*** tiago has joined #buildstream10:08
*** ernestask has joined #buildstream10:10
*** valentind has quit IRC10:49
*** jude has quit IRC11:31
*** jude has joined #buildstream11:59
*** mcatanzaro has joined #buildstream12:40
*** harshitaneja has quit IRC12:49
gitlab-br-botbuildstream: issue #183 ("Plugin loading documentation is confusing") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/18313:02
tristanIn the hope of expediting these patch reviews... any thoughts on https://gitlab.com/BuildStream/buildstream/merge_requests/181#note_53643337 ?13:14
tristanI feel like whitelisting overlaps in the sense that "I am an element but I dont care if some of what I install is later overwritten" is conceptually backwards13:15
tristanbut lets think about it; first, it would seem that you might have to write less YAML to whitelist things that way13:15
tristanBut then, maybe more than one element overwrites this file, that should be a separate warning anyway right ?13:16
tristanYeah - maybe my talking to myself here is enough... I think that's correct13:16
tristanLater it might be interesting to some to have a project wide whitelist too, but that could be a separate feature that maybe noone will ever ask for13:17
jonathanmawHi tristan, I'm writing a response in the merge request right now13:19
jonathanmawonce I've gotten my head around some of the use-cases of overlaps, and how that affects whether we'd prefer to have configuration as "this can overlap" or "this can be overlapped", I'll send it.13:20
tristanJust added my comment to the bug report13:22
tristanjonathanmaw, it's pretty clear I think; the current approach is lossy and error prone13:23
tristanSee the example I made in the last comment13:23
* jonathanmaw refreshes13:23
tristanIntuitively it felt wrong - because an element by itself can be stand alone, intuitively I think it should not have knowledge of it's reverse dependencies; that lead me to the line of thinking13:24
tristanbut looking further it's clear that it's a different warning; i.e. there is a warning for every artifact which overwrites a given file that already exists, those are separate warnings13:25
tristanwe should not be allowed to silence them all at once13:26
tristanotherwise undesired overlaps stemming from newly added elements later fall through the cracks13:26
*** tristan has quit IRC13:38
*** tristan has joined #buildstream13:41
jonathanmawtristan: okay, seems sensible to do it the other way. I've got some questions about your other questions, though13:44
jonathanmawthat I asked in the merge request.13:44
ssam2i fixed a pet hate of mine13:45
ssam2https://buildstream.gitlab.io/ now redirects to the docs13:45
ssam2also, https://buildstream.gitlab.io/ exists with a really minimal website setup that does the redirect13:45
ssam2bah, the second link should be https://gitlab.com/BuildStream/buildstream.gitlab.io13:45
*** xjuan has joined #buildstream13:46
tlaterAh, ta ssam2, that also annoyed the hell out of me13:47
*** xjuan has quit IRC15:02
*** xjuan has joined #buildstream15:02
*** xjuan has quit IRC15:05
*** xjuan has joined #buildstream15:05
gitlab-br-botbuildstream: merge request (sam/meson->master: WIP: use Meson instead of setuptools to install and distribute BuildStream) #196 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/19615:53
*** noisecell has quit IRC15:53
mcatanzaroHi, anyone (jjardon[m]) around to help a fool try to learn how to use BuildStream? I'm trying to open a workspace to hack on epiphany but getting a weird error: git source at core/epiphany.bst [line 3 column 2]: expected ref 'None' was not found in git repository: 'https://git.gnome.org/browse/epiphany'15:57
mcatanzaroFull output: https://paste.gnome.org/p8qlxbodd/u3h32j/raw15:57
mcatanzaroI'll ping tristan on the off chance he's not asleep in Asia right now....15:59
ssam2mcatanzaro, could you paste the core/epiphany.bst file ?16:00
ssam2or is it unchanged from a commit of gnome-modulesets.git ?16:00
mcatanzarossam2: It's unchanged... actually maybe that's the problem, I probably need to track it first16:00
ssam2ah yes16:01
ssam2bit of a rubbish error message though16:01
mcatanzaroYes, error message needs to be better16:01
ssam2do you have time to file an issue ?16:01
mcatanzaroSure can16:01
mcatanzaroAlso, next question: are you Sam Thursfield? Or different Sam?16:02
ssam2i am that sam :-)16:02
mcatanzaroAlso, next question: where's the bugtracker?16:02
ssam2we're on gitlab.com16:02
ssam2so https://gitlab.com/BuildStream/buildstream/issues16:02
mcatanzaroOK16:02
ssam2once upon a time i used an irc client that would set realname, but no longer apparently16:03
mcatanzaroAlso, next question: is modifying my local checkout of the repos, and only executing buildstream from within that checkout, really the way that development with buildstream is supposed to work? It's not very convenient relative to jhbuild.16:03
ssam2I don't quite follow what you mean16:03
ssam2i think that is how things are supposed to work, but i'm interested what the issues are vs. jhbuild16:04
mcatanzaroTo use buildstream, I have to cd into its git repo, right?16:04
ssam2you have to run it in the git repo of the *project*16:04
ssam2you can also do `bst -C $projectdir` to run it from somewhere else16:04
mcatanzaroAh yes, well that would be gnome-modulesets... so I 'cd ~/Projects/gnome-modulesets' first before doing anything else. I guess that's OK.16:05
mcatanzaroI can get used to that... it's a step that I don't have to do in jhbuild, though. Maybe it'd be nice to be able to configure a default project?16:05
ssam2that might be nice16:05
ssam2although you could equally `alias bstgnome=`bst -C /path/to/gnome-modulesets.git`16:06
mcatanzaroAnyway, that workflow seems to interact badly with 'bst track' which modifies the local projects16:06
ssam2(except with correct shell syntax :-)16:06
ssam2how so ?16:06
mcatanzaroI guess I'm just not super happy about how 'bst track' modifies my local projects16:06
ssam2there is a way to avoid that16:06
ssam2i can't remember the exact arguments, it changed at some point16:07
mcatanzaroSay I want to pull the new version of the modulesets... I can't 'git pull' because I have to 'git reset --hard @' first and that undoes all the work of 'bst track'16:07
ssam2`bst build --track-all`16:07
ssam2says "run a build, track everything, don't save the tracked refs`16:07
ssam2where `bst build --track-all --track-save` does the same effectively as `bst track --all; bst build`16:07
ssam2or, would it help to use `git stash` before pulling ?16:08
ssam2i've not used this feature in anger yet ... personally i like having the refs written to disk, so that i don't unexpectedly get new changes16:08
mcatanzaroMaking modifications to a git repository just seems ugly... I would have saved the modifications somewhere else, maybe under ~/.local/share/buildstream16:08
mcatanzaroI do have to regularly pull the new modulesets manually, right? E.g. if someone adds a new dependency to some module, buildstream is not going to magically know about that until I 'git stash' and then 'git pull' (and 'git stash pop')16:10
ssam2that would be quite a drastic change conceptually; not that it couldn't happen, but the best person to discuss with is Tristan16:10
ssam2and yeah, you do have to pull new modulesets16:10
ssam2the self-modifying is definitely on an ugly/convenient spectrum16:11
ssam2some projects like to have exact refs stored in Git16:11
ssam2for that use case, having something that updates the actual files with new refs is super useful16:11
ssam2but we are trying to use the same mechanism to emulate jhbuild's "always build master" functionality, which is not quite equivalent16:12
ssam2(personally i quite like having refs committed to Git ... fewer surpises that way ... but it's not how jhbuild has ever worked)16:12
mcatanzaroI think it's too hard: we don't want to have to teach users about git stash in order to get the new GNOME projects. Honestly, ideally it would pull the projects from the *server* and not look locally at all, so that they are always up-to-date. Like JHBuild. Only experienced developers need to modify the projects usually.16:14
ssam2we're going to recommend `bst build --track-all` as the default16:14
mcatanzaroOK, can you update https://wiki.gnome.org/action/show/Newcomers/BuildSystemComponent with that information? Right now the instructions there are different.16:15
mcatanzaroMaybe if --track-all is going to be recommended, that should be the default behavior...?16:15
ssam2other projects won't want that16:16
ssam2but perhaps it could be enabled in the project config16:16
ssam2i didn't write that doc, i'd rather leave updating it to jjardon or tristan... but will make a note to follow up tomorrow16:16
* tristan appears and reads backlog...16:18
mcatanzaroThanks! (and hi tristan!)16:18
mcatanzaroI will have more questions soon. I tried using it to develop libsoup tests yesterday, but couldn't figure out how to run 'make check' or get into the build directory at all. I was planning to review the buildstream documentation before asking for help with that, but, eh, I'm here now...!16:19
tristanYeah it's certainly important to be able to store the very precise and exact ref of every source in a project16:21
tristanthat's how you can tag a release and be absolutely sure that your project state reflects something that is exactly reproducible elsewhere16:21
tristanthat said, we offer (build)+(track)-(save) as a solution for those who just want to build something on a project that has no refs stored yet, and get "whatever is latest" every time16:22
tristan(at the expense of course, of querying every single repo in the build plan over the network for it's latest version)16:23
tristanmcatanzaro, what it looks like to me, is that opening a workspace could do with some automatic tracking functionality built in16:24
tristanif I understand from a quick read of the backlog, this seems to be what caused the frustration16:24
mcatanzarotristan: I was going to report a bug to improve the error message, like ssam2 suggested, but yeah, automatic tracking might work too16:25
tristanmcatanzaro, regarding the wiki fwiw; the --track-all stuff was one of the final things I wanted to land before 1.0 which we rolled out last week16:26
tristanthat's why it's not on the wiki yet, indeed16:26
* tristan puts that in his immediate todo list for tomorrow16:26
mcatanzarotristan: FYI: I moved the wiki page to replace the original jhbuild page. I added a redirect, should be seamless.16:28
tristanNice16:28
mcatanzaro"I was going to report a bug to improve the error message, like ssam2 suggested, but yeah, automatic tracking might work too" --> but I discovered I need to register an account on GitLab. And the username "mcatanzaro" is already taken. So I need to think long and hard before hastily selecting a username.16:29
tristanOh, and that last one... you probably want `bst shell --build <element>`16:29
mcatanzaroSo maybe one of you can report the bug instead, sorry. :P16:29
mcatanzaroI'll try 'bst shell --build', thanks16:30
jjardon[m]Hi! seems buildstream needs the fuse module: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/47092921 It seems this is not documented? Should I open an issue?16:30
tristanThere are 2 kind of shells, the default expects the element to be built, and stages it's runtime dependencies16:30
tristanthe build shell stages the elements build dependencies and the sources and drops you into a shell where you can run the build yourself16:31
tristanthats meant for debugging the build stage16:31
tristanjjardon[m], dont open an issue no16:31
tristanjjardon[m], I believe the issue you are looking for, sounds like: https://gitlab.com/BuildStream/buildstream/issues/11616:32
jjardon[m]ah ok, thanks tristan !16:32
tristanjjardon[m], the fuse module, well we used to use fuse.py and then later merged a copy of it, because bugs... in both cases we fail to depend directly on fusermount16:33
tristanbecause distros can package it separately from libfuse16:33
ssam2looking at the log jjardon posted16:33
ssam2the actual issue seems to be that the kernel on the machine running the build had fuse support as a module, which wasn't loaded16:33
tristanbut it looks like now since we dont depend on fuse.py, we additionally need to depend directly on libfuse16:33
ssam2and because the build was running inside a container, the module couldn't be loaded with `modprobe fuse`16:33
ssam2or maybe that's a side effect of needing fusermount16:33
tristanAh, interesting, looks a bit more elaborate ;-)16:34
ssam2all we can really do is document that your kernel needs FUSE support16:34
tristanssam2, perhaps at that point, once we've done all the requiring we can do; we can say hey; your host is a bit messed up, are you in a vm or smth ?16:34
tristanyeah, documenting it will do :)16:34
*** adds68_ has quit IRC16:35
jjardon[m]tristan: ok, so I was already installing fuse in that machine, which includes fusermount, so I guess this a separate "issue"?16:37
* tristan added some notes to https://gitlab.com/BuildStream/buildstream/issues/11616:38
ssam2jjardon[m], the issue is that containers can't cause modules in the host to be loaded16:38
ssam2jjardon[m], running `modprobe fuse` on the host should fix it16:38
tristanjjardon[m], meh, so very closely related I'd just treat it as the same issue, it's all about documenting the fuse needs16:38
jjardon[m]tristan: rigth16:39
jjardon[m]ssam2: tristan thanks for the comments!16:39
mcatanzarotristan: What's the recommended workflow for running tests ('jhbuild buildone -n --check')? Is it to use 'bst shell --build' and run them manually? That's fine I guess.16:43
tristanmcatanzaro, for now yes, and very soon the UX for that should be greatly improved/optimized for workspaced builds16:48
tristanmcatanzaro, https://gitlab.com/BuildStream/buildstream/merge_requests/126 <-- there was already a lot of work done there, we're waiting for juergbi to land a little internal state handling refactor before massaging that branch in16:49
tristanSo what should happen for a workspace build, is once you've built it once, you can build it again16:49
tristanand keep the .o files and everything littered around in the workspace16:49
tristanBut, there is also another alternative, which we didnt do "just just yet", but is probably worth filing a bug for16:50
tristanWhich is to include `test-commands` in the build elements to be run post build; we would probably want a variable to turn it on and off16:50
tristanThere is some conflicting ideas on how to do that though; i.e. from a build optimization point of view, I'd really like to have separate elements running `make check`, and unblock later builds before running tests16:51
tristanThat approach, would require https://gitlab.com/BuildStream/buildstream/issues/2116:53
gitlab-br-botbuildstream: merge request (jjardon/fuse->master: doc/source/install.rst: BuildStream depends on 'fuse' (for fusermount) and libfuse) #214 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/21416:53
tristan(I.e. theres no reason why I have to wait for glib tests to complete in order to build it's reverse deps; I can run them in parallel and fail the build as a whole after successfully building some reverse deps)16:54
mcatanzaroOK16:55
mcatanzarotristan: Last question: what's your current timezone? Either you are up late, or you must not be in Korea?16:55
tristanmcatanzaro, it's about 2am16:56
tristanI should be ignoring IRC, but ya know... it gets my attention and cant escape it ;-)16:56
mcatanzaroGoodness. ZzzzzZZZZzzz. Well now I know you're a night owl, I won't feel bad about asking questions this late. :P16:56
*** xjuan has quit IRC17:02
*** ssam2 has quit IRC17:04
*** xjuan has joined #buildstream17:15
*** tristan has quit IRC17:16
kailueke[m]Who can push to the artifact cache? Still the case that only one machine is doing it or do individual people also have the possibility to upload their stuff?17:17
*** tristan has joined #buildstream17:17
*** xjuan has quit IRC17:18
persiakailueke[m]: I'm not authoritative, but I believe that A) there is restricted access to "the" artifact cache, being the default one, and that anyone can publish "an" artifact cache, and use that to upload their stuff.17:44
persiaErr, s/and that/and (B) that/17:44
kailueke[m]persia: yes, just thought it would be nice to have one of e.g. those who have commit access anyway17:48
persiaI don't disagree: I just don't think it exists today :)17:51
kailueke[m]the cache already saved me some time, but even with just one project using Rust it is already a bit long to wait17:54
kailueke[m]but congrats to the 1.0 release, all worked fine so far17:56
*** ernestask has quit IRC17:58
*** jonathanmaw has quit IRC17:59
mcatanzaro"ERROR   Inconsistent pipeline18:01
mcatanzaro    Exact versions are missing for the following elements    Try tracking these elements first with `bst track`"18:01
mcatanzaro'bst track' doesn't help18:01
mcatanzarohttps://paste.gnome.org/pcw0qaqnf/ju8dbf/raw <-- what to do?18:01
mcatanzaroI tried 'bst track core/epiphany.bst' and it ran fine, but didn't change anything when I tried to build18:02
mcatanzaroI do have a workspace checked out for epiphany, maybe that's causing problems?18:02
*** xjuan has joined #buildstream18:05
nexusSo. I'm trying to update the meta data that is stored in artifacts, and i've been able to add the new information to the code, however, when i rebuild/install BuildStream and then build an element, the meta data isn't there, and the sha for the artifact isn't changing, even if i delete it and recreate it18:22
nexusany thoughts?18:22
*** ahayzen has left #buildstream18:33
gitlab-br-botbuildstream: merge request (juerg/element-state->master: WIP: element state updates) #215 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/21518:42
juergbinexus: if you want to force rebuild of artifacts, you have to increment BST_ARTIFACT_VERSION18:43
*** valentind has joined #buildstream18:43
*** valentind has quit IRC18:46
*** paulsherwood has quit IRC19:33
*** nexus has quit IRC19:33
*** benbrown has quit IRC19:34
*** tlater has quit IRC19:34
*** ltu has quit IRC19:34
tristanmcatanzaro, sorry missed that, and dozing off soon... `bst track` was at one time recursive by default, but proved unflexible; instead it will only track one element unless the --deps arg is given19:51
tristanone can combine `--deps all` with `--except <element>` as well, short hand to track a lot of elements while avoiding say, updating the base system and rebuilding the whole world19:52
tristanbst track --help should be helpful, too19:52
*** tlater has joined #buildstream19:59
*** laurenceurhegyi has joined #buildstream20:00
m_22[m]is there anything similar to jhbuild uninstall in buildstream? i built a module i don't need and ended up running out of disk space so i need to uninstall it20:13
m_22[m]also, are there any known issues with puseaudio working in the bst shell?20:14
persiapulse (or any component that needs to talk to the kernel) isn't going to work very well in a sandbox,.20:15
m_22[m]hmm, then i guess i should go back to using jhbuild for now20:15
persiaThere are documented tricks to get things like pulse working in a chroot, but I believe bubblewrap protects against most of those working.20:16
persiaOr maybe someone with more experience will be able to answer your question better.20:18
*** juergbi has quit IRC20:36
*** juergbi has joined #buildstream20:38
jjardon[m]Hi! any idea why the integration commands are running before the build commands here? https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/47128247 Maybe Im missing something?21:00
jjardon[m](for the llvm5 element, I meant)21:00
juergbijjardon[m]: the integration commands for the build dependencies have to be executed before the build, otherwise the build dependencies might not work21:01
jjardon[m]juergbi: ah rigth. Sorry IIRC in another tool (ybd/baserock) they are always in the end thats why the confusion21:03
*** noisecell has joined #buildstream21:43
*** noisecell has quit IRC21:47
*** luc14n0 has quit IRC22:28
*** luc14n0 has joined #buildstream22:28
luc14n0Hey gents! I'm making an openSUSE Tumbleweed BuildStream package, and I need to figure out what are the runtime deps. I can already say that python 3, ruamel.yaml, ostree gi bindings are needed to run, but I suspect that bubblewrap and pygobject gi bindings are possible runtime candidates as well. If someone could shed a light here for me.23:50
*** xjuan has quit IRC23:50

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!