IRC logs for #buildstream for Tuesday, 2019-07-16

*** tristan has quit IRC03:51
*** tristan has joined #buildstream03:56
gitlab-br-botmarge-bot123 closed issue #1033 (Allow customisation of max-jobs instead hardcore to a maximum of 8) on buildstream https://gitlab.com/BuildStream/buildstream/issues/103304:06
gitlab-br-botmarge-bot123 merged MR !1473 (tristan/config-max-jobs->master: Add max-jobs configuration and command line option) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147304:06
gitlab-br-botmarge-bot123 merged MR !1474 (tristan/bst-1/config-max-jobs->bst-1: Add max-jobs configuration and command line option (bst 1)) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147404:50
juergbibenschubert: strip-args was removed from default project.conf a while ago. freedesktop-sdk should already have that fixed, though, at least in the branch that's targeting bst master04:52
*** ChanServ sets mode: +o tristan05:04
tristanI am a bit clueless at how this is intended to work, and so I probably messed this up in some way: https://gitlab.com/BuildStream/buildstream-docker-images/merge_requests/11805:04
tristanAny help ?05:04
tristanmaybe jjardon or cs-shadow ?05:04
tristanAhhh, the .gitlab-ci.yml is *also* generated05:06
tristanjuergbi, I think I'm going to re-tag 1.3.0 and roll a dev release of it off of the new bst-1 branch, leading up to 1.405:22
juergbihm, retaging sounds like a very bad idea05:24
tristanI know05:24
juergbidid we announce the 1.3.0 snapshot?05:24
tristanBut in this case, we only ever had the 1.3.0 tag for the purpose of making the badges on the websites work05:24
tristanWe never did no05:25
juergbiah ok, but then let's just tag it as 1.3.105:25
juergbiand ignore 1.3.0 existed05:25
tristanWe just abandoned stability and moved on05:25
tristanjuergbi, I'm alright with that too yeah05:25
tristanIn other news, I'm baffled as to how https://gitlab.com/BuildStream/buildstream/blob/bst-1/buildstream/_ostree.py#L274 has ever worked in the past :-S05:26
tristanI'm looking at https://github.com/ostreedev/ostree/blob/master/src/libostree/ostree-repo.c#L2047, which has seen some docs annotation changes in the last months... but has otherwise not changed since 201505:27
* tristan has recently upgraded to buster05:27
tristanAnd I'm comparing /usr/local/share/gir-1.0/OSTree-1.0.gir (a couple year old build I have locally) with /usr/share/gir-1.0/OSTree-1.0.gir (recently distro installed), but there are the same arguments there (just some clarifications of their types)05:28
tristanYet... on my laptop I get a stack trace telling me: TypeError: OSTree.Repo.remote_gpg_import() takes exactly 5 arguments (6 given)05:29
tristanWhile it doesnt appear to be a problem in CI (hence my trying to add buster to the images)05:29
tristanThis happens in tests/examples/flatpak-autotools.py::test_autotools_build and tests/examples/flatpak-autotools.py::test_autotools_run05:30
tristangirs... enter API breaks where it is hard to know where to point fingers05:30
tristanOr... wait... I have GI_TYPELIB_PATH set05:32
tristanSo... it appears to be a problem of my local build not playing well with the system installed pygobject stuff perhaps ?05:33
tristanThat would be a relief05:33
tristanyeah, anyway definitely not hitting that error anymore05:38
tristannothing is broke except my distro :)05:39
tristan(and now a bit less so :))05:39
*** tristan has quit IRC05:42
*** tristan has joined #buildstream06:27
gitlab-br-botmarge-bot123 merged MR !1477 (tristan/backport-previous-sources->bst-1: Backport ability to see previous sources in fetch() and track()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147706:29
*** toscalix has joined #buildstream07:09
benschubertjuergbi: thanks!07:44
benschuberttristan: did you encounter any difficulties with the docker-images CI? Is there anything that would make it easier?07:44
*** bochecha has joined #buildstream08:01
*** alexandrufazakas has joined #buildstream08:12
cs-shadowtristan: also, the Docker images repo is suffering from CI related issues at the moment. I plan on fixing it today. Will land your MR after that08:18
alexandrufazakastristan: Hey, can you please help me understand https://gitlab.com/BuildStream/buildstream/issues/178 a bit when you've some time?08:21
bochechamorning, bst 1.2.8 is in Fedora Rawhide, currently building in F30 and F2908:42
bochechaI didn't follow much lately, is there already a roadmap for the next breaking release?08:43
*** ChanServ sets mode: +o tristan08:48
tristanbenschubert, cs-shadow I think I got it figured out, which files I needed to modify in templates/08:48
tristanbenschubert, cs-shadow but I cannot explain why the CI fails really: https://gitlab.com/BuildStream/buildstream-docker-images/pipelines/7118350408:49
tristanreview appreciated :)08:49
tristanbochecha, I'm about to release a dev snapshot of 1.3.1 (1.3.0 was just a tag so leaving it there)... details will be in the mail08:50
bochechacool, will read then :)08:50
tristanbochecha, it's not "leading up to 2.0", it's rather a small feature release for some desperately needed features08:50
tristanalexandrufazakas, https://developer.gnome.org/glib/ <-- look at that page for example08:51
tristanalexandrufazakas, the ultimate goal, is that we have a landing page like that where we decide *which* docs to browse08:52
tristanalexandrufazakas, this has become all the more pressing since we decided to break API and work on BuildStream 208:52
alexandrufazakasOh, okay, that's a good example08:53
alexandrufazakasI see the master documentation is at https://buildstream.gitlab.io/buildstream/main_about.html08:53
alexandrufazakasAnd the one at https://docs.buildstream.build/main_about.html is the latest stable?08:54
alexandrufazakastristan: All the information there is generated using `tox man`, right? So the idea is to have everything somewhere for all stable versions + the development version?08:57
tristanalexandrufazakas, First question, yes... second question... no09:00
tristansecond question `tox -e docs` I believe09:00
tristanman updates the man pages only09:00
tristanwhich are currently statically committed, anyway man pages are irrelevant to the full published html docs09:01
*** phildawson_ has joined #buildstream09:01
alexandrufazakasThen what's the documentation at https://docs.buildstream.build/index.html for?09:03
alexandrufazakastristan: So which versions we'd want to have documentation for there?09:04
benschuberttristan: Kinnison if I remember correctly, YAML from buildstream ought to always have "\n" and whitespaces stripped, correct?09:05
KinnisonI believe arbitrary strings are always trimmed, yes09:05
KinnisonBut not necessarily anything interpreted as a boolean09:05
Kinnisonor number09:05
KinnisonAnd there was talk of removing that09:05
benschubertOk, because before my rework it was not correcy done, and BST master break cache keys on freedesktop by removing newlines09:07
benschubertSo new master has the expected behavior09:09
tristanTrailing and leading whitespace is stripped indeed09:12
tristanWe tried changing that but it was a headache as I recall09:12
benschubertperfect09:12
tristanalexandrufazakas, The latest of every minor point09:13
tristanalexandrufazakas, For added fancyness, I would group major points together (so that BuildStream 1 and BuildStream 2 are clearly separated)09:14
tristanalexandrufazakas, For BuildStream 2 however, your first entry will be 1.91.009:15
tristanwe probably need a tag for that09:15
tristanalexandrufazakas, You might take a look at badges.py in the docs subdir to find a script which introspects release tags09:15
alexandrufazakastristan: So all the tags on the buildstream repository?09:15
tristanyou can use that to do some arithmetic to "list all minor points" and derive "what is the greatest micro point for each minor point"09:16
alexandrufazakasAlthough yes, there's no 1.91.009:16
tristanright, we probably want one for that sake09:16
tristanalexandrufazakas, Oh wait... there are some more logistics...09:17
tristanalexandrufazakas, Asides from release tags, we'd want a "latest" at the end of each major point, that is branch driven instead of tag driven09:17
tristanI think that mechanics would work for all eternity :)09:17
tristanalexandrufazakas, Anyway, you could put branches and "latest" aside for the moment09:19
tristanalexandrufazakas, Assuming you can get the tag driven stuff done, adding a "latest" to that will become trivial09:19
alexandrufazakastristan: So we want a list with the tags from gitlab, e.g. "1.0.0, 1.0.1, ..., 1.3.1" each with their own documentaiton09:20
alexandrufazakasWhich should be generated by checking out that and `tox -e docs`09:20
tristanalexandrufazakas, Right... your task I suppose is figuring out and designing how that will all come together09:21
tristanalexandrufazakas, I.e. how the site automatically updates as a result of pushing a new tag, etc09:21
alexandrufazakastristan: Oh, okay, so we want this to be automated09:22
*** lachlan has joined #buildstream09:22
alexandrufazakastristan: And which site should we use?09:23
tristanalexandrufazakas, docs.buildstream.build of course :)09:24
alexandrufazakastristan: Alright. How can I make changes it?09:25
tristanalexandrufazakas, it's currently published with a hack here: https://gitlab.com/BuildStream/docs-website09:25
tristanalexandrufazakas, So that is currently a job which yanks the latest stable artifact and publishes it (it doesnt build the docs)09:26
tristanalexandrufazakas, Please feel completely free to either use that repo as a part of your mechanics for publishing docs, or nuke it to hell if possible09:26
alexandrufazakaslol09:26
gitlab-br-botaevri opened (was WIP) MR !1478 (aevri/nodews->master: node.pyx: rm trailing whitespace) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147809:26
tristanalexandrufazakas, I think it's a separate repo for permissions purposes (we need a special key to publish to the website, and we don't want "all developers" to have access to it)09:27
gitlab-br-botBenjaminSchubert approved MR !1478 (aevri/nodews->master: node.pyx: rm trailing whitespace) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147809:27
*** jonathanmaw has joined #buildstream09:33
tristanalexandrufazakas, Sorry that I don't have time today to brainstorm a proper design with you09:34
alexandrufazakastristan: It's alright09:34
alexandrufazakasI'm just trying to make sense of everything here09:34
* tristan started earlier today, and it's already 6:30pm, going to meet folks for dinner09:34
tristanalexandrufazakas, I hope I've clarified the global scope of the thing :)09:34
tristanalexandrufazakas, There are a few ways to fry the cat of course09:35
alexandrufazakastristan: You did help me understand it quite a bit, yes :D09:35
tristanMaybe build it all in one CI... or keep the approach of collecting prebuilt artifacts and then aggregating them on the website ?09:35
tristanThings for you to consider09:35
alexandrufazakasI'll comment on the issue should I have other questions, maybe you can have a look at them when you're in tomorrow09:35
tristanaggregating prebuilt artifacts might be easier because of possibly differing CI images09:35
tristani.e. maybe the docker images used to generate 1.0 documentation can no longer be used to generate docs for BuildStream 5.6, which will be released in the year 232909:36
tristan(it's already there on the roadmap !)09:36
alexandrufazakashahaha09:37
alexandrufazakasDefinitely worth considering09:37
tristanyou might have a git repo checkout in the docs-website repo which just lets you introspect the tags09:38
tristanand use the tags to derive gitlab artifact URLs09:39
tristananyway, not thinking about this anymore I promise !09:39
* tristan heads out to dinner :)09:39
alexandrufazakasEnjoy, tristan :)09:39
*** tristan has quit IRC09:43
benschubertjjardon: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/merge_requests/1559 :)09:46
gitlab-br-botmarge-bot123 merged MR !1449 (jennis/update_ready_for_runtime->master: Remove __update_state_recursively()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/144909:52
*** tristan has joined #buildstream09:57
bochecha> BuildStream 1.4 will be compatible in every way with previous BuildStream 1 releases10:00
bochechacool10:00
bochechadoes it make sense to package the 1.3 series then or should I wait for 1.4? maybe for Rawhide?10:00
*** lachlan has quit IRC10:11
*** lachlan has joined #buildstream10:17
alexandrufazakashttps://gitlab.com/BuildStream/docs-website/blob/master/.gitlab-ci.yml anyone have any idea where ACME_CHALLENGE is set here? Or is it just an envrionment variable which contains a key, therefore you don't want it in the code?10:26
gitlab-br-botmarge-bot123 merged MR !1478 (aevri/nodews->master: node.pyx: rm trailing whitespace) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147810:26
*** lachlan has quit IRC10:29
*** phoenix has joined #buildstream10:31
*** phoenix has quit IRC10:34
*** lachlan has joined #buildstream10:35
juergbialexandrufazakas: yes, that's an environment variable that is set in gitlab CI config for the project10:45
alexandrufazakasjuergbi: gitlab CI config? Isn't this yaml what that is?10:46
alexandrufazakasOh, I guess it's set through the UI?10:49
SotKyeah, there is also config you can edit in-browser if you have sufficient permissions, for things like variables to set in the environment for CI runs10:49
alexandrufazakasAh, okay. Thanks SotK, juergbi10:49
alexandrufazakasCan someone help me understand how thi https://gitlab.com/BuildStream/docs-website/blob/master/.gitlab-ci.yml publishes documentation to docs.buildstream.build, please?11:04
alexandrufazakasI understand most of the stuff here and the script itself, but it's not clear to me how the files end up at that url. I guess it's identified directly through the provided key?11:05
gitlab-br-botcs-shadow opened MR !1479 (chandan/stack-stricter-warnings->master: elements/stack: Better communicate its requirements) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147911:29
gitlab-br-botwillsalmon approved MR !1476 (becky/artifact_checkout_directory->master: Artifact checks out to default dir if no --tar or --directory) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147611:51
gitlab-br-botmarge-bot123 closed issue #1079 (bst artifact checkout should default to directory if neither --tar nor --directory given) on buildstream https://gitlab.com/BuildStream/buildstream/issues/107911:52
gitlab-br-botmarge-bot123 merged MR !1476 (becky/artifact_checkout_directory->master: Artifact checks out to default dir if no --tar or --directory) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/147611:52
tpollard\o/12:00
gitlab-br-botwillsalmon opened (was WIP) MR !951 (buildbox->master: Add BuildBox backend for sandboxing) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/95112:35
tpollardaevri: any chance that your pickle work will include making queues picklable?12:38
aevritpollard: unfortunately not - I'm trying to avoid pickling as much as possible.12:39
aevriWhat's you're use-case for that I wonder?12:39
aevriOne thing the pickling will enable is replays of jobs, which I've personally found useful for debugging.12:40
tpollardaevri: this front end process separation, the failure handling wants to query queue objects12:43
tpollardfor now I can get around it (because of having no interactive tests...) but it's extra pain12:44
aevriI did a quick test, and it looks like BuildQueue is picklable at least12:45
aevriMight be lucky and get the others for free too :)12:45
aevriFor that use-case, we'll probably have to address the "Element._get_args_for_child_job_pickling" elephant though - I suspect you'll be needing the callbacks and __reverse_dependencies.12:47
valentindtristan, I was wondering if it was fine to merge !1146. The rebasing was quite complex, so I wonder if we need a second review.12:50
gitlab-br-botMR !1146: Deduplicate files in local cache with or without exec rights https://gitlab.com/BuildStream/buildstream/merge_requests/114612:50
*** dtf has joined #buildstream13:03
gitlab-br-botaevri opened (was WIP) MR !1463 (aevri/pickle->master: pickle jobs) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/146313:11
*** lachlan has quit IRC13:22
*** lachlan has joined #buildstream13:22
*** phil has joined #buildstream13:31
*** phildawson_ has quit IRC13:31
*** lachlan has quit IRC13:34
*** phildawson_ has joined #buildstream13:34
*** phil has quit IRC13:36
*** lachlan has joined #buildstream13:39
gitlab-br-botjonathanmaw approved MR !1451 (becky/tar_compression->master: Allowing `bst artifact checkout --tar` to use compression) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/145113:49
*** lachlan has quit IRC13:51
*** bochecha has quit IRC14:02
*** bochecha_ has joined #buildstream14:03
*** bochecha has joined #buildstream14:07
gitlab-br-botjuergbi opened MR !1480 (juerg/platform->master: Store Platform reference in Context instance variable) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/148014:08
*** bochecha_ has quit IRC14:09
juergbibenschubert: are you ok with this approach? https://gitlab.com/BuildStream/buildstream/merge_requests/1465#note_19236346914:11
juergbiif so, I'll marge !146514:11
gitlab-br-botMR !1465: Make Context class a Python context manager https://gitlab.com/BuildStream/buildstream/merge_requests/146514:11
*** lachlan has joined #buildstream14:12
*** bochecha_ has joined #buildstream14:12
*** bochecha has quit IRC14:15
*** bochecha_ has quit IRC14:17
*** lachlan has quit IRC14:17
benschubertjuergbi: seems good!14:22
juergbita14:22
gitlab-br-botjuergbi opened MR !1481 (juerg/unused-cache-keys->master: Remove unused _cache_key variables and get_cache_key() methods) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/148114:22
gitlab-br-botmarge-bot123 merged MR !1465 (juerg/context->master: Make Context class a Python context manager) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/146514:27
*** bochecha has joined #buildstream14:36
*** lachlan has joined #buildstream14:40
*** lachlan has quit IRC14:46
*** lachlan has joined #buildstream15:00
*** phoenix has joined #buildstream15:01
*** dftxbs3e has quit IRC15:03
*** dftxbs3e has joined #buildstream15:04
*** dftxbs3e has quit IRC15:04
*** dftxbs3e has joined #buildstream15:04
*** lachlan has quit IRC15:09
alexandrufazakasAre documentation artifacts generated only for stable versions and master?15:11
alexandrufazakasNot sure if that ^ question makes sense. I want to have documentation for all the tags we have on the repo and download them similar to how we do it here https://gitlab.com/BuildStream/docs-website/blob/master/.gitlab-ci.yml#L1115:12
alexandrufazakasbst-1.2 is the documentation for 1.2.8. I also got master using */master/*, but not sure about 1, 1.0.1 etc15:13
*** phoenix has quit IRC15:13
*** lachlan has joined #buildstream15:17
alexandrufazakasI think I found a way, although it's differnet to the one in that yaml15:19
*** phoenix has joined #buildstream15:22
gitlab-br-botmarge-bot123 merged MR !1481 (juerg/unused-cache-keys->master: Remove unused _cache_key variables and get_cache_key() methods) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/148115:30
*** lachlan has quit IRC15:41
*** phildawson_ has quit IRC15:41
*** phildawson_ has joined #buildstream15:41
*** lachlan has joined #buildstream15:56
juergbiaevri: the build/test logs here seem to be incomplete: https://gitlab.com/BuildStream/buildstream/pipelines/71260439/builds16:02
juergbiany idea what's happening there? I don't see the issue in other branches but maybe it's coincidence16:03
*** lachlan has quit IRC16:03
*** lachlan has joined #buildstream16:04
*** bochecha_ has joined #buildstream16:06
aevriHow odd, I don't see the problem on the previous run: https://gitlab.com/BuildStream/buildstream/pipelines/7124996416:06
aevriThe changes introduced since then don't seem significant: https://gitlab.com/BuildStream/buildstream/merge_requests/1463/diffs?diff_id=48325312&start_sha=bcd204f1a3b4db59282b720f73e5b02d4162c50a16:07
juergbimight just be a gitlab issue16:07
aevriMy guess is coincidence, I'll make them rerun16:07
aevriGood to know you dive into the logs :)16:07
*** lachlan has quit IRC16:11
*** toscalix has quit IRC16:12
*** phoenix_ has joined #buildstream16:12
*** phoenix has quit IRC16:13
*** phoenix_ is now known as phoenix16:14
*** lachlan has joined #buildstream16:15
*** lachlan has quit IRC16:19
gitlab-br-botjonathanmaw approved MR !1451 (becky/tar_compression->master: Allowing `bst artifact checkout --tar` to use compression) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/145116:20
*** rdale has quit IRC16:25
*** alexandrufazakas has quit IRC16:27
*** rdale has joined #buildstream16:35
gitlab-br-botjonathanmaw opened MR !1482 (jonathan/job-progress->master: Render progress information for loading and processing elements) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/148216:36
*** lachlan has joined #buildstream16:41
*** lachlan has quit IRC16:45
*** jonathanmaw has quit IRC16:54
*** phildawson_ has quit IRC16:55
aevrijuergbi - those seem actually run on the second run, spooky.16:58
*** toscalix has joined #buildstream17:03
*** toscalix_ has joined #buildstream17:04
*** toscalix has quit IRC17:04
*** toscalix_ has quit IRC17:06
*** toscalix has joined #buildstream17:06
*** phoenix has quit IRC17:09
*** toscalix has quit IRC17:11
*** tiagogomes has quit IRC17:15
*** toscalix has joined #buildstream17:16
benschubertvalentind: are you around? I don't seem to have permission to push to freedesktop-sdk as per https://gitlab.com/freedesktop-sdk/freedesktop-sdk/merge_requests/155917:16
*** lachlan has joined #buildstream17:19
*** lachlan has quit IRC17:21
*** toscalix has quit IRC17:22
*** phoenix has joined #buildstream18:19
*** xjuan has joined #buildstream18:22
*** phoenix has quit IRC18:29
*** phoenix has joined #buildstream18:34
gitlab-br-botBenjaminSchubert opened MR !1483 (bschubert/optimize-downloadable-sources->master: Optimize downloadable sources) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/148319:02
benschubert^ under 60s for bst show on debian-stack19:03
benschuberto/19:03
valentindbenschubert, Sorry, I added the wrong person.19:23
valentindIt should work now.19:23
benschubertthanks, let me check19:32
benschubertvalentind: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/merge_requests/1560 here you go :D19:34
*** xjuan has quit IRC19:42
gitlab-br-botBenjaminSchubert opened MR !1484 (bschubert/api-improvements->master: node: Add 'get_str_list' on 'MappingNode') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/148419:47
*** xjuan has joined #buildstream20:51
*** xjuan has quit IRC21:34
*** bochecha_ has quit IRC21:49

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!