IRC logs for #buildstream for Friday, 2018-03-09

*** sstriker has quit IRC00:12
*** Prince781 has quit IRC00:20
*** Prince781 has joined #buildstream00:58
*** mcatanzaro has quit IRC01:18
*** tristan has joined #buildstream07:00
*** Prince781 has quit IRC07:09
*** i0d3l4y has joined #buildstream07:16
*** noisecell has joined #buildstream07:36
*** i0d3l4y has quit IRC08:16
*** aiden has left #buildstream09:13
*** aiden has joined #buildstream09:13
*** valentind has joined #buildstream09:21
jennisjuergbi, tlater were you referring to using this to deal with dir_?09:24
jennishttps://github.com/PyCQA/pylint/blob/master/pylint/extensions/bad_builtin.py09:24
tlaterYep09:24
tlaterWe should probably use that in general09:24
tlaterIt seems it was mostly disabled because too many people didn't understand why you shouldn't use certain builtins09:25
tlaterskullman: I suspect you haven't enabled CI on your buildstream fork: https://gitlab.com/BuildStream/buildstream/merge_requests/31209:37
skullmantlater: having a look09:40
tristanskullman, did you request developer access ? I recommend working on the main repo directly, it's less hassle than a fork09:43
skullmantristan: it was pretty late by the time I got around to it and assumed nobody would respond. Is that a button on GitLab I can press or something?09:45
tristanyeah, hold on...09:49
skullmanah, I hadn't noticed the button since I wasn't logged in when I loaded the page09:50
tristanah you found it :)09:50
tristanfaster than me09:50
tristanpolicy is just accept developer access; as it's non-destructive09:51
skullmanah cool09:51
* tlater wonders if MRs can be migrated to new branches09:51
tristannew policy should be, prefix your branch names with user name; I'm going to follow that too now since we're more active than before09:51
tristantlater, probably not worth worrying about; if it's a long standing MR just create a new one to obsolete the old09:52
skullmanI tend to put my name on my branches anyway. While you're around. I noticed there's provenance tracking in the data structures. Should I be adding some provenance data for new fields?09:52
tristanskullman, that is entirely automated by _yaml module at load time09:53
tristanThe Provenance objects is just filename/line/column info09:53
skullmanI meant fields added to those structures. e.g. the dict was a yaml document that was loaded, but I add a new field, should I try to add something to say it came from somewhere else, pretend that it came from the same file but without a line or column, or just leave it without any provenance for that field?09:54
tristanIn what case do you need to do this ?09:55
*** slaf- has joined #buildstream09:55
tristanskullman, most likely answer is going to be just leave it without09:56
skullmantristan: adding a new version field to a loaded dict.09:56
*** slaf- has joined #buildstream09:56
tristanJust leave it, then09:56
skullmanok, none of the tests have complained about them missing, I'm just worried because I'm not familiar enough with the code to know if it'll cause bugs elsewhere09:57
tristanskullman, the provenance is only useful for reporting errors about things which were loaded09:57
tristani.e. the node_get() family of functions will use it to make better errors09:57
*** slaf has quit IRC09:57
*** slaf- is now known as slaf09:57
tlaterHm, I wonder how frequently we should change workspace.yml versions10:06
tlaterI think we have 3 version changes that will happen basically in the same week10:07
tlaterIs there any way we could better coordinate this?10:07
gitlab-br-botbuildstream: merge request (richardmaw-271-workspaces-versioning->master: WIP: Add versioning to workspaces yaml configuration) #313 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/31310:08
gitlab-br-botbuildstream: merge request (richardmaw-271-workspaces-versioning->master: WIP: Add versioning to workspaces yaml configuration) #312 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/31210:09
tristantlater, I am very, very not fussed about that version being incremented with every change10:11
tristantlater, I would say, if three version changes happen in one week; ok I dont care... if they only change once in that week, sure... as the first patch, or the last one; I also dont care10:12
tristanThat version check is: If you have a BuildStream installation one day and open a workspace... and then another day; you *downgrade* buildstream... or say a year later you install an older version of buildstream...10:13
tristanThen instead of exploding, it tells you that your local state .bst/workspaces.yml file requires a newer version of BuildStream10:13
* tlater wasn't sure if anything might break with updates10:13
tlaterBut I really can't think of a scenario10:14
tristanno, workspaces.yml is internally managed user local state, it's not like project format that way10:14
tristanyou can't pull an update of your project and then have a problem10:15
* tlater boldly changes version to "2" on his branch then10:15
*** dominic has joined #buildstream10:15
tristanI honestly would have been okay with an unversioned thing; others had a nitpick on that so lets do it10:16
tlaterHm, I am starting to wonder if a separate class would be better for workspaces. It looks like Project will gain a handful of methods every time we change the workspace format.10:19
tristantlater, I suggested that yes10:20
tristanI dont mind the patch either way for now, but as it grows, that's exactly what we should do10:20
*** ernestask has joined #buildstream10:25
gitlab-br-botbuildstream: merge request (239-use-pylint-for-linting->master: Resolve "Use pylint for linting") #283 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/28310:59
gitlab-br-botbuildstream: merge request (tristan/project-refs->master: WIP: Optionally load and store source references in project.refs) #314 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/31411:02
*** valentind has quit IRC11:06
gitlab-br-botbuildstream: merge request (issue-89_unique_build_dirs->master: WIP: Made modification to generate unique subdirs for built elements) #310 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/31011:14
tlaterNexus: Let's go through updating host tools here11:18
tlaterCurrently the CI uses this image: https://gitlab.com/BuildStream/buildstream/blob/master/.gitlab-ci.yml#L111:19
tlaterThe source for it lives here: https://gitlab.com/BuildStream/buildstream-docker-images/tree/master/fedora11:19
tlaterWhat you'll need to do is add installing your host tool (dpkg I think) to that image11:20
tlaterIf you push to buildstream-docker-images/master it will automatically build and push the new image to dockerhub11:21
tlaterAfter that you only need to update .gitlab-ci.yml to refer to your new tag11:21
tlaterNexus: You might want to take up whether this is actually required in master with tristan first though, I'm not sure why we would have dpkg as a hard buildstream dependency.11:23
Nexustlater: due to the dpkg plugin11:23
Nexusit's the cleanest way to have it, without writing my own dpkg in python, which is no better11:24
tristantlater, it sounds better suited to bst-external, but in either case it will not be a hard dependency11:24
tristantlater, we have preflight stage to tell the user they need host tools if their project needs them11:25
tlaterAh, alright11:25
tlaterI've been thinking of anything in the docker image as a hard dependency, but I suppose the CI needs even these kinds of tools.11:26
tristanthe CI needs to test everything yeah11:26
gitlab-br-botbuildstream: merge request (239-use-pylint-for-linting->master: Resolve "Use pylint for linting") #283 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/28311:27
*** tristan has quit IRC11:31
*** tristan has joined #buildstream11:50
tlaterHm, there's no nice way to access artifact metadata for a given key yet, is there?11:58
*** Prince781 has joined #buildstream13:42
*** Prince781 has quit IRC14:02
*** Prince781 has joined #buildstream14:06
millonihello14:10
milloniI've been wanting to get into buildstream14:10
milloniin my free time, where do I start?14:10
milloni(as a developer)14:11
milloniare there any simple issues I could work at?14:12
*** mcatanzaro has joined #buildstream14:14
tlatero/ milloni14:18
*** Prince781 has quit IRC14:19
*** Prince781 has joined #buildstream14:20
tlaterThere are a ton of issues, quite a few simple, tristan can probably point out the fun ones. Still, some less fun ones I can see at a glance:14:21
tlaterhttps://gitlab.com/BuildStream/buildstream/issues/257 https://gitlab.com/BuildStream/buildstream/issues/271 https://gitlab.com/BuildStream/buildstream/issues/288 https://gitlab.com/BuildStream/buildstream/issues/28914:21
tlatermilloni: That last one would probably be a massive improvement :)14:21
Nexusanyone know why i got an error saying "ERROR: Job failed: execution took longer than 3h0m0s seconds"14:24
Nexusinb4 the job took over 3h joke14:24
tlaterNexus: The job took over 3h14:25
jjardon[m]Nexus: what repo?14:25
Nexusjjardon[m]: https://gitlab.com/BuildStream/buildstream/merge_requests/31014:25
tlaterNexus: "Duration: 180 minutes 4 seconds"14:26
* Nexus slaps tlater 14:26
tlaterFor some reason your CI just took that long... Did you write a really convoluted test?14:26
Nexusi don't think so!14:26
jjardon[m]Nexus: CI is configured to fail after 180 minutes14:26
tlaterIt also sometimes happens that gitlab gitches a bit14:26
jjardon[m]Nexus: any idea why that job is taken more than normal?14:26
millonithanks tlater14:26
Nexusbut it passes fine on mine and takes ~1014:27
* Nexus runs it again14:27
*** Prince781 has quit IRC14:27
tlaterThat might just solve it :)14:27
tlaterThough do keep track of how long it takes, if it's anywhere near that long again you should take a closer look.14:27
Nexusyep14:27
*** noisecell has quit IRC14:29
*** Prince781 has joined #buildstream14:31
millonitlater, in issue #289, what's a runtime in this context?14:36
tlatermilloni: A sysroot, in essense14:36
tlaterA buildstream project always needs a base sysroot, which is used to set up the initial sandbox14:36
tlaterAnything else can be run from there14:36
tlatermilloni: If you'd like to tackle that issue, it would probably make sense to try and set up a buildstream project from scratch first - Nexus can probably link some docs to help with that14:37
Nexustlater: jjardon[m] looks like it stopped again14:38
tlaterNexus: Hrm, so it's an issue with that test case, hm14:39
millonitlater, so do i understand correctly, there's a configuration option for setting up a runtime, but if you don't specify that option and try to run shell commands anyway, it won't complain that you didn't set that option14:39
milloniit will only say that /bin/sh is not available14:39
tlatermilloni: That's the gist of the issue, yes14:39
tlaterThe configuration in this case is a buildstream element14:40
Nexushmmmm14:40
* Nexus has a secret thought14:40
tlaterNexus: I think your docs would really help milloni here :)14:40
Nexuswut,14:40
Nexuswhich ones, whats going on?14:40
milloniNexus, yes please14:40
milloniNexus, do you have docs to set up a buildstream project?14:41
Nexushttps://gitlab.com/BuildStream/buildstream/issues/16914:41
Nexusinstall buildstream and build a project14:41
Nexusare linked in there14:41
millonithanks Nexus14:41
Nexusnp14:41
tlatermilloni: This *is* the topic most starters struggle with initially, so shout if you get stuck14:42
milloniwhere are the docs for buildstream in general?14:42
tlatermilloni: These are our reference docs: https://buildstream.gitlab.io/buildstream/14:42
tlaterWe also have some gnome wiki pages, not sure where they are exactly14:43
Nexusyou know what would be great?... if my docs got merged.14:43
millonicool, I'll look into this over the weekend. thanks14:43
tlater:)14:43
tlaterNexus: Back to failing tests14:44
tlaterSecret thoughts?14:44
tlaterI can't see anything obvious about that test case, best guess is that that f.read() call somehow fails to terminate14:45
*** noisecell has joined #buildstream14:45
jmacNexus: Is there an outstanding merge request? I can see !183 but that's closed.14:46
Nexusjmac: i think tristan cancelled it14:47
tlaterIt's kinda cute because your docs are still the best thing to link someone to14:47
Nexusoh wait14:47
Nexushttps://gitlab.com/BuildStream/buildstream/merge_requests/206      https://gitlab.com/BuildStream/buildstream/merge_requests/236     https://gitlab.com/BuildStream/buildstream/merge_requests/237    https://gitlab.com/BuildStream/buildstream/merge_requests/23814:48
Nexusall my docs14:48
Nexustlater: ikr?14:48
* jmac can't see any outstanding reason not to merge 20614:50
Nexustristan:14:51
*** Prince781 has quit IRC14:56
jmacHmm, I'm seeing source_dist take ages on gitlab now, might be the same thing Nexus had14:57
Nexusmine was `test_script_cwd`14:58
tlaterjmac: That *might* be the case, but Nexus' tests hang on the same exact case14:58
tlaterSo I think it's only Nexus' branch14:58
jmacI have a merge request for the benchmarks repo, https://gitlab.com/BuildStream/benchmarks/merge_requests/215:10
jmacAny chance someone could take a look, maybe Nexus? Since jonathanmaw isn't here15:11
jmacNot urgent15:11
Nexussure15:11
jmacDoes anyone have any views on adding type annotations to python code in BuildStream? I'm quite keen on them as it makes it easier for me to read code, even if we never do any type-checking on them15:42
tlaterjmac: I thought type annotations weren't supported in python3.4?15:43
jmacFunction argument annotations are valid in Python 3.0 so far as I know15:43
jmacThey have a more specific meaning in 3.415:43
jmacSorry, 3.515:44
* tlater would like having them, if only to make code completion handier15:45
Nexusapparently typing was added in 3.5, according to https://docs.python.org/3/library/typing.html  but i'm by no means an authority on it, just a casual googler of the matter15:45
Nexus*unrelated note* How hard would it be to update buildstream to the latest python/15:46
Nexus?15:46
tlaterNexus: It's already fully supported15:47
Nexusthen why use 3.4?15:47
tlaterBecause some people run python3.415:47
Nexusic15:47
* juergbi likes type annotations15:47
tlaterNotably, the previous major version of debian (which we support)15:47
tlaterIf type annotations don't break 3.4 support they'd be a great addition I think, but for consistency we'd need someone to do a massive overhaul on our repo :/15:48
juergbii don't think we actually support jessie anymore but other distros might still be on 3.415:48
* tlater has seen the doc comments get out of sync with actual types in places, too15:48
tlaterSo a conversion tool probably wouldn't work15:49
jmachttps://www.python.org/dev/peps/pep-3107/ was in 3.0, and that allows you to put anything you like in the annotation; the direct use of that to mean types only comes in in 3.515:49
jmacI don't think it's likely that we'll overhall the whole repo to add types everywhere15:49
jmacoverhaul!15:50
* tlater finds the idea of mixing styles ugly15:51
juergbimaybe one file at a time?15:52
jmacThat's a bit like saying you shouldn't add a comment unless you add comments on all functions15:52
juergbiwell, we should be clear about whether we want them or not. having people write new code, some with and some without annotations, sounds bad15:53
juergbiand assuming we want them, the aim should definitely be to consistently add them to existing functions as well15:54
juergbihowever, this could be done in multiple steps to make it more feasible15:54
juergbitristan might want it in a single step, though15:55
jmacIt does not sound bad to me15:55
* tlater wonders if pylint has a way to enforce type hints15:56
juergbiwe have to be careful, there may be places where we accept multiple types (list or string)15:57
jmacOf course.15:57
jmacThat in itself is a source of confusion, so all the more reason to make it explicit that we accept both15:58
tlaterIt would be great if we could also express whether set() is supported15:58
Nexusjmac: other than what i mentioned, no problems with what you've pushed up, just trying to test it locally now16:00
jmacThanks Nexus16:01
Nexusit'd help if i wasn't looking in the buildstream main repo...16:11
Nexusis there even any kind of test for this?16:12
jmacFor the feature I'm adding? No.16:13
jmacThere's only one test for the repo at the moment, just to run the perf test with default settings and see if it exits with value 016:14
*** mcatanzaro has quit IRC16:17
milloniNexus, links in https://gitlab.com/knownexus/buildstream/blob/buildproject/doc/source/buildproject.rst seem broken16:19
*** mcatanzaro has joined #buildstream16:20
Nexusmilloni: sadly yes, it's intended for use when merged. The linked files are (for the most part)also in that repo under the appropriate branch names16:20
milloniah, ok thanks16:20
Nexusnp16:20
Nexusmilloni: i think your best option is to clone the repo, build the docs and read them that way16:22
Nexusthen all the links should work :)16:23
millonigood idea16:23
Nexusjmac: i see16:24
Nexusi have no idea how to do that16:24
Nexusdoes the perf laptop run those?16:24
jmacI suppose I should add some then16:24
Nexusif so has this been tested on it?16:24
Nexus:)16:25
jmacNexus: The perf laptop isn't running anything at the moment. I've done all the tests using my desktop.16:25
Nexuskk16:25
milloniit seems sphinx is needed to build the docs, but is not listed as a dependency in setuptools? is that intentional?16:27
Nexusiirc it's because it's not a buildstream dependency, and the docs are only build for the site16:28
jennisIf I want a project.conf that uses two different plugins, and each require a different `path:`, what's the solution for this?16:28
jennisCan you have two plugins:...?16:28
Nexusbuilt*16:28
Nexusi think so16:28
Nexusyou just put them one beneith the other16:29
Nexusoh dear god i can't spell16:29
* Nexus quits english16:29
Nexusbeneath*********16:29
jennislol16:29
Nexus(╯°□°)╯︵ɥʇɐǝuǝq16:29
jennisdoesn't look like that's worked :o16:30
* Nexus wanders over to jennis 16:32
milloniNexus, https://pastebin.com/jSBbk86e16:44
millonican't build docs from your branch, but I can from main branch16:44
Nexusinteresting16:45
Nexusmilloni: which branch was that?16:47
Nexusmodandtest?16:48
milloniuhmmm actually16:48
milloniok, so I never even changed the branch16:48
Nexuslol16:48
Nexusthat's a relief, because i spent several hours removing all of those errors16:49
milloniNexus, I'm on modAndTest and, still getting an error16:50
Nexusis ssssam[m] around?16:51
*** schmorris has joined #buildstream16:52
Nexusmilloni: interesting, which error?16:52
milloniNexus, https://pastebin.com/ZuetKF2916:54
milloniI think it's the same error16:54
Nexusdid you clean up the repo when you moved?16:56
Nexusif might be looking at a file that didn't exist at the time i wrote this16:57
*** schmorris has quit IRC16:59
milloniNexus, I ran git clean -f -d17:02
* Nexus usually runs git clean -dfx just to be sure but that's unlikely to change anything17:03
Nexushow odd17:03
Nexusit builds on mine17:03
millonicloned again into a separate dir to be sure, checkout'd modAndTest, ran make -C doc17:04
millonistill fails17:04
Nexusrun `make` from the doc dir and see if it works17:05
Nexusbut that worked for me too :/17:05
Nexuscommit 98eb7381c8713157b549b6b18e07a3ffab987b6917:05
Nexus?17:05
millonididn't work from doc dir either17:06
milloniyes, that commit17:06
*** valentind has joined #buildstream17:06
Nexusmilloni: can you do a `grep -rn . -e "sandboxing" for me pls and tell me what you find?17:10
milloni./build/html/buildstream.sandbox.sandbox.html:40:<p>See also: <span class="xref std std-ref">sandboxing</span>.</p>17:11
milloniBinary file ./build/doctrees/buildstream.sandbox.sandbox.doctree matches17:11
Nexusi don't have that line  in ./build/html/buildstream.sandbox.sandbox.html ._.17:12
milloniwat17:13
Nexus 40         <div class="bodywrapper">17:13
milloniNexus, `git remote get-url origin` gives https://gitlab.com/knownexus/buildstream17:15
Nexusi have that in master though...17:15
millonigit branch17:15
millonimodAndTest17:15
Nexushttps://gitlab.com/knownexus/buildstream.git17:15
Nexus* modAndTest17:15
Nexusmilloni: sandboxing is only in the current master, not in the repo you're using :p17:16
millonicurrent master on buildstream/buildstream?17:16
Nexusyes17:16
milloniwtf17:17
NexusSOMEHOW it looks like you've merged the two17:17
Nexusmilloni: do you have a sandboxing.rst file in your doc/source ?17:17
milloniNexus, I don't have that file17:18
Nexusgood, you shouldn't17:19
Nexusso why does it think you do17:19
Nexushmm17:19
Nexustry downlodaing a tar of the repo and extracting it17:19
Nexusinstead of using git17:19
milloniNexus, before I do that17:24
millonidifferent sphinx versions?17:24
Nexuserm, what's the sphinx package called?17:24
milloniI installed from pip17:24
millonido17:25
millonipython317:25
milloniimport sphinx17:25
millonisphinx.__version__17:25
Nexus'1.6.5'17:25
millonimine is 1.7.117:25
millonithat's probably what is causing this17:27
Nexusmilloni: what package did you install? just pip install sphinx?17:27
Nexuspip3*17:27
milloniyes17:27
milloniyou might have not upgraded in a while17:27
Nexusfacinating17:27
Nexusi have multiple installed17:28
milloniheh17:28
Nexusi just uninstalled it and it still exists17:28
millonithat's probably because there's pip install --user17:28
millonibut there's no pip uninstall --user17:28
*** dominic has quit IRC17:31
Nexusok, completely removed sphinx and reinstalling17:31
Nexus1.7.117:32
Nexusbuild succeeded.17:32
Nexus:)17:32
Nexusmilloni: it was a good theory :p17:32
milloni\o/17:32
millonioh17:32
millonino :(17:32
Nexuslol17:33
Nexustar idea? :)17:33
Nexusatleast i have the new sphinx :)17:33
Nexusso not a complete loss17:33
millonido you think it might be possible taht you still have multiple sphinx versions?17:33
* milloni downloads the tar17:33
Nexusi tried importing sphinx after removing it manually and it failed17:34
Nexusany luck?17:40
*** ernestask has quit IRC17:43
*** ernestask has joined #buildstream17:47
milloniNexus, nope17:47
Nexushttps://goo.gl/pHPFVb17:48
milloniit looks like there must be a difference between my build env and yours17:48
milloniwe've checked sphinx, let me see what else is involved17:49
Nexusthat's literally the thing buildstream is trying to fix :P17:49
milloniyou could build docs using buildstream... but how will you know how to use buildstream without the docs?17:49
Nexusyou need to build buildstram using buildstream so you can read the docs to create a build17:49
millonithat too lol17:50
milloniwell I mean people are using gcc to build gcc so that's not that crazy of an idea17:50
Nexus:D17:50
Nexusi'm afraid you may need more assistance than i can provide you atm :/17:51
Nexuswithout actually seeing your local repo anyway17:51
Nexushave you tried diffing your pull and master?17:52
Nexusor your pull and my repo17:52
milloniNexus, could you please do17:55
millonipip3 install --upgrade sphinx-click17:56
Nexus0upgraded17:56
Nexusbuild succeeded.17:56
milloniI know my modAndTest is rebased on top of master, but master hasn't been updated in a while17:56
milloniin that fork17:56
Nexusyes, it's very far behind master17:57
Nexusdon't rebase17:57
milloniI didn't17:58
Nexushmmmm17:58
* Nexus is going for the day17:59
Nexusbest of luck, if you're still having this issue on monday, i'll get others involved17:59
millonithanks18:00
*** xjuan has joined #buildstream18:26
*** tiago has quit IRC18:39
*** toscalix has joined #buildstream20:32
*** ernestask has quit IRC20:35
*** xjuan has quit IRC21:20
*** toscalix has quit IRC21:32
*** noisecell has quit IRC22:20
*** tristan has quit IRC22:25
*** toscalix has joined #buildstream22:35
*** valentind has quit IRC23:12
*** valentind has joined #buildstream23:12
*** valentind has quit IRC23:38
*** toscalix has quit IRC23:38

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!