IRC logs for #buildstream for Tuesday, 2019-06-25

*** tristan has quit IRC03:08
*** Trevinho has quit IRC03:18
*** Trevinho has joined #buildstream03:19
*** Trevinho has quit IRC03:51
*** Trevinho has joined #buildstream03:52
jenniscs-shadow, around now08:09
*** raoul has joined #buildstream08:25
benschubertlaurence: I don't remember, is it our policy to delete all merged branches on gitlab? We have quite a few  still there. I can take care of that, just want to make sure I'm not doing a mistake :)08:43
* tlater[m] wonders if the "not providing sh" message could be enhanced to suggest changing base-dir for tarball elements08:53
tlater[m]I spent 3 hours trying to figure out why my tarball didn't contain what it contains yesterday |:08:53
*** tristan has joined #buildstream08:55
*** jonathanmaw has joined #buildstream08:55
*** tpollard has joined #buildstream09:03
juergbibenschubert: I was wondering about this myself. imo, we should delete merged branches (I always enable the option for my MRs)09:03
juergbithere is even a button to delete all merged branches in gitlab, iirc09:03
laurencebenschubert, honestly I do not know, I can have a look in contributing - otherwise juergbi / valentind / jjardon - do you know about old branches?09:03
laurencewhat's our policy, delete them?09:03
benschubertjuergbi: I agree, once merged, I'm not sure a branch is useful09:03
* tlater[m] hasn't seen anyone talk about a policy for this09:04
jonathanmawtristan: would you mind looking at https://gitlab.com/BuildStream/buildstream/merge_requests/1409 when you have time? I think I've made changes in line with all of your review comments, now.09:04
tlater[m]I always just click the delete button, but I've definitely forgotten in the past09:04
juergbiyes, especially as we use merge commits now, we can easily get the full SHA / latest history09:05
benschubertI'll shoot a post to the ML if we don't have a policy, in order to add one09:07
laurenceis there even any downside to deleting old branches, post merge?09:07
benschubertI don't see any, especially with merge commits09:08
tlater[m]benschubert: technically a developer might want to reset to their branch state at some point, and not know when it was merged.09:09
tlater[m]But I suppose you can still grep the git log for that09:09
benschubertfair enough. Not running 'git remote prune origin' locally would also keep your branch on your local system09:10
tlater[m]Yeah, it's an odd enough use case that I don't really see a point.09:10
benschuberthey tristan I updated !1401 to answer your comments. Could you have another look?09:28
gitlab-br-botMR !1401: introduce a Node api, and remove node_get https://gitlab.com/BuildStream/buildstream/merge_requests/140109:28
laurencetristan, with two approvals, can we assign this one to marge ? https://gitlab.com/BuildStream/buildstream/merge_requests/141109:29
tlater[m]valentind: Would you mind helping me get a new base image into the digital ocean host? I have an x86 one, I'd need to build an aarch64 one somehow.09:31
*** ChanServ sets mode: +o tristan09:33
tristanlaurence, Yeah lets marge those, they are dead simple09:33
laurenceok, cool]09:33
tristanbenschubert, jonathanmaw ... lemme get to that in a moment09:33
* tristan working through a bit of a small fire in freedesktop-sdk09:34
valentindtlater[m], sure, I have a board. Give me a branch I can build it.09:36
benschuberttristan: no worries, I just realize I missed the 'allow_none' part anyways. I'll fix that09:36
tlater[m]valentind: https://gitlab.com/BuildStream/buildstream/tree/tlater/freedesktop - in tests/integration/base, as previously09:36
valentindtlater[m], base.bst [line 5 column 4]: Could not find element 'freedesktop-sdk.bst:public-stacks/buildsystems.bst' in elements directory '/home/valentin/repos/buildstream/tests/integration/base/elements'09:38
tlater[m]That is interesting. Are you building with buildstream master?09:39
*** lachlan has joined #buildstream09:40
valentindAh maybe not.09:40
*** rdale has quit IRC09:41
*** lachlan has quit IRC09:46
*** rdale has joined #buildstream10:01
*** alexandrufazakas has joined #buildstream10:08
*** lachlan has joined #buildstream10:21
gitlab-br-botmarge-bot123 merged MR !1414 (juerg/fetch-subprojects->master: Improve subproject fetching) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/141410:23
*** lachlan has quit IRC10:25
gitlab-br-bottpollard approved MR !1412 (bschubert/remove-useless-sanitize->master: _yaml: Remove useless calls to '_yaml.node_sanitize') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/141210:28
gitlab-br-botjennis opened issue #1054 (Remove the concept of update_state) on buildstream https://gitlab.com/BuildStream/buildstream/issues/105410:30
tlater[m]valentind: I assume using master worked?10:52
valentindYes. But I updated the board before launching the build.10:52
*** lachlan has joined #buildstream10:54
valentindI launched the build now.10:55
valentindIt will take some time I think. I will send you the result tomorrow probably.10:56
tlater[m]Awesome, thanks10:56
*** lachlan has quit IRC11:06
*** raoul_ has joined #buildstream11:11
*** bochecha has joined #buildstream11:11
*** raoul has quit IRC11:12
*** raoul_ has quit IRC11:15
*** raoul_ has joined #buildstream11:15
gitlab-br-botmarge-bot123 merged MR !1412 (bschubert/remove-useless-sanitize->master: _yaml: Remove useless calls to '_yaml.node_sanitize') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/141211:23
*** lachlan has joined #buildstream11:52
gitlab-br-botjennis opened MR !1416 (jennis/do_not_leak_project_specific_remotes->master: Do not leak subproject remotes) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/141612:01
jennisjuergbi ^ I've re-fixed that regression12:02
tristanjonathanmaw, Another batch of comments up... lemme see if I have replies to the previous existing conversations...12:10
jonathanmawtvm tristan12:10
tristanjonathanmaw, I think https://gitlab.com/BuildStream/buildstream/merge_requests/1409/diffs#note_185075506 is the most important comment12:12
tristanjonathanmaw, I.e. what I'm seeing here... is there is still manual marshalling of events, which means the State object is not really implementing an observer model12:13
tristanevents are not being delivered in a single focal point12:13
tristanjonathanmaw, The gist of what I was aiming for is basically that (A) There is a State object (or group of objects, including TaskGroup)... which is passed around in the core... and parts of the core *drive* this object by calling it's API....  (B) This same State object is exposed to the frontend, allowing parts of the frontend to register interests on specific events and get notifications12:14
tristanThis model eliminates the need to marshal things around through modules12:15
tristanNote that it is a 2 faceted API... receiving events is for the frontend... driving state is for the core12:15
tlater[m]I was wondering if I could get some review on https://gitlab.com/BuildStream/buildstream/merge_requests/1413 while I'm waiting for the aarch64 build. It's WIP, because it'd be a PITA to try and locally run any tests or anything (and definitely fails CI), but is mostly done otherwise.12:48
tlater[m]Would be good to get some opinions soon so I'm not just twiddling my thumbs waiting for the build ;p12:49
tpollardtlater[m]: might be mis-remembering, but will this baseimage also have a cached artifact that's public too?13:16
tlater[m]tpollard: We ultimately decided that that's not the way to go.13:17
tlater[m]Though if you want to use that artifact cache for something else, the infrastructure is there13:17
tlater[m]For the time being, at least13:17
benschuberttristan: for !1401, do you think you'll have time today or should I expect this later ? So I know how to organize :)13:24
gitlab-br-botMR !1401: introduce a Node api, and remove node_get https://gitlab.com/BuildStream/buildstream/merge_requests/140113:24
tpollardtlater[m]: thanks, it'll be nice to consume that we've produced in our own examples!13:25
tristanbenschubert, hmmmm, ok so I think this is another full review, I don't think I can get that done today13:25
tristanbenschubert, lemme see if anything jumps out at me though13:26
benschuberttristan: ok perfect thanks!13:26
* tristan will just look at the plugin changes and how they use the new API for now13:26
tristanbenschubert, from that perspective it's looking very nice indeed :)13:27
benschubertIt's also slightly faster, so all in all I'm very happy with this new solution, even though it's more to write :)13:28
tristannot more for plugin authors though afaics13:28
tristaneven some parts have become less wordy13:29
benschubertagreed13:29
benschubertand the API is much less confusing13:29
laurenceNot something I see as hugely urgent, but I'd appreciate reviews on this committer policy patch - v13:36
laurencegah13:36
laurencehttps://gitlab.com/BuildStream/buildstream/merge_requests/141513:36
laurenceAnd input into the general conversation also13:36
laurencethe original list post never got much response - https://mail.gnome.org/archives/buildstream-list/2018-September/msg00044.html13:37
laurencealthough that was during an insanely busy period for that ML13:37
alexandrufazakasHey, everyone. I'm trying to work on https://gitlab.com/BuildStream/buildstream/issues/702 now that I've made some changes, how can I test those?14:06
tlater[m]o/ alexandrufazakas14:07
jonathanmawalexandrufazakas: running `tox` in the root of the buildstream repo should check for any breakages caused by your changes14:07
alexandrufazakastlater: hey o/14:07
alexandrufazakasjonathanmaw: I see, thank you14:07
alexandrufazakasSo that's what I should start with14:07
tlater[m]alexandrufazakas: If you can't get tox to run properly (although it should be simple), try to fork the project and push your changes to a branch on gitlab14:08
tlater[m]The CI should be set up automatically and it will run our tests for you as well.14:08
jonathanmawalexandrufazakas: yep, and to make sure the changes you made keep working, we recommend you write a test that checks the new functionality works, too14:08
alexandrufazakasYeah, i've forked it and I can do that, was thinking of trying locally first heh14:08
alexandrufazakasjonathanmaw: Yep, I read that. I'll definitely write one if what I tried actually works haha14:09
alexandrufazakasjonathanmaw, tlater: thanks for the help14:09
tpollardthat'll be a nice change to have!14:09
alexandrufazakasUgh, `tox` in the root of the repository results in `command not found`14:14
tlater[m]alexandrufazakas: Yup, it's a command you need to install separately14:15
tlater[m]What distro are you on? Arch's would be installed with `pacman -S python-tox`14:16
tpollardhttps://gitlab.com/BuildStream/buildstream/blob/master/CONTRIBUTING.rst#L1464 :)14:17
alexandrufazakastlater: oh, I didn't know14:17
alexandrufazakasI'm on Debian14:17
tlater[m]`apt install tox` should do the trick then. I *believe* everything else will be installed automatically once you start running tox.14:19
tlater[m]Our contributing.rst should probably be updated to mention that dependency ;)14:19
tpollardyou might need some extra deps, the link I posted should have everything14:19
alexandrufazakasUhh, this doesn't look like dependencies14:21
alexandrufazakasTraceback (most recent call last):14:22
alexandrufazakas  File "/usr/bin/tox", line 11, in <module>14:22
alexandrufazakas    load_entry_point('tox==2.5.0', 'console_scripts', 'tox')()14:22
alexandrufazakas  File "/usr/lib/python3/dist-packages/tox/session.py", line 38, in main14:22
alexandrufazakas    config = prepare(args)14:22
alexandrufazakas  File "/usr/lib/python3/dist-packages/tox/session.py", line 26, in prepare14:22
alexandrufazakas    config = parseconfig(args)14:22
alexandrufazakas  File "/usr/lib/python3/dist-packages/tox/config.py", line 239, in parseconfig14:22
alexandrufazakas    parseini(config, inipath)14:22
alexandrufazakas  File "/usr/lib/python3/dist-packages/tox/config.py", line 760, in __init__14:22
alexandrufazakas    self.make_envconfig(name, section, reader._subs, config)14:22
alexandrufazakas  File "/usr/lib/python3/dist-packages/tox/config.py", line 791, in make_envconfig14:22
alexandrufazakas    res = meth(env_attr.name, env_attr.default)14:22
alexandrufazakas  File "/usr/lib/python3/dist-packages/tox/config.py", line 959, in getbool14:22
alexandrufazakas    "boolean value %r needs to be 'True' or 'False'")14:22
alexandrufazakastox.ConfigError: ConfigError: boolean value %r needs to be 'True' or 'False'14:22
alexandrufazakasSorry for the wall of text everyone, should've used some paste tool :/14:22
alexandrufazakasOkay, I think it works now for some reason? It's installing more dependencies, so that's better than the error messages I got earlier14:26
* alexandrufazakas sighs14:28
alexandrufazakasAnyone familiar with this https://pastebin.com/iGuiRgw0 ?14:28
juergbino, incompatible version of click?14:29
juergbialthough, odd as that's tox-installed14:29
alexandrufazakasI think it just installed 7.014:30
alexandrufazakasIs that not the one I need?14:30
juergbiclick 7.0 should be correct14:32
alexandrufazakasYeah, it's listed as installed14:33
alexandrufazakasAnyone else have any idea what I could do? :(14:34
tpollardis the bst command working for you outside of tox?14:34
alexandrufazakastpollard: yes14:34
* phil notes that apt install on debian stable gets you tox 2.5, but installing tox via pip get's you tox 3.6. Could the issue be that you've got a really old version of tox?14:34
alexandrufazakasphil: good point, I think it got past the first initial issue because I `sudo apt-get auto-remove tox`14:35
alexandrufazakasand installed it via pip314:35
alexandrufazakasLooks like my current version is 3.13.0?14:36
alexandrufazakastox --version14:36
alexandrufazakas3.13.0 imported from /home/alexandrufazakas/.local/lib/python3.5/site-packages/tox/__init__.py14:36
*** tristan has quit IRC14:37
philAnd after doing that you got the last error you pasted?14:41
alexandrufazakasphil: yes14:41
philhmmm14:41
alexandrufazakasNot sure why I'm on 3.1314:42
tlater[m]alexandrufazakas: Is that running the test suite?14:42
alexandrufazakastlater: The error message?14:42
tlater[m]Oh, nvm, the command is listed at the end14:42
philalexandrufazakas, me neither, but I installed 3.13 and it worked for me, so I suspect the issue is somewhere else14:42
alexandrufazakasphil: Oh, I see14:43
philBut I'm not sure where :/14:43
tpollardI was wondering if you'd made a change to cli.py that was causing the click error. But I'd expect that to be thrown if your bst install was tracking the source too14:43
tlater[m]alexandrufazakas: I've seen that error before, it usually happens when I have junk in my project directory.14:43
alexandrufazakasHmm14:43
tlater[m]pytest attempts to parse all python files, and if it encounters something it doesn't expect it will do this.14:43
alexandrufazakasOkay, i'll just save what I did14:43
alexandrufazakasAnd `reset --hard`14:43
tlater[m]Try `git clean -fxd`14:44
alexandrufazakastlater: alright, thanks14:44
*** lachlan has quit IRC14:44
jjardonlaurence: I always delete them14:44
alexandrufazakastlater: looks like it does the same thing :(14:46
philAh. That's a really unhelpful stack trace. Is there any more output from your attempt at running tox, or does it crash like that immediately?14:48
alexandrufazakasphil: https://pastebin.com/3ATpfHsR14:49
*** tristan has joined #buildstream14:50
alexandrufazakasI saved my work and `git fetch origin && git reset --hard origin/master` and tox works now14:50
phil\o/14:51
alexandrufazakas...and now that I put my changes back in it does the same thing14:51
alexandrufazakasSo yeah, I messed something up good heh14:51
tlater[m]alexandrufazakas: Good job stumping everybody with your stacktrace anyway :p14:54
tlater[m]If you're stuck, push up your branch somewhere - happy to help debug (well, after meetings).14:54
alexandrufazakastlater: Yeah, heh, sorry about that :P14:54
alexandrufazakasI'd appreciate some help so yeah, I'd push everything to a branch. Thank you :)14:55
alexandrufazakasAnd everyone else who stepped in to help, ofc :D14:55
alexandrufazakastlater: Should I open a WIP MR on it or just link the branch to you?14:58
*** lachlan has joined #buildstream15:05
alexandrufazakastlater: I just opened a WIP MR on it, it's !1418. Have a look when you have some spare time :)15:12
gitlab-br-botMR !1418: WIP: app.py, cli.py: Make bst init take an argument https://gitlab.com/BuildStream/buildstream/merge_requests/141815:12
*** tristan has quit IRC15:17
tpollardalexandrufazakas: click.option can't have a help arg15:36
tpollardwait sorry click.argument15:36
alexandrufazakastpollard: hmm, that makes sense15:37
alexandrufazakastpollard: hey, now tests run, thank you!!15:38
alexandrufazakasI mean they fail, but still :D15:38
*** phildawson_ has joined #buildstream15:40
alexandrufazakasHmm, why are all [bzr] tests being skipped?15:41
tpollardalexandrufazakas: your local system doesn't have bzr installed15:41
*** phil has quit IRC15:42
alexandrufazakasMakes sense15:42
tpollardthat's expected test behaviour, you could install bzr if you're specifically interested in running those tests (that will be covered in the CI)15:42
alexandrufazakasI think I should install that too so they're as thorough as possible?15:42
alexandrufazakasOr let the CI do it, sure15:43
alexandrufazakasSo, if tests pass, I should write on specifically for this feature, right?15:43
tpollardalexandrufazakas: yep, in tests/frontend/init.py probably for what you're adding15:44
alexandrufazakastpollard: alright. thanks a bunch for the help15:45
tpollardnp!15:46
*** bochecha has quit IRC15:47
alexandrufazakasCan I run only a set of tests using `tox tests/frontent/init.py` ?15:57
alexandrufazakasor something similar15:57
tlater[m]alexandrufazakas: tox -- tests/frontend/init.py15:59
alexandrufazakasSweet, thanks tlater15:59
tlater[m]For reference, anything you type after -- is passed on to pytest, so if you want to figure out how to do other weird things, check here: https://docs.pytest.org/en/latest/16:00
tlater[m]Personal favorites are -x and --lf16:00
tlater[m]I'd also recommend reading CONTRIBUTING.rst closely (tpollard linked it earlier), it has a number of such snippets.16:01
tpollard-lf ++16:01
tpollardalso -n16:02
alexandrufazakasI'll read through these, sure!16:02
alexandrufazakasIt's still not very clear to me what "@pytest.mark.datafiles" does16:27
alexandrufazakasDoes it actually create a file/directory or is it only like a reference?16:27
tlater[m]alexandrufazakas: In essence, it creates a temporary directory with which we can work so that our tests stay consistent (and don't suddenly start passing the second time you run them).16:28
alexandrufazakasUhh16:29
alexandrufazakasYeah, I was asking because I bump into this "Error: Invalid value for "[DIRECTORY]": Path "/home/alexandrufazakas/Documents/buildstream/tests/frontend/foobar" does not exist."16:29
alexandrufazakasI used @pytest.mark.datafiles(DATA_DIR) where DATA_DIR is that location the error message says is missing16:30
tlater[m]alexandrufazakas: Did you run this through cli.run?16:30
alexandrufazakasyes16:31
alexandrufazakastlater: this is what I wrote https://pastebin.com/S3dzy1kN16:31
alexandrufazakasDidn't include these, sorry: TOP_DIR = os.path.dirname(os.path.realpath(__file__))16:31
alexandrufazakasDATA_DIR = os.path.join(TOP_DIR, 'foobar')16:31
tlater[m]Ah16:32
tlater[m]Have you come accross decorators before?16:32
alexandrufazakasI'm not sure what those are, no16:32
tlater[m]They're interesting little things and modify the way your actual function headers look. it's the @ thing with which you prefix your function.16:32
tlater[m]That pytest mark in particular makes the first argument a path to a temporary directory containing things dictated by DATA_DIR.16:33
alexandrufazakasSo I should parametrize tmpdir to DATA_DIR?16:33
tlater[m]If you look at the other tests, the first argument is always "datafiles"16:33
tlater[m]Pytest passes its directory into there - you should take that first argument as the project directory (and make it a string).16:34
tlater[m]Oh16:34
tlater[m]I get what you're trying to do, haha16:34
tlater[m]You've basically got the arguments the wrong way around and you should use tmpdir instead of DATA_DIR16:35
alexandrufazakasHmm16:35
alexandrufazakasSo I can remove that TOP_DIR, DATA_DIR stuff16:35
alexandrufazakasAnd just use tmpdir straight up16:35
tlater[m]No, you still need the DATA_DIR16:36
tlater[m]The DATA_DIR points to the files you're using for the test16:36
alexandrufazakasOh16:36
tlater[m]pytest just copies them to a temporary location16:36
alexandrufazakasYes16:36
tlater[m]For reference, this is the pytest feature: https://docs.pytest.org/en/latest/fixture.html16:36
tlater[m]The cli argument is actually our own little fixture, it's just provided test-wide instead of with one of those mark things, iirc.16:37
tlater[m]You can find the code for it in `buildstream/testing/integration.py` - it has a lot of neat helper methods.16:37
tlater[m]But well, in general you shouldn't need to know what this stuff does in too much detail. It's there to keep people from writing inconsistent tests, just repeat the patterns you see elsewhere.16:38
alexandrufazakasYeah, that's how I got the TOP_DIR, DATA_DIR stuff16:39
alexandrufazakasI was reproducing code from a different test I saw used .datafiles16:39
tlater[m]Yup, gotcha. Have you figured it out then?16:39
alexandrufazakasNot really :(16:39
tlater[m]Where are you stuck?16:40
alexandrufazakasNevermind16:40
alexandrufazakasI think I got it16:40
tlater[m](It might help just to print() your arguments ;p)16:41
alexandrufazakasEarlier when I just told you not really it was because the test failed, but I just trying to join something that was not 2 strings :P16:41
tlater[m]Ah, right16:41
alexandrufazakasI was doing something like os.path.join(tmpdir, 'project.conf')16:41
alexandrufazakasBut your pointers helped me out16:41
alexandrufazakasIt seems to work fine now16:41
tlater[m]Awesome - tell me when it's ready, I'll give it a quick review, perhaps we can even get it merged today16:42
*** phildawson_ has quit IRC16:42
alexandrufazakastlater: Sure, I'll write a commit on it and push my changes :)16:42
*** phildawson_ has joined #buildstream16:44
alexandrufazakastlater: I've just pushed everything to !141816:47
gitlab-br-botMR !1418: WIP: app.py, cli.py: Make bst init take an argument https://gitlab.com/BuildStream/buildstream/merge_requests/141816:47
tlater[m]Cool, taking a look16:50
alexandrufazakastlater: I'll probably get going. I'll have a look at this tomorrow morning and fix what needs further fixing :D17:00
tlater[m]alexandrufazakas: Yep, no worries, I'll have my comments done by then17:00
tlater[m]o/17:00
alexandrufazakasAwesome. Thank you for all the help \o/17:00
*** phildawson_ has quit IRC17:01
*** alexandrufazakas has quit IRC17:01
*** jonathanmaw has quit IRC17:03
*** lachlan has quit IRC17:42
*** tristan has joined #buildstream17:58
*** tristan_ has joined #buildstream17:59
*** cynthia has joined #buildstream18:39
*** cynthia has quit IRC18:41

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!