| *** tristan has joined #buildstream | 04:00 | |
| *** kapil___ has joined #buildstream | 06:31 | |
| gitlab-br-bot | juergbi opened issue #843 (Follow-up from "Introduce new "source" command group") on buildstream https://gitlab.com/BuildStream/buildstream/issues/843 | 07:03 |
|---|---|---|
| gitlab-br-bot | juergbi opened MR !1036 (juerg/source-guess-target->master: _frontend/cli.py: Reinstate support for guessing targets) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1036 | 07:14 |
| *** alatiera has joined #buildstream | 07:28 | |
| gitlab-br-bot | BenjaminSchubert approved MR !1035 (chandan/update-requirements-one-liner->master: Add Makefile to update requirements files) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1035 | 09:34 |
| *** rdale has joined #buildstream | 09:42 | |
| *** raoul has joined #buildstream | 10:01 | |
| juergbi | cs-shadow: fyi, https://gitlab.com/BuildStream/buildstream/merge_requests/1036 | 10:23 |
| gitlab-br-bot | willsalmon opened (was WIP) MR !1018 (willsalmon/outOfSourecBuilddocs->master: Adding Out of Source Build Example) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1018 | 10:24 |
| *** jonathanmaw has joined #buildstream | 10:34 | |
| cs-shadow | juergbi: good catch, thanks. It makes me wonder if we can add a test for it | 10:46 |
| juergbi | yes, maybe jonathanmaw can add the missing tests | 10:47 |
| juergbi | context: !1036 | 10:47 |
| cs-shadow | They’ll be good. Your MR looks good to me for the actual fix | 10:48 |
| *** juergbi has quit IRC | 10:51 | |
| *** juergbi has joined #buildstream | 10:51 | |
| jonathanmaw | juergbi: ok, I'll have a look at that | 11:06 |
| gitlab-br-bot | cs-shadow approved MR !1036 (juerg/source-guess-target->master: _frontend/cli.py: Reinstate support for guessing targets) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1036 | 11:20 |
| juergbi | ta | 11:27 |
| gitlab-br-bot | juergbi merged MR !1036 (juerg/source-guess-target->master: _frontend/cli.py: Reinstate support for guessing targets) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1036 | 11:27 |
| benschubert | If I want to benchmark some of my changes compared to master, what is the way to go? | 11:57 |
| *** alatiera_ has joined #buildstream | 12:00 | |
| *** alatiera has quit IRC | 12:01 | |
| laurence | benschubert, hmmmm, resident benchmarks chief lachlan isn't around today | 12:05 |
| laurence | are you aware of the specific benchmarks repo? https://gitlab.com/BuildStream/benchmarks | 12:05 |
| laurence | specifically, the README points to the Benchmarks Concept Document which may be of use? https://gitlab.com/BuildStream/benchmarks/blob/master/Benchmark_concept.md/ | 12:05 |
| *** alatiera has joined #buildstream | 12:07 | |
| benschubert | laurence: I'm aware of it, and couldn't find how to test something locally, and a direct comparison of two branches. I might be asking for something that doesn't exist yet though :/ | 12:09 |
| laurence | I think so, yes. Although I know that's it's been designed so that adding more benchmarking tests isn't difficult. sorry i can't be of more help | 12:12 |
| benschubert | No worries, do you happen to know when lachlan will be back? :) | 12:20 |
| jmac | Monday according to my calendar | 12:21 |
| benschubert | thanks! | 12:24 |
| *** kapil___ has quit IRC | 12:41 | |
| *** nimish has joined #buildstream | 13:08 | |
| *** raoul has quit IRC | 13:18 | |
| *** nimish has quit IRC | 13:22 | |
| *** nimish has joined #buildstream | 13:23 | |
| *** nimish has quit IRC | 13:37 | |
| *** nimish has joined #buildstream | 13:38 | |
| cs-shadow | hi, does anyone know if we are presenting something at FOSDEM this year? | 13:45 |
| *** raoul has joined #buildstream | 13:51 | |
| *** bochecha has joined #buildstream | 13:57 | |
| *** nimish has quit IRC | 14:03 | |
| *** nimish has joined #buildstream | 14:03 | |
| laurence | cs-shadow, I don't think so, sadly - I have asked before and haven't received any information on a submission, anyway | 14:05 |
| cs-shadow | laurence: thanks for confirming. For some reason, I was under the impression that we were doing a talk, but then I wasn't able to find anything about it, so checked here | 14:06 |
| *** nimish has quit IRC | 14:23 | |
| *** nimish has joined #buildstream | 14:24 | |
| *** nimish has quit IRC | 14:30 | |
| *** raoul has quit IRC | 14:32 | |
| *** alatiera_ has joined #buildstream | 14:32 | |
| *** phildawson_ has quit IRC | 14:32 | |
| *** alatiera_ has joined #buildstream | 14:33 | |
| *** alatiera has quit IRC | 14:33 | |
| *** alatiera has joined #buildstream | 14:34 | |
| *** alatiera has joined #buildstream | 14:47 | |
| *** tristan has quit IRC | 15:04 | |
| *** phildawson_ has joined #buildstream | 15:19 | |
| *** raoul has joined #buildstream | 15:19 | |
| *** tristan has joined #buildstream | 15:23 | |
| *** mieszko has joined #buildstream | 15:45 | |
| gitlab-br-bot | tristanvb merged MR !1035 (chandan/update-requirements-one-liner->master: Add Makefile to update requirements files) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1035 | 15:50 |
| *** ChanServ sets mode: +o tristan | 15:56 | |
| tristan | cs-shadow, chandan/no-sdist ? | 15:57 |
| tristan | man you are on top of it | 15:57 |
| tristan | every time I come to do something, chandan is doing it already, on fire ;-) | 15:58 |
| cs-shadow | tristan: haha :D | 15:58 |
| cs-shadow | do you have any other improvements planned? | 15:58 |
| tristan | ah you solved the nightlies with pip install -r | 15:59 |
| tristan | cs-shadow, I've been struggling a bit with coverage but my motivation is low | 16:00 |
| tristan | seeing as currently in the CI nothing is actually broken | 16:00 |
| cs-shadow | coverage has been giving me problems lately, but for slightly different reasons. According to coverage, all our external plugins have exactly 0% coverage. This is likely due to the way we launch subprocess/threads but in any case, coverage is decidedly not happy | 16:02 |
| benschubert | tristan: what are the coverage problem? Do we have a summary somewhere? | 16:03 |
| tristan | benschubert, now that we use tox to invoke tests, it can happen that multiple runs occur as a result of running `tox` (if you have multiple python versions installed) | 16:03 |
| benschubert | oh right, I know how to fix that | 16:04 |
| tristan | benschubert, the ".coverage" data file at the end of running tox is only collected from the last run | 16:04 |
| benschubert | Do you want them combined or one per run? | 16:04 |
| tristan | Some techniques I've seen included setting "envlist" to something with a :before and :after, but I didnt like that approach | 16:05 |
| tristan | because its not compatible with the freedom of executing `tox -e foo` | 16:05 |
| cs-shadow | one way would be to set coverage dir based on the name of the env, and then have a separate coverage env that combines the reports | 16:05 |
| tristan | benschubert, right now it's not a big deal because each CI run only runs one python version, and the results of separate runs are collated in another job | 16:06 |
| tristan | only an issue for the .coverage file output of a single `tox` invocation | 16:07 |
| tristan | So I tries setting `parallel = true` | 16:07 |
| tristan | which is supposed to output .coverage.${pid}.${random_number} | 16:07 |
| tristan | but since we've already told coverage about multiprocessing, turning that on does nothing | 16:08 |
| benschubert | So do we want separate coverage reports? Or do we want a single one at the end end? | 16:08 |
| tristan | We can live with separate reports | 16:08 |
| benschubert | Ok, I'll add this to my todo list :) | 16:09 |
| tristan | guess I'll file an issue for it | 16:09 |
| benschubert | thanks! | 16:09 |
| tristan | benschubert, maybe with --append we can do it ? | 16:14 |
| tristan | benschubert, but I didnt find out how to enable append mode in the config file | 16:14 |
| benschubert | That would require some care, like wiping the file before everything :) | 16:14 |
| tristan | and right now we have it automatically launched by pytest | 16:14 |
| tristan | Append would mean we couldnt use detox | 16:15 |
| tristan | I've been wanting to try that | 16:15 |
| tristan | I have python 3.[5-7] installed and it would be nice to parallel run them all | 16:15 |
| tristan | Oh | 16:16 |
| tristan | I expect coverage to fail in that regard every time | 16:16 |
| tristan | or perhaps succeed by accident... | 16:17 |
| tristan | It will litter the tree with .coverage.${pid}.${rand} files while tasks run, and collate them | 16:17 |
| tristan | (because of multiprocessing, it already does that) | 16:17 |
| tristan | and then separate runs in the same tree will pick up unrelated coverage | 16:18 |
| benschubert | we would need to setup a coverage directory for each | 16:18 |
| benschubert | mutliprocessing parallel tests are a hell in python -_-' | 16:18 |
| tristan | heh | 16:26 |
| *** tpollard has quit IRC | 16:29 | |
| gitlab-br-bot | tristanvb opened issue #844 (Coverage reports inacurate with multiple runs of `tox`) on buildstream https://gitlab.com/BuildStream/buildstream/issues/844 | 16:32 |
| tristan | benschubert, ^^ | 16:32 |
| benschubert | thansk! | 16:34 |
| gitlab-br-bot | cs-shadow closed issue #843 (Follow-up from "Introduce new "source" command group") on buildstream https://gitlab.com/BuildStream/buildstream/issues/843 | 16:35 |
| gitlab-br-bot | cs-shadow opened MR !1037 (chandan/no-sdist->master: .gitlab-ci.yml: Remove prepare stage) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1037 | 16:39 |
| valentind | What is pycairo used for? | 16:43 |
| cs-shadow | ostree needs it | 16:44 |
| valentind | That is surprising. | 16:45 |
| cs-shadow | sorry, i meant the ostree plugin and not the ostree package | 16:45 |
| valentind | Yes, I understood. | 16:45 |
| valentind | AH no | 16:46 |
| valentind | OK | 16:46 |
| valentind | cs-shadow, do you mean buildstream/plugins/sources/ostree.py depends on it? | 16:47 |
| tristan | valentind, I believe it's needed specifically by pygobject | 16:49 |
| valentind | Is it building pygobject that depends on it? | 16:50 |
| cs-shadow | yeah, we only directly depend on pygobject | 16:50 |
| tristan | exact | 16:51 |
| valentind | tox is only used for testing, right? | 16:52 |
| valentind | (and building doc) | 16:52 |
| cs-shadow | yep, testing and coverage and doc etc | 16:52 |
| tristan | cs-shadow, hmmm, coverage you know I kept thinking, - I didnt want to hold the MR on that but would it make sense to move all that report collation logic out of .gitlab-ci.yml and into tox.ini ? | 16:54 |
| tristan | Well, no worry about that now I guess | 16:55 |
| cs-shadow | tristan: I was thinking about that. To do it naively, we can just copy paste the commands in a tox environment. But, it might be better to come up with something that would work both locally and on CI, and that can handle multiple coverage dirs | 16:55 |
| cs-shadow | that's why I haven't added it just yet | 16:56 |
| tristan | exactly, I mean that is the worst part of .gitlab-ci.yml, it's real ugly | 16:56 |
| tristan | and it doesnt really belong there (we shouldn't depend too hard on the infra I think) | 16:56 |
| tristan | if we had it cleaner at the tox entry point, we can reuse it in other scenarios easier | 16:57 |
| cs-shadow | definitely agree. We'll get there slowly :) | 16:58 |
| tristan | I guess it makes sense to solve #844 first, if we're being perfectionists about coverage | 16:59 |
| gitlab-br-bot | Issue #844: Coverage reports inacurate with multiple runs of `tox` https://gitlab.com/BuildStream/buildstream/issues/844 | 16:59 |
| tristan | Oh | 16:59 |
| tristan | cs-shadow, I'm curious - will the lint and docs environments just use whatever python environments they find ? | 16:59 |
| tristan | maybe in the order they were specified in envlist ? | 17:00 |
| tristan | It happened to do that for me | 17:00 |
| cs-shadow | tristan: I am not sure about whether the order affects it or not but AFAICT tox doesn't guarantee anything about the version this way. We can set basepython for these jobs if we want to be sure | 17:02 |
| cs-shadow | https://tox.readthedocs.io/en/latest/config.html#conf-basepython | 17:02 |
| tristan | yeah but doing that means that we require specific python versions to run those | 17:03 |
| tristan | The order doesnt effect it :-( | 17:03 |
| valentind | is pycodestyle run for all files and pylint only for buildstream directory? | 17:04 |
| cs-shadow | tristan: we can say things like "python3" | 17:04 |
| cs-shadow | or simply "python" even | 17:04 |
| *** max has joined #buildstream | 17:04 | |
| tristan | I swapped it to be 3.[7,6,5], cleaned and ran `tox -e docs`, and it chose 3.5 | 17:04 |
| cs-shadow | valentind: that's accurate | 17:05 |
| valentind | cs-shadow, why have we stopped running pylint on the tests? | 17:05 |
| valentind | Or did we not do it? | 17:05 |
| tristan | valentind, I think we never did | 17:05 |
| cs-shadow | i don't think we ever ran pylint on the tests, although I don't really know why | 17:05 |
| tristan | I tried running it there and there is just a bunch of stuff to fix first | 17:06 |
| valentind | Ah ok. I thought we did. But I am probably wrong. It might have been pycodestyle that reported errors. | 17:06 |
| tristan | cs-shadow, because of the contorted setup.py -> setup.cfg config layout I think | 17:13 |
| tristan | cs-shadow, the different pytest plugins offered different ways of configuring stuff, so it didnt match up well | 17:13 |
| jonathanmaw | hmm, while looking at why my tests failed to catch that we had a regression that requires https://gitlab.com/BuildStream/buildstream/merge_requests/1036, I'm curious about the proper fetching behaviour for elements that have open workspaces | 17:24 |
| jonathanmaw | specifically, when I `bst fetch` an element that has an open workspace, that element's sources aren't fetched (though its dependencies may be fetched) | 17:25 |
| jonathanmaw | this is a weird edge-case because the only way to have an element that hasn't been fetched, but has had a workspace opened is if you opened the workspace with "--no-checkout" | 17:25 |
| tristan | jonathanmaw, interesting, I have an opinion that that is probably wrong | 17:25 |
| *** max has quit IRC | 17:31 | |
| *** raoul has quit IRC | 17:38 | |
| gitlab-br-bot | cs-shadow merged MR !1037 (chandan/no-sdist->master: .gitlab-ci.yml: Remove prepare stage) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1037 | 17:41 |
| *** max has joined #buildstream | 17:53 | |
| *** max has quit IRC | 18:20 | |
| *** mieszko has quit IRC | 18:30 | |
| *** jonathanmaw has quit IRC | 18:48 | |
| *** juergbi has quit IRC | 18:48 | |
| *** juergbi has joined #buildstream | 18:54 | |
| tristan | juergbi, I'm curious about reinstating the element guessing, https://gitlab.com/BuildStream/buildstream/merge_requests/1036 | 19:57 |
| tristan | juergbi, it appears that Jonathan originally added a test for that, does the test need to be fixed too ? | 19:58 |
| tristan | (test added from https://gitlab.com/BuildStream/buildstream/merge_requests/954) | 19:58 |
| tristan | Also... running on master... I am doing a great deal of fetches and have not built anything yet :-S | 19:59 |
| tristan | Have we regressed the build depth sort ? | 19:59 |
| * tristan should ensure there is a test for that | 19:59 | |
| *** tristan has quit IRC | 20:00 | |
| *** tristan has joined #buildstream | 20:00 | |
| *** ChanServ sets mode: +o tristan | 20:01 | |
| *** tristan has quit IRC | 20:27 | |
| *** tristan has joined #buildstream | 20:27 | |
| *** ChanServ sets mode: +o tristan | 20:27 | |
| tristan | Yeah this is a regression I can see it clearly now | 20:48 |
| tristan | I'm not sure if the sorting is related... I have (A): A constant flow of active fetch jobs (B): 3 buildable elements in the pipeline (checking with `bst show` in separate terminal) ... (C) Zero built elements.... (D) I can observe that every so often when a fetch completes, a *Pull* job is getting tried... it skips and then back to another fetch job | 20:50 |
| tristan | But we are supposed to observe eligible elements in the queues from right to left | 20:50 |
| *** tristan has quit IRC | 21:00 | |
| *** bochecha has quit IRC | 21:03 | |
| *** slaf_ has joined #buildstream | 21:08 | |
| *** slaf_ has joined #buildstream | 21:08 | |
| *** slaf_ has joined #buildstream | 21:09 | |
| *** slaf_ has joined #buildstream | 21:09 | |
| *** slaf_ has joined #buildstream | 21:09 | |
| *** slaf has quit IRC | 21:09 | |
| *** slaf_ has joined #buildstream | 21:09 | |
| *** slaf has joined #buildstream | 21:10 | |
| *** max has joined #buildstream | 21:49 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!