| *** valentind has quit IRC | 01:01 | |
| *** valentind has joined #buildstream | 01:02 | |
| *** tristan has quit IRC | 07:59 | |
| *** tristan has joined #buildstream | 08:23 | |
| gitlab-br-bot | push on buildstream@master (by Jürg Billeter): 1 commit (last: _ostree.py: Fix string formatting in fetch()) https://gitlab.com/BuildStream/buildstream/commit/b9418b495ca5b12953c97724d8c10f2ac8db5767 | 08:23 |
|---|---|---|
| tristan | juergbi, ah good, one less glitch to think about :) | 08:43 |
| * tristan thinks this is the week for 1.0 | 08:44 | |
| juergbi | is there a way we can run our tests in parallel? running them one at a time seems pretty wasteful | 09:15 |
| tristan | Not sure | 09:18 |
| tristan | could be nice | 09:18 |
| tristan | juergbi, on the other hand I kind of like that it can be ordered, and would rather strive for less randomness in test ordering | 09:19 |
| tristan | juergbi, when testing specific things, you can always do for example: ./setup.py test --addopts -k --addopts 'tests/pipeline/load.py' | 09:19 |
| tristan | during development | 09:19 |
| tristan | if you want to test a specific thing quickly | 09:19 |
| tristan | regarding randomness, what I would prefer is to test _yaml first; test lower level / simpler functionality first; so that you know that if a lot of later tests are failing; fixing the first failing ones most likely causes the others to pass again | 09:20 |
| juergbi | well, we wouldn't have to force parallelism. but if you just want to quickly check whether the changes break any tests, parallelism would be very useful | 09:22 |
| tristan | parallelization might be tricky to achieve with python and pytest anyway; especially considering the coverage reports which need to hook into parallelization techniques | 09:22 |
| tristan | nod, they are taking close to 2 minutes now | 09:23 |
| tristan | would be nice to have | 09:23 |
| juergbi | yes, and they could just be a few seconds with maximum parallelism on a fast desktop | 09:24 |
| tristan | true | 09:24 |
| juergbi | maybe we could have two different invocations | 09:24 |
| juergbi | one for just testing, as quickly as possible | 09:24 |
| juergbi | and the other for reasonable order and coverage | 09:24 |
| tristan | well, if it can be done without severely contorting the setup, I wont hate it :) | 09:25 |
| juergbi | :) | 09:25 |
| tristan | We should also have a new set testing that in the CI | 09:25 |
| juergbi | i may take a look at this but not right now | 09:25 |
| tristan | if there is a separate entry point, we risk it bitrotting otherwise | 09:25 |
| juergbi | well, i would frequently test it locally ;) | 09:25 |
| tristan | still, it costs little to add some lines to the .gitlab-ci.yml (and there would be no need for collecting the coverage for those runs either) | 09:26 |
| juergbi | we might not want to run all tests twice in the CI just to test the test runner. as i guess the CI will be CPU-limited | 09:26 |
| * tristan is thinking of a bright future where you and I can eventually work on inventing cool new stuff | 09:27 | |
| tristan | so I'm hoping to make the CI and everything as hardened as I can :D | 09:27 |
| juergbi | :) | 09:28 |
| paulsherwood | :-) | 09:29 |
| *** bochecha has joined #buildstream | 09:30 | |
| *** jude has joined #buildstream | 09:32 | |
| juergbi | tristan: unrelated syntax question. i added variable substitution for junction/subproject options last week with the usual %{...} syntax. my implementation was actually broken and it's a bit less straight forward to handle than i expected | 09:33 |
| juergbi | i'm now wondering whether it even makes sense to use that syntax as options are not arbitrary strings | 09:33 |
| juergbi | and it's only usable if you export the top-level project options as variables | 09:34 |
| juergbi | it might make more sense to just support a convenient way to directly inherit a project option | 09:34 |
| juergbi | however, not sure what syntax to use for that | 09:34 |
| juergbi | theoretically, we could interpret an option name without value in the junction options dict as 'inherit option with the same name from parent' but it seems rather odd | 09:36 |
| tristan | hrmm | 09:40 |
| tristan | juergbi, my instinct says that your toplevel project options expose conditionals in that toplevel project; which in turn can be used to conditionalize variable values | 09:41 |
| tristan | and that variable values can be used in junction configurations to assign project options to subprojects | 09:41 |
| tristan | yes, stings vs multitype... but... meh ? | 09:41 |
| tristan | Also, do you need to have a separate variable resolution algo for that ? or can the junction element not just call self.node_subst_member() in the regular way ? | 09:42 |
| tristan | I guess what this all boils down to is just, ability to create non-string values ? | 09:43 |
| tristan | juergbi, maybe all that is required here is for the junction element to perform some string->bool conversions when feeding the options to the sub project ? | 09:43 |
| juergbi | tristan: if you think the %{...} still makes sense even though we don't really need that flexibility, i'll fix that implementation | 09:46 |
| juergbi | as it's not completely straight forward, i wanted to check whether this is sensible before refactoring related stuff | 09:46 |
| juergbi | (each option type has their own loading implementation which can't directly be combined with Element's subst mehtods) | 09:47 |
| tristan | juergbi, right so; maybe we need to relax the loading implementation of options ? | 09:52 |
| tristan | juergbi, the reason I think it's valid to allow variable expansion is more for consistency of the API than anything else | 09:53 |
| tristan | juergbi, currently anyway; the options allow loading from strings for the sake of the command line too | 09:53 |
| tristan | so, *maybe* (not sure ?) we can even reduce the options code paths by relaxing the loading mechanism a bit | 09:53 |
| * tristan notes that he doesnt expect to ever have a need for creating options for complex types like lists or dictionaries | 09:54 | |
| tristan | conditionals for those would be weird anyway | 09:55 |
| juergbi | not sure what you mean with relaxing in this context | 09:55 |
| tristan | juergbi, well; the option classes each have 2 codepaths for loading a value | 09:55 |
| juergbi | yes, yaml and string | 09:55 |
| tristan | one from yaml and one from a string provided by the command line | 09:55 |
| tristan | so they could just use the string method | 09:55 |
| tristan | and do their assertions based on the string | 09:56 |
| tristan | it's "relaxing" because we dont use the additional type check from _yaml.node_get() | 09:56 |
| tristan | ensuring that say, a boolean was specified as a valid yaml boolean | 09:56 |
| juergbi | right, that'a possibility | 09:56 |
| *** jonathanmaw has joined #buildstream | 09:59 | |
| *** bethw has joined #buildstream | 10:25 | |
| *** WSalmon has joined #buildstream | 10:27 | |
| *** jude has quit IRC | 10:33 | |
| *** WSalmon has quit IRC | 10:38 | |
| tristan | tlater, I need to add an integration test | 10:51 |
| tristan | tlater, and I'm confused | 10:51 |
| tristan | tlater, is it one test per directory ? | 10:51 |
| tristan | no, cant be | 10:51 |
| tristan | but the run scripts dont enumerate the test that they run | 10:52 |
| tristan | tlater, it's based on the element names in the element subdirectories ? | 10:52 |
| * tristan wishes this was all just a lot more verbose and explicit, and not functional and automated and hence, difficult to understand | 10:53 | |
| tlater | tristan: I think so, if I recall correctly | 10:53 |
| gitlab-br-bot | buildstream: Jonathan Maw created branch 132-loading-external-plugins-works-without-explicit-requirement-in-project-conf | 11:00 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 11:00 |
| *** valentind has quit IRC | 11:02 | |
| jonathanmaw | tristan: the loading of external plugins is currently blocked because when the tests run with coverage it segfaults. tlater had a similar problem. | 11:02 |
| tristan | jonathanmaw, I backed out a change I merged this weekend due to a similar problem | 11:06 |
| tristan | seems it was consistently failing in the cmake test | 11:07 |
| tristan | jonathanmaw, if it helps you to track it down; this patch was causing it to trigger: https://gitlab.com/BuildStream/buildstream/merge_requests/151 | 11:08 |
| jonathanmaw | tristan: hrm, that's nowhere near where my changes were :/ | 11:09 |
| tristan | thats nowhere near *anything* I would expect to have that side effect | 11:09 |
| tristan | but it did very consistently cause the cmake test to fail, with a segfault from coverage | 11:09 |
| tristan | so, perhaps it's a good lead to figuring out the real underlying cause | 11:10 |
| jonathanmaw | tristan: it was the cmake test where it failed for me, too | 11:11 |
| jonathanmaw | I wonder what's special about that... | 11:11 |
| tristan | yup, you and me both | 11:11 |
| *** jude has joined #buildstream | 11:12 | |
| tristan | hrm... this is disturbing | 11:14 |
| tristan | looks like `./setup.py test` is failing sporadically while running `./run-test.sh run <test-case>` in parallel | 11:14 |
| tristan | in the integration tests | 11:14 |
| tristan | I'm getting "BUG: Message handling out of sync, unable to retrieve failure message for element import element at workspace-test-tar.bst [line 1 column 0]" | 11:15 |
| tristan | in random places | 11:15 |
| tristan | Think maybe the message queue implementation tries to create a socket name based on the program name or something horrible like that ? | 11:15 |
| tristan | i.e. Queue() from multiprocessing | 11:16 |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 1 commit (last: Test whether removing whitelist checking is enough) https://gitlab.com/BuildStream/buildstream/commit/a54352c1e9d4ddcea0d033b3ed0bd7a077a7d546 | 11:19 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 11:19 |
| gitlab-br-bot | buildstream: merge request (workspace-build-count->master: WIP: Skip configure-commands when building workspaced elements a second time) #154 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/154 | 11:25 |
| *** tlater has left #buildstream | 11:27 | |
| *** tlater has joined #buildstream | 11:27 | |
| tristan | crap | 11:33 |
| * tristan not happy with integration tests | 11:33 | |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 11 commits (last: main.py: Fix app initialization) https://gitlab.com/BuildStream/buildstream/commit/1909c686f5bba4e6df398d4a7399834f3f6be503 | 11:40 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 11:40 |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 1 commit (last: Test whether not passing the whielist makes tests pass) https://gitlab.com/BuildStream/buildstream/commit/e328988a0c8d7037681ca659fcf76bc8c255d1d0 | 11:45 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 11:45 |
| *** bethw has quit IRC | 11:54 | |
| *** bethw has joined #buildstream | 11:54 | |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 14 commits (last: _yaml.py: Remove extra brackets in Provenance class) https://gitlab.com/BuildStream/buildstream/commit/44b00de090348f94d27998e5378c1beef0db6296 | 12:00 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 12:00 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 8 commits (last: load.py: Add test to check intersection exceptions) https://gitlab.com/BuildStream/buildstream/commit/dc3169ec201db997574025963736c85aea71befc | 12:07 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 12:07 |
| gitlab-br-bot | push on buildstream@compose-integration-delete (by Tristan Van Berkom): 4 commits (last: Handle removed files from integration in compose plugin) https://gitlab.com/BuildStream/buildstream/commit/cecbbec7ae5a9e6f9021bb087427c41cdb2bd633 | 12:11 |
| gitlab-br-bot | buildstream: merge request (compose-integration-delete->master: Compose integration delete) #156 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/156 | 12:14 |
| tristan | grrr, ok so that out of sync message thing seems to be happening at random places regardless of concurrently running things | 12:19 |
| tristan | nobody else seeing this error in `./setup.py test` ? | 12:20 |
| tristan | maybe it's only happening in `bst push` tests | 12:22 |
| tristan | looks like it | 12:22 |
| * tristan wonders if it's another case of what juergbi fixed this morning | 12:23 | |
| * tlater doesn't currently | 12:25 | |
| gitlab-br-bot | push on buildstream@compose-integration-delete (by Tristan Van Berkom): 1 commit (last: integration-tests: Adding a compose test ensuring removed files in integration are removed) https://gitlab.com/BuildStream/buildstream/commit/82291261e6630b46c009378804a6703471f2b42c | 12:29 |
| gitlab-br-bot | buildstream: merge request (compose-integration-delete->master: Compose integration delete) #156 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/156 | 12:29 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 10 commits (last: tests/pipeline/load.py: Enhanced and added test cases for exceptions) https://gitlab.com/BuildStream/buildstream/commit/0214e4c5be01412466c19be83e6b97bc0a441f26 | 12:30 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 12:30 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 1 commit (last: Fix tests) https://gitlab.com/BuildStream/buildstream/commit/4f8a116c0b5021f92a3a77c5321aa7a802e8783a | 12:31 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 12:31 |
| tristan | I think it's related to the coverage crashes | 12:37 |
| tristan | I just ran one which just did "Segmentation Fault" at some point | 12:37 |
| tristan | and no report, just crash | 12:37 |
| tristan | if a child process crashes without getting the exception back to the parent, that would cause the out of sync thing to happen | 12:38 |
| *** tristan has quit IRC | 12:40 | |
| *** tristan has joined #buildstream | 12:42 | |
| tristan | wat | 12:44 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/jobs/41101537 | 12:45 |
| tristan | tlater, how come that is happening ? | 12:45 |
| tristan | tar source at dependencies/amhello-full.bst [line 9 column 2]: Error mirroring file:///home/tristanmaat/Documents/Projects/buildstream/buildstream-tests/compose-test/src/amhello.tar.gz: <urlopen error [Errno 2] No such file or directory: '/home/tristanmaat/Documents/Projects/buildstream/buildstream-tests/compose-test/src/amhello.tar.gz'> | 12:45 |
| *** tristan has quit IRC | 12:48 | |
| gitlab-br-bot | push on buildstream@compose-integration-delete (by Tristan Van Berkom): 1 commit (last: integration-tests: Adding a compose test ensuring removed files in integration are removed) https://gitlab.com/BuildStream/buildstream/commit/81203b63e0cdbe3ed0b0696168a79764816b047f | 12:50 |
| gitlab-br-bot | buildstream: merge request (compose-integration-delete->master: Compose integration delete) #156 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/156 | 12:51 |
| *** tristan has joined #buildstream | 12:51 | |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 1 commit (last: Test removing get_expected_plugins) https://gitlab.com/BuildStream/buildstream/commit/3ac2d3583568287cfb43e67462bfad33d0d48dfd | 12:51 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 12:51 |
| * tristan tried a reboot... maybe my system has become prone to these crashes | 12:55 | |
| tristan | nope, one successful run, second run had a failure, lets see | 12:56 |
| tristan | we are not out of the woods | 12:56 |
| *** tristan has quit IRC | 13:16 | |
| jonathanmaw | erk, there's no changes left for me to pare down in search of the cause of coverage crashing - the only changes I have left are changing function signatures. | 13:24 |
| tlater | jonathanmaw: Changing function signatures may well be it | 13:30 |
| tlater | Sorry, was out for lunch for a bit | 13:30 |
| jonathanmaw | hmm, I'll juggle things around and see if a method that sets the whitelist will work | 13:30 |
| * tlater wonders what trips up the url for that tar | 13:33 | |
| tlater | Ah, tristan looks to have already fixed it | 13:34 |
| tlater | Ack | 13:37 |
| tlater | I have a segmentation fault again | 13:37 |
| *** tristan has joined #buildstream | 13:38 | |
| tlater | tristan: Do you get those crashes in a docker container? | 13:39 |
| tlater | I haven't seen a segmentation fault outside of sam's image | 13:39 |
| tristan | I'm getting mixed results | 13:45 |
| tristan | I see it only while tracking or while interacting with a remote artifact cache (push/pull) | 13:45 |
| tristan | I got one time a "segfault" | 13:45 |
| tristan | Many times I got the out of sync message, which could be a segfault in a child thread | 13:46 |
| tristan | And two times I got a hang in the push_pull_all test | 13:46 |
| tristan | I tried to check if it's due to the branch I'm working on.... | 13:46 |
| tristan | but I've gotten hangs from master too | 13:47 |
| tlater | Hm, I don't have an artifact cache server set up, do the tests require that? | 13:47 |
| tristan | my branch is for #147, based on valentind's branch but adds integration tests | 13:47 |
| tristan | tlater, the tests simulate it by creating a local one | 13:48 |
| * tlater attempts this on his machine | 13:48 | |
| tristan | tlater, and the artifact cache code supports a local artifact share with file:// uris, with slightly different code paths | 13:48 |
| tristan | so actually, the code which my branch touches is not touched at all by the cases I've been seeing | 13:49 |
| tristan | still; I have a sneaking suspicion it might be due to the formatting changes | 13:49 |
| tristan | which have done things like {:d} instead of just raw {} | 13:49 |
| tristan | not sure what the idea behind attempting to format explicit types was, but it looked like a no-brainer merge so I merged that | 13:50 |
| tlater | I noted a few hiccups after that merge too, it was an odd one. Testing with an older branch the push/pull works fine | 13:51 |
| * tlater hopes that pytest caches buildstream so I can check out master without my other run breaking down | 13:51 | |
| tlater | tristan: It doesn't seem like the test hangs on my machine... | 13:52 |
| tristan | I dont understand about pytest caching buildstream | 13:53 |
| tristan | tlater, push_pull_all ? | 13:53 |
| tristan | oh | 13:53 |
| tristan | doesn't | 13:53 |
| tlater | tristan: Ah, might be running the wrong test | 13:53 |
| tristan | yeah; I dont know what's up | 13:53 |
| tlater | Aww | 13:53 |
| tristan | I've seen it mostly happen in the frontend tests | 13:53 |
| tristan | but push_pull_all is the only one I've seen produce a hang (twice) | 13:53 |
| tlater | tristan: What file is that in? | 13:54 |
| tlater | Ah, nevermind, found it | 13:54 |
| gitlab-br-bot | buildstream: issue #147 ("Compose element loses symlink modifications from integration commands") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/147 | 13:55 |
| gitlab-br-bot | buildstream: issue #147 ("Compose element loses symlink modifications from integration commands") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/147 | 13:55 |
| * tristan thinks that tests/sandboxes/mounting should be an integration test exclusively | 13:55 | |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 4 commits (last: Handle removed files from integration in compose plugin) https://gitlab.com/BuildStream/buildstream/commit/cecbbec7ae5a9e6f9021bb087427c41cdb2bd633 | 13:55 |
| gitlab-br-bot | buildstream: merge request (compose-integration-delete->master: Compose integration delete) #156 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/156 | 13:55 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch compose-integration-delete | 13:55 |
| gitlab-br-bot | buildstream: merge request (compose-integration-delete->master: Handle removed files from integration in compose plugin) #142 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/142 | 13:57 |
| * tristan notes that he is regularly running `./setup.py test` without buildstream installed | 13:58 | |
| tristan | I wonder also if this could be related to coverage, and if there might be a regression in upstream coverage causing us grief | 13:59 |
| tlater | tristan: I have run `./setup.py test --addopts k --addopts 'tests/frontend/pull.py'` with the current master a few times now, I don't seem to get any hangs | 14:00 |
| tristan | hmmm, well ... would be nice if this one just disappeared of it's own accord I guess | 14:01 |
| * tristan rebooted and hasnt had the out of sync message problem since then | 14:01 | |
| tristan | but did see one hang in push_pull_all | 14:02 |
| tlater | Might have been an oddity in network setup things? | 14:02 |
| tlater | jonathanmaw: Any luck in the meantime btw? It looks like the pycoverage maintainer wants a minimal test case, unsurprisingly. | 14:04 |
| tristan | possibly | 14:04 |
| tristan | nedbat, he's one of the good python developers | 14:05 |
| tristan | coverage maintainer | 14:05 |
| jonathanmaw | tlater: no luck yet | 14:05 |
| tristan | tlater, anyway the except branch went pretty well, I would have looked deeper into your other branch but rebase didnt work out so well after the other tests I added | 14:06 |
| tlater | tristan: I fixed that :) Unfortunately I got a segfault again after... | 14:07 |
| * tlater liked the changes you made to the tests though | 14:07 | |
| * tristan notes out loud that the next thing he would work on if he had the freedom to just hack instead of an accumulating todo list; would be to refactor the integration tests more | 14:08 | |
| tristan | probably convert to pytest | 14:08 |
| tristan | fix the base runtime assumptions | 14:08 |
| tristan | Sam is not here... but if he were I would tell him that we now have Element.compute_manifest() which unblocks his work on another branch | 14:09 |
| tristan | test_push_pull hanging again; ps axf tells me: [python3] <defunct> | 14:10 |
| tristan | one zombie | 14:10 |
| tristan | looks like I have the machine that reproduces this | 14:10 |
| *** WSalmon__ has quit IRC | 14:19 | |
| *** WSalmon__ has joined #buildstream | 14:19 | |
| tristan | there it is again | 14:23 |
| tristan | ok, dont think I can fix tonight, but I'll give it a shot after midnight pizza | 14:23 |
| *** mcatanzaro has joined #buildstream | 14:32 | |
| *** ecruzreyes has joined #buildstream | 14:35 | |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 4 commits (last: Revert "Test removing get_expected_plugins") https://gitlab.com/BuildStream/buildstream/commit/297894490e848e938db21d0a233d07f234387718 | 14:35 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 14:35 |
| *** ecruzreyes has quit IRC | 14:38 | |
| jonathanmaw | :tableflip: reworked my code to not change the function signature, but instead to have a separate setter that's called | 15:04 |
| jonathanmaw | still segfaulting in cmake-test | 15:04 |
| jonathanmaw | so I can conclude it's no individual part that's causing segfaults | 15:04 |
| jonathanmaw | so I'm going to revert everything, see if that still fails, and if not, find the minimal change that'll cause it | 15:05 |
| tlater | :| | 15:06 |
| * tlater also resorted to that last week | 15:06 | |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 3 commits (last: Revert "Test an implementation that doesn't affect function signatures") https://gitlab.com/BuildStream/buildstream/commit/fdbcd5aa699c64ba2f2da3f8c0f39cd9f0439b97 | 15:06 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 15:06 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 1 commit (last: Only run the cmake test) https://gitlab.com/BuildStream/buildstream/commit/0e800e0954debe194dbd213ea7c0e3662da5c65f | 15:12 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 15:12 |
| tristan | well, I'm often getting both the hang and the out-of-sync error message thing | 15:14 |
| tristan | both before and after my branch merge | 15:14 |
| tristan | so I can at least conclude that I did not introduce it (even though it only started happening to me today it seems) | 15:14 |
| tristan | eek, and we're raising OSError in utils.py | 15:20 |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 1 commit (last: Test minimal change to make the CI complain) https://gitlab.com/BuildStream/buildstream/commit/e80de2df71599de69154499fb027f7f03a7f76cf | 15:29 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 15:29 |
| *** jude has quit IRC | 15:31 | |
| gitlab-br-bot | push on buildstream@another-segfault (by Tristan Maat): 1 commit (last: Only run the cmake test) https://gitlab.com/BuildStream/buildstream/commit/1eb7f616f0a1fcaaa64d8699a22fee8db5af15f2 | 15:37 |
| tlater | Oh, whoops | 15:37 |
| tlater | That was supposed to be on my private repo | 15:37 |
| *** WSalmon__ has quit IRC | 15:39 | |
| tristan | tlater, theres no problem with pushing to main repo | 15:42 |
| tristan | we only wish the gitlab bot would shut up about pushes to non-protected branches ;-) | 15:42 |
| tristan | maybe at some point we'll want to do some house cleaning of old stale work branches | 15:43 |
| * tlater hates the spam so avoids it when possible x) | 15:53 | |
| tristan | ya know, I'm not totally against just disabling the `git push` notifications until we can reduce it to only notify protected branches | 15:54 |
| tristan | I think I mentioned that | 15:54 |
| tristan | just that I would *prefer* to keep the notifications for pushes to protected branches | 15:55 |
| tristan | I.e., for the time being if everyone's happier without them, until we can tune it better; lets just disable them | 15:55 |
| tristan | ooooo strange | 15:56 |
| tristan | ah, nope; false alarm | 15:57 |
| tristan | didnt track it down to the commit introducing this | 15:57 |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 1 commit (last: Test minimal change to make the CI complain) https://gitlab.com/BuildStream/buildstream/commit/2427d315f15f6c99d1b1c2d9fc926c922e7d04f5 | 16:03 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 16:03 |
| * tristan is starting to see a trend | 16:08 | |
| * tlater just executed an event hook :) | 16:08 | |
| tristan | tlater, try this if you can... `./setup.py test`... and wait till you get around the _yaml tests at the end, then interrupt with ^C | 16:09 |
| tristan | Then, check to ensure that you have a metric ton of .coverage.hostname.pid files lying around | 16:09 |
| tristan | tlater, you should have > 300 files | 16:10 |
| tristan | I *think* the hangs are a problem with coverage not dealing well with stale data lying around from previous runs | 16:10 |
| tristan | ah, maybe not that many | 16:10 |
| tristan | looks like ~80 is enough | 16:11 |
| * tristan will run the cycle a bunch of times to verify | 16:11 | |
| * tlater is waiting... | 16:12 | |
| * tristan thinks he is on the wrong path | 16:13 | |
| tristan | tlater, sorry for the noise :-/ | 16:13 |
| tlater | Hmm, I don't even see the metric ton of tempfiles | 16:14 |
| * tristan was bisecting back to old revisions and bug still happens, even since last week when we landed the pep thing by jonathanmaw | 16:14 | |
| tristan | even since before that it reproduces | 16:14 |
| tristan | then suddenly I removed the .coverage files, and it stopped happening | 16:14 |
| tristan | came back to master, still not happening, but happened again; only when there is a .coverage output file | 16:15 |
| tristan | but, it doesnt yet prove that that is what is to blame | 16:15 |
| tlater | Well, it was worth a try :) | 16:15 |
| tristan | I will instead try many runs with the .coverage files removed first | 16:15 |
| tristan | and see if it never happens | 16:16 |
| tristan | if this is the case; it might mean we need per-bst-invocation coverage collection in the integration-tests | 16:16 |
| tristan | ok so, uninstalled coverage 4.4.2 from ~/.local | 16:32 |
| tristan | and I have older 4.2 installed in /usr/lib/python3.5/dist-packages | 16:32 |
| tristan | so far so good | 16:33 |
| * tlater hasn't attempted the segfault branches with old pycoverages either | 16:37 | |
| tlater | It would explain why we can't reproduce this locally | 16:37 |
| tristan | using 4.4.2 without any installed, also reproduces... | 16:38 |
| tristan | Now trying it with tests_require pinning it to == 4.2 | 16:38 |
| tristan | looks promising | 16:38 |
| * tlater wonders if he can script docker to try it with all kinds of versions | 16:39 | |
| tristan | I think we need to look at the git log of coverage between 4.2 and 4.4.2 | 16:40 |
| tristan | possibly something related to multiprocessing | 16:40 |
| * tlater checks if his current segfault is fixed by switching to 4.2 | 16:41 | |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 2 commits (last: Removing unneeded str() statements in "{}".format() statements) https://gitlab.com/BuildStream/buildstream/commit/1f39c63ca8f18291b43ad0dd0a007e3951b0557d | 16:45 |
| * tristan expects that might fail | 16:47 | |
| * tristan filed https://gitlab.com/BuildStream/buildstream-docker-images/issues/1 | 16:47 | |
| tristan | tlater, in the absence of sam, do you have any idea how to move the latest tag on the docker images ? | 16:51 |
| gitlab-br-bot | push on buildstream@132-loading-external-plugins-works-without-explicit-requirement-in-project-conf (by Jonathan Maw): 1 commit (last: Require a different version of coverage) https://gitlab.com/BuildStream/buildstream/commit/fa250a4832ab4cbfa40558b972dc6d8443616e3e | 16:52 |
| gitlab-br-bot | buildstream: merge request (132-loading-external-plugins-works-without-explicit-requirement-in-project-conf->master: WIP: Resolve "Loading external plugins works without explicit requirement in project.conf") #125 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 16:52 |
| tristan | jonathanmaw, same ? | 16:52 |
| jonathanmaw | tristan: hrm, not sure. I'll have a look. | 16:53 |
| tristan | jonathanmaw, https://gitlab.com/BuildStream/buildstream-docker-images/merge_requests/6 will *probably* work around that cmake issue | 16:53 |
| tristan | then maybe I can unrevert that patch I reverted | 16:53 |
| tlater | tristan: Move the latest tag to what? | 16:53 |
| tristan | tlater, there is this thing called a "docker repository" | 16:53 |
| tristan | at least | 16:53 |
| tlater | I think it was supposed to update automatically, didn't sam set up the CI to do that? | 16:53 |
| tristan | I think thats what people who know anything about docker, call it | 16:53 |
| tlater | The docker hub thing :) | 16:54 |
| * tlater knows that doing it wrong can cause issues, so is not too comfortable saying anything | 16:54 | |
| tristan | tlater, I accepted that we use the "latest" tag from the buildstream repository CI, because of the promise that updating that latest tag was always going to be an explicit activity | 16:54 |
| tristan | I'm not sure that that activity is "merging to master" in the buildstream-docker-images repo | 16:54 |
| tristan | it might be | 16:54 |
| tristan | gitlab ci yaml seems to indicate it might be yes | 16:55 |
| * tlater thinks so too | 16:57 | |
| tristan | it's on the way... needs to execute again once it hits master | 16:59 |
| tlater | That will take a while, unfortunately :/ | 17:00 |
| tristan | will it ? | 17:00 |
| tristan | https://gitlab.com/BuildStream/buildstream-docker-images/-/jobs/41145883 | 17:00 |
| tristan | first run didnt take that long | 17:00 |
| tristan | now it's blocking on creating the cache or smth | 17:00 |
| tlater | Damn, that's a lot faster than I expected | 17:01 |
| tristan | only 221161 files | 17:01 |
| tristan | peanuts :) | 17:01 |
| tlater | Docker tends to take its sweet time on my private machines, but I guess it's faster if you keep all intermediate containers | 17:01 |
| tristan | as expected, buildstream pipeline fails for now | 17:05 |
| tristan | it's uploading... ok so; things should come back to life I guess in less than an hour | 17:06 |
| * tlater still can't confirm this fixes the segfault T.T | 17:06 | |
| tristan | or sooner, it's merged and we're on the way: https://gitlab.com/BuildStream/buildstream-docker-images/-/jobs/41149343 | 17:08 |
| tristan | tlater, because you only reproduce it in a docker image ? | 17:09 |
| tlater | tristan: Because I keep screwing up the pip remove line in the .gitlab-ci.yml x) | 17:11 |
| tristan | anyway, soon the docker image will be with coverage 4.2 | 17:12 |
| tristan | funny, seeing as the docker image is created with fedora, you would think that *somebody* knows the package names needed for the dnf install line | 17:12 |
| tristan | yet, we *still* have no patch for fedora instructions at: http://buildstream.gitlab.io/buildstream/install.html#installing ?? | 17:12 |
| * tristan mystified | 17:12 | |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: doc/source/install.rst: Added instructions to install on Fedora) https://gitlab.com/BuildStream/buildstream/commit/380dff80ff2a6d79cf8d656a204b35a56abfdae5 | 17:20 |
| * tristan adds the dnf install line and crosses fingers that it's the right one | 17:21 | |
| tristan | at least people will complain with "Hey that's missing the package for python3 !" instead of "Hey how come you have no install instructions for Fedora !" | 17:22 |
| tlater | "Hey how come you have no install instructions for Gentoo !" | 17:23 |
| * tlater will just wait until the image updates naturally, it's too hard to get pip to uninstall something in CI | 17:24 | |
| tristan | Well, gentoo is another beast | 17:31 |
| tristan | the only important ones are really debian and fedora | 17:31 |
| tristan | we have this strange thing called arch... just because jjardon[m] :) | 17:31 |
| *** valentind has joined #buildstream | 17:32 | |
| jonathanmaw | hrm, the new docker image failed to push https://gitlab.com/BuildStream/buildstream-docker-images/-/jobs/41149344 | 17:32 |
| tristan | jonathanmaw, I made a retry | 17:32 |
| tristan | but it looks a bit doomed without sam indeed | 17:32 |
| jonathanmaw | fingers crossed | 17:32 |
| jonathanmaw | googling around the problem, "tag does not exist" happens when the image doesn't exist locally | 17:32 |
| tristan | jonathanmaw, maybe it's because I actually did a merge request from a separate branch | 17:33 |
| tristan | maybe the thing has to properly run directly from master only | 17:33 |
| tristan | or, maybe the cache wasnt in sync | 17:34 |
| tristan | Dunny | 17:34 |
| tristan | Dunno | 17:34 |
| tristan | jonathanmaw, I also just created https://gitlab.com/BuildStream/buildstream-docker-images/pipelines/14160724 to spam it for good measure | 17:35 |
| tristan | hopefully it'll get hammered through | 17:35 |
| benbrown | tristan: seems like the buildstream repo was pushed between the build and push job | 17:41 |
| benbrown | the CI fetches latest buildstream to form the tag | 17:41 |
| tristan | oh weird | 17:49 |
| tristan | why would it do that ? | 17:49 |
| tristan | worth filing a bug | 17:50 |
| tristan | benbrown, you sure about that ? | 17:50 |
| benbrown | tristan: Yeah. I assume it's done to create a unique tag. But it should only be done once (during build), then passed on to the push stage. | 17:50 |
| benbrown | Otherwise this happens :) | 17:51 |
| tristan | benbrown, i.e.; the buildstream-docker-images basically fails to push a latest tag, if even the buildstream project master gets updated inbetween the buildstream-docker-images build & push tasks ? | 17:51 |
| tristan | ok | 17:51 |
| * benbrown nods | 17:51 | |
| tristan | https://gitlab.com/BuildStream/buildstream-docker-images/issues/2 | 17:55 |
| * tristan thinks there is an automatic git sha for the docker images repo that is already exported as a variable for gitlab CI | 17:58 | |
| tristan | which seems like a more suitable "unique thingy" | 17:58 |
| *** bethw has quit IRC | 18:01 | |
| tristan | and we're back! | 18:01 |
| tristan | tlater, now you can run pipelines again :) | 18:02 |
| gitlab-br-bot | push on buildstream@130-interactive-prompt-prefix (by Tristan Van Berkom): 1 commit (last: Accept the first character as shortcut on interruption prompts) https://gitlab.com/BuildStream/buildstream/commit/22d804df6e5d08fb84be590da173164171365374 | 18:38 |
| tristan | benbrown, thanks for the patch ! | 18:55 |
| tristan | benbrown, I have a theory that the tag format may however be wrong | 18:55 |
| tristan | I think that maybe a random seed or such might be better; the point being: I *think* the idea here is that manually creating a pipeline creates a new docker image; which *might* be desirable regardless of whether the buildstream or buildstrea-docker-images repo has changed or not | 18:56 |
| tristan | i.e. the process fetches the latest from fedora | 18:57 |
| tristan | anyway; I think we leave this for sam to look at when he resurfaces | 18:57 |
| *** mcatanzaro has quit IRC | 19:30 | |
| *** bethw has joined #buildstream | 20:07 | |
| valentind | GNU ed is distributed as tar lzip file. But python tarfile does not support it. | 20:34 |
| valentind | That is a little bit annoying. | 20:38 |
| valentind | I am wondering what is the best way to work around that. | 20:38 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Accept the first character as shortcut on interruption prompts) https://gitlab.com/BuildStream/buildstream/commit/22d804df6e5d08fb84be590da173164171365374 | 20:58 |
| gitlab-br-bot | buildstream: issue #130 ("Interactive prompts requiring full words is annoying") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/130 | 20:58 |
| gitlab-br-bot | buildstream: merge request (130-interactive-prompt-prefix->master: Accept the first characters as shortcut on interruption prompts) #151 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/151 | 20:59 |
| *** mcatanzaro has joined #buildstream | 21:03 | |
| *** tristan has quit IRC | 21:21 | |
| gitlab-br-bot | buildstream: issue #157 ("Missing accept header in http resquests leads to "HTTP Error 406: Not Acceptable"") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/157 | 21:47 |
| gitlab-br-bot | buildstream: issue #158 ("Support for lzip is missing") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/158 | 21:50 |
| *** bethw has quit IRC | 23:38 | |
| *** valentind has quit IRC | 23:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!