| *** tristan has quit IRC | 02:21 | |
| *** bochecha has quit IRC | 02:44 | |
| *** bochecha has joined #buildstream | 02:45 | |
| *** bochecha has quit IRC | 02:49 | |
| *** bochecha has joined #buildstream | 02:49 | |
| *** tristan has joined #buildstream | 02:50 | |
| *** jude has joined #buildstream | 07:42 | |
| *** jude has quit IRC | 08:31 | |
| *** jude has joined #buildstream | 08:35 | |
| *** bochecha has quit IRC | 08:39 | |
| *** bochecha has joined #buildstream | 08:40 | |
| *** bochecha has quit IRC | 08:44 | |
| *** bochecha has joined #buildstream | 08:44 | |
| *** jonathanmaw has joined #buildstream | 08:46 | |
| *** tlater has joined #buildstream | 08:58 | |
| *** ssam2 has joined #buildstream | 09:11 | |
| *** jonathanmaw has quit IRC | 09:55 | |
| *** jonathanmaw has joined #buildstream | 09:56 | |
| *** ssam2 has quit IRC | 10:01 | |
| *** jonathanmaw has quit IRC | 10:24 | |
| *** ssam2 has joined #buildstream | 10:24 | |
| *** jonathanmaw has joined #buildstream | 10:25 | |
| gitlab-br-bot | push on buildstream@options-refactor (by Tristan Van Berkom): 15 commits (last: project.py: Support project options) https://gitlab.com/BuildStream/buildstream/commit/0df3d75e9de524b8bc8165b2055b4008724064c1 | 10:30 |
|---|---|---|
| tristan | Hmmmm | 10:35 |
| tristan | I think there is no chance of breakage here | 10:36 |
| tristan | ssam2, juergbi ... I'm literally about to land project options | 10:37 |
| tristan | Sigh... deep breath ? | 10:37 |
| ssam2 | what could possibly go wrong | 10:37 |
| juergbi | hehe | 10:37 |
| tristan | :) | 10:37 |
| tristan | Yeah... the one thing is; I didnt do the (<) (=) and (>) thing yet | 10:38 |
| *** jude has quit IRC | 10:38 | |
| ssam2 | so you can only append stuff to dicts ? | 10:38 |
| tristan | So composition resulting from project option conditionals will just use whatever buildstream decides | 10:38 |
| tristan | ssam2, to arrays yes | 10:38 |
| ssam2 | oh yeah. dicts are unordered of course | 10:38 |
| tristan | But, I suppose the worst that can happen, is that people use options extensively this week | 10:38 |
| tristan | before implementing declarative list composition | 10:39 |
| tristan | and have expectations that dont match | 10:39 |
| ssam2 | probably less of a big deal than the arch conditionals breakage that is about to ensue :-) | 10:39 |
| tristan | variants are anyway not in use - they go away with this branch (which is actually split into 2 branches) | 10:39 |
| tristan | Well, regarding that; I actually have 3 branches | 10:40 |
| tristan | - remove-variants | 10:40 |
| tristan | - options-refactor (adds options) | 10:40 |
| tristan | - remove-arches | 10:40 |
| tristan | My plan is to land the first 2 now | 10:40 |
| tristan | And we migrate everything over to use the new arch conditionals | 10:40 |
| tristan | then remove the 'arches' stuff | 10:40 |
| ssam2 | ah ok | 10:41 |
| ssam2 | that seems a better plan | 10:41 |
| juergbi | makes sense | 10:42 |
| tristan | Alrighty, lets push the button on this | 10:42 |
| tristan | Unfortunately the CI is taking ages for the unix platform | 10:43 |
| tristan | So, I will have to coax it and retry... otherwise the documentation wont update | 10:43 |
| * tristan did HUGE documentation refactor today | 10:43 | |
| tristan | complete restructure - now much more coherent, broken down into Installing, Running, Project Format, Built-in Plugins, Writing custom Plugins | 10:44 |
| ssam2 | nice | 10:44 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 14 commits (last: userconfig.py: Default element format doesnt include %{variant} anymore) https://gitlab.com/BuildStream/buildstream/commit/dfa5186e77a74c928ce4ab679c532e442ac33d43 | 10:45 |
| tristan | Here comes the first | 10:45 |
| tristan | remove-variants branch is landed | 10:45 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 20 commits (last: _yaml.py: Dont blindly coerse strings to bool in _yaml.node_get()) https://gitlab.com/BuildStream/buildstream/commit/1405bbb9018550223f8f75ae705afe2a160bc42b | 10:46 |
| tristan | here comes options-refactor | 10:46 |
| * tristan makes CI only run on the tip | 10:46 | |
| tristan | If you want a preview of the docs, just can build them | 10:47 |
| tristan | There are 5 option types | 10:47 |
| tristan | - bool | 10:47 |
| tristan | - enum | 10:47 |
| tristan | - flags | 10:47 |
| tristan | - arch (derived from bool, convenience thingy) | 10:47 |
| tristan | - element-mask (derived from flags, convenience thingy) | 10:48 |
| tristan | ssam2, one thing I tried to figure out today but didnt succeed... is how you do things like (('literal1', 'literal2') & flags_variable) | 10:48 |
| tristan | with jinja2 expressions | 10:49 |
| ssam2 | hmm | 10:49 |
| tristan | seems undocumented, they support lists but no mention of sets in literals | 10:49 |
| ssam2 | so flags_variable is a list of strings, or an integer ? | 10:49 |
| tristan | or rather: (('literal1' | 'literal2) & variable) | 10:49 |
| tristan | list of strings yes | 10:49 |
| ssam2 | ah right | 10:49 |
| ssam2 | i would have thought '&' is for boolean AND | 10:50 |
| ssam2 | probably the best we can do is `('literal1' in variable) or ('literal2' in variable)` | 10:50 |
| tristan | I think it's pythons way of mimicking how we do bitmasks in C but generically with sets | 10:50 |
| ssam2 | i can't do list & string in Python | 10:50 |
| tristan | Well, I asked for the real pythonic ways, and they run a bit differently, but also dont work in jinja2 | 10:51 |
| tristan | i.e. the AND way is like `{'literal1', 'literal2'} >= set_variable` | 10:51 |
| tristan | the OR way is `{'literal1', 'literal2'} & set_variable` | 10:52 |
| tristan | but anyway; not in jinja it seems | 10:52 |
| tristan | ssam2, which is actually comforting in a way | 10:52 |
| ssam2 | ah yeah, seems python set() overloads various operators to be fancy | 10:52 |
| tristan | Because it seems to indicate, at least we're not doing something crazy like blindly evaluating user provided strings with a python interpretor | 10:53 |
| ssam2 | we could have done that ourselves in 1 line :-) | 10:53 |
| ssam2 | (and of course i'm glad we didn't) | 10:53 |
| tristan | heh | 10:53 |
| tristan | well, what I did do, but didnt document, cause I think something better should exist... | 10:54 |
| tristan | is I sort the selected flags in the loaded list | 10:54 |
| tristan | So, if you want to do (['bar', 'foo'] == flags_variable) | 10:54 |
| tristan | you can | 10:55 |
| tristan | But (['foo', 'bar'] == flags_variable) will never match, because ordering | 10:55 |
| ssam2 | hmm.. i'm certainly in favour of having everything in stable sort order | 10:56 |
| tristan | yeah but I'm not digging into jinja and mucking with how the expression side of things are | 10:56 |
| tristan | So for arch conditionals | 10:56 |
| tristan | What I've done; is I've just created an option which derives from the `enum` type | 10:57 |
| ssam2 | i guess to avoid relying on that you'd need to do `len(flags_variable)==2 and 'bar' in flags_variable and 'foo' in flags_variable` | 10:57 |
| tristan | So you can declare what arches you support | 10:57 |
| tristan | And it has the added nicety of defaulting to the result of `uname -a` | 10:57 |
| ssam2 | that sounds good | 10:57 |
| tristan | So you can define one or two or three of those if you like | 10:57 |
| *** jude has joined #buildstream | 10:58 | |
| tristan | And deriving from the same logic... if your default is not one of the supported ones, and the user does not provide a `--option arch arm` (or configuration equivalent)... | 10:59 |
| tristan | then you get an unsupported option error | 10:59 |
| ssam2 | excellent | 10:59 |
| tristan | looks like https://gitlab.com/BuildStream/buildstream/-/jobs/35695223 started without burning minutes on waiting for the docker or some other irrelevancy | 11:01 |
| tristan | So there is a hope in hell that docs will be published :) | 11:01 |
| tristan | looks like I typo'd there but nobody noticed | 11:04 |
| tristan | <tristan> - arch (derived from bool, convenience thingy) | 11:04 |
| tristan | s/bool/enum | 11:04 |
| ssam2 | hah, makes more sense | 11:14 |
| ssam2 | i dunno how to speed up the docker fetches significantly... in the baserock CI we actually set up a local docker registry in the same cloud provider, but it still takes a minute or two to start the tests | 11:15 |
| ssam2 | shrinking the docker base would no doubt help | 11:15 |
| ssam2 | i did try to do an alpine-based image last week, but they don't seem to package GI .typelib files at all | 11:16 |
| ssam2 | at least, they didn't for OSTree, which is the one we need | 11:16 |
| ssam2 | it was only about 200MB compared to the 600MB fedora one though | 11:16 |
| tlater | ssam2: Getting rid of the `dnf install` calls could help a lot, creating intermediate containers takes a lot of time. | 11:19 |
| tlater | It could also help to chain a few of the setup commands with && rather than specifying everything individually. | 11:20 |
| ssam2 | the first of those is done actually | 11:20 |
| ssam2 | https://gitlab.com/BuildStream/buildstream/merge_requests/105 | 11:20 |
| ssam2 | ci for that branch took 43 minutes though, so it's not a huge win ... | 11:21 |
| tlater | Ah, cool :) | 11:21 |
| tlater | Well, baby steps... | 11:22 |
| persia | && doesn't make the shell faster: it's the same as set -e with carriage returns in a script. | 11:24 |
| ssam2 | given a list of shell commands where each command is run in a new shell, sticking them all on one line with && does provide a speedup | 11:25 |
| persia | ssam2: For the desktop-docker use case, is there a way to setup a local provider so that only the first operation is slow(ish), or is that also already done. | 11:25 |
| gitlab-br-bot | push on buildstream@sam/ci-update-docker-image (by Tristan Van Berkom): 20 commits (last: setup.py: Added jinja2 dependency) https://gitlab.com/BuildStream/buildstream/commit/0ec8a1e623c07d8096a0aa1118daec0fdf259620 | 11:25 |
| gitlab-br-bot | buildstream: merge request (sam/ci-update-docker-image->master: .gitlab-ci.yml: Follow latest BuildStream Docker image) #105 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/105 | 11:25 |
| ssam2 | persia, surely that's up to the desktop user to do? | 11:25 |
| persia | ssam2: In that situation (e.g. in make), I tend to either use escaped newlines or here documents for ease-of-reading). But yes, not spawning N subshells is faster. | 11:26 |
| ssam2 | desktop docker is fast apart from the initial pull | 11:26 |
| persia | ssam2: Yes. I just wonder if it can be done easily. I may be overoptimising. | 11:26 |
| ssam2 | i think you are :-) | 11:26 |
| tristan | the bottleneck to CI is the freedesktop runtime really | 11:27 |
| ssam2 | that said, setting up a local docker registry is (of course) just a matter of pulling the correct docker image | 11:27 |
| ssam2 | its something about gitlab CI's setup that is causing the 1-5 minute delay on CI running | 11:27 |
| tristan | With that nifty tiny runtime you've been cooking up, things will go much more quickly | 11:27 |
| ssam2 | also true | 11:27 |
| tlater | persia: It *is* faster in docker, since docker wants to create intermediate containers for each command you run. It does not distinguish between things concatted with &&, though, which means you avoid a fair amount of drive bottleneck. | 11:27 |
| tlater | I'm not sure how the gitlab CI uses that, though. | 11:27 |
| ssam2 | tlater, it's only `docker build` which does that I think | 11:28 |
| ssam2 | when you use `docker run`, you get a new container that is then abandoned or deleted afterwards (depending on whether you pass --rm) | 11:29 |
| ssam2 | but since the backend is copy-on-write, spawning a new container is very cheap | 11:29 |
| ssam2 | it's committing changes that has a cost | 11:29 |
| tlater | I was under the impression that gitlab CI created containers for each individual command for some reason | 11:30 |
| ssam2 | it must concat everything into one script, because env vars persist once set | 11:31 |
| tlater | I frankly don't even know why I thought that. It seems entirely pointless and stupid for performance reasons in this case. | 11:34 |
| tlater | Ignore that then x) | 11:34 |
| tristan | w00t http://buildstream.gitlab.io/buildstream/ ! | 11:38 |
| tristan | relevant bits to project options... Project Format -> Introduction -> Directives | 11:39 |
| tristan | and Project Format -> Project Configuration -> Options | 11:39 |
| tristan | whooops | 11:40 |
| tristan | looks like I missed committing something added | 11:40 |
| tristan | (anyway for the project options, it's not missing) | 11:41 |
| * tristan adds | 11:41 | |
| ssam2 | looks good | 11:42 |
| ssam2 | someone needs to do a round of "it's" fixes :-) | 11:42 |
| tristan | oh those | 11:43 |
| tristan | yeah | 11:43 |
| tristan | I developed a habit of caring about that a few months ago | 11:43 |
| tristan | then I lost that habit :-/ | 11:43 |
| tristan | So a really annoying part of the slowness of CI... is serializing merges :-/ | 11:55 |
| tristan | Like I just rebased ssam2's branch to improve the CI, but have to wait for it to finish before I can push my fix for the missing doc .rst file :-S | 11:55 |
| tristan | Well, seeing as the pytest ran, and the CI *is running*, I suppose I could logically make the call that the .gitlab-ci.yml didnt break after the rebase; and just not pay attention | 11:57 |
| persia | This is why I like zuul. It deals with serialisation, and automerges anything that can be fast-forwarded that has been approved. | 11:57 |
| persia | But it's annoying to set up :/ | 11:57 |
| tristan | sounds like a decent feature yes | 11:58 |
| tristan | if fastforward is possible, please do that, re-run CI and then merge (or fastforward again at the end of CI if need be, repeating until merge or fastforward is no longer possible) | 11:59 |
| tristan | oh wait :-/ | 12:06 |
| tristan | ssam2, why are we following the "latest" tag in our CI with https://gitlab.com/BuildStream/buildstream/merge_requests/105 ? | 12:06 |
| tristan | thats exactly what we didnt want right ? | 12:06 |
| ssam2 | not sure | 12:07 |
| ssam2 | the 'latest' tag only gets updated when someone merges stuff to master in https://gitlab.com/BuildStream/buildstream-docker-images | 12:07 |
| ssam2 | or someone runs the CI pipeline | 12:07 |
| ssam2 | i can change it to use a fixed version if preferred, just that then there will be an extra step needed every time the docker image changes | 12:07 |
| tristan | That's preferred please | 12:08 |
| ssam2 | the gitlab logs show the sha256 of the exact container that was used, so there is traceability | 12:08 |
| ssam2 | ok | 12:08 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Added missing file from documentation makeover) https://gitlab.com/BuildStream/buildstream/commit/b98eb1f0d3f34c9ac35a9518c108e3fe8960c139 | 12:09 |
| tristan | ssam2, wait a sec | 12:09 |
| tristan | ssam2, dont bother, maybe we can live with that, as long as there are no surprises | 12:10 |
| tristan | The goal is only "Never have a random image" | 12:10 |
| tristan | Maybe I'm overthinking this - if anyway we have to make an explicit action for the "latest" tag to change | 12:10 |
| tristan | ssam2, thoughts ? | 12:10 |
| ssam2 | i think following "latest" should be fine | 12:11 |
| ssam2 | as we control it | 12:11 |
| tristan | yeah ok, have to just think a bit more "distributed" | 12:11 |
| tristan | and it makes sense | 12:11 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: .gitlab-ci.yml: Follow latest BuildStream Docker image) https://gitlab.com/BuildStream/buildstream/commit/7704f58146ef266e135425e11572df434573c370 | 12:13 |
| gitlab-br-bot | buildstream: merge request (sam/ci-update-docker-image->master: .gitlab-ci.yml: Follow latest BuildStream Docker image) #105 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/105 | 12:13 |
| tristan | screw a second round of passing CI | 12:13 |
| tristan | what will be interesting, is the day that our CI actually *does* fail because of a new docker, and that we have to modify buildstream to support a newer base (although that is hopefully very unlikely) | 12:16 |
| tristan | it happens though specifically with docs | 12:16 |
| tristan | cause when sphinx changes, we can get breakage, I guess cause... python culture :-/ | 12:17 |
| ssam2 | since the docker images are Fedora based, ideally every 6 months we'll update to the next release | 12:17 |
| tristan | point I guess is, when we find that an update is breaking buildstream, that is a good time to start testing both the old and new images | 12:18 |
| ssam2 | right, so buildstream is tested on multiple platforms | 12:18 |
| ssam2 | part of the reason for calling the docker image 'buildstream/buildstream-fedora' (which is of course pretty longwinded) is that we can in future maintain debian, ubuntu or whatever in parallel and test there too | 12:19 |
| tristan | yeah :) | 12:19 |
| * tristan looks forward to 'buildstream/buildstream-win32' ;-) | 12:19 | |
| gitlab-br-bot | push on buildstream@74-prevent-artifacts-from-containing-files-in-buildstream-build (by Tristan Van Berkom): 20 commits (last: _options: OptionPool implementation, core project options module) https://gitlab.com/BuildStream/buildstream/commit/058f8ced73042d28472b3270210d625fa6bc8e10 | 12:26 |
| gitlab-br-bot | buildstream: merge request (74-prevent-artifacts-from-containing-files-in-buildstream-build->master: Add warnings when staging to /buildstream/build) #93 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/93 | 12:26 |
| ssam2 | ok to use the new docker image in the buildstream-tests repo as well? it's https://gitlab.com/BuildStream/buildstream-tests/merge_requests/18 | 12:41 |
| gitlab-br-bot | buildstream: issue #100 ("Avoid package upgrades during CI") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/100 | 12:42 |
| tristan | ssam2, yes please fire away ! | 12:50 |
| * tristan presses merge button | 12:51 | |
| tristan | beat you to it I guess | 12:51 |
| *** tiagogomes has quit IRC | 12:53 | |
| ssam2 | ah thanks | 12:55 |
| persia | +1 on using "latest" with logs, for repos under control of the same project team. Doing that with foreign repos is dangerous (as those aren't under project control), but for native repos, it makes life so much easier not to have to fuss about upgrading foo every few days and untangling the resulting integration issues because latest bar hasn't been tested with latest baz before. | 13:01 |
| tristan | persia, my main concern was that latest tag moving is an explicit action | 13:02 |
| tristan | not something that changes (under our control is still better, but different meaning) | 13:02 |
| persia | tristan: In that case, I'm not +1 anymore. I believe CI should try to integrate master of A against master of B, as otherwise one has a potential headache when moving one of the goalposts. | 13:05 |
| tristan | No. | 13:05 |
| tristan | Unaccepted :) | 13:05 |
| persia | To me, the entire point of CI is that one doesn't have a manual integration step when e.g. redefining "latest". That said, I know I'm not in a majority school of thought here, so :) | 13:05 |
| tristan | persia, They both have to CI against eachother - but if a CI for a merge request of buildstream code fails - I need to know with certainty that it was tested against the same thing which passed the last time | 13:06 |
| tristan | I.e. "The content of this commit broke the CI" | 13:06 |
| tristan | Not "The random content on which this was tested" | 13:06 |
| persia | I think you and I are violently agreeing here. | 13:06 |
| tristan | :) | 13:06 |
| persia | I define "random content" as "things in repos outside project control". | 13:07 |
| persia | As such, any test should be against static snapshots of "random content", occasionally updated. | 13:07 |
| tristan | In this case, we're talking about a docker image that is auto generated from upstream repos | 13:07 |
| persia | But, for two repos under project control, both should be automatically constantly updated as master. | 13:07 |
| tristan | We control the creation of the image... but if it's "rolling" by itself... I dont want to point to "latest" of it. | 13:07 |
| persia | And if a commit breaks, it doesn't matter if it is a commit to foo or bar, which ever commit doesn't pass the test is the bad one. | 13:07 |
| persia | Ah, if the image isn't deterministic, then I agree. | 13:08 |
| persia | If it is, then I don't. | 13:08 |
| persia | I suspect you agree with me, despite the apparent argument :) | 13:08 |
| tristan | Right, so the situation now, is the other gitlab should do CI when we trigger the thing... and if that fails; we gate the update of "latest" on passing the buildstream tests | 13:08 |
| tristan | iiuc | 13:08 |
| jonathanmaw | tristan: I'm picking up https://mail.gnome.org/archives/buildstream-list/2017-September/msg00003.html. Is "buildstream-plugins" a good name for this repo? iirc there was discussion on whether to have gstreamer-like *-plugins-{base,good,bad} | 13:09 |
| tristan | However of course, noticing that the latest build of fedora packages breaks buildstream, would probably also mean we need to fix buildstream in a back-compat way | 13:09 |
| *** tiagogomes has joined #buildstream | 13:09 | |
| tristan | Myeeaaaaah... I dunny | 13:09 |
| persia | Where the migration of "latest" must pass against latest BuildStream, and a merge to BuildStream must pass to latest "latest". Yes. I presume there is tooling that regularly rebuilds "latest" and proposes merges? | 13:09 |
| tristan | jonathanmaw, I was skeptical of that name | 13:09 |
| tristan | especially if there was going to be more than one | 13:10 |
| jonathanmaw | "buildstream-external-plugins" probably describes what it's going to be most accurately for the present | 13:10 |
| tristan | Also, I wonder if we want separation between debian specific stuff... and "stuff that we dont think is stable enough yet" | 13:10 |
| persia | buildstream-contrib? | 13:10 |
| persia | Maybe with "devel" and "stable" directories? | 13:10 |
| tristan | both buildstrem-contrib and buildstream-external-plugins say nothing of the content except that they are external to the main repo | 13:11 |
| persia | tristan: Yes :) Maximum flexibility. Once the content is comprehended, we can refactor, no? | 13:11 |
| tristan | But then again... maybe we dont have to care so much - because we can change that with a bit less friction than changing stuff with buildstream repo | 13:11 |
| tristan | yeah I think I'm saying what persia just said | 13:11 |
| * persia also | 13:12 | |
| tristan | we have a tendency of violent agreement :) | 13:12 |
| persia | Helps iron out the details. | 13:12 |
| tristan | And keep life spicy | 13:12 |
| tristan | *keeps | 13:12 |
| persia | Why don't lions eat elephants? Have you seen elephants dance? | 13:13 |
| tristan | How about 'bst-external' for now ? | 13:14 |
| persia | +1 | 13:14 |
| tristan | I like the bst abbreviation, and external captures the loooooooooooooooooooooong buildstream-external-plugins meaning | 13:14 |
| persia | tristan: Do you want devel/stable subdirs? Some other taxonomy? | 13:14 |
| tristan | Also project using these plugins will have to type out the entire name of the repository they are using plugins from | 13:15 |
| tristan | which is a good reason to not have suuuuuper long names | 13:15 |
| tristan | persia, not for now; and I dont believe the current structure supports that | 13:15 |
| tristan | currently plugin discovery from pip installed modules (only discovery we have for now) - is one name, one directory of plugins, in one repo that gets installed | 13:16 |
| tlater | tristan: Multiple repos are supported :) | 13:17 |
| tristan | tlater, "repos" ? | 13:17 |
| tristan | explain | 13:17 |
| tlater | You can have multiple plugin repositories | 13:18 |
| tristan | yes | 13:18 |
| tristan | And you can depend on multiple external plugin repositories | 13:18 |
| tristan | Correct ? | 13:18 |
| tlater | Yes | 13:18 |
| * tlater thought you were implying that you can't | 13:18 | |
| tristan | tlater, can a single repository have plugins living in separate subdirectories ? | 13:18 |
| tlater | Yes | 13:19 |
| tristan | Ah, so it is supported | 13:19 |
| tristan | We *could* have some split there then, but I dont really care for it if anyway we might change it | 13:19 |
| tlater | The plugin project structure is pretty free form, anything that you can point to using python modules. | 13:19 |
| tristan | Also "devel / stable" would not be the right structure | 13:19 |
| tristan | persia, it would rather be "devel / debian" | 13:20 |
| persia | WIth the idea that any coherent collection of plugins from devel would get a new top-level directory? | 13:21 |
| persia | If so, I'd hope that was documented in the README, as otherwise it makes "debian" look sorta awkward and lonely. | 13:21 |
| tristan | or be migrated to buildstream proper when we consider it stable, if it's widely used enough and general purpose enough | 13:21 |
| persia | Right, of course. I meant only the things that clearly didn't belong in core. | 13:21 |
| tristan | Yeah; I guess I like using a purpose-named directory for purpose-specific things, because... | 13:22 |
| tristan | When if one day we decide we want to chop it up into separate repos; we'll already have that structure | 13:23 |
| tristan | but... I'm really nit picking about something I'm *mostly* ambivalent about at this point | 13:23 |
| jonathanmaw | ok, shall we stick with bst-external for the time being, containing both the dpkg elements and the x86image element? | 13:23 |
| persia | Yes, plus git, making chopping and reorganising history easy. | 13:24 |
| tristan | I do *feel* that devel / stable sends an incorrect message about the purpose of the repo, though | 13:24 |
| persia | jonathanmaw: Please. | 13:24 |
| tristan | jonathanmaw, yes +1 | 13:24 |
| persia | tristan: If you don't consider "debian" stable, why isn't it "devel"? | 13:24 |
| tristan | persia, Actually, it's a good point that I dont really think about the "debian" stuff as "stable", I wasnt really thinking of that aspect | 13:25 |
| tristan | But it's not the driving factor for me to not want it in buildstream repo | 13:25 |
| tristan | (but it indeed, would be another good reason to not want it _yet_, if we wanted it at all) | 13:25 |
| persia | I think BuildStream core shouldn't contain anything specific to any distro or project. | 13:26 |
| tristan | Exactly, thats the driving factor | 13:26 |
| persia | So I agree with that, but I think that if one uses "devel", all the devel stuff belongs there, and anything non-devel is inherently understood as different by folk looking at the repo. | 13:26 |
| tristan | persia, ok so in this case, we can actually say that all of this stuff is experimental | 13:26 |
| persia | and just toss it all in one bucket, with a READMe talking about experimentality? | 13:27 |
| tristan | And maybe we're going to want to put distro specific stuff in their distro specific repos, and split them up, when they are better tested and API more stable | 13:27 |
| persia | If there are good bits, people should adopt them and maintain them elsewhee, in a repo under stricter governance and stronger CI? | 13:27 |
| tristan | So maybe when we get debian stuff more tested and refined, it can graduate to somewhere else ? | 13:27 |
| tristan | persia, that approach also solves some API issues, I.e. once we're done with it; and put it somewhere that is "stable"... projects need to explicitly ask for the new plugin set | 13:28 |
| tristan | which is a good thing in a way right ? (less surprises) | 13:28 |
| tristan | Then we could change `bst-external` for `bst-experimental` | 13:29 |
| tristan | And seems all neat and tidy ? | 13:29 |
| tristan | Maybe x86Image will come back into buildstream, or maybe it will go into a repo called... heheh ... "bic" ! | 13:30 |
| persia | Let's stick with -external. Reduces the amount of jonathanmaw's patience we burn today. | 13:30 |
| tristan | then there will be mic, wic... and bic ! | 13:30 |
| * persia had happily spent many years not thinking about mic, and now needs to step away from the computer | 13:30 | |
| tristan | :) | 13:30 |
| *** tristan has quit IRC | 14:04 | |
| gitlab-br-bot | buildstream: Tristan Maat created branch 108-integration-tests-not-idempotent-and-self-contained | 14:41 |
| *** bochecha has quit IRC | 15:00 | |
| *** bochecha has joined #buildstream | 15:04 | |
| tlater | In regards to issue #98 (C-c doesn't kill interactive processes) - it seems that what's happening is that we kill the process running inside a child python process, but said child python process doesn't die for some reason, which means that our process never realizes its grandchild died. | 16:26 |
| tlater | So it seems like all of that is entirely outside our control | 16:27 |
| tlater | The child process is launched by Popen(), and new_session doesn't change anything... Maybe shell does? | 16:28 |
| jonathanmaw | hmm, while trying to make the dpkg element work as an external package, I'm getting https://pastebin.com/zjddC8K0 when I try to build a dpkg_build element | 16:28 |
| jonathanmaw | I'm not entirely sure I did it correctly, but I changed the element name from "dpkg_build" to "BuildStream-external:dpkg_build" | 16:28 |
| jonathanmaw | I know the documentation builder couldn't handle package names with hyphens in, maybe something similar's happened here | 16:29 |
| tlater | jonathanmaw: BuildStream-external is supposed to be the pip package name, I'm not sure if that's correct - though the traceback definitely shouldn't be happening. | 16:30 |
| jonathanmaw | it seemed to be unable to find the superclass. As an experiment, I tried to import sys and make it sys.exit() with a useful message, and it complained it couldn't find "sys" | 16:30 |
| jonathanmaw | and when I tried importing sys in the body of the text, it claimed it couldn't find _import_ | 16:31 |
| jonathanmaw | which is very strange | 16:31 |
| *** tristan has joined #buildstream | 16:31 | |
| tristan | jonathanmaw, from the logs... yeah, if we're not allowed hyphens, I guess underscores | 16:35 |
| tristan | I dont know if that's the problem or not, fwiw; just saying, its not the first time I wanted a hyphen but python stole it away from me | 16:36 |
| tlater | It looks like this is might happen if the element plugin isn't defined in the plugin package. | 16:37 |
| tristan | regarding ^C... I think it will abort for certain if we unshare the pid | 16:39 |
| tristan | because our killing of the indirect subprocess of bwrap ensures this | 16:40 |
| tristan | tlater, however; I have a sneaking suspicion that you may have left things behind with cross_platform because of file renames, and not checking that you had exactly the last copy before the merge | 16:40 |
| tristan | that happened for something else already | 16:41 |
| tristan | tlater, it would be good to verify for sure that this ^C thing is not one of those | 16:41 |
| tristan | job control is the hardest part of all of this so far | 16:41 |
| * tristan curses unix signals | 16:41 | |
| tlater | Alright, I'll double check, though I made sure to use a fresh copy of the file during the rebase | 16:42 |
| tristan | another thing to *note*: We're not doing it right | 16:42 |
| tristan | I am sure I've debugged this numerous times, and was pretty confident that we had a pretty functional terminal enough | 16:42 |
| tristan | (the side effects were not this severe at all) | 16:42 |
| tlater | What's the not doing it right part? | 16:43 |
| tristan | The proper way to do this... is some weird thing about providing a proper console device to the child namespace, and having a side process doing IPC with it | 16:43 |
| tristan | that was from Dr Richards in #kernelnewbies on OFTC | 16:43 |
| tristan | which, if you can manage to tease good info from that channel... is a decent resource for all this namespace complexity | 16:44 |
| tristan | short of digging through kernel code :) | 16:44 |
| tlater | Heh, let's hope that's not required for python multiprocessing. | 16:45 |
| tristan | well its required for having a fully functional terminal | 16:45 |
| tristan | in a child namespace | 16:45 |
| tristan | the definition of that is ... not sure | 16:45 |
| tristan | but it does cause either bash or dash to complain about it | 16:46 |
| tristan | job control is at least no disabled with bash, and that's because of the start_new_session thing | 16:46 |
| tristan | (i.e. if we inherit the same process group, then bash can deal) | 16:47 |
| tristan | creating a new group iirc means no job control in interactive shells | 16:47 |
| tristan | which had the opposite effect of... I'm running a shell, and when I launch something... and press ^C... EVERYTHING goes down immediately | 16:47 |
| tristan | (lets just say... the water is deep and there be dragons) | 16:48 |
| tlater | Kernel dragons, in fact! | 16:48 |
| * tlater will look into fixing the potential regression first, we can worry about figuring out the "proper" tomorrow/later :) | 16:49 | |
| tristan | yeah if you could verify it's not a regression that would be really good | 16:50 |
| tristan | and verify that your port has not fallen behind what was in master at the time in any other way either, also important :) | 16:50 |
| *** ssam2 has quit IRC | 16:52 | |
| tlater | jonathanmaw: You may have also forgotten to set the 'entry_points' in setup.py | 16:55 |
| tlater | I suspect the issue is that you have a plugin package, but no entry point for the actual plugin | 16:56 |
| jonathanmaw | tlater: I'll have to check that it's correct, but it was your work originally :P | 16:56 |
| * tlater didn't consider this edge case | 16:56 | |
| tlater | Damn, that's weird | 16:57 |
| jonathanmaw | it's definitely an avenue of investigation for tomorrow. ta tlater. | 16:57 |
| tlater | Alright :) | 16:57 |
| *** jonathanmaw has quit IRC | 16:58 | |
| *** anahuelamo has quit IRC | 17:03 | |
| *** tlater has quit IRC | 17:11 | |
| *** persia has quit IRC | 17:30 | |
| *** persia has joined #buildstream | 18:07 | |
| *** jude has quit IRC | 18:18 | |
| *** tristan has quit IRC | 18:31 | |
| *** jude has joined #buildstream | 20:30 | |
| *** jude has quit IRC | 22:06 | |
| *** jude has joined #buildstream | 22:07 | |
| *** jude has quit IRC | 22:17 | |
| *** david-mora_ has joined #buildstream | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!