| *** mohan43u has quit IRC | 01:46 | |
| *** mohan43u has joined #buildstream | 01:50 | |
| *** mohan43u has quit IRC | 03:02 | |
| *** mohan43u has joined #buildstream | 03:06 | |
| *** mohan43u has quit IRC | 03:11 | |
| *** mohan43u has joined #buildstream | 03:15 | |
| *** mohan43u has quit IRC | 03:31 | |
| *** mohan43u has joined #buildstream | 03:35 | |
| *** tristan has quit IRC | 06:23 | |
| *** tristan has joined #buildstream | 06:27 | |
| *** juergbi has quit IRC | 07:08 | |
| *** juergbi has joined #buildstream | 07:11 | |
| *** bochecha has joined #buildstream | 07:37 | |
| *** raoul_ has joined #buildstream | 08:25 | |
| jennis | Mhmm I'm working on a branch which prohibits junction elements being pulled from / pushed to the parent project's remote. This is going to be configurable on the project level. At first, I had the boolean `push-junction-elements`... Decided against this because we're also trying to pull. Then after a review suggestion, I changed this to "inherit-ju | 08:29 |
|---|---|---|
| jennis | nction-caches", but now I'm thinking about it, that doesn't make sense either. We're not inheriting the junction's cache, we're just preventing junction elements from communicating with the parent project's cache | 08:30 |
| gitlab-br-bot | marge-bot123 merged MR !1407 (aevri/smallerjobs->master: jobs/job: send ChildJob the context, not scheduler) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1407 | 08:30 |
| jennis | Would appreciate if someone could just agree with me here | 08:30 |
| jennis | Also open to suggestions of better names | 08:30 |
| *** ChanServ sets mode: +o tristan | 08:32 | |
| tristan | jennis, I think already each artifact cache configuration has a boolean `push` attribute | 08:32 |
| tristan | jennis, at least it is like this in the user configuration... so given that a project also provides a recommendation of artifact caches (overridable by user configuration)... and that the project certainly has knowledge of the projects it junctions, why would we not simply mirror the same configuration style at the project level ? | 08:33 |
| jennis | Yes, but if my parent project has an artifact cache, configured with `push: True`, BuildStream will try to pull/push junction elements from/to this parent project's cache | 08:33 |
| tristan | What is a parent project ? | 08:33 |
| jennis | The `push` attribute also exists on the project config level | 08:33 |
| tristan | You mean a project "which junctions this project" ? | 08:34 |
| jennis | The top level project | 08:34 |
| tristan | Right, so it should be up to *that* project to decide | 08:34 |
| jennis | yep, this is what i've implemented | 08:34 |
| *** phildawson has quit IRC | 08:34 | |
| jennis | But I don't think the name "inherit-junction-caches" makes sense, because that's not what we're doing, I'm just trying to think of a not so verbose, accurate name | 08:35 |
| tristan | I don't see why we need such a configuration | 08:35 |
| tristan | still not getting it | 08:35 |
| tristan | the *user* configuration says... "For each project... here is an artifact cache address... and whether to push or pull to/from it" | 08:35 |
| tristan | If the project can have the same, then where is the need for even more configurability ? | 08:36 |
| jennis | Yes, but say for the top level project, you say "I want to push to this" | 08:36 |
| jennis | When you build, BuildStream will build elements in your project, and elements from the junction, and push *both* into the remote | 08:36 |
| jennis | Now, say you control the junction, and the junction's remote, you may not want to duplicate caching, especially if you have a lot of elements and caching is expensive | 08:37 |
| tristan | jennis, What if in the toplevel project.... you configure the *subproject* with your toplevel project's artifact cache... and say "push: false" ? | 08:37 |
| tristan | I mean, that is the same style as we do in user configuration | 08:37 |
| tristan | jennis, i.e. the project.conf has not only *one* artifact cache configuration, but an artifact cache configuration *for every subproject* | 08:38 |
| tristan | mirroring exactly the same configuration style we have at the user configuration level | 08:38 |
| tristan | but with a lower priority | 08:38 |
| jennis | ok, then that still sounds like we'll try and pull subproject elements from the top level project's remote | 08:38 |
| jennis | which is a waste of time | 08:38 |
| jennis | Especially as we pull element by element | 08:39 |
| tristan | Ummm, I'm not sure that is true | 08:39 |
| tristan | You could just as well also say "pull: false" in the same place right ? | 08:39 |
| tristan | effectively disabling the pulling/pushing of a specific project's elements from a specific artifact cache | 08:40 |
| tristan | The parent project's configuration has priority over the child project's configuration... and the user config has priority over everything | 08:40 |
| tristan | finally you resolve which elements are allowed to push/pull from which artifact server at startup time | 08:41 |
| tristan | and then go :) | 08:41 |
| tristan | jennis, I'm not saying that the code reads *exactly* this way as it stands, but I assume that you are going to change things anyway - so you have some freedom here to make better sense of artifact cache configuration than the current state | 08:42 |
| jennis | Ok it sounds like, I've essentially implemented this but with a one line boolean | 08:42 |
| jennis | Albiet, not as granular as your suggestion | 08:42 |
| tristan | So then how about "enabled: true/false" as an alias for both "push" and "pull" ? | 08:42 |
| tristan | We have to assume that there are many projects in the same pipeline, considering the toplevel vs others is certainly not a good idea | 08:43 |
| tristan | the toplevel is not special | 08:43 |
| jennis | mhmm, so if we had a toplevel project, with lets say 5 junctions, we should be able to say "ok I only want to be remotely caching 3 of those" | 08:45 |
| jennis | ? | 08:45 |
| *** tristan has quit IRC | 08:50 | |
| *** jonathanmaw has joined #buildstream | 08:51 | |
| *** tristan has joined #buildstream | 08:55 | |
| *** ChanServ sets mode: +o tristan | 08:59 | |
| tristan | jennis, Sure why not ? | 08:59 |
| * tristan may have missed some part of this | 08:59 | |
| * tristan lost connection for a moment | 08:59 | |
| tristan | jennis, More than that even... we can say "I want to be caching 2 of these... and I want to be caching the sub-sub-project of one of those" | 09:00 |
| tristan | And more importantly... the day that I create a *new* project and junction this so called "toplevel" (which is not toplevel anymore)... things *still* work in the way that that "no longer toplevel" project decided they would | 09:01 |
| tristan | *unless* | 09:01 |
| tristan | *unless* the new project has overridden that behavior, as it has priority in the new context | 09:01 |
| tristan | jennis, My concerns are the following (A) Historically artifact cache configuration has been very messy, it would be good to clean house and make sure that project configuration mirrors user configuration very well, and not add any *more* semantics if we can avoid it at all... | 09:03 |
| tristan | and (B) That we make sure that we are considering that what is a toplevel project today, might not be a toplevel project tomorrow, and that it's configuration still makes sense in any context; whether it is being built as toplevel or as a sub-project of something else | 09:04 |
| tristan | jennis, Frankly, I'm not concerned with having such fine grained control | 09:04 |
| tristan | It is more (A) and (B) that are important, and for (A), it seems logical that configuration is a simple/same semantic which can be applied on a per-project/per-artifact-cache level | 09:05 |
| tristan | and that at load time we just resolve those configurations where subprojects and parent projects and user configuration might disagree | 09:06 |
| tristan | so that we have clear priorities of who has the last word on what | 09:06 |
| tristan | jennis, That said... there is one hole in all of this which needs further thought, and that is: Projects actually do *not* have knowledge of their subprojects | 09:07 |
| tristan | that was a misstep in my logic | 09:07 |
| tristan | While a project might know which projects it junctions directly... we should consider that projects regularly run `bst track` on their junctions in CI... and that those subprojects may reorganize themselves and in turn grow new junctions which the calling project may never need to be aware of | 09:08 |
| tristan | All of that should be considered when designing how to best express configuration of which projects are pushing/pulling from which artifact servers | 09:09 |
| *** phildawson has joined #buildstream | 09:12 | |
| juergbi | does anyone know what marge means with this? 'I couldn't merge this branch: failed on filter-branch; check my logs!' | 09:12 |
| tristan | heh | 09:12 |
| tristan | I suppose the logs might say | 09:12 |
| tristan | but a wild guess is that there is some regex or something in her configuration named 'filter-branch', and it doesnt like your branch name ? | 09:13 |
| tristan | heh | 09:13 |
| tristan | s/it/she ... sorry for being so insensitive marge... | 09:14 |
| *** phildawson has quit IRC | 09:14 | |
| *** phildawson has joined #buildstream | 09:14 | |
| juergbi | I thought that might be the case but I can't think of what she might dislike about juerg/grpc | 09:15 |
| jennis | tristan, I see what you're saying. However, say I make this change in my toplevel project, and then further down the line someone junctions my project. What I've implemented won't change the ability for this "new" top level to be able to cache my (former toplevel) project's elements, and its subproject elements | 09:16 |
| jennis | It seems to me that the only difference is that I've done a sweeping statement of "no subproject elements can communicate with this project's remotes" | 09:16 |
| jennis | Whereas your suggestion is much more granular | 09:16 |
| tristan | jennis, it doesnt sound bad... I would suggest that it at least be tied to the definition of an artifact cache... i.e. "local-push-only: true" or "local-pull-only: true" (or "local-only" for both ?) be defined for a specific artifact cache declaration in a project.conf | 09:21 |
| tpollard | I like the idea of it being more granular in the sense of pull/pull/'none' approaches, but as it stands having it as a project option doesn't concern me | 09:21 |
| tpollard | *push/pull/none | 09:22 |
| tristan | Because anyway a project declares a list of artifact caches | 09:22 |
| tristan | And limiting one or more of the artifact caches it declares to operate only with it's own elements, has the advantage of not needing to know about it's subprojects | 09:23 |
| tristan | in fact... given that we probably *already* have push:/pull: configurations on artifact cache definitions... it might make sense to allow "local" as an option, in addition to "true" and "false" ? | 09:25 |
| jennis | Ok, I'm trying to think about the usecase of local-push-only: True and local-pull-only: False and vice versa. Does this make sense? Why would we want to try pulling subproject elements from a cache we don't want to push them too? | 09:25 |
| tristan | Sure | 09:26 |
| jennis | I do like the idea of having this defined under the artifact specs | 09:26 |
| tpollard | you might have CI setup to feed that cache, and that instance might be the only thing that's designated to populate it | 09:26 |
| tristan | One example is when working on a feature branch, and you want to try pulling from mainline, but you only want to push to a dedicated artifact cache related to your side branch | 09:26 |
| tpollard | but you still want to pull from it | 09:27 |
| jennis | Ok, it looks like we're all getting on the same page here | 09:27 |
| tpollard | but there's always going to be 'mights' for any perceived situation | 09:27 |
| tristan | Right, I think it would be better to not limit this by what we predict are valid use cases | 09:28 |
| jennis | Fair point | 09:28 |
| tristan | Rather, we know that the caches can exist, and things we can do with them are push and pull | 09:28 |
| tristan | Ah right... jonathanmaw sent an email and I have to look at a branch... | 09:33 |
| jonathanmaw | thanks, tristan | 09:33 |
| jennis | Yeah, ok, I will see what I can do about what we've just discussed | 09:34 |
| jennis | I like the `local-push`/`local-pull` option | 09:34 |
| tristan | jennis, I think I like reducing the amount of options possible and allowing "local" as a value for "push" and "pull", if it indeed amounts to the same thing | 09:35 |
| tristan | seems like | 09:35 |
| *** lachlan has joined #buildstream | 09:35 | |
| *** tristan has quit IRC | 09:44 | |
| *** benschubert has joined #buildstream | 09:50 | |
| gitlab-br-bot | juergbi merged MR !1405 (juerg/grpc->master: Guard against gRPC channels in the main process) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1405 | 09:52 |
| *** lachlan has quit IRC | 10:01 | |
| *** tristan has joined #buildstream | 10:14 | |
| *** lachlan has joined #buildstream | 10:26 | |
| *** ChanServ sets mode: +o tristan | 10:27 | |
| tristan | jonathanmaw, Is the description up to date on that MR !1409 ? | 10:27 |
| gitlab-br-bot | MR !1409: WIP: Separate frontend state handling from core state https://gitlab.com/BuildStream/buildstream/merge_requests/1409 | 10:27 |
| tristan | Looks like the description is right on target yeah | 10:27 |
| jonathanmaw | the description should be accurate, yeah | 10:28 |
| *** lachlan has quit IRC | 10:37 | |
| *** lachlan has joined #buildstream | 10:40 | |
| *** lachlan has quit IRC | 10:52 | |
| tristan | jonathanmaw, Commented on some details... in general the approach is looking good | 10:57 |
| tristan | still some untangling to do I think | 10:57 |
| jonathanmaw | okie doke, ta | 10:58 |
| *** lachlan has joined #buildstream | 11:10 | |
| gitlab-br-bot | raoul.hidalgocharman opened MR !1410 (raoul/915-capabilities-service->master: Capabilities service) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1410 | 11:11 |
| *** bochecha_ has joined #buildstream | 11:15 | |
| *** bochecha has quit IRC | 11:17 | |
| *** bochecha_ is now known as bochecha | 11:17 | |
| *** lachlan has quit IRC | 11:17 | |
| *** bochecha_ has joined #buildstream | 11:43 | |
| *** bochecha has quit IRC | 11:45 | |
| *** bochecha_ is now known as bochecha | 11:45 | |
| jennis | haha do it | 11:49 |
| jennis | whoops | 11:50 |
| *** rdale has quit IRC | 12:00 | |
| *** benschubert has quit IRC | 12:06 | |
| *** rdale has joined #buildstream | 12:12 | |
| *** ahmed89 has joined #buildstream | 12:28 | |
| *** ahmed89 has quit IRC | 12:53 | |
| *** rdale has quit IRC | 12:55 | |
| gitlab-br-bot | raoul.hidalgocharman approved MR !1389 (tristan/exit-on-nonblock-terminal-1.2->bst-1.2: _frontend/cli.py: Exit with error if output streams are set to nonblocking) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1389 | 13:11 |
| *** lachlan has joined #buildstream | 13:18 | |
| *** rdale has joined #buildstream | 13:25 | |
| *** lachlan has quit IRC | 13:29 | |
| gitlab-br-bot | tristanvb opened MR !1411 (tristan/exit-on-nonblock-terminal->master: _frontend/cli.py: Exit with error if output streams are set to nonblocking) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1411 | 13:39 |
| *** lachlan has joined #buildstream | 13:41 | |
| tpollard | anyone tried bst on wsl2 yet? | 13:47 |
| *** lachlan has quit IRC | 13:49 | |
| Kinnison | is wsl2 out? | 13:50 |
| Kinnison | I thought it wasn't due for preview until later this year | 13:50 |
| tpollard | it's in the latest windows insider preview build | 13:50 |
| Kinnison | Coo! | 13:51 |
| tpollard | https://blogs.windows.com/windowsexperience/2019/06/12/announcing-windows-10-insider-preview-build-18917 | 13:51 |
| Kinnison | If you have access, please do test. It *should* be significantly better | 13:51 |
| tpollard | would be nice to try :) | 13:53 |
| Kinnison | WSL1 has a serious IO problem around locking the FD table | 13:53 |
| Kinnison | WSL2 ought to mitigate that by being a real Linux kernel | 13:54 |
| samkirkham | I'm getting a weird error when building a compose element | 13:58 |
| samkirkham | Of some folder that should be there not being in the cache | 13:58 |
| *** tristan has quit IRC | 13:58 | |
| *** lachlan has joined #buildstream | 14:07 | |
| gitlab-br-bot | danielsilverstone-ct closed MR !1386 (danielsilverstone-ct/microopts->master: Just a couple of microoptimisations) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1386 | 14:11 |
| *** lachlan has quit IRC | 14:18 | |
| *** lachlan has joined #buildstream | 14:22 | |
| *** lachlan has quit IRC | 14:25 | |
| *** tristan has joined #buildstream | 14:26 | |
| gitlab-br-bot | jennis approved MR !1392 (tpollard/shellbuildtree->master: _frontend/cli.py: Tweak 'try' & 'always' bst shell buildtree handling) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1392 | 14:39 |
| * tpollard looks at marge | 14:56 | |
| *** lachlan has joined #buildstream | 14:57 | |
| tpollard | ah, my bad | 14:57 |
| *** lachlan has quit IRC | 15:05 | |
| gitlab-br-bot | marge-bot123 merged MR !1392 (tpollard/shellbuildtree->master: _frontend/cli.py: Tweak 'try' & 'always' bst shell buildtree handling) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1392 | 15:06 |
| *** lachlan has joined #buildstream | 15:07 | |
| *** Kinnison has left #buildstream | 15:10 | |
| *** lachlan has quit IRC | 15:11 | |
| gitlab-br-bot | raoul.hidalgocharman approved MR !1411 (tristan/exit-on-nonblock-terminal->master: _frontend/cli.py: Exit with error if output streams are set to nonblocking) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1411 | 15:19 |
| *** bochecha has quit IRC | 15:49 | |
| *** phildawson_ has joined #buildstream | 15:50 | |
| *** lachlan has joined #buildstream | 15:50 | |
| gitlab-br-bot | juergbi approved MR !1411 (tristan/exit-on-nonblock-terminal->master: _frontend/cli.py: Exit with error if output streams are set to nonblocking) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1411 | 15:51 |
| *** phildawson has quit IRC | 15:51 | |
| *** lachlan has quit IRC | 16:10 | |
| *** lachlan has joined #buildstream | 16:19 | |
| *** raoul_ has quit IRC | 16:34 | |
| *** jonathanmaw has quit IRC | 17:05 | |
| *** phildawson_ has quit IRC | 17:16 | |
| *** lachlan has quit IRC | 17:31 | |
| *** bethw has quit IRC | 17:37 | |
| *** bethw has joined #buildstream | 17:38 | |
| *** adds68 has quit IRC | 17:44 | |
| *** paulsherwood has quit IRC | 17:44 | |
| *** benbrown has quit IRC | 17:44 | |
| *** ikerperez has quit IRC | 17:44 | |
| *** Becky has quit IRC | 17:44 | |
| *** samkirkham has quit IRC | 17:44 | |
| *** ikerperez has joined #buildstream | 17:45 | |
| *** paulsherwood has joined #buildstream | 17:45 | |
| *** Becky has joined #buildstream | 17:45 | |
| *** samkirkham has joined #buildstream | 17:45 | |
| *** adds68 has joined #buildstream | 17:45 | |
| *** benbrown has joined #buildstream | 17:45 | |
| *** abderrahim has joined #buildstream | 17:46 | |
| *** lachlan has joined #buildstream | 17:47 | |
| *** paulsherwood has quit IRC | 17:47 | |
| *** benbrown has quit IRC | 17:47 | |
| *** jennis has quit IRC | 17:49 | |
| *** valentind has quit IRC | 17:49 | |
| *** laurence has quit IRC | 17:49 | |
| *** WSalmon has quit IRC | 17:49 | |
| *** bethw has quit IRC | 17:49 | |
| *** johnward has quit IRC | 17:49 | |
| *** bethw has joined #buildstream | 17:50 | |
| *** jennis has joined #buildstream | 17:53 | |
| *** laurence has joined #buildstream | 17:53 | |
| *** valentind has joined #buildstream | 17:53 | |
| *** WSalmon has joined #buildstream | 17:55 | |
| *** johnward has joined #buildstream | 17:55 | |
| *** ikerperez has quit IRC | 17:56 | |
| *** samkirkham has quit IRC | 17:56 | |
| *** adds68 has quit IRC | 17:56 | |
| *** Becky has quit IRC | 17:56 | |
| *** lachlan has quit IRC | 18:26 | |
| *** lachlan has joined #buildstream | 18:31 | |
| *** lachlan has quit IRC | 18:42 | |
| *** lachlan has joined #buildstream | 18:58 | |
| *** abderrahim has quit IRC | 19:09 | |
| *** abderrahim has joined #buildstream | 19:10 | |
| *** abderrahim has joined #buildstream | 19:11 | |
| *** abderrahim has quit IRC | 19:12 | |
| *** abderrahim has joined #buildstream | 19:12 | |
| *** abderrahim_ has joined #buildstream | 19:21 | |
| *** abderrahim has quit IRC | 19:22 | |
| *** abderrahim_ is now known as abderrahim | 19:22 | |
| *** abderrahim has quit IRC | 19:30 | |
| *** lachlan has quit IRC | 19:31 | |
| *** lachlan has joined #buildstream | 19:36 | |
| *** lachlan has quit IRC | 19:38 | |
| *** abderrahim has joined #buildstream | 19:45 | |
| *** benschubert has joined #buildstream | 20:08 | |
| *** abderrahim has quit IRC | 21:25 | |
| *** paulsherwood has joined #buildstream | 22:35 | |
| *** benschubert has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!