| *** tristan has quit IRC | 06:40 | |
| *** tristan has joined #buildstream | 07:02 | |
| *** benschubert has quit IRC | 07:33 | |
| *** bochecha has joined #buildstream | 07:38 | |
| *** Trevinho has quit IRC | 07:40 | |
| *** Trevinho has joined #buildstream | 07:43 | |
| *** benschubert has joined #buildstream | 07:46 | |
| *** bozcan has joined #buildstream | 08:01 | |
| *** benschubert has quit IRC | 08:01 | |
| *** benschubert has joined #buildstream | 08:01 | |
| *** tristan has quit IRC | 08:04 | |
| *** aevri has quit IRC | 08:14 | |
| *** aevri has joined #buildstream | 08:14 | |
| *** bochecha has quit IRC | 08:16 | |
| *** bochecha has joined #buildstream | 08:20 | |
| *** toscalix has joined #buildstream | 08:29 | |
| *** bochecha_ has joined #buildstream | 08:34 | |
| *** bochecha has quit IRC | 08:36 | |
| *** bochecha_ is now known as bochecha | 08:36 | |
| *** bochecha_ has joined #buildstream | 08:37 | |
| *** jonathanmaw has joined #buildstream | 08:39 | |
| *** bochecha has quit IRC | 08:39 | |
| *** bochecha_ is now known as bochecha | 08:39 | |
| *** bozcan has quit IRC | 08:53 | |
| *** alatiera_ has joined #buildstream | 09:01 | |
| *** tristan has joined #buildstream | 09:13 | |
| *** lachlan has joined #buildstream | 09:43 | |
| *** lachlan has quit IRC | 09:51 | |
| *** lachlan has joined #buildstream | 10:04 | |
| *** lachlan has quit IRC | 10:10 | |
| gitlab-br-bot | tristanvb opened MR !1355 (tristan/fix-workspaced-junctions-1.2->bst-1.2: Fix workspaced junctions (1.2)) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1355 | 10:15 |
|---|---|---|
| jjardon | Hi, are you aware on any change in the CAS that make it incompatible with the one in bst-1.2.x? Seems bst master can not use it anymore: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/218506007 | 10:17 |
| Kinnison | the CAS no, the artifact services, yes. | 10:18 |
| jjardon | Ok, so bst master can not use an bst-1.2 artifact cache? | 10:20 |
| Kinnison | Correct | 10:20 |
| jjardon | Is that intentional or is that something needs fixing? | 10:20 |
| Kinnison | bst2 (which master is) is intentionally replacing the artifact stuff | 10:20 |
| Kinnison | This is part of the 'Artifact as a Proto' work | 10:21 |
| jjardon | Ok, so for projects using bst-1.2, we will have to deploy a new artifact cache server when moving to bst2 ? | 10:22 |
| Kinnison | Yes | 10:22 |
| juergbi | master bst-artifact-server still supports the old mechanism, however, we might not keep this for bst2 and it's probably better to have a separate installation | 10:23 |
| juergbi | it should become possible to use the bst2 artifact server together with an arbitrary CAS server, though | 10:24 |
| juergbi | if object sharing is desired | 10:24 |
| juergbi | (that's not supported right now, though, afaik) | 10:25 |
| adds68 | juergbi, tpollard mentioned that the new cache can still find hits in the old cache, but you have to manually accept the artifacts, is that true? | 10:28 |
| *** ChanServ sets mode: +o tristan | 10:28 | |
| juergbi | adds68: the new bst-artifact-server should still work fine with bst-1.2 clients | 10:29 |
| tristan | juergbi, right... I think that's what I've been telling people | 10:29 |
| tristan | juergbi, So basically, master artifact server still supports bst1 clients right ? | 10:29 |
| juergbi | adds68: however, the artifact references stored for 1.2 and master are independent (completely different format) | 10:29 |
| tristan | The opposite, I wouldn't expect | 10:29 |
| adds68 | ok thanks | 10:29 |
| juergbi | tristan: yes but I don't think we've decided yet how long we want to keep this | 10:30 |
| juergbi | (not much code, so not really an issue to keep) | 10:30 |
| tristan | Right, I think that from what I understood the effort to keep it compatible is minimal enough | 10:30 |
| juergbi | although with the planned move to casd, it might require a bit of effort to keep supporting it | 10:31 |
| tristan | :-/ | 10:31 |
| tristan | I should get the ball rolling on parallel installability again soon | 10:31 |
| juergbi | the question is whether it actually makes sense to have such a combined installation | 10:31 |
| tristan | and figure out if we need it for the artifact server also | 10:32 |
| adds68 | tristan ++ | 10:32 |
| jjardon | bst-1.2 server is incompatible with bst master in the client rigth now already | 10:32 |
| jjardon | Have anyone here tested buildstream with another cache server based in the Remote Execution protocols? Like BuildBarn | 10:32 |
| juergbi | not sure about buildbarn | 10:33 |
| tristan | jjardon, However that was always going to be the case, regardless of bst2 separation... i.e. if we went forward without breaking API, we expect that we might need to *upgrade* the artifact server, but we also expect the new one to support older clients | 10:33 |
| juergbi | CAS and artifact service had been somewhat tightly linked until this change, so we couldn't use a random CAS server as CAS for artifacts | 10:34 |
| jjardon | I see | 10:34 |
| tristan | From what I understand a CAS doesnt support named refs, while the artifact servers are a superset which support named refs | 10:35 |
| tristan | not sure how else it differs | 10:35 |
| tristan | jjardon, Are we doing that anywhere outside of BuildStream nightly CI ? | 10:36 |
| tristan | oops wrong channel hehe ;-) | 10:36 |
| tristan | jjardon, Anyway, are we building freedesktop-sdk using bst master anywhere outside of BuildStream nightly CI ? | 10:37 |
| jjardon | We are doing it in freedesktop-sdk CI | 10:38 |
| tristan | jjardon, I think anyway... we should be able to keep the CI working | 10:38 |
| tristan | Ummm, why ? | 10:38 |
| juergbi | yes, bst-artifact-server currently provides CAS + ReferenceStorage service (the old protocol for artifacts) + new Artifact service (storing artifact protos) | 10:38 |
| tristan | jjardon, What is the difference between fdsdk CI of BuildStream master and BuildStream's CI, and why would we want to do it twice ? | 10:41 |
| tristan | jjardon, afaics, it should be up to BuildStream to (A) Keep track of what changes need to be made to bst1 projects... (B) Keep it working | 10:42 |
| jjardon | Yeah the bst CI is broken as well | 10:42 |
| tristan | Looks like I ran into another weird bug | 10:42 |
| tristan | jjardon, Right, but at least that can be BuildStream's problem | 10:43 |
| tristan | no need to double our trouble | 10:43 |
| tristan | So different topic... (A) why does BuildStream 2 require workspaces to be specified as relative paths ? ... (B) why are those relative paths relative to the *elements* directory, and not the project directory ?? | 10:44 |
| tristan | I thought we *support* relative paths from the *project* directory, not *require* relative paths from the *elements* directory | 10:45 |
| jjardon | I can remove the fdsk one; but it takes very little effort to keep it | 10:45 |
| juergbi | I would also expect relative to project directory but haven't been working in that code area recently | 10:46 |
| jjardon | All overnight tests are failing: seems a problem with the bastion runner: https://gitlab.com/BuildStream/buildstream/pipelines/62889746 (and the aarch64 one) | 10:50 |
| tristan | juergbi, I think I messed up my test case and I'm wrong about the weird behavior | 10:56 |
| *** lachlan has joined #buildstream | 10:57 | |
| *** lachlan has quit IRC | 11:01 | |
| *** lachlan has joined #buildstream | 11:11 | |
| *** lachlan has quit IRC | 11:17 | |
| tristan | Ok so how are you supposed to open a workspace ? | 11:20 |
| tristan | On an existing directory ? In BuildStream 2 ? | 11:20 |
| tristan | it wants a project relative directory, but keeps assuming CWD for some reason | 11:21 |
| jjardon | tristan: how could we fix this in bst CI? AFAIK Bst doesn't have a bst cache server setup at the moment | 11:22 |
| tristan | jjardon, That sounds like part of fixing it indeed, setting up a cache | 11:24 |
| tristan | Ok so seems absolute paths are still supported in bst workspace open --directory, and without specifying an absolute path, CWD is assumed as starting point | 11:25 |
| tristan | all makes sense now and test case works | 11:25 |
| tristan | whew | 11:25 |
| *** lachlan has joined #buildstream | 11:26 | |
| *** lachlan has quit IRC | 11:34 | |
| gitlab-br-bot | tristanvb opened MR !1356 (tristan/fix-workspaced-junctions->master: Fix workspaced junctions) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1356 | 11:37 |
| *** lachlan has joined #buildstream | 11:38 | |
| tristan | Someone care to take a look at !1356, to fix #1030 ? | 11:40 |
| gitlab-br-bot | Issue #1030: bst show asks the user to fetch a workspaced junction https://gitlab.com/BuildStream/buildstream/issues/1030 | 11:40 |
| tristan | juergbi, the above does one thing in addition to !1355 (for bst-1.2), which is to use Element._fetch() from the loader (bst-1.2 never had an Element._fetch()) | 11:40 |
| gitlab-br-bot | MR !1355: Fix workspaced junctions (1.2) https://gitlab.com/BuildStream/buildstream/merge_requests/1355 | 11:41 |
| *** lachlan has quit IRC | 11:45 | |
| *** toscalix has quit IRC | 11:56 | |
| *** lachlan has joined #buildstream | 12:01 | |
| benschubert | tristan: for !1356, did you try it by any change with nested junctions? I'm not sure we have tests for that, but it might be a good addition there (or separately) | 12:32 |
| tristan | benschubert, Hmmm, I think that is separate issue https://gitlab.com/BuildStream/buildstream/issues/691 | 12:34 |
| benschubert | tristan: ah, I was thinking about ensuring the fetch of a subjunction from a workspaced junction is fetching everything correctly :) I don't htink we have this, but it actually can be done separately | 12:35 |
| gitlab-br-bot | BenjaminSchubert approved MR !1356 (tristan/fix-workspaced-junctions->master: Fix workspaced junctions) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1356 | 12:36 |
| tristan | benschubert, definitely we need to take a closer look at recursive subproject autofetch :-/ | 12:39 |
| benschubert | I might have some time to allocate to that once I'm more done with Cython :) | 12:40 |
| tristan | Is marge doing her thing these days ? | 12:40 |
| benschubert | I think so | 12:40 |
| * tristan thinks not, yeah | 12:40 | |
| benschubert | I'm tackling a last problem with getting cython files coverage and it should be good to unwip :) | 12:40 |
| benschubert | tristan: btw, what do you prefer, check in generated code or not? | 12:41 |
| benschubert | The cython files can be quite large | 12:41 |
| tristan | I was going to comment on #691 I think the problem there is reduced, at least we improved error reporting and provenance about subproject errors | 12:41 |
| gitlab-br-bot | Issue #691: BuildStream does not automatically fetch a junction within a junction / displays an inaccurate helper https://gitlab.com/BuildStream/buildstream/issues/691 | 12:41 |
| tristan | benschubert, I think I prefer not committing generated code, and integrating all code generation into the build process in general | 12:41 |
| tristan | it cant be that onerous for developers to have the tools for generating these right ? | 12:42 |
| tristan | juergbi, what about grpc, would that be fine with you too ? | 12:42 |
| benschubert | tristan: ok :) The only problem is when testing with setup.py or pyest, to remember to do it when changing branches for example | 12:42 |
| * tristan recalls juergbi integrated the grpc as committed files initially | 12:42 | |
| benschubert | tox will take care of everything for us anyways | 12:42 |
| juergbi | tristan: fine with me | 12:43 |
| * tlater[m] wonders how IDE stuff handles missing generated code | 12:44 | |
| gitlab-br-bot | tristanvb closed issue #1030 (bst show asks the user to fetch a workspaced junction) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1030 | 12:44 |
| gitlab-br-bot | tristanvb merged MR !1356 (tristan/fix-workspaced-junctions->master: Fix workspaced junctions) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1356 | 12:44 |
| gitlab-br-bot | tristanvb merged MR !1355 (tristan/fix-workspaced-junctions-1.2->bst-1.2: Fix workspaced junctions (1.2)) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1355 | 12:44 |
| juergbi | the grpcio-tools package is required and the command to generate is: ./setup.py build_grpc | 12:44 |
| juergbi | would have to be integrated into the rest | 12:44 |
| benschubert | tlater[m]: Pycharm can rebuild for you before running tests if you want :) | 12:45 |
| benschubert | juergbi: so the "build" command should do it, that's good news :) | 12:45 |
| benschubert | I'll also have to update the contributing guide | 12:45 |
| juergbi | benschubert: I'm not sure. build_grpc is a custom command | 12:45 |
| benschubert | juergbi: if I remember well, "build" will call all "build_*" commands too | 12:46 |
| juergbi | does 'build' pick those up implicitly? | 12:46 |
| benschubert | that's my understanding | 12:46 |
| juergbi | I suspected it does this only for known build subcommands | 12:46 |
| juergbi | but if it works, the better | 12:46 |
| benschubert | I'll double check and will create a custom Build otherwise | 12:47 |
| juergbi | ta | 12:47 |
| *** tristan has quit IRC | 13:07 | |
| gitlab-br-bot | raoul.hidalgocharman opened MR !1357 (raoul/1023-grpc-forking->master: Ensure grpc channels are in separate process) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1357 | 13:10 |
| raoul | ^^ may help with hanging tests | 13:10 |
| raoul | I thought I'd got it to hang again, but have not been able to reproduce it a second time | 13:11 |
| *** tristan has joined #buildstream | 13:15 | |
| benschubert | juergbi: grpc is using cython for itnerfacing with python right? | 13:16 |
| juergbi | not sure, has been a while since I've looked at these details | 13:17 |
| raoul | Yeah I think it does | 13:17 |
| benschubert | Ok | 13:17 |
| raoul | all the grpcio core stuff is cython | 13:17 |
| benschubert | with cython we might be able to interface better with it, so we don't have those annoying process problems anymore :) | 13:17 |
| benschubert | (that being a long term goal, I don't expect that to be easy) | 13:18 |
| juergbi | hm, don't know how that would help with the fork issue | 13:18 |
| benschubert | juergbi: depending how they export their stuff, we might be able to not get that imported (wild guess) | 13:18 |
| juergbi | unless it's only the python binding that has trouble with fork | 13:18 |
| Kinnison | cython adds a bunch of horror to process handling | 13:19 |
| juergbi | established sockets can't ever work across forks | 13:19 |
| Kinnison | because of forks etc | 13:19 |
| juergbi | but maybe it would behave more reasonably | 13:19 |
| raoul | I think grpcio uses some cython/c background threads which is why it behaves so poorly with fork so I'm not sure how we'd be able to get around that | 13:22 |
| benschubert | oh, that's sad | 13:22 |
| raoul | indeed :( | 13:23 |
| juergbi | well, as I said, it's not like connection sockets can reasonably be shared across forks anyway | 13:24 |
| juergbi | independent of whether grpc uses background threads or not | 13:25 |
| juergbi | however, the threads may be responsible for hangs | 13:25 |
| benschubert | I wonder if we should try to have a worker only for grpc | 13:25 |
| juergbi | and how do you communicate with it? | 13:26 |
| benschubert | mp.queues | 13:26 |
| juergbi | which internally uses pipes or sockets, I presume? | 13:26 |
| raoul | Yeah with the test hanging it's not like we were trying to share sockets, its the following forks getting confused | 13:26 |
| benschubert | juergbi: those would get closed on forks, so that is clean | 13:26 |
| benschubert | python will close all fd in the forked process | 13:27 |
| juergbi | I don't think it's worth the effort | 13:28 |
| juergbi | probably a lot less effort to add some kind of safeguard to avoid accidental use of grpc in the main process | 13:28 |
| juergbi | and to avoid establishing tons of connection to remote servers we'll soon have buildbox-casd | 13:29 |
| benschubert | right | 13:29 |
| benschubert | then we probably don't need it :) | 13:29 |
| *** ChanServ sets mode: +o ironfoot | 13:42 | |
| ironfoot | Turns out, if you are not identified on the network, you can't write on this channel.. and these error messages are a bit hidden on my IRC client.. | 13:44 |
| ironfoot | jennis: I'm and the one behind the bots | 13:44 |
| *** ironfoot has quit IRC | 13:46 | |
| *** ironfoot has joined #buildstream | 13:46 | |
| *** ChanServ sets mode: +o ironfoot | 13:46 | |
| tlater[m] | ironfoot: I run into that every couple of weeks too :| | 13:47 |
| ironfoot | there's a way to configure the channel to bounce unregistered people to another channel | 13:47 |
| Kinnison | Have your client auto-identify? | 13:47 |
| Kinnison | bouncing unregistered people makes the asynchronous nature of the irc services a pain | 13:48 |
| ironfoot | Kinnison: just done so, no SASL, right? | 13:48 |
| ironfoot | plain /msg nickserv? | 13:48 |
| * Kinnison checks | 13:48 | |
| Kinnison | I think I use client certificate fingerprinting | 13:48 |
| Kinnison | [05-24 14:48] * [Whois] Kinnison has client certificate fingerprint 958263f5f3cc43db00e617b4d836776237401ad7b8ac272c135a0e1c972c1fd7 | 13:48 |
| ironfoot | interesting, thanks | 13:49 |
| *** bochecha has quit IRC | 14:44 | |
| *** bochecha has joined #buildstream | 14:44 | |
| *** bochecha_ has joined #buildstream | 14:50 | |
| *** bochecha has quit IRC | 14:53 | |
| *** bochecha_ is now known as bochecha | 14:53 | |
| gitlab-br-bot | juergbi closed issue #1023 (CI test suite hanging) on buildstream https://gitlab.com/BuildStream/buildstream/issues/1023 | 15:14 |
| gitlab-br-bot | juergbi merged MR !1357 (raoul/1023-grpc-forking->master: Ensure grpc channels are in separate process) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1357 | 15:14 |
| *** kapip has joined #buildstream | 15:28 | |
| *** phildawson_ has joined #buildstream | 15:30 | |
| *** bochecha has quit IRC | 15:31 | |
| *** phil has quit IRC | 15:32 | |
| *** alatiera has quit IRC | 15:37 | |
| *** alatiera_ is now known as alatiera | 15:37 | |
| *** lachlan has quit IRC | 15:40 | |
| gitlab-br-bot | jonathanmaw approved MR !1351 (bschubert/normalize-path-in-context-tests->master: Fix: tests/context.py: Normalize path of XDG_CACHE) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1351 | 15:46 |
| gitlab-br-bot | marge-bot123 merged MR !1351 (bschubert/normalize-path-in-context-tests->master: Fix: tests/context.py: Normalize path of XDG_CACHE) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1351 | 15:50 |
| *** alatiera9 has joined #buildstream | 15:55 | |
| *** lachlan has joined #buildstream | 16:03 | |
| *** lachlan has quit IRC | 16:12 | |
| *** lachlan has joined #buildstream | 16:24 | |
| *** lachlan has quit IRC | 16:31 | |
| *** phildawson_ has quit IRC | 16:32 | |
| *** phildawson_ has joined #buildstream | 16:32 | |
| *** alatiera has quit IRC | 16:33 | |
| *** alatiera9 has quit IRC | 16:37 | |
| *** dftxbs3e has joined #buildstream | 16:39 | |
| *** alatiera9 has joined #buildstream | 16:46 | |
| *** lachlan has joined #buildstream | 16:47 | |
| *** phildawson_ has quit IRC | 16:48 | |
| *** raoul has quit IRC | 16:56 | |
| *** jonathanmaw has quit IRC | 17:00 | |
| *** bochecha has joined #buildstream | 18:13 | |
| *** lachlan has quit IRC | 18:40 | |
| *** lachlan has joined #buildstream | 19:46 | |
| *** lachlan has quit IRC | 20:17 | |
| *** kapip has quit IRC | 20:26 | |
| *** bochecha has quit IRC | 21:25 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!