| *** narispo has quit IRC | 03:24 | |
| *** narispo has joined #buildstream | 03:24 | |
| *** narispo has quit IRC | 03:47 | |
| *** narispo has joined #buildstream | 03:49 | |
| *** tristan has quit IRC | 06:05 | |
| *** tristan has joined #buildstream | 06:18 | |
| *** ChanServ sets mode: +o tristan | 07:09 | |
| tristan | junctions are fun | 07:09 |
|---|---|---|
| tristan | I just realized that if I can override the junction configuration in a subproject explicitly, that does _not_ mean we can just simply drop the junction in the subproject | 07:09 |
| tristan | I think it still means that we have to parse that junction for the overrides it makes on *it's* subprojects | 07:10 |
| tristan | juergbi, any thoughts on that ? | 07:11 |
| tristan | heh | 07:11 |
| tristan | maybe not | 07:11 |
| tristan | if its overridden from a higher level, that probably means disregarding any overrides made in an inbetween project | 07:12 |
| tristan | yeah, I'll run with that for now | 07:12 |
| *** benschubert has joined #buildstream | 07:26 | |
| *** aminbegood has joined #buildstream | 07:28 | |
| WSalmon | using yesterdays master, should we still be stopping a build over grpc errors? did we decide we did like this behavoir? https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/539214163 | 07:33 |
| tristan | Can we not get stdout/stderr printed to the console while tests are running anymore ? Or were we never able to do that ? | 07:38 |
| *** aminbegood has quit IRC | 07:48 | |
| *** seanborg has joined #buildstream | 07:53 | |
| benschubert | tristan: pytest -s works for some tests at least, but I think that since we capture the output in the clirunner, we have some weird stuff happening there | 07:54 |
| juergbi | tristan: junction, if I understand you correctly, I agree. the an overridden junction should not be used at all | 07:54 |
| WSalmon | has anyone run bst-1 lately? i dont know why this back port would cause that error, https://gitlab.com/BuildStream/buildstream/-/jobs/539240401 | 07:56 |
| tristan | benschubert, yeah it seems that we don't see the output while the test is running, if we pass `-s` then we only see the output when the test completes (if it completes) | 08:00 |
| tristan | I have an interesting little double failure here | 08:02 |
| tristan | since I'm no longer coalescing junctions by name, by default every junction is separate | 08:02 |
| benschubert | tristan: we can try to fix our cli runner to passthrough / print what it gets as we go, it would be really neat :) | 08:03 |
| tristan | This appears to mean that for tests/format/junctions.py::test_nested_double, multiple tasks get scheduled with the same name | 08:04 |
| tristan | So I get first an assertion from _state.py:State.add_task() | 08:04 |
| tristan | But then the fun starts | 08:04 |
| tristan | The scheduler handles this and says "Ok terminating tasks !" | 08:05 |
| tristan | and then self._process.terminate() gets called from job.py | 08:05 |
| tristan | And self._process is None at this point, triggering another assertion, causing the scheduler to say "heeeey there time to terminate some tasks again !" | 08:06 |
| tristan | and on we go | 08:06 |
| tristan | around and around :) | 08:06 |
| tristan | I think that the task identifier should include the element id | 08:07 |
| *** aminbegood has joined #buildstream | 08:08 | |
| tristan | Ah right now I remember what State is, that is the nice frontend abstraction we got from jonathanmaw | 08:08 |
| tristan | Hmmmmm, but this might not be a good idea, because if they are legitimately different tasks they should definitely show up differently in the frontend | 08:11 |
| tristan | juergbi, there is a hitch in the plan... which revolves around the old assumption that it's always okay to only print two levels of depth for element names in the UI | 08:12 |
| tristan | foo.bst:base.bst:target.bst and bar.bst:base.bst:target.bst are not guaranteed to be the same, and there is no requirement for the toplevel to have a base.bst junction defined | 08:13 |
| juergbi | right, hm | 08:13 |
| * tristan takes a moment to digest this, I don't like long element names in the UI :-S | 08:13 | |
| tristan | Although, it might all around be more clear what is going on if we do that | 08:14 |
| tristan | (where doing that = printing full depth for every element) | 08:14 |
| juergbi | what do we currently do if we have really long element names? ellipsize the end? | 08:14 |
| tristan | No, we horribly wrap | 08:14 |
| tristan | without any sense to it either | 08:14 |
| tristan | juergbi, we could make frontend enhancements though | 08:15 |
| tristan | with interactive mode, we can wrap the elements on the `:` if they are too wide and ensure well tabulated lines for multiline messages | 08:16 |
| tristan | and when running on some CI machine, we don't have a terminal and don't have any terminal width | 08:17 |
| tristan | in which case the sensible thing is the horizonally scroll (although some views might wrap instead) | 08:17 |
| robjh | ive observed an odd problem with buildstream where a build will fail when the remote cache is fully primed but the local cache is totally empty. https://gitlab.com/celduin/buildstream-bazel/minimal-rootfs/-/jobs/539255497 | 08:17 |
| juergbi | tristan: sounds reasonable | 08:18 |
| robjh | which lead me to write this little workaround that does give me green ticks in CI, but at the cost of failures being retried until the runner times out https://gitlab.com/celduin/buildstream-bazel/minimal-rootfs/-/commit/4b1f20e1b914f793e86ac6a0808a07f24abc1b07 | 08:18 |
| robjh | is this an issue people are aware of? | 08:18 |
| juergbi | robjh: which buildstream and buildbox version? | 08:18 |
| *** tpollard has joined #buildstream | 08:18 | |
| robjh | the ci is using whatever is in buildstream/buildstream:dev-extra | 08:19 |
| tristan | robjh, if you see a log file, you should be able to get the exact version at the beginning of the session log | 08:20 |
| robjh | locally. im using 1.93.1+206.g10de4185 and buildbox-casd from a download | 08:20 |
| juergbi | robjh: BuildStream Version 1.93.0.dev0. that sounds vary old (for an unstable version) | 08:20 |
| robjh | helpfully r@mayall-build:~/work/minimalroot$ buildbox-casd --version | 08:20 |
| robjh | buildbox-casd unknown | 08:20 |
| juergbi | https://gitlab.com/BuildStream/buildstream-docker-images/-/merge_requests/168 is still pending | 08:20 |
| juergbi | it probably doesn't make sense to debug this and first check whether this still happens after the above MR is merged | 08:21 |
| robjh | ok :) | 08:22 |
| *** jude has joined #buildstream | 08:22 | |
| juergbi | robjh: or do you see it locally as well? | 08:23 |
| robjh | juergbi, i do see it locally, but thats a 1.93.1 build too | 08:23 |
| juergbi | 1.93.1+206.g10de4185 is a lot more recent than 1.93.0. not completely up-to-date but I wouldn't expect such problems with it | 08:24 |
| juergbi | how easy is it to reproduce? | 08:24 |
| robjh | it takes about 10m, as i need to clear the cache completely | 08:24 |
| robjh | r@mayall-build:~/src/buildstream$ bst --version | 08:25 |
| robjh | 1.93.3+25.gceedc481 | 08:25 |
| robjh | just upgraded :) | 08:25 |
| juergbi | :) | 08:25 |
| juergbi | you may also have to fetch the latest buildbox binary tarball if you haven't already | 08:25 |
| robjh | unless upgrading is going to trigger the remote cache to be invalid, in which case; this'll take about an hour | 08:26 |
| robjh | lets do that too | 08:26 |
| robjh | this one? https://buildbox-casd-binaries.nyc3.cdn.digitaloceanspaces.com/buildbox-x86_64-linux-0.0.7-f938a187.tar.xz | 08:27 |
| juergbi | oh, I missed the buildbox-casd exit code at the end. -9 means it was killed with SIGKILL... | 08:28 |
| *** aminbegood has quit IRC | 08:28 | |
| robjh | oh aye? ram then? | 08:28 |
| juergbi | yes, could be OOM | 08:28 |
| coldtom | hi, i'm seeing "[buildboxcasd_localcasproxyinstance.cpp:47] [INFO] LocalCasProxyInstance::FindMissingBlobs request for instance name "7b76fb173740440fe0497943622dd60e891668ad2a3b4a727b33823da57daa62" for 512 digest(s)" in some casd logs, to clarify is the instance name here the one for the remote or the local one? | 08:28 |
| juergbi | buildbox-casd is not supposed to use that much RAM but there might still be bugs with large file in certain operations | 08:28 |
| juergbi | coldtom: that's the local buildbox-casd instance name (generated for the dynamic remote feature) | 08:29 |
| juergbi | (hash of the remote URL etc.) | 08:29 |
| coldtom | ta juergbi, i thought it would be | 08:30 |
| robjh | r@mayall-build:~/work/minimalroot$ free -h | 08:30 |
| robjh | total used free shared buff/cache available | 08:30 |
| robjh | Mem: 11Gi 155Mi 11Gi 0.0Ki 387Mi 11Gi | 08:30 |
| robjh | Swap: 0B 0B 0B | 08:30 |
| juergbi | if it uses too much RAM in a repeatable case, I can look into this to optimize/fix buildbox-casd | 08:31 |
| robjh | ahhh, suddenly the project doesnt meet the the min-version /o\ | 08:31 |
| juergbi | as you've tested this on 1.93.1+ before, it may be as simple as replacing format-version:.... with min-version: 2.0 | 08:31 |
| juergbi | as you junction to freedesktop-sdk, you may also have to update to a more recent freedesktop-sdk bst2 branch (valentind's branch should be up-to-date) | 08:32 |
| coldtom | (the up to date one is valentindavid/bst2 i think) | 08:33 |
| robjh | Element plugin 'docker_image' did not specify BST_MIN_VERSION ... crap | 08:37 |
| WSalmon | willsalmon/bst2 gets furethest so far but im hoping to merge that in to valentindavid/bst2 sson | 08:39 |
| WSalmon | soon | 08:39 |
| coldtom | robjh, you'll have to update bst-plugins-experimental | 08:39 |
| WSalmon | robjh, registry.gitlab.com/freedesktop-sdk/infrastructure/freedesktop-sdk-docker-images/bst2/amd64:dev-willsalmon-useragent may or may not have all you need | 08:41 |
| robjh | WSalmon, 404 error? | 08:43 |
| WSalmon | robjh, thats a docker image | 08:43 |
| * tristan finds it odd to have Element class holding a project_name instance variable, I suppose this optimizes things somehow | 08:45 | |
| *** santi has joined #buildstream | 08:47 | |
| robjh | i ordered more ram last night, and i look forward to using virtual machines with lots of ram to spare | 08:55 |
| tristan | we have more inconsistencies with element names, all over again :-S | 08:57 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/blob/master/src/buildstream/plugin.py#L758 (which does some ugliness of having knowledge of it's subclasses), and https://gitlab.com/BuildStream/buildstream/-/blob/master/src/buildstream/plugin.py#L758... both appear to do the same thing | 09:00 |
| * tristan will see if he can clean it up | 09:00 | |
| robjh | buildbox-run not found. i have buildbox-run-bubblewrao | 09:02 |
| robjh | updating my binaries | 09:03 |
| *** seanborg has quit IRC | 09:05 | |
| WSalmon | anyone know off the top of there heads why https://paste.gnome.org/p8rs9r4ic would work for bst1.4 but not bst1.9x? | 09:11 |
| *** tristan has quit IRC | 09:11 | |
| *** lachlan has joined #buildstream | 09:12 | |
| coldtom | https://gitlab.com/BuildStream/buildstream-docker-images/-/merge_requests/168 could do with a review if anyone has a minute | 09:12 |
| *** seanborg has joined #buildstream | 09:14 | |
| tpollard | WSalmon: I assume it's not giving a nice error? | 09:15 |
| *** devcurmudgeon has quit IRC | 09:16 | |
| *** ikerperez has quit IRC | 09:16 | |
| *** cphang has quit IRC | 09:16 | |
| *** cphang has joined #buildstream | 09:17 | |
| *** ikerperez has joined #buildstream | 09:17 | |
| *** lachlan has quit IRC | 09:24 | |
| *** tristan has joined #buildstream | 09:25 | |
| *** aminbegood has joined #buildstream | 09:34 | |
| robjh | so i saw a similar problem with buildbox dieing, i had 6gb in use, i have 12gb assigned to this vm (no thin provisioning) | 09:36 |
| robjh | im unsure which log file is going to be useful | 09:36 |
| robjh | latest everything | 09:37 |
| WSalmon | it gives d`eploy/filesystem.bst [line 7 column 2]: Value of 'build-depends' is not of the expected type 'list'` | 09:40 |
| WSalmon | it gives `deploy/filesystem.bst [line 7 column 2]: Value of 'build-depends' is not of the expected type 'list'` | 09:41 |
| tpollard | have you tried it without `filename:` ? | 09:43 |
| *** devcurmudgeon has joined #buildstream | 09:46 | |
| *** lachlan has joined #buildstream | 09:47 | |
| *** aminbegood has quit IRC | 09:49 | |
| juergbi | robjh: the casd log file may be useful, is in ~/.cache/buildstream/logs/_casd | 09:50 |
| juergbi | the full filename should be at the end of the bst output when casd dies | 09:51 |
| juergbi | if it's reproducible on a particular project, I can give it a try locally as well | 09:51 |
| robjh | okay, looking at that log makes me think installing fusermount might help. let me retry this | 09:53 |
| juergbi | fuse3 is now required for normal Linux installations. however, the absence of fusermount3 should not result in buildbox-casd dying | 09:56 |
| *** lachlan has quit IRC | 09:59 | |
| robjh | yeah, this error is different from before | 10:05 |
| *** lachlan has joined #buildstream | 10:13 | |
| robjh | to update you juergbi, compiling libfuse from source has got rid of the error. | 10:23 |
| robjh | im now trying to compel bst-plugins-container into working with master | 10:23 |
| juergbi | ok, good. fyi: fuse3 is also available as a package in most somehwat recent distros | 10:24 |
| juergbi | robjh: nice. please note that there is already an oci element plugin with docker support in bst-plugins-experimental | 10:25 |
| juergbi | not sure whether there is a need for both | 10:25 |
| juergbi | nothing like the docker source plugin is in bst-plugins-experimental, though | 10:26 |
| robjh | juergbi, please tell me more! :) | 10:26 |
| robjh | so it is. i guess i didnt need to install it from source | 10:27 |
| juergbi | there is some documentation here https://gitlab.com/BuildStream/bst-plugins-experimental/-/blob/master/src/bst_plugins_experimental/elements/oci.py | 10:27 |
| *** lachlan has quit IRC | 10:32 | |
| *** tpollard has quit IRC | 10:34 | |
| *** tpollard has joined #buildstream | 10:34 | |
| robjh | kind: OciElement? kind: oci? | 10:38 |
| robjh | in both cases; No Element type registered for kind 'oci' :/ | 10:39 |
| juergbi | robjh: you need to add it to the bst-plugins-experimental element plugin list in project.conf | 10:42 |
| juergbi | kind: oci is correct | 10:42 |
| robjh | ooh | 10:43 |
| robjh | okay, getting there :) | 10:45 |
| WSalmon | I still cant work out why https://gitlab.com/celduin/bsps/example-app/-/blob/0a38bb9d9674b5d4416c4713ecd05045edf6d49b/elements/deploy/filesystem.bst gives https://gitlab.com/celduin/bsps/example-app/-/jobs/540589180 i looked at the bst2 tests and they seem happy with this sysntax.. | 10:52 |
| WSalmon | juergbi, benschubert | 10:52 |
| robjh | does it think line4 is a key value pair because of the colon? | 10:54 |
| robjh | put it in quotes? | 10:54 |
| tpollard | yes, try without filename, it's not needed anyway | 10:55 |
| WSalmon | - system/userland.bst <- dose not work | 10:56 |
| tpollard | same error? | 10:56 |
| WSalmon | deploy/filesystem.bst [line 7 column 2]: Value of 'build-depends' is not of the expected type 'list' | 10:57 |
| tpollard | hmm | 10:57 |
| WSalmon | if i remove the dash i get a serverly mall formed yaml error | 10:57 |
| robjh | line 7 | 10:58 |
| tpollard | what about if you just do `(@): bsp.bst:elements/deploy/filesystem.bst` | 10:59 |
| tpollard | and then add the build-depends | 11:00 |
| WSalmon | robjh, that gives suveraly mall formed yaml | 11:00 |
| *** seanborg has quit IRC | 11:01 | |
| *** seanborg has joined #buildstream | 11:01 | |
| robjh | WSalmon, 4 spaces on line 8? | 11:01 |
| WSalmon | nope and nope | 11:02 |
| WSalmon | this feels odd as this works just fine in bst1.4... and the tests have things like tests/format/list-directive-error-element/config.bst | 11:03 |
| robjh | infact, is (>): needed at all? | 11:03 |
| WSalmon | i will work round this for this case but in some cases i think it will be | 11:04 |
| robjh | i mean, it might be complaining because build-depends hasnt been previously defined and so its got nothing to append to? i dunno im just guessing | 11:05 |
| WSalmon | I have worked it out | 11:05 |
| WSalmon | My fault | 11:05 |
| WSalmon | thanks for all the good sugestions tho | 11:05 |
| *** lachlan has joined #buildstream | 11:05 | |
| WSalmon | I had updated the example but not the bsp to master silly me | 11:06 |
| WSalmon | that error really needs to be better tho | 11:06 |
| WSalmon | now i understand i will write it up as a issue | 11:06 |
| tpollard | 95% of the yaml errors I get need to be better | 11:11 |
| *** lachlan has quit IRC | 11:11 | |
| *** lachlan has joined #buildstream | 11:36 | |
| *** lachlan has quit IRC | 11:42 | |
| *** lachlan has joined #buildstream | 11:45 | |
| *** lachlan has quit IRC | 11:45 | |
| *** lachlan has joined #buildstream | 11:50 | |
| *** aminbegood has joined #buildstream | 11:55 | |
| *** lachlan has quit IRC | 11:57 | |
| benschubert | tpollard: please send reproductions and open issues :) It's hard to "test" each case, and the more un-intuitive failures we get, the better we'll get tests and behavior | 12:17 |
| *** aminbegood has quit IRC | 12:30 | |
| *** seanborg has quit IRC | 12:31 | |
| *** seanborg has joined #buildstream | 12:31 | |
| *** seanborg has quit IRC | 12:37 | |
| WSalmon | benschubert, that one i was stuggling with was trying to extend something that wasnt in the include, i will write up | 12:38 |
| *** seanborg has joined #buildstream | 12:38 | |
| WSalmon | why do we list out each group of plugins per plugin source but not label the source? | 12:43 |
| *** lachlan has joined #buildstream | 13:20 | |
| *** lachlan has quit IRC | 13:29 | |
| *** lachlan has joined #buildstream | 13:55 | |
| *** lachlan has quit IRC | 13:56 | |
| *** seanborg has quit IRC | 14:11 | |
| *** seanborg has joined #buildstream | 14:11 | |
| *** seanborg_ has joined #buildstream | 15:02 | |
| *** seanborg_ has quit IRC | 15:03 | |
| *** seanborg_ has joined #buildstream | 15:03 | |
| *** seanborg has quit IRC | 15:03 | |
| *** seanborg_ has quit IRC | 15:05 | |
| *** seanborg_ has joined #buildstream | 15:05 | |
| *** lachlan has joined #buildstream | 15:10 | |
| *** lachlan has quit IRC | 15:12 | |
| *** pointswaves has joined #buildstream | 15:39 | |
| valentind | WSalmon, Did you see the tests for your backport to bst-1 did not pass? | 16:04 |
| valentind | repo.remote_gpg_import(remote, stream, None, None) | 16:04 |
| valentind | TypeError: Must be number, not NoneType | 16:04 |
| juergbi | seems like an issue independent of the branch | 16:07 |
| juergbi | no idea what changed to trigger this, though | 16:07 |
| valentind | juergbi, was there an update in the docker image for the build? | 16:08 |
| juergbi | no | 16:08 |
| valentind | According to ostree, the parameters can be none. | 16:08 |
| valentind | According to the doc. | 16:08 |
| valentind | Except the first which must be a string. | 16:08 |
| valentind | Test passes on my machine. | 16:11 |
| valentind | Ah, it is integration. | 16:11 |
| *** aminbegood has joined #buildstream | 16:11 | |
| valentind | juergbi, Maybe the gpg key is expired. | 16:18 |
| juergbi | that would be a very odd error message | 16:20 |
| valentind | It is the same file. | 16:21 |
| valentind | And it expires 2027-06-14 | 16:21 |
| * tpollard wonders what has happened to the doc pages https://buildstream.gitlab.io/bst-plugins-experimental/index.html | 16:21 | |
| valentind | It passed on my machine | 16:21 |
| juergbi | it also passes on non-debian-based distros in CI, right? | 16:22 |
| valentind | fedora yes | 16:24 |
| valentind | But it works on my machine that is debian. | 16:24 |
| valentind | Why are the docker images not public? | 16:27 |
| valentind | unauthorized: HTTP Basic: Access denied | 16:27 |
| juergbi | hm, not sure, I've pulled docker testsuite images successfully before | 16:36 |
| valentind | Did it pull a wrong version of python-gi? | 16:44 |
| valentind | I do not see gi in the requirements. Is it installed in the docker image? | 16:45 |
| *** tpollard has quit IRC | 16:46 | |
| valentind | Ah yes PyGObject==3.32.2 | 16:47 |
| valentind | Bumping pygobject does not seem to change anything. | 16:57 |
| valentind | juergbi, Is the time wrong on the builders? | 17:03 |
| juergbi | Session Start: Tuesday, 05-05-2020 at 17:02:22 | 17:05 |
| juergbi | looks ok | 17:05 |
| juergbi | matches job trigger time + 2 minutes | 17:05 |
| valentind | The date is right. | 17:07 |
| *** santi has quit IRC | 17:07 | |
| valentind | I would suggest to disable the test. | 17:07 |
| pointswaves | valentind, juergbi I spent a small amount of time this morgning working out that i had no idea what was casuing that issue with the back port | 17:14 |
| pointswaves | juergbi, were do i ask about buildbox-casd errors? | 17:14 |
| juergbi | pointswaves: there is a slack channel but you can also ask here, at least if it's somewhat buildstream related | 17:16 |
| pointswaves | juergbi, ish, im trying to get buildgrid to work so i can remoteX | 17:18 |
| pointswaves | /usr/local/bin/ldbuildbox-casd --cas-remote=http://localhost:50051 --bind=127.0.0.1:50011 /buildbox/cached/ | 17:18 |
| pointswaves | this craps out | 17:18 |
| pointswaves | ubuntu@ip-172-31-22-241:~$ sudo -u casduser /usr/local/bin/buildbox-casd --cas-remote=http://localhost:50051 --bind=127.0.0.1:50011 /buildbox/cached/ | 17:18 |
| pointswaves | terminate called after throwing an instance of 'std::invalid_argument' | 17:18 |
| pointswaves | what(): std::invalid_argument exception thrown at [buildboxcasd_fslocalcas.cpp:169], errMsg = "`path` is not in /buildbox/cached//cas/tmp" | 17:18 |
| pointswaves | Aborted | 17:18 |
| pointswaves | sorry for the spam i mean https://paste.gnome.org/pk5x6rdhy | 17:19 |
| juergbi | pointswaves: oh, maybe there is an issue with path normalization. can you try removing the trailing slash from the path? | 17:19 |
| pointswaves | juergbi, i think that did it | 17:20 |
| pointswaves | its now complaining about the upstream cache | 17:20 |
| pointswaves | but with a nice error message | 17:20 |
| juergbi | ok, good. would you mind opening an issue? | 17:20 |
| pointswaves | if i remove the remote cache it seems to work | 17:21 |
| pointswaves | juergbi, no probs | 17:21 |
| juergbi | ta | 17:21 |
| pointswaves | juergbi, https://gitlab.com/BuildGrid/buildbox/buildbox-casd/-/issues/68 | 17:26 |
| pointswaves | let me know if you need more detail | 17:27 |
| pointswaves | juergbi, is ther a wait for cas argument or am i going to have to be careful how i bring these up? | 17:29 |
| juergbi | pointswaves: it needs to connect early on, retrieving the CAS server capabilities | 17:37 |
| juergbi | not sure how to best improve this | 17:37 |
| *** jude has quit IRC | 17:40 | |
| pointswaves | i think systemd will just retry eveything till it all works, systemd can do the order but the waiting bit can be more tricky | 17:44 |
| *** narispo has quit IRC | 17:46 | |
| *** narispo has joined #buildstream | 17:46 | |
| juergbi | pointswaves: on the same system we could theoretically use socket activation to avoid races. however, gRPC doesn't support that yet | 17:49 |
| juergbi | and in typical larger setups you have multiple workers each with their own buildbox-casd and BuildGrid running on separate server(s). in that case socket activation would obviously not be possible | 17:49 |
| pointswaves | juergbi, yep, so far there on the same host but my plan is to put the buildgrid bits some were persistant so it shouldnt be a issue | 17:57 |
| pointswaves | but on a cluster when you start the casd's might go up faster then buildgrid so k8 or something would just keep restarting the docker image until it was up i gues | 17:59 |
| *** aminbegood has joined #buildstream | 18:07 | |
| *** seanborg_ has quit IRC | 18:37 | |
| *** tpollard has joined #buildstream | 18:38 | |
| *** tpollard has quit IRC | 18:42 | |
| *** pointswaves has quit IRC | 18:55 | |
| *** pointswaves has joined #buildstream | 18:56 | |
| pointswaves | juergbi, the chatter on slack is a bit hit and miss.. | 18:59 |
| *** aminbegood has quit IRC | 19:07 | |
| pointswaves | Do we have a minimal buildgrid config some were | 19:43 |
| pointswaves | i am getting | 19:43 |
| pointswaves | [20:41:55][--:--:--][ ][ main:core activity ] WARNING Failed to initialize remote http://3.250.149.126:50051: Configured remote does not have the BuildStream capabilities service. Please check remote configuration. | 19:43 |
| pointswaves | I thought i had enough in my config | 19:43 |
| pointswaves | scottclarke, ^? | 19:44 |
| juergbi | pointswaves: BuildGrid does not implement a BuildStream artifact server | 19:53 |
| juergbi | for the BuildStream artifact protos you still need bst-artifact-server | 19:53 |
| juergbi | BuildGrid can be used for the CAS part | 19:53 |
| juergbi | however, this is somewhat orthogonal from remote execution, i.e., you don't need an artifact server for remote execution, and you can set up an artifact server without remote execution | 19:54 |
| juergbi | and the plan is to migrate BuildStream towards using the new Remote Asset API as replacement for the BuildStream-specific protocl | 19:55 |
| juergbi | when that happens, there won't be a BuildStream-specific server anymore | 19:55 |
| juergbi | you could take a look at the config we use for testing remote execution in CI | 19:56 |
| pointswaves | ah right, i knew this was a issue with buildbarn didnt realise buildgrid didnt have one ether | 19:56 |
| juergbi | .gitlab-ci/buildgrid-compose.yml in buildstream sets up the docker containers | 19:56 |
| pointswaves | juergbi, is https://gitlab.com/BuildStream/buildstream/-/blob/master/.gitlab-ci/buildgrid-compose.yml#L56 the config coming from the docker image | 19:59 |
| pointswaves | ? | 19:59 |
| juergbi | pointswaves: yes but that's actually from the upstream buildgrid repo | 20:00 |
| pointswaves | yep | 20:01 |
| juergbi | https://gitlab.com/BuildGrid/buildgrid/-/tree/master/data/config | 20:01 |
| juergbi | the ReferenceStorage part is outdated, though, we should remove it from BuildGrid | 20:01 |
| pointswaves | oh i assumed that was the magic | 20:02 |
| juergbi | it's a bit confusing as the protocol has changed over time | 20:02 |
| juergbi | we used to have a generic protocol but it was defined as part of BuildStream, not used outside | 20:03 |
| juergbi | then we switched to a fully BuildStream-specific protocol | 20:03 |
| juergbi | and now the plan is to switch to a new generic protocol that was developed in the bazel remote execution group and should thus be used outside BuildStream as well | 20:04 |
| juergbi | BuildGrid implemented that first generic protocol (reference storage) | 20:04 |
| pointswaves | so thats good for now? | 20:04 |
| juergbi | no, it's not useful anymore | 20:05 |
| juergbi | it remains a bit messy until we complete the planned migration to the new Remote Asset API | 20:05 |
| juergbi | until that's the case you need to use bst-artifact-server for the artifact proto part | 20:06 |
| juergbi | you can still use arbitrary CAS servers for actual payload storage, though | 20:06 |
| juergbi | I can hopefully start working on the migration to the Remote Asset API within a couple of weeks or so. however, this will take a while | 20:07 |
| pointswaves | juergbi, no pressure from me, if i can get something that just about works to unlock RE i will then focus on using my little projects | 20:12 |
| pointswaves | thanks for all the help | 20:12 |
| juergbi | yes, it should definitely already be in a working state, and we keep testing this also in CI | 20:13 |
| pointswaves | right it looks like its working for now | 20:20 |
| pointswaves | but 2gb is not gona last long vs FDSDK | 20:20 |
| pointswaves | ahhh miss read the log | 20:21 |
| *** pointswaves has quit IRC | 22:34 | |
| *** lachlan has joined #buildstream | 22:46 | |
| *** lachlan has quit IRC | 22:55 | |
| *** rdale has quit IRC | 23:28 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!