| *** rdale has quit IRC | 03:05 | |
| *** benbrown_ has quit IRC | 04:16 | |
| *** benbrown_ has joined #baserock | 04:23 | |
| *** gtristan has quit IRC | 04:50 | |
| *** gtristan has joined #baserock | 05:14 | |
| *** jude_ has joined #baserock | 07:30 | |
| *** ctbruce has joined #baserock | 07:47 | |
| *** toscalix has joined #baserock | 07:59 | |
| *** rdale has joined #baserock | 08:11 | |
| *** rdale has quit IRC | 08:13 | |
| *** rdale has joined #baserock | 08:17 | |
| *** jonathanmaw has joined #baserock | 08:41 | |
| *** ssam2 has joined #baserock | 09:05 | |
| *** ChanServ sets mode: +v ssam2 | 09:05 | |
| *** gtristan has quit IRC | 09:32 | |
| *** gtristan has joined #baserock | 09:37 | |
| *** locallycompact has joined #baserock | 09:59 | |
| *** noisecell has joined #baserock | 10:05 | |
| ssam2 | jjardon, benbrown_: hey, how can I get access to the cache at digitalocean ? | 10:19 |
|---|---|---|
| ssam2 | i have access to the manager machine but i don't know how to get from there to the cache | 10:19 |
| ssam2 | and would like to debug why none of the BuildStream sources are being shared between jobs in CI | 10:19 |
| jjardon | let me refresh my memory ... | 10:23 |
| gitlab-br-bot | infrastructure: merge request (kill-obsolete->master: Kill obsolete system definitions) #8 changed state ("opened"): https://gitlab.com/baserock/infrastructure/merge_requests/8 | 11:51 |
| benbrown_ | ssam2: I don't think the cache is *at* digitalocean | 11:52 |
| benbrown_ | it's a docker volume on the runner | 11:52 |
| benbrown_ | which you could inspect via docker-machine ssh runner-bla | 11:52 |
| ssam2 | ok, i'll try that | 11:52 |
| ssam2 | we did try to set up a distributed cache though | 11:53 |
| ssam2 | https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching | 11:54 |
| ssam2 | as part of this investigation i will document the actual setup in baserock infrastructure.git | 11:54 |
| gitlab-br-bot | infrastructure: merge request (kill-obsolete->master: Kill obsolete system definitions) #8 changed state ("opened"): https://gitlab.com/baserock/infrastructure/merge_requests/8 | 14:22 |
| *** gtristan has quit IRC | 14:23 | |
| gitlab-br-bot | infrastructure: merge request (kill-obsolete->master: Kill obsolete system definitions) #8 changed state ("opened"): https://gitlab.com/baserock/infrastructure/merge_requests/8 | 14:25 |
| ironfoot | ssam2: ^ looks good, ready to merge? | 14:38 |
| ironfoot | I can't just "approve" | 14:39 |
| ssam2 | yeah ready i think | 14:39 |
| ssam2 | we probably didn't configure that repo to require approvals | 14:39 |
| gitlab-br-bot | infrastructure: merge request (kill-obsolete->master: Kill obsolete system definitions) #8 changed state ("merged"): https://gitlab.com/baserock/infrastructure/merge_requests/8 | 14:40 |
| ssam2 | i've changed the config to require approvals | 14:41 |
| ironfoot | ta! | 14:41 |
| ssam2 | and added you, me, jjardon and benbrown_ as the approvers | 14:41 |
| *** gtristan has joined #baserock | 14:45 | |
| *** locallycompact has quit IRC | 14:45 | |
| *** locallycompact has joined #baserock | 14:47 | |
| *** adds68 has quit IRC | 14:57 | |
| *** jude_ has quit IRC | 15:01 | |
| *** paulwaters_ has joined #baserock | 15:01 | |
| ssam2 | seems that the distributed cache is working, but all it stores is an empty directory tree | 15:24 |
| ssam2 | /cache/ybd_base/gits | 15:25 |
| ssam2 | what's the correct way to tell YBD "store gits here" ? | 15:25 |
| ssam2 | we are setting XDG_CACHE_HOME which i thought should work | 15:25 |
| ssam2 | in fact, I do see this in the ybd output | 15:26 |
| ssam2 | gits=/cache/ybd_base/gits | 15:26 |
| ssam2 | i guess it didn't build anything recently, but seems odd that it would have never downloaded any sources at all... | 15:27 |
| ssam2 | aha | 15:28 |
| ssam2 | as in https://gitlab.com/baserock/definitions/-/jobs/31740842, ybd is doing as told, it's just that gitlab is ignoring everything in /cache | 15:29 |
| ssam2 | i've tried creating a `pwd`/cache symlink in the hope that we can trick it | 15:29 |
| ssam2 | got to wait for the whole freedesktop SDK to download yet again before i'll see if that worked though | 15:29 |
| *** noisecell has quit IRC | 15:32 | |
| * benbrown_ notes that, assuming the hack works, it will upload and pass around a .zip with everything in that path | 15:32 | |
| benbrown_ | Which, I expect, will be *huge*. | 15:32 |
| ssam2 | yeah | 15:32 |
| ssam2 | but it's just going from one digitalocean droplet to another, so hopefully pretty quick to transfera nyway | 15:33 |
| ssam2 | *anyway | 15:33 |
| ssam2 | and it's only gits, so i don't expect it to be more than 10s of GBs | 15:33 |
| ssam2 | well, gits and ostree binaries | 15:33 |
| * benbrown_ nods | 15:34 | |
| ssam2 | if the hack doesn't work, perhaps we could change the /cache volume to be at /build | 15:35 |
| ssam2 | and then we could store everything inside the builddir without running out of space | 15:35 |
| gtristan | caching build dir is... well it will mean failed builds will accumulate I suspect | 15:54 |
| gtristan | ssam2, ybd.conf sets up the directories, in a similar way to buildstream.conf (although I suspect you figured that by now) | 15:55 |
| ssam2 | yeah i don't want to cache the whole build dir | 15:57 |
| ssam2 | just need to satisfy gitlab-ci's seemingly arbitrary requirement that i only cache things that are within the build dir | 15:57 |
| ssam2 | i guess it's a sanity check to stop people caching / or whatever | 15:57 |
| gtristan | I thought it already was successfully caching ybd's sources | 15:58 |
| gtristan | in which case, all you need is to configure ybd & buildstream to put their sources under the same cached subdir | 15:58 |
| ssam2 | turns out it isn't | 15:58 |
| gtristan | really ? | 15:58 |
| gtristan | wow | 15:58 |
| ssam2 | look at the bottom of https://gitlab.com/baserock/definitions/-/jobs/31740842 for example | 15:58 |
| ssam2 | its helpfully ignoring all the sources ybd downloaded because they're in /cache | 15:59 |
| gtristan | weird message | 15:59 |
| ssam2 | my symlink trick seems to work though | 16:00 |
| ssam2 | https://gitlab.com/baserock/definitions/-/jobs/35082679 | 16:00 |
| gtristan | ah right, in buildstream we cache things in cache/ | 16:00 |
| gtristan | not /cache/ | 16:01 |
| gtristan | seems probably a sensible limitation | 16:01 |
| ssam2 | yeah | 16:01 |
| gtristan | allowing .gitlab-ci.yml to walk outside of it's cwd probably invites disaster, whenever you want to change your mounts and surrounding layout or whatever | 16:02 |
| gtristan | (or whenever gitlab wants to refactor how the workers work) | 16:02 |
| gitlab-br-bot | infrastructure: merge request (document-ci->master: Document how the Baserock runners for GitLab CI are set up) #9 changed state ("opened"): https://gitlab.com/baserock/infrastructure/merge_requests/9 | 16:09 |
| *** ctbruce has quit IRC | 16:13 | |
| gitlab-br-bot | definitions: merge request (sam/fix-ci->master: Fix caching in CI) #62 changed state ("opened"): https://gitlab.com/baserock/definitions/merge_requests/62 | 16:21 |
| gitlab-br-bot | infrastructure: merge request (document-ci->master: Document how the Baserock runners for GitLab CI are set up) #9 changed state ("merged"): https://gitlab.com/baserock/infrastructure/merge_requests/9 | 16:24 |
| gitlab-br-bot | definitions: merge request (sam/fix-ci->master: Fix caching in CI) #62 changed state ("opened"): https://gitlab.com/baserock/definitions/merge_requests/62 | 16:24 |
| ironfoot | ssam2: I found some issues in https://gitlab.com/baserock/infrastructure/merge_requests/9 but has already been merged | 16:29 |
| ssam2 | oh sorry | 16:29 |
| ssam2 | my badf | 16:30 |
| ssam2 | *bad | 16:30 |
| ssam2 | wait, this makes no sense | 16:30 |
| ssam2 | ok, one of the commits in that branch should have been in the previous MR :_)( | 16:30 |
| ssam2 | :-) | 16:30 |
| ssam2 | ok yes, I see my mistake | 16:31 |
| ssam2 | will do another MR | 16:31 |
| ssam2 | argh, i just pushed the fix to master instead | 16:37 |
| * ssam2 alters settings to make that less possible | 16:37 | |
| ssam2 | i've enabled merge request removals again, too. somehow the box was unticked even though i did tick it before | 16:38 |
| ssam2 | s/removals/approvals/ | 16:38 |
| *** toscalix has quit IRC | 16:39 | |
| ssam2 | hmm, my changes still don't work | 16:43 |
| * ssam2 watches as buildstream pulls the freedesktop sdk all over again | 16:43 | |
| *** jonathanmaw has quit IRC | 16:44 | |
| ssam2 | I guess the cache is extracted before the symlink is created | 16:45 |
| gitlab-br-bot | definitions: merge request (sam/fix-ci->master: Fix caching in CI) #62 changed state ("opened"): https://gitlab.com/baserock/definitions/merge_requests/62 | 16:53 |
| *** paulwaters_ has joined #baserock | 16:54 | |
| *** paulwaters_ has quit IRC | 16:56 | |
| gitlab-br-bot | definitions: merge request (sam/fix-ci->master: Fix caching in CI) #62 changed state ("opened"): https://gitlab.com/baserock/definitions/merge_requests/62 | 17:00 |
| *** locallycompact has quit IRC | 17:07 | |
| *** ssam2 has quit IRC | 17:12 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!