| *** mohan43u_ is now known as mohan43u | 02:05 | |
| *** slaf has quit IRC | 02:52 | |
| *** mohan43u has quit IRC | 04:19 | |
| *** mohan43u has joined #buildstream | 04:48 | |
| *** slaf has joined #buildstream | 06:12 | |
| *** tristan has quit IRC | 06:34 | |
| *** tristan has joined #buildstream | 07:12 | |
| *** pointswaves has joined #buildstream | 07:45 | |
| *** pointswaves has quit IRC | 08:13 | |
| *** ChanServ sets mode: +o tristan | 08:22 | |
| tristan | grrr | 08:22 |
|---|---|---|
| tristan | How come I cannot push a new bst-1.4 branch ?? | 08:22 |
| tristan | ! [remote rejected] bst-1.4 -> bst-1.4 (pre-receive hook declined) | 08:23 |
| tristan | Seems like you need to change the protected branches setting in https://gitlab.com/BuildStream/buildstream/-/settings/repository to work around this | 08:32 |
| *** aminbegood has joined #buildstream | 08:38 | |
| abderrahim[m] | or you can create the branch from the UI | 08:39 |
| abderrahim[m] | without actually "pushing" things to it | 08:39 |
| tristan | abderrahim[m], can you ? | 08:40 |
| tristan | maybe you can, I didnt dig that deep | 08:40 |
| abderrahim[m] | I think I did it once | 08:40 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/branches has a "New branch" button indeed | 08:40 |
| * tristan is starting to cook up some 1.6 related material | 08:41 | |
| tristan | The junction include backport: !1912 | 08:42 |
| tristan | hmmm, might have issues, I was surprised that the format tests passed on the first try | 08:43 |
| *** aminbegood has quit IRC | 08:47 | |
| tristan | hmmm, I thought we had fixed that issue with ostree gpg API | 09:00 |
| tristan | jjardon, fixed it in 1318bb7f99086013e24fddd20965af60e1c7f706, but half of our CI is broken ?? | 09:02 |
| *** benschubert has joined #buildstream | 09:31 | |
| tristan | Did we recently change the docker images used in bst-1 CI ? | 09:33 |
| benschubert | not sure, why? | 09:35 |
| benschubert | oh tristan since you're here and if you have time :) I'm battling with design questions around cache key tests | 09:36 |
| tristan | benschubert, the bst-1 branch CI is totally borked | 09:36 |
| benschubert | let me see | 09:37 |
| tristan | it's that repo.remote_gpg_import() back to haunt us from the dead | 09:37 |
| tristan | https://gitlab.com/BuildStream/buildstream/pipelines/144325901 | 09:37 |
| tristan | That is a pipeline which triggered by side effect of creating the bst-1.4 branch for example | 09:37 |
| tristan | fedora tests pass, and debian/ubuntu tests fail | 09:37 |
| tristan | With an incorrect error message I'm presuming | 09:38 |
| *** aminbegood has joined #buildstream | 09:38 | |
| tristan | We're getting 'TypeError: Must be number, not NoneType' from repo.remote_gpg_import(remote, stream, None, None) | 09:38 |
| *** toscalix has joined #buildstream | 09:38 | |
| tristan | But the last 2 parameters in the underlying C API are return pointer locations | 09:39 |
| benschubert | so the debian 10 is new (4th of april) | 09:39 |
| benschubert | the others havent moved for quite a bit and are actually not in sync with the rest :/ | 09:40 |
| tristan | it must have passed it's CI when it was merged ? | 09:40 |
| benschubert | correct | 09:40 |
| * tristan thinks it would be nice to see "What is the last passing CI run for this branch" | 09:40 | |
| benschubert | you can see that normally | 09:40 |
| tristan | Or rather, do we have some distortion in here ? if everything passed CI to land, how can it stop working on a later date ? | 09:41 |
| benschubert | I don't think we are pinning bst-plugins-experimental dependnecies? | 09:42 |
| benschubert | so maybe that? | 09:42 |
| tristan | This is bst-1 though | 09:42 |
| benschubert | ah good point | 09:42 |
| *** aminbegood has quit IRC | 09:43 | |
| benschubert | it's the bst-1 branch right? Not bst-1.4 | 09:43 |
| tristan | they are currently exactly the same | 09:44 |
| tristan | I just branched bst-1.4 for the sake of https://gitlab.com/BuildStream/buildstream/-/merge_requests/1912 | 09:44 |
| benschubert | according to https://gitlab.com/BuildStream/buildstream/-/commit/62eee7be681c74c6b6aa679acac9d27bf1b871e9/pipelines, there hasn't been a single successful pipeline there | 09:45 |
| tristan | then !1912 failed CI, for this unrelated change... looking at .gitlab-ci.yml and requirements files, there is no variables | 09:45 |
| tristan | i.e. PyGobject is pinned and I suppose ostree is from the docker | 09:45 |
| benschubert | and https://gitlab.com/BuildStream/buildstream/-/merge_requests/1898 was force pushed | 09:45 |
| tristan | How did that happen ? | 09:46 |
| tristan | I had to reach into settings just to create bst-1.4, because apparently, no stabled branches are allowed to be created, by anyone (as per protected branch rules) | 09:47 |
| tristan | in fact 62eee7be681c74c6b6aa679acac9d27bf1b871e9 | 09:48 |
| tristan | there is a gitlab generated fluff commit, so it was probably not force pushed | 09:48 |
| benschubert | I'm not sure? seems like juergbi merged it? | 09:48 |
| benschubert | or am I reading all of this right? | 09:49 |
| tristan | unless juergbi hand crafted the fluff commit but I doubt he would have done it so perfectly | 09:49 |
| benschubert | https://gitlab.com/BuildStream/buildstream/-/merge_requests/1898 the pipeline here seems to have failed right? Though it went it | 09:49 |
| tristan | benschubert, I think the confusing part is that when you look at a merge request, the pipeline linked to it changes over time | 09:50 |
| benschubert | https://gitlab.com/BuildStream/buildstream/-/merge_requests?scope=all&utf8=%E2%9C%93&state=merged&target_branch=bst-1 | 09:50 |
| tristan | it was possibly merged with a successful pipeline, but it might have been the branch tip when a nightly pipeline failed ? | 09:50 |
| juergbi | tristan: I simply added a line to link to the MR... | 09:50 |
| juergbi | yes, I manually merged it as it was blocked by unrelated CI issue which we couldn't immediately track down | 09:51 |
| tristan | juergbi, you disabled the setting and force pushed it with a fabricated merge commit ? | 09:51 |
| tristan | Ok | 09:51 |
| benschubert | ah :) that explains, thanks | 09:51 |
| juergbi | i temporarily gave myself permission to push directly | 09:51 |
| benschubert | and it's the first one doing so | 09:51 |
| benschubert | the previous commit was fine | 09:51 |
| tristan | I don't really feel comfortable doing anything at all to bst-1 sans fixing the broken CI | 09:52 |
| juergbi | does the manual merge cause any issues? | 09:52 |
| tristan | especially that it is a clear indication that BuildStream 1 is broken :-S | 09:52 |
| juergbi | we definitely need to fix the bst-1 CI properly. however, it was also clearly not related to the branch and non-debian-based CI passes | 09:52 |
| juergbi | it seems debian-based CI broke without any change on our side (also no docker image change) | 09:53 |
| tristan | Yeah, it's the gobject introspection horror show | 09:53 |
| juergbi | but do you know which component changed? | 09:53 |
| juergbi | the pip install list from the logs still look the same | 09:53 |
| tristan | Certainly something must have changed right ? | 09:53 |
| juergbi | and the same docker image versions | 09:53 |
| benschubert | https://gitlab.com/BuildStream/buildstream/-/jobs/515691731 last successful build, compared to last build https://gitlab.com/BuildStream/buildstream/-/jobs/539240401 | 09:54 |
| tristan | yeah the requirements.txt pin PyGObject and other than that everything comes from docker | 09:54 |
| benschubert | same image | 09:55 |
| juergbi | PyGObject==3.32.2 in both | 09:55 |
| tristan | there is 0e1ede11c7a82443a49d9beba1f111f44c03287a on April 4th which does a bunch of docker image swapping | 09:56 |
| tristan | but that's a while back | 09:56 |
| benschubert | tristan: both builds I linked are using the same imag | 09:56 |
| benschubert | and same pygobject | 09:56 |
| benschubert | one fails the other not | 09:56 |
| benschubert | only difference I see there is one is using the permanent-runner the other not -_-' | 09:57 |
| tristan | The docker image change is !1853 and that pipeline passed | 09:57 |
| tristan | benschubert, Weird | 09:57 |
| tristan | https://gitlab.com/BuildStream/buildstream/pipelines/144331848 | 09:59 |
| tristan | Ok so that is my horrible patch ^^^^ | 09:59 |
| tristan | Which seems to be passing at the tip now, but it's really horrible | 09:59 |
| benschubert | yep... well, it works I guess? | 09:59 |
| benschubert | mind adding a comment above though? I would not bet we are not hitting that again in a month :) | 10:00 |
| tristan | If I look at v2019.2-10-gaa5df899, the ostree commit which "fixes" the introspection data | 10:00 |
| tristan | I see this: https://paste.centos.org/view/131b83bd | 10:01 |
| benschubert | pygobject might not be doing the conversion correctly? 'None' is a python object, not a null pointer | 10:01 |
| tristan | This is the commit which passes the CI: https://gitlab.com/BuildStream/buildstream/-/commit/1ba3884ec12561cf3a163cd9d52209c88b6c66b4 | 10:01 |
| tristan | Well, you'd think that pygobject would know to do the right thing with the None object when it comes to pointers | 10:02 |
| benschubert | that's true | 10:02 |
| tristan | I have a feeling that passing 0 there is fooling pygobject to pass along a valid pointer with the value 0, which luckily means NULL | 10:02 |
| tristan | or NULL luckily means, rather | 10:02 |
| tristan | benschubert, yeah I'll prepare an MR and add a comment | 10:04 |
| benschubert | thanks :) | 10:04 |
| benschubert | if you have a moment after that, I'd like to discuss cache key tests :) | 10:04 |
| abderrahim[m] | tristan: your change works because 0 can be either the out parameter (in old ostree) or the cancellable (in new ostree) | 10:04 |
| tristan | Honestly I was just firing in the dark when I pushed that commit, curious about whether pygobject would be "happier with a number" since that's what it asked for | 10:04 |
| tristan | however I wouldn't call this "getting to the bottom of it" | 10:05 |
| tristan | Uhhhh | 10:05 |
| tristan | abderrahim[m], I thought the API didn't *actually* change, only the GIR data was clarified, no ? | 10:05 |
| abderrahim[m] | at least that's how I see it | 10:05 |
| abderrahim[m] | which means the python api did change | 10:05 |
| tristan | abderrahim[m], but nothing involving cancellables changed as far as I can see: https://paste.centos.org/view/131b83bd | 10:06 |
| tristan | allow-none (previously) became nullable and optional respectively, for the last 2 parameters | 10:06 |
| abderrahim[m] | going from 6 arguments (including self and the optional cancellable) to 5 | 10:07 |
| abderrahim[m] | out_imported became an out parameter | 10:07 |
| abderrahim[m] | so it's no longer an argument in python, but rather a return value | 10:07 |
| tristan | The cancellable is still there | 10:07 |
| tristan | What !!!! | 10:07 |
| tristan | Holy shit | 10:07 |
| tristan | "became an out parameter" caused the return to change, how... crazy | 10:08 |
| tristan | So that *was* an API break after all | 10:09 |
| tristan | I was always thinking that it was okay, because it was only clarifying or strengthening the declaration of the params | 10:09 |
| tristan | so older incorrect python might have a clearer warning instead of exploding unexpectedly | 10:10 |
| tristan | but if pygobject is changing the whole signature that seem... horrible :-S | 10:10 |
| abderrahim[m] | I think at the time we said that the old ostree is quite old and no longer "in the wild" | 10:10 |
| tristan | *seems | 10:10 |
| tristan | Myeah that's not something I would be comfortable with I think | 10:11 |
| abderrahim[m] | do we still test against debian 9? as that would probably have the old ostree | 10:11 |
| tristan | I rather think we should ideally have a try/except block in there and handle every mistake our dependencies have made | 10:11 |
| tristan | abderrahim[m], currently CI breaks in both debian 9 and 10 with the same error | 10:12 |
| tristan | but then, there was the older error | 10:12 |
| juergbi | do you know what caused it to suddenly break? | 10:12 |
| tristan | No idea | 10:12 |
| tristan | that is also frustrating | 10:12 |
| abderrahim[m] | yup, debian 10 has ostree 2019.1 | 10:13 |
| juergbi | yes, I don't like sudden changes | 10:13 |
| tristan | benschubert, appears to have found 2 pipelines with same image and same pygobject, one fails and the other passes | 10:13 |
| abderrahim[m] | pygobject version shouldn't affect this | 10:13 |
| juergbi | afaict, at some point in April or May it suddenly broke with the exact same setup | 10:13 |
| juergbi | but ostree version should be fixed in the Docker image | 10:14 |
| juergbi | including the .gir/.typelib | 10:14 |
| tristan | So what comes to mind but is painful to say... | 10:14 |
| tristan | is that possibly ostree got upgraded in the remote hosting ? | 10:14 |
| tristan | Or the state of the ostree repo changed somehow ? | 10:14 |
| tristan | That is really stretching it, that line of thinking really hurts :-S | 10:15 |
| tristan | right, docker image is static data, we don't change the image without an explicit commit to .gitlab-ci.yml | 10:15 |
| tristan | so docker image changes need to also go through CI | 10:15 |
| juergbi | the docker image should definitely be unmodified | 10:17 |
| juergbi | the only thing that makes sense to me is that something implicitly gets updated. but not sure by what | 10:17 |
| tristan | there is no ostree install happening afaics | 10:18 |
| abderrahim[m] | there couldn't be a buildstream artifact cache, right? | 10:20 |
| abderrahim[m] | or ~/.cache/buildstream/sources | 10:21 |
| benschubert | those are normally hold in a temp directory, which shouldget cleanedup | 10:21 |
| tristan | Ah !!! | 10:26 |
| tristan | good catch | 10:26 |
| tristan | abderrahim[m], benschubert ... I think the "integration tests" are optimized by persisting the artifact cache in the gitlab cache right ? | 10:26 |
| tristan | that might be a contributing factor ? but I'm not entirely sure... | 10:26 |
| tristan | The source cache maybe ? which is relevant here because it's an ostree source in the autotools test ? | 10:27 |
| benschubert | ah we do have the INTEGRATION_CACHE, not 100% sure anymore what it does | 10:27 |
| tristan | benschubert, it's there to cache sources from CI runs | 10:31 |
| abderrahim[m] | that would explain it | 10:32 |
| abderrahim[m] | the error is in fetch, so if the source is already cached we don't get an error | 10:32 |
| tristan | So when we do the integration tests, we're actually testing sandbox functionality which requires external things, but we're not actually testing the downloads | 10:32 |
| tristan | Right | 10:32 |
| tristan | So if we zapped the caches and ran a new CI run, we would expect that... fedora would equally fail ? | 10:33 |
| tristan | I guess ? | 10:33 |
| abderrahim[m] | probably not, if fedora has the new ostree | 10:33 |
| tristan | lets try that anyway, gitlab caches deserve a healthy dose of zaps | 10:33 |
| abderrahim[m] | actually fedora should have the new ostree | 10:33 |
| tristan | but both debian 9 and 10 fail, I still run debian 9 | 10:34 |
| abderrahim[m] | but the cache is reused for other tests using other oses | 10:34 |
| abderrahim[m] | we could make the cache be per-os, it would take more space but at least we shouldn't have such issues | 10:35 |
| tristan | How do you zap the caches anyway ? | 10:35 |
| tristan | abderrahim[m], do you think they are reused ? | 10:35 |
| abderrahim[m] | i'll take a look and see how they are stored | 10:35 |
| benschubert | https://gitlab.com/BuildStream/buildstream/pipelines -> clear runner cache? | 10:35 |
| tristan | Ah | 10:35 |
| abderrahim[m] | no, they aren't | 10:36 |
| * tristan runs https://gitlab.com/BuildStream/buildstream/pipelines/144340670 with naked caches | 10:36 | |
| abderrahim[m] | tristan: this one is for master? | 10:37 |
| abderrahim[m] | and btw, the caches are shared across branches | 10:38 |
| abderrahim[m] | they probably shouldn't | 10:38 |
| tristan | waaaait a sec no it's not that one | 10:39 |
| tristan | why did I get that one ? | 10:40 |
| tristan | So weird | 10:47 |
| tristan | I started a pipeline for bst-1 twice and they both chose to do master instead | 10:47 |
| tristan | Again ! | 10:48 |
| tristan | gitlab is broken yay | 10:48 |
| tristan | abderrahim[m], I don't know if theres a way to have caches for bst-1 and master separate, because you definitely want the caches to be shared with any branch *off* of bst-1 or master | 10:49 |
| abderrahim[m] | worked for me https://gitlab.com/BuildStream/buildstream/pipelines/144341897 | 10:50 |
| tristan | i.e. you want to optimize merge requests, or branches which will become merge requests, targetting master/bst-1 respectively | 10:50 |
| tristan | nice | 10:50 |
| tristan | I did "Run Pipeline" from the pipelines page, selected bst-1 and clicked the Run button and I kept getting master :-S | 10:51 |
| * tristan is going to abandon ship for the evening, will have to revisit the introspection nightmare tomorrow :-S | 10:51 | |
| abderrahim[m] | we can do it manually, i.e. have 1 and 2 in the cache key | 10:52 |
| tristan | abderrahim[m], ah that would work yeah | 10:52 |
| tristan | encoded into .gitlab-ci.yml I suppose | 10:52 |
| abderrahim[m] | yup | 10:52 |
| *** pointswaves has joined #buildstream | 11:12 | |
| benschubert | tristan: I'm trying to extract the 'cachekey' test for sources so that external plugins can also implement it. I've got https://gitlab.com/BuildStream/buildstream/-/commit/576e3da33e8b977a33e997d4ce80dc3c17d5da15 in mind. I'm not sure if we should keep it as a `file.bst, file.expected` instead and the `repo` would just return the path to a project that contains all the files the plugin wants to test? | 11:31 |
| benschubert | I'm also not 100% sure how we would like to handle element plugins. any thoughts? | 11:32 |
| *** pointswaves has quit IRC | 12:33 | |
| *** aminbegood has joined #buildstream | 14:36 | |
| *** aminbegood has quit IRC | 14:45 | |
| *** phoenix has joined #buildstream | 15:33 | |
| tristan | benschubert, not a good time for me but; what I think we want is to not discriminate between sources and elements, and offer an API which allows external repos to make explicit calls to a generic fixture, providing expected results for each element | 15:42 |
| benschubert | ah let's talk about it next week then? :) | 15:43 |
| tristan | yeah that's better, it's late here and I'm a little tipsy | 15:43 |
| benschubert | haha no worries, enjoy your weekend! | 15:44 |
| tristan | thanks :) | 15:44 |
| *** toscalix has quit IRC | 16:10 | |
| *** toscalix has joined #buildstream | 16:10 | |
| *** phoenix has quit IRC | 16:15 | |
| *** phoenix has joined #buildstream | 16:15 | |
| *** toscalix has quit IRC | 17:29 | |
| *** narispo has quit IRC | 18:55 | |
| *** phoenix has quit IRC | 18:55 | |
| *** narispo has joined #buildstream | 18:56 | |
| *** narispo has joined #buildstream | 18:56 | |
| *** toscalix has joined #buildstream | 20:05 | |
| *** narispo has quit IRC | 20:49 | |
| *** narispo has joined #buildstream | 20:49 | |
| *** toscalix has quit IRC | 21:42 | |
| *** narispo has quit IRC | 21:49 | |
| *** narispo has joined #buildstream | 21:49 | |
| *** narispo has quit IRC | 21:54 | |
| *** narispo has joined #buildstream | 21:54 | |
| *** aminbegood has joined #buildstream | 23:12 | |
| *** benschubert has quit IRC | 23:35 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!