| *** Camila5448 has joined #buildstream | 01:35 | |
| *** Camila5448 has quit IRC | 01:37 | |
| *** tristan has quit IRC | 05:50 | |
| *** tristan has joined #buildstream | 06:34 | |
| *** benschubert has joined #buildstream | 07:32 | |
| *** cphang has joined #buildstream | 07:56 | |
| *** phildawson has joined #buildstream | 08:04 | |
| *** jude has joined #buildstream | 08:06 | |
| *** rdale has joined #buildstream | 08:07 | |
| cphang | juergbi after having a little think on https://gitlab.com/BuildGrid/buildbox/buildbox-common/-/commits/juerg/grpc-retry I think you're right, it's best not to think of theoretical examples of different error codes when most of them probably demonstrate non-compliance anyway. | 08:24 |
|---|---|---|
| cphang | WSalmon juergbi has it been shown that https://gitlab.com/BuildStream/buildstream/-/issues/1310 has been fixed? | 08:27 |
| WSalmon | Looks like it, but it looked like it last time, FD has not built with master but its got a long way, https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/574850923 | 08:36 |
| *** tpollard has joined #buildstream | 08:42 | |
| cphang | is there a smaller test that we can do without having to resort to a full end to end fdsdk build? | 08:43 |
| WSalmon | https://gitlab.com/BuildStream/buildstream/-/issues/1318#note_352723829 I would think its worth keeping this open until the fix is in upstream (buildbox-*) and there are new installable bins on the buildstream website? but we could close it and make new issues for buildbox and the website? | 08:43 |
| WSalmon | cphang, not that i know, i can see the POV of saying that the error is no longer observable so lets close it | 08:44 |
| coldtom | i'd have expected to see a failure on that by now without the fix | 08:44 |
| *** santi has joined #buildstream | 08:44 | |
| cphang | It'd be nice to have a small integration test, but https://gitlab.com/BuildStream/buildstream/-/merge_requests/1929/diffs#a01481ca4ec6523776100e756c661eab1db49c9e_120_122 looks fairly convincing to me as well. | 08:46 |
| WSalmon | yep, im sure benschubert has fix the exact case and the build has got further than it had but i worry there are other edge case's with variables that could have been missed | 08:58 |
| WSalmon | that job is so far in that it is looking more and more unlikely tho | 08:59 |
| cphang | WSalmon that aren't covered by the unit tests? If you can think of some, that sounds like patches for BuildStream imo? | 09:01 |
| *** tristan has quit IRC | 09:05 | |
| WSalmon | not that i know | 09:10 |
| *** Allison6011 has joined #buildstream | 09:44 | |
| *** Allison6011 has quit IRC | 09:46 | |
| *** tristan has joined #buildstream | 09:56 | |
| *** tristan_ has joined #buildstream | 09:57 | |
| *** tristan has quit IRC | 09:58 | |
| *** tristan_ has quit IRC | 10:21 | |
| *** Ariana8648 has joined #buildstream | 10:28 | |
| *** Ariana8648 has quit IRC | 10:30 | |
| *** tristan has joined #buildstream | 10:39 | |
| *** ChanServ sets mode: +o tristan | 11:29 | |
| tristan | juergbi, benschubert... care to take a look at !1948 for me ? | 11:30 |
| benschubert | tristan: sure. I'll do that this afternoon | 11:30 |
| benschubert | I've also started reading your email on artifact server config | 11:30 |
| benschubert | and will try to have an answer by today, but that's a tough one :D | 11:31 |
| tristan | benschubert, thanks ! | 11:31 |
| tristan | benschubert, yeah... it's a tough one, I think you might at least agree it's confusing as hell and needs to be revisited, at least ? | 11:32 |
| benschubert | ah I definitely agree there :) | 11:32 |
| tristan | heh | 11:32 |
| benschubert | There is just a few assumptions made that I might not agree with :) | 11:32 |
| tristan | Yeah | 11:32 |
| tristan | Fair :) | 11:32 |
| benschubert | (like loosing the fine-grained per-junction config ) | 11:32 |
| benschubert | and the no 'push' in the project.conf config | 11:33 |
| benschubert | which is useful for development caches | 11:33 |
| benschubert | where you have a team that wants to collaborate and share build failures | 11:33 |
| benschubert | but that will come in my email in a better form :) | 11:33 |
| tristan | The logic that is based in is that: We have a very confusing config which is even hard to define/specify what it means, and that much of the flexibility might be based on assumptions | 11:33 |
| tristan | that we "might" need something, not necessarily proven use cases | 11:34 |
| tristan | benschubert, for "push" in project.conf particularly, I think I clarified that we might keep that as it's low cost and mirrors the user config | 11:34 |
| benschubert | oh haven't read tha tpart yet :) | 11:35 |
| benschubert | anyways, I'll continue after lunch | 11:35 |
| tristan | Anyway, please take your time to digest it, it's indeed a tough one :) | 11:35 |
| *** tpollard has quit IRC | 12:10 | |
| *** tpollard has joined #buildstream | 12:11 | |
| *** tristan has quit IRC | 12:13 | |
| valentind | traveltissues, I am having an issue with this magic value in a test: https://gitlab.com/BuildStream/buildstream/-/blob/master/tests/internals/storage_vdir_import.py#L53 | 12:48 |
| valentind | How was this value calculated? | 12:48 |
| valentind | Is that TIMESTAMP in seconds since epoch? | 12:49 |
| valentind | Yes, I think it is that. | 12:51 |
| traveltissues | yes, it should be | 12:52 |
| *** Elizabeth5953 has joined #buildstream | 12:53 | |
| *** Elizabeth5953 has quit IRC | 12:55 | |
| valentind | traveltissues, This fails on my machine. Is there some requirements for this to work? | 12:56 |
| *** Piper3025 has joined #buildstream | 12:57 | |
| *** Piper3025 has quit IRC | 12:58 | |
| traveltissues | in what way does it fail valentind | 13:01 |
| valentind | I get a different mtime for line 222 https://gitlab.com/BuildStream/buildstream/-/blob/master/tests/internals/storage_vdir_import.py#L222 | 13:01 |
| valentind | https://paste.gnome.org/psstzgzr0 | 13:02 |
| traveltissues | that looks like the stamp from today | 13:03 |
| valentind | Yes | 13:03 |
| valentind | Why does master fail like that on my computer? | 13:03 |
| valentind | Does this go through buildbox-casd? Do I need a specific snapshot? | 13:04 |
| traveltissues | i'm not sure but it seems like it's updating the mtime, i remember seeing a similar issue at one point but it's been quite a while since i looked at this particular part of it | 13:04 |
| traveltissues | it does but should have been working with 0.0.6 | 13:07 |
| valentind | I am on 0.0.7 | 13:07 |
| valentind | Does not master use 0.0.7? | 13:08 |
| traveltissues | i'm not sure, i know that 0.0.8 was released recently | 13:17 |
| *** Cora6751 has joined #buildstream | 13:20 | |
| *** Cora6751 has quit IRC | 13:22 | |
| traveltissues | valentind: it fails locally for me too | 13:31 |
| traveltissues | master seems to be using master-149783478 which looks like it failed https://gitlab.com/BuildStream/buildstream-docker-images/-/jobs/568215330 | 13:38 |
| *** tristan has joined #buildstream | 13:50 | |
| benschubert | valentind: nice PR :) I'll have a look at it and run a few benchmarks of my own too on it, thanks :) | 13:59 |
| robjh | how can i get buildstream to retry a failed build? (i want it to offer me a shell again). so far ive tried `bst artifact delete python3-pillow.bst` but it still seems to be caching the result | 14:14 |
| robjh | lessons from history tells me that m -rf ~/.cache/buildstream* will work, but it will also add 20m to my turnaround | 14:16 |
| benschubert | robjh: do you have an artifact cache? if so it would be stored there and then pulled. You could do --remote "" to disable it IIRC | 14:21 |
| robjh | oooh, i do! | 14:23 |
| *** ChanServ sets mode: +o tristan | 14:25 | |
| tristan | benschubert, Interesting comment, I have a meeting now and it will be late (already late actually) | 14:25 |
| benschubert | tristan: we can discuss this tomorrow if you want? | 14:26 |
| tristan | benschubert, I'll check to see if I can get away with that, it would be nice | 14:26 |
| benschubert | I believe it would make things both simpler and faster | 14:26 |
| tristan | Oh yeah I'll try it first and see if it's possible | 14:26 |
| tristan | I have a feeling that there are obstacles but maybe you're right :) | 14:26 |
| * tristan wishes he had a final decision about how to allow multiple instances of the same project to coexist in the same pipeline ... | 14:27 | |
| tristan | that would give me more stuff to work on before people out west start to appear ... | 14:27 |
| * tristan must poke the hornets nest on that topic daily | 14:28 | |
| robjh | is there any way to make a `kind: pip` use pip3? | 14:29 |
| benschubert | tristan: which ML threads is that again? | 14:30 |
| tristan | benschubert, https://mail.gnome.org/archives/buildstream-list/2020-May/msg00005.html | 14:31 |
| tristan | branch has been in limbo for close to a month I think | 14:31 |
| benschubert | tristan: what's more urgent, this one or the artifact config one? | 14:31 |
| tristan | I have overrides working perfectly, can't land it until this ominous decision gets made | 14:31 |
| tristan | this one | 14:31 |
| benschubert | ok I'll look at it today then and will do the other afterwards | 14:32 |
| tristan | artifact thing isn't blocking me, I think we should consider blocking 2.0 on the artifact cache thing though | 14:32 |
| benschubert | right | 14:32 |
| benschubert | robjh: https://buildstream.gitlab.io/buildstream/sources/pip.html this one? | 14:33 |
| robjh | benschubert, yeah, thats the guy | 14:33 |
| robjh | im making my depends include symlink from pip to pip3 as a workaround | 14:33 |
| benschubert | robjh: if you create an issue we should probably be able to add it https://gitlab.com/BuildStream/buildstream/-/blob/master/src/buildstream/plugins/sources/pip.py#L77, already has a quite long list (and is really ugly :/) | 14:34 |
| benschubert | or a MR if you can, that would be even better! | 14:36 |
| benschubert | though I think we need a more intelligent thing there | 14:37 |
| traveltissues | i thought `python3 -m pip` was equivalent to `pip3` | 14:37 |
| benschubert | it is, but not all systems have a 'python3' | 14:38 |
| traveltissues | usually it's a symlink, so `python3.x -m pip` | 14:39 |
| benschubert | traveltissues: that's why we go over all possible known versions, but that's not idea | 14:39 |
| traveltissues | true, and that could be done in a more elegant way | 14:40 |
| traveltissues | but if python3.x is installed then it will already be using pip3 right? | 14:40 |
| traveltissues | https://gitlab.com/BuildStream/buildstream/-/blob/master/src/buildstream/plugins/sources/pip.py#L125 | 14:40 |
| benschubert | not necessarily | 14:41 |
| benschubert | it will be using pip3.7 or 3.8 | 14:41 |
| benschubert | though robjh: what's hte problem with `python3.X -m pip` is that not found on your system? | 14:41 |
| robjh | i dont know? its trying to use pip instead of pip3 | 14:42 |
| traveltissues | so is python symlinked to python2.7? | 14:43 |
| traveltissues | that's fair benschubert | 14:44 |
| benschubert | that should only happen if you don't have python3.8 to pyhon3.0 in your path | 14:44 |
| robjh | i wouldnt have thought so? im using FDSDK's components/python3.bst and components/python3-pip.bst | 14:44 |
| benschubert | this part of the code runs on the host machine, not in the project | 14:44 |
| robjh | i see, well adding a symlink in my dependencies fixed it | 14:45 |
| robjh | r@mayall-build:~/work/freedoom$ which python | 14:46 |
| robjh | r@mayall-build:~/work/freedoom$ which python3 | 14:46 |
| benschubert | ah wait, is that a `pip` source or a `pip` element? | 14:46 |
| robjh | pip element | 14:46 |
| benschubert | ah then that's not the right doc | 14:46 |
| robjh | :O | 14:47 |
| benschubert | see https://gitlab.com/BuildStream/bst-plugins-experimental/-/blob/master/src/bst_plugins_experimental/elements/pip.yaml | 14:48 |
| benschubert | if you set `pip: pip3` in your variables, that hsould work | 14:48 |
| robjh | alright, i'll try it, thanks :) | 14:48 |
| robjh | success! thanks benschubert :) | 14:50 |
| benschubert | perfect! | 14:50 |
| *** toscalix has joined #buildstream | 15:38 | |
| *** phildawson has quit IRC | 16:33 | |
| *** narispo has joined #buildstream | 16:51 | |
| *** aav1o2 has joined #buildstream | 17:01 | |
| *** tpollard has quit IRC | 17:04 | |
| benschubert | tristan: are you still around? | 17:15 |
| benschubert | I'm trying to get my head around the explanation on the ML about the junciton override | 17:16 |
| benschubert | I'm not sure how to follow the conversation thread and not sure what's the outcome you have in mind at that point | 17:16 |
| benschubert | is !1901 implementing all of the proposal? | 17:16 |
| tristan | benschubert, sec... lemme refresh... | 17:18 |
| tristan | benschubert, so !1901 is part 1, what that does is (A) implement the "overrides" semantic on junctions, ensuring that any overrides of a junction configuration in a subproject is explicit and never implicit... and (B) Starts finding conflicts if two instances of the same project occur in the same pipeline | 17:20 |
| tristan | benschubert, the original point of this whole exercise is to ensure that you always know what's going on with your junctions, but you are not forced to know about a subproject's junction unless there is a potential conflict | 17:21 |
| tristan | So, this leaves us with the problem of explicitly *allowing* multiple projects to occur in the same pipeline, for whatever reason | 17:21 |
| benschubert | and then a second MR would add a way of renaming a junction to prevent such a conflict, right? | 17:21 |
| tristan | It was probably going to end up on the same MR, as juergbi argued we aught to postpone landing until we solved the second part | 17:22 |
| tristan | otherwise we introduce a situation where you cannot have 2 of the same project at once at all | 17:22 |
| tristan | benschubert, "renaming a project" is the last idea I had about making this possible | 17:23 |
| tristan | The first mail I linked to had some use cases | 17:23 |
| benschubert | gotcha | 17:23 |
| tristan | An interesting one is the "internal project" usecase, where you know you're only using it internally and reverse dependency projects should not end up with runtime dependencies on that internal project | 17:23 |
| tristan | that makes the "internal" keyword on a junction interesting, cause there could be some additional protections around there | 17:24 |
| benschubert | that's something that would not be trivial to validate easily though no? | 17:24 |
| benschubert | I'd be wary to implement something like that that could be verified with a linter :) | 17:24 |
| benschubert | but I get the point | 17:24 |
| tristan | The first idea was to have just a "whitelist" where a project which ends up with a conflict, would decide to either override, link (previously target), or whitelist it as allowed | 17:25 |
| tristan | Well, about validation; I'm not entirely convinced that it needs to be in BuildStream's contract, but that validation could be done at runtime, for the "internal" usecase, which is a bonus | 17:25 |
| tristan | benschubert, my personal view is that BuildStream should not make assumptions about how people want to use artifacts from subprojects | 17:26 |
| tristan | So while "internal" is an interesting usecase, it's totally plausible that people end up staging two versions of the same project, even in the same element at build time | 17:27 |
| tristan | I don't want us to care, but I want us to ensure project authors are aware of it when creating such constructs, at an early stage | 17:27 |
| benschubert | yep definitely, that seem correct, I was more wary about the 'internal' thing | 17:27 |
| benschubert | but the overrides and renaming otherwise looks good to me | 17:28 |
| tristan | Of course, there are late stage overlap errors which will occur if you open up the door to multiple instances of the same project | 17:28 |
| benschubert | (wonder if we should have it under a single key ( overrides: {key: project name -> {name | element}}) | 17:28 |
| tristan | Well, "will probably occur if an element decides to stage the same element at the same sandbox relative location" I should say | 17:28 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/merge_requests/1901/diffs#ae21ae20fa7c1172671e2b7cd0c6ba979c572eaf_58_58 | 17:29 |
| tristan | Ah you mean, use overrides to also override the name | 17:30 |
| benschubert | correct | 17:30 |
| benschubert | but that's a detail | 17:31 |
| tristan | right, it's a detail | 17:31 |
| benschubert | all in all I don't think I've got much to add to the ML thread. It seems generally good to me | 17:31 |
| benschubert | and I don't see any case where that double override possiblity would not work | 17:31 |
| tristan | I think that we need to be able to whitelist without overriding too | 17:32 |
| tristan | i.e. from a toplevel, just decide "I'm okay with this" without redefining any junction in a subproject | 17:32 |
| benschubert | ah one thing: how would the override work with multiple nested overrides? | 17:32 |
| tristan | Also, with the new `link`, it will be interesting to, e.g.: Override one subproject's junction with a link to another subproject's junction | 17:32 |
| benschubert | Like if a top level project include a junctions that overrides stuff already? | 17:32 |
| tristan | The override overrides everything, not considering anything which was in place | 17:33 |
| tristan | That was also briefly discussed on IRC but not in depth | 17:33 |
| benschubert | ok so the most toplevel override wins | 17:34 |
| tristan | Well it's all explicit | 17:34 |
| benschubert | but if there are multiple overrides, is it a merge, or a top level does all? | 17:34 |
| tristan | if a toplevel overrides a subproject junction and that subproject junction also overrides, the whole junction is anyway replaced by the toplevel one | 17:34 |
| tristan | topevel overwrites completely, no merge | 17:34 |
| benschubert | so if I have A, B a junction of A, which contains C and D, two other junctions. If B overrides something in D, and A overrides something on B and C, will D get overriden? | 17:35 |
| tristan | I think it's correct, the toplevel can still override a subproject junction, with a local junction which itself has overrides | 17:35 |
| benschubert | ok | 17:36 |
| benschubert | It seems sensible to me, not sure what I can add to the ML, but I can say that seems fine and summaries my understanding. will do that tomorrow morning though | 17:37 |
| benschubert | and then I'll answer your other ml thread | 17:37 |
| tristan | after this is done, I will make an effort in the user guide with examples to layout (A) Overrides, (B) Using links to inherit junctions, (C) Using renames to allow duplication | 17:37 |
| tristan | With that all in place we should have a very clear story of what's going on I think | 17:38 |
| tristan | benschubert, That would be helpful yes, I've been ping ponging this with juergbi who had some other ideas, I don't feel that feeling of "consensus" to go ahead and implement things | 17:38 |
| tristan | I'm still glad to have held off for a while to think about this all | 17:39 |
| benschubert | ok! I'll do that tomorrow, sorry for the delay | 17:39 |
| tristan | Thanks :) | 17:39 |
| *** santi has quit IRC | 18:14 | |
| *** finnb0 has joined #buildstream | 18:27 | |
| *** narispo has quit IRC | 18:28 | |
| *** aav1o2 has quit IRC | 18:28 | |
| *** jswagner has quit IRC | 18:28 | |
| *** krichter[m] has quit IRC | 18:28 | |
| *** cgm[m] has quit IRC | 18:28 | |
| *** awacheux[m] has quit IRC | 18:28 | |
| *** asingh_[m] has quit IRC | 18:28 | |
| *** Demos[m] has quit IRC | 18:28 | |
| *** jjardon[m] has quit IRC | 18:28 | |
| *** dbuch[m] has quit IRC | 18:28 | |
| *** doras has quit IRC | 18:28 | |
| *** abderrahim[m] has quit IRC | 18:28 | |
| *** mattiasb has quit IRC | 18:28 | |
| *** segfault1[m] has quit IRC | 18:28 | |
| *** reuben640[m] has quit IRC | 18:28 | |
| *** theawless[m] has quit IRC | 18:28 | |
| *** Trevio[m] has quit IRC | 18:28 | |
| *** walterve[m][m] has quit IRC | 18:28 | |
| *** m_22[m] has quit IRC | 18:28 | |
| *** tlater[m] has quit IRC | 18:28 | |
| *** finnb has quit IRC | 18:28 | |
| *** ironfoot has quit IRC | 18:28 | |
| *** douglaswinship has quit IRC | 18:28 | |
| *** jjardon has quit IRC | 18:28 | |
| *** benbrown has quit IRC | 18:28 | |
| *** jude has quit IRC | 18:28 | |
| *** devcurmudgeon has quit IRC | 18:28 | |
| *** pointswaves has quit IRC | 18:28 | |
| *** Trevinho has quit IRC | 18:28 | |
| *** cphang has quit IRC | 18:28 | |
| *** SotK has quit IRC | 18:28 | |
| *** finnb0 is now known as finnb | 18:28 | |
| *** jswagne- has joined #buildstream | 18:30 | |
| *** jude has joined #buildstream | 18:30 | |
| *** cphang has joined #buildstream | 18:30 | |
| *** devcurmudgeon has joined #buildstream | 18:30 | |
| *** pointswaves has joined #buildstream | 18:30 | |
| *** Trevinho has joined #buildstream | 18:30 | |
| *** SotK has joined #buildstream | 18:30 | |
| *** jswagne- is now known as jswagner | 18:30 | |
| *** douglaswinship has joined #buildstream | 18:31 | |
| *** narispo has joined #buildstream | 18:42 | |
| *** krichter[m] has joined #buildstream | 18:42 | |
| *** cgm[m] has joined #buildstream | 18:42 | |
| *** awacheux[m] has joined #buildstream | 18:42 | |
| *** m_22[m] has joined #buildstream | 18:42 | |
| *** walterve[m][m] has joined #buildstream | 18:42 | |
| *** Trevio[m] has joined #buildstream | 18:42 | |
| *** asingh_[m] has joined #buildstream | 18:42 | |
| *** Demos[m] has joined #buildstream | 18:42 | |
| *** reuben640[m] has joined #buildstream | 18:42 | |
| *** mattiasb has joined #buildstream | 18:42 | |
| *** doras has joined #buildstream | 18:42 | |
| *** jjardon[m] has joined #buildstream | 18:42 | |
| *** dbuch[m] has joined #buildstream | 18:42 | |
| *** tlater[m] has joined #buildstream | 18:42 | |
| *** segfault1[m] has joined #buildstream | 18:42 | |
| *** abderrahim[m] has joined #buildstream | 18:42 | |
| *** theawless[m] has joined #buildstream | 18:42 | |
| *** ironfoot has joined #buildstream | 18:42 | |
| *** jjardon has joined #buildstream | 18:42 | |
| *** irc.acc.umu.se sets mode: +oo ironfoot jjardon | 18:42 | |
| *** toscalix has quit IRC | 19:24 | |
| *** rdale has quit IRC | 19:42 | |
| *** jude has quit IRC | 20:00 | |
| *** toscalix has joined #buildstream | 20:47 | |
| *** Emily9314 has joined #buildstream | 21:14 | |
| *** Emily9314 has quit IRC | 21:16 | |
| *** seanborg has joined #buildstream | 21:35 | |
| *** seanborg has quit IRC | 22:17 | |
| *** toscalix has quit IRC | 22:26 | |
| *** Scarlett8896 has joined #buildstream | 22:36 | |
| *** Scarlett8896 has quit IRC | 22:38 | |
| *** benschubert has quit IRC | 22:50 | |
| *** Genesis9929 has joined #buildstream | 23:19 | |
| *** Genesis9929 has quit IRC | 23:21 | |
| *** Arianna6953 has joined #buildstream | 23:54 | |
| *** Arianna6953 has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!