| *** tristan has quit IRC | 03:42 | |
| *** tristan has joined #buildstream | 03:58 | |
| tristan | jjardon[m], it's on by default when connected to a terminal | 03:59 |
|---|---|---|
| tristan | jjardon[m], gitlab is an exception; it's not a real terminal but it has some hack to interpret ANSI color codes (but not other ANSI control codes, just colors) | 03:59 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 3 commits (last: _pipeline.py: Handle exceptions when checking out elements) https://gitlab.com/BuildStream/buildstream/commit/b9cfc6399d71db9764f1e167709dfe423b972b66 | 04:33 |
| gitlab-br-bot | buildstream: merge request (jonathan/36-fix-ugly-stack-trace->master: Fix ugly stack trace) #53 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/53 | 04:38 |
| gitlab-br-bot | buildstream: issue #36 ("`bst checkout` prints a stack trace when it lacks permission to make symlinks") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/36 | 04:39 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: _frontend/widget.py: Fixed stack trace found by Tristan Maat) https://gitlab.com/BuildStream/buildstream/commit/e54c28df429db38dd721c17fccc1229e729a526d | 04:44 |
| gitlab-br-bot | buildstream: merge request (inconsistent_trace->master: widget.py: Fix stacktrace on early pipeline failure) #54 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/54 | 04:48 |
| gitlab-br-bot | push on buildstream@dual-cache-keys (by Jürg Billeter): 20 commits (last: widget.py: Use _get_full_display_key()) https://gitlab.com/BuildStream/buildstream/commit/948f19f2e0cd483022f6b39fa17e96b185fc7926 | 05:32 |
| gitlab-br-bot | buildstream: merge request (dual-cache-keys->master: Dual cache keys) #55 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/55 | 05:44 |
| juergbi | tristan: ^^ it could probably use some more testing but feedback would be appreciated | 05:46 |
| tristan | alright | 06:00 |
| * tristan has some weird phoning to do, might have to reschedule flights :-/ | 06:00 | |
| tristan | juergbi, looking at element.py... looks... a bit crazy :) | 06:47 |
| tristan | juergbi, So I wonder if we can reduce the codepaths for querying cache keys | 06:50 |
| tristan | Is there a reason that we want a separate path for strong_cache_key_for_build(), rather than an added KeyStrength type ? | 06:50 |
| tristan | I guess the tricky thing is that, cache key for build is resolved later on | 06:51 |
| * tristan is just looking at this and trying to think how we can make it a bit more... readable and simple to understand, if at all possible | 06:51 | |
| tristan | Also I've been thinking a bit about the user experience, I think it makes sense in the codebase to be talking about weak and strong cache keys | 07:03 |
| tristan | But I think we can use simpler jargon for the user | 07:03 |
| tristan | Maybe `bst --strict/--no-strict` would be nice | 07:03 |
| tristan | and in the config file, either strict-build-plan: True or False ? | 07:04 |
| * tristan will add comments to the MR about these UX things, they are a bit more clear... I think I like 'strict' as a word for this | 07:05 | |
| juergbi | tristan: strictly speaking, cache_key_for_build is not a different key strength | 07:44 |
| juergbi | feel free to suggest different names, function structure. i wasn't sure what made most sense | 07:44 |
| juergbi | i tried to be as descriptive as possible | 07:45 |
| tristan | juergbi, yeah I understand... kinda sorta :) | 08:13 |
| tristan | juergbi, the angle I'm thinking with that is basically... | 08:14 |
| tristan | o Who are the callers of _get_cache_key() ? | 08:14 |
| tristan | o If a caller of _get_cache_key() wanted a strong cache key, is it the majority of cases that they actually want a _cache_key_for_build() ? | 08:15 |
| tristan | o If that is the case, maybe we should be having the cache key for build being the default cache key, except for the few exceptions where we want an idealistic cache key ? | 08:16 |
| tristan | instead of the reverse ? | 08:16 |
| tristan | juergbi, you are closer to this code than I am presently, but does that line of thinking make sense ? | 08:16 |
| juergbi | as it is, get_cache_key_for_build is actually used very rarely | 08:17 |
| juergbi | from_artifact is used slightly more frequently because that also covers the case of reusing a weakly cached artifact | 08:17 |
| juergbi | ('weakly cached' might be a bit confusing but i take it you know what i mean) | 08:17 |
| tristan | I hope we're not saying "weakly cached" in the code, but yeah I know what you mean :) | 08:19 |
| juergbi | in the end all of these are mainly used in the artifact cache code or in element.py itself | 08:19 |
| juergbi | not much outside | 08:19 |
| juergbi | actually, except for one call to _get_cache_key_from_artifact(), none of _get_cache_key* are used outside these two files, afaict | 08:20 |
| juergbi | so it's mainly a question about what the best function names are, not really about how frequently they are called | 08:21 |
| juergbi | there is one place where key strength is really used as variable and then passed to get_cache_key(). for that it's convenient to have idealistic strong and weak keys via single function. but it's just one call site, afaict, so it could be changed, if desired | 08:24 |
| *** jonathanmaw has joined #buildstream | 08:30 | |
| *** jude has joined #buildstream | 08:37 | |
| tristan | juergbi, so I gave some comments about the user facing interfaces, those I think are good important changes to make (and completely orthogonal to the inner workings of cache keys I think) | 08:38 |
| tristan | maybe we can reconsider wording, but I think --strict/--no-strict CLI option overriding whatever is in the configuration, defaulting to `strict: True` in userconfig.yaml is the right approach | 08:39 |
| tristan | it will also help us poor devs in conversation; because we can speak of strict mode and weak/strong cache keys as separate things | 08:40 |
| juergbi | yes, sounds fine to me | 08:40 |
| * tristan has found it confusing to use the same language for cache keys and for build planning modes | 08:40 | |
| juergbi | i definitely want to be able to set it on the command-line but like that we can have both | 08:40 |
| tristan | yeah we mostly have that pattern | 08:40 |
| tristan | except for some things which dont make much sense to put in config | 08:41 |
| tristan | but it's mostly CLI option -> overrides user's config -> overrides default config | 08:41 |
| *** tlater has joined #buildstream | 08:41 | |
| juergbi | yes, that definitely makes sense | 08:42 |
| jonathanmaw | tristan: I don't know if you saw https://gitlab.com/BuildStream/buildstream/merge_requests/37#note_34841737 | 08:49 |
| jonathanmaw | but it looks like elements named with hyphens cause problems for documentation generation | 08:49 |
| jonathanmaw | i.e. documentation doesn't build correctly and the linked warnings are raised during doc generation. | 08:50 |
| jonathanmaw | changing the filename of the plugin to use underscores instead of hyphens fixes that | 08:50 |
| tristan | jonathanmaw, python module names should not have dashes in them, actually | 08:50 |
| jonathanmaw | ah, I didn't know that | 08:50 |
| tristan | you cannot `import foo-bar` | 08:51 |
| jonathanmaw | that clears that up, underscores only. I'll change my generation script as well. | 08:51 |
| * tristan has been referring to it as 'dpkg-deploy` and `dpkg-build`, and never thought of that detail either | 08:52 | |
| tristan | but I ran into that very early on as well; I like dashes :) | 08:52 |
| *** ssam2 has joined #buildstream | 09:14 | |
| tristan | tlater, so I've added some comments on your branch... most important of those are a.) since we're running a scheduler anyway; fetch if the source is not there, only add track queue on user request.... b.) click.prompt() user in the cases where their actions can delete data | 09:22 |
| tristan | tlater, and there is one, I think/hope final consideration for that branch; which is how this plays into the artifact cache sharing | 09:22 |
| tlater | Alright | 09:23 |
| tlater | I'm working through the comments right now | 09:23 |
| tlater | Should be done quite quickly I think, these aren't too big :) | 09:23 |
| tristan | tlater, for that part, we probably want to avoid uploading A.) Any artifact that is tainted by a local workspace, and B.) Any artifact that depends on an artifact that is tainted by a local workspace | 09:24 |
| tristan | It might seem like weak keys uploaded for reverse dependencies of workspaced elements can be reusable by peers; but I *think* that it's all around wiser to avoid that altogether | 09:25 |
| tristan | juergbi, thoughts on the above ^^^^ ? | 09:25 |
| juergbi | tristan: as workspaced dependencies might differ quite a bit, i tend to agree | 09:26 |
| tristan | So quick touch base... juergbi you have been master of all things regarding encoding metadata into artifacts | 09:36 |
| tristan | juergbi, I think that what makes sense and is forward viable in case of churn (I could envision a `bst push` command for instance, in the case one wants to upload results of a pipeline they built while offline)... I guess that workspaced state belongs in artifact metadata | 09:37 |
| tristan | And the decision of whether to upload an artifact or not should be artifact based, and not based on any kind of transient state (like whether it was built in this session or not) | 09:38 |
| juergbi | yes, i agree. when we can use the summary file, we should (be able to) remove 'built' state | 09:38 |
| juergbi | (as planned from the start) | 09:38 |
| tristan | yes I understand, but with or without that; should we still be tainting the artifacts themselves in the local cache ? | 09:39 |
| tristan | I guess that is the right way to make that decision of upload-or-not, right ? | 09:39 |
| juergbi | do you mean tainting the cache key or just noting it in embedded metadata? | 09:40 |
| tristan | the latter | 09:40 |
| juergbi | i guess we have to | 09:40 |
| juergbi | there could be some odd consequences, though | 09:40 |
| tristan | like ? | 09:41 |
| juergbi | the strong cache key with and without workspaces could theoretically be become the same at some point | 09:41 |
| tristan | Ah | 09:41 |
| tristan | Right | 09:41 |
| tristan | No | 09:41 |
| tristan | juergbi, they cannot | 09:41 |
| juergbi | although, i don't think it will be an issue in practice | 09:41 |
| juergbi | calculated in a different way? | 09:41 |
| juergbi | commit vs contents? | 09:42 |
| tristan | juergbi, because cache key for workspaces is calculated based on contents exactly | 09:42 |
| juergbi | ok, i see | 09:42 |
| tristan | so, possible, astronomically unlikely, though | 09:42 |
| tlater | Not entirely sure how artifacts work, but couldn't they for local sources? | 09:42 |
| tristan | juergbi, ok so the point of touching base here, is please keep an eye on tlater and walk him through that part :) | 09:43 |
| tristan | tlater, I thought of that yes; but there is an easy solution for that; add something custom to the key | 09:43 |
| juergbi | we should anyway do that for all keys that have different structure | 09:44 |
| tristan | tlater, like; do a recursive checksum on content, and then just make get_unique_key() be { "workspace-contents": checksum } | 09:44 |
| juergbi | to avoid accidental match | 09:44 |
| tristan | true | 09:44 |
| tristan | we could ensure that say, plugin name is always accounted | 09:44 |
| tristan | that would put a stop to it | 09:44 |
| tristan | there is not really much room for error currently, though, I think | 09:45 |
| gitlab-br-bot | buildstream: merge request (jonathan/dpkg-build->master: WIP: Jonathan/dpkg build) #37 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/37 | 10:16 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 12 commits (last: main.py: Add workspace commands) https://gitlab.com/BuildStream/buildstream/commit/7a4992f96c16accc55c7a0ae50e454de6bc3e7db | 10:52 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 10:52 |
| tristan | ooooo | 10:57 |
| tristan | no space left on device | 10:57 |
| tristan | on gnome7 | 10:57 |
| tristan | ssam2, I suppose ~builder/gits/gnome-modulesets -> ../gnome-modulesets is an integral part of cgit config ? | 11:01 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 8 commits (last: _pipeline.py: Add workspace commands) https://gitlab.com/BuildStream/buildstream/commit/c3a1e843295c2e398b794bca1c6e5788c51cbd9d | 11:12 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 11:12 |
| ssam2 | tristan, ah, no it's not any more | 11:16 |
| ssam2 | ~builder/gits can be deleted | 11:16 |
| tristan | ssam2, ah thanks, was just cleaning house there | 11:17 |
| tristan | looks like back in the days, I installed some flatpak runtime for testing, that cleared up some space | 11:17 |
| tristan | still | 11:17 |
| tristan | there is 78GB used | 11:17 |
| tristan | and the artifacts is only like 4GB right now | 11:18 |
| tristan | and the export is not huge either | 11:18 |
| tristan | the buildstream cache is 20GB, artifact cache 4.1GB and debootstrap export and logs and all, collectively 8GB | 11:19 |
| tristan | thats only 32GB | 11:19 |
| tristan | I cant believe that the system installation is 46GB :-/ | 11:20 |
| tristan | cant be | 11:20 |
| tristan | ssam2, if you're taking care of that gnu-toolchain thing, I just learned something interesting | 11:32 |
| tristan | ssam2, flatpak sdks are actually a superset of the platform, and my original staging stuff is bogus and wrong | 11:32 |
| tristan | we dont need to stage both the platform + sdk runtimes, just the sdk runtime | 11:33 |
| tristan | *but*, there is something weird about the rootfs | 11:33 |
| ssam2 | ok | 11:33 |
| tristan | it seems that it needs to be staged into /usr with a manual /usr merge done in / | 11:33 |
| ssam2 | you mean, /bin and /lib and /lib64 symlinks created by us ? | 11:34 |
| tristan | I have to look more closely what flatpak is doing to assemble the environment, but seems there are hard coded assumptions about slash there | 11:34 |
| tristan | something like that yeah | 11:34 |
| ssam2 | that's how OSTree-as-host expects to work | 11:34 |
| ssam2 | Fedora atomic is the same -- / isn't committed to an OSTree repo | 11:34 |
| tristan | ssam2, for gnu-toolchain, this only means for the part which provides the tools we bootstrap from | 11:34 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 4 commits (last: Add `workspace reset` command) https://gitlab.com/BuildStream/buildstream/commit/e0cffe036ff2fcbc0ae05d311bfdb91e536672b4 | 11:35 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 11:35 |
| ssam2 | i can look at this, sure | 11:35 |
| tristan | What I think is very weird about this... is that things are built with --prefix=/usr | 11:35 |
| tristan | but then only the content of user is collected into artifacts | 11:35 |
| tlater | tristan: Would you like to have another look at my MR or should I start working on the artifact stuff first? | 11:36 |
| tristan | tlater, keep trucking | 11:37 |
| gitlab-br-bot | buildstream: merge request (jonathan/dpkg-build->master: Jonathan/dpkg build) #37 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/37 | 12:32 |
| ssam2 | juergbi, how do I get buildstream to push to an artifact cache ? | 12:54 |
| ssam2 | i've added push-url and push-port to my config file | 12:54 |
| juergbi | that should suffice | 12:55 |
| ssam2 | https://pastebin.com/QZv5WkSt | 12:55 |
| ssam2 | when I run `bst build`, the builds succeed but nothing is pushed, and no errors | 12:55 |
| ssam2 | bah, maybe i didn't pull | 12:56 |
| ssam2 | ah, things look different now indeed | 12:56 |
| ssam2 | for one thing, the build is starting from scratch | 12:56 |
| ssam2 | but I also see pulled and pushed counters | 12:56 |
| * ssam2 goes for lunch while rebuilding | 12:57 | |
| juergbi | enjoy | 12:57 |
| ssam2 | oh, new issue: https://pastebin.com/qRqyuzif | 12:59 |
| ssam2 | looks like confusion between remote_path and remotepath, I'll do a fix | 13:00 |
| jonathanmaw | hrm, I'm trying to replicate https://gitlab.com/BuildStream/buildstream/issues/41 and when I exit the shell, it handles it worse than SIGSTP https://pastebin.com/BKsGRX5r | 13:19 |
| *** xjuan has joined #buildstream | 13:32 | |
| tlater | juergbi: I've been looking for a while now, where is metadata for artifacts stored? | 13:33 |
| juergbi | tlater: in Element._assemble() | 13:33 |
| tristan | ssam2, you need to specify the port too | 13:34 |
| tristan | ssam2, 'push-port' in 'artifacts' in the config | 13:34 |
| tristan | ssam2, it's 10007 | 13:34 |
| tristan | remote path :-/ | 13:35 |
| tlater | juergbi: Ah, thanks | 13:35 |
| jonathanmaw | hmm, it refuses to terminate after keyboard interrupts on exit from shell, but can still be killed | 13:36 |
| tristan | jonathanmaw, behaves differently for you ? | 13:41 |
| tristan | strange, did you hit enter perhaps on shell exit ? | 13:41 |
| tristan | maybe you are using a different runtime, too ? | 13:41 |
| jonathanmaw | tristan: mashing enter doesn't do anything interesting, either. | 13:42 |
| jonathanmaw | not sure what you mean by a different runtime, but I'm using python3.4, whereas iirc you're using 3.5 | 13:42 |
| tristan | jonathanmaw, I dont recommend <enter>, what happens for me is... I get backgrounded | 13:43 |
| tristan | and then I hit fg | 13:43 |
| tristan | and see no feedback | 13:43 |
| tristan | if at that point I enter dskfj;dsof;sdof<ENTER> | 13:43 |
| tristan | I will be asked again what I want to do | 13:43 |
| tristan | If I just hit <ENTER>, it does the default action and continues to build non-failing elements | 13:43 |
| tristan | jonathanmaw, what I mean about runtime is... that I suspect the result will be different depending on what you have staged in the sandbox | 13:44 |
| tristan | I have mostly been testing this with the GNOME builds against the debootstrap-ostree base | 13:44 |
| tristan | in there, default is dash too | 13:45 |
| tristan | not bash | 13:45 |
| tristan | ssam2, sorry I didnt look your paste properly | 13:46 |
| tristan | ssam2, so did you install buildstream on the host you are pushing to ? | 13:47 |
| tristan | ssam2, and did you follow the instructions at https://buildstream.gitlab.io/buildstream/artifacts.html#artifacts ? | 13:47 |
| ssam2 | i did | 13:48 |
| tristan | ssam2, if you have configured it with the ForceCommand as recommended, then any 'ssh -p <port> -l artifacts <url> command' invocation will be turned into the ForceCommand, and I would expect to see some output in the terminal from that | 13:48 |
| ssam2 | i've got a new error | 13:49 |
| ssam2 | pushreceive: INFO: Executing ssh -l ostree -p None ostree.baserock.org bst-artifact-receive --verbose ssh://ostree@ostree.baserock.org/cache/ | 13:49 |
| ssam2 | pushreceive: INFO: Receiving repository information | 13:49 |
| ssam2 | Bad port 'None' | 13:49 |
| tristan | but it's a good place to start debugging, fondle the port | 13:49 |
| tristan | oh | 13:49 |
| tristan | that's a bug | 13:49 |
| tristan | and I know why | 13:49 |
| tristan | ssam2, push-port: ostree@ostree.baserock.org:cache | 13:50 |
| tristan | ssam2, when you specify ssh://, the original ostree-push code uses urllib to try to do something... I dont know, a bit crazy | 13:50 |
| ssam2 | oh, push-url you mean | 13:50 |
| ssam2 | OK, will try that | 13:50 |
| tristan | it thinks there is some kind of way to specify the port in the ssh:// url | 13:50 |
| tristan | and tries to resolve it from the url | 13:51 |
| tristan | even though, as far as I know, there is no such thing as specifying the port in an ssh:// url | 13:51 |
| tristan | (admittedly, it's not the kind of thing I would know, but I didnt find out how with some googling) | 13:51 |
| tristan | it does 'if netloc', so... there is apparently, some kind of obscure way called 'netloc' that elite people that I never heard of... use to specify URIs | 13:53 |
| ssam2 | I think you can do ssh://ostree@ostree.baserock.org:22200/cache | 13:53 |
| ssam2 | netloc is part of the uri, I've only ever seen the term in python's url parsing code though | 13:54 |
| tristan | I tried that | 13:55 |
| tristan | and failed | 13:55 |
| tristan | I was adding the 'push-port' attribute and thought, maybe we dont need it ? but I was unable to make that work | 13:56 |
| ssam2 | ok, this time: host key verification failed | 13:57 |
| tristan | ssam2, if you can make that work, feel free to patch buildstream to remove the push-port attribute and document how to specify it in the URI | 13:57 |
| ssam2 | not much buildstream can do about that, but might be nice to remind in the docs that you should ensure host keys are valid | 13:57 |
| tristan | well, that's pretty indicative | 13:57 |
| tristan | We documented that you need to authorize the users with their public keys, is that not enough ? | 13:58 |
| tristan | What is a "host key" ? | 13:58 |
| ssam2 | probably | 13:58 |
| ssam2 | host key is the identity of the machine itself | 13:58 |
| tristan | that you push from ? | 13:58 |
| ssam2 | the first time you connect interactively to a machine by ssh, you see something like this: | 13:58 |
| ssam2 | The authenticity of host 'ostree.baserock.org (185.43.218.170)' can't be established. | 13:58 |
| ssam2 | ECDSA key fingerprint is SHA256:xTCKfsH5tNZSUN6tSw6IuIHlgBAWxYpktOk2lrbW3Nc. | 13:58 |
| ssam2 | ECDSA key fingerprint is MD5:df:14:78:e0:a8:c3:83:ef:06:a2:bb:5f:a5:c8:ba:14. | 13:58 |
| ssam2 | Are you sure you want to continue connecting (yes/no)? yes | 13:58 |
| tristan | Right | 13:58 |
| ssam2 | in this case it's the machine i'm pushing *to* | 13:59 |
| tristan | Ah, ssam2 so then you need to remove it from your known hosts | 13:59 |
| ssam2 | well, I need to add it there | 13:59 |
| tristan | ? | 13:59 |
| ssam2 | I guess since BuildStream's SSH session is non-interactive, if the host key isn't in known_hosts then it tries to ask "Are you sure you want to continue connecting", can't because there's no interactive stdin, and raises an error instead | 13:59 |
| tristan | I thought the failure was the case where, the host you are contacting has changed | 14:00 |
| ssam2 | that can happen too | 14:00 |
| tristan | I.e. you are resolving url to new machine | 14:00 |
| ssam2 | but if you never contacted the host, the default SSH config is to ask, or fail if it's not possible to ask | 14:00 |
| ssam2 | it's a minor thing really, but i'm sure i'm not the last person who will hit it | 14:00 |
| tristan | I thought maybe, in the course of your setting up this cache, that may have happened | 14:00 |
| tristan | ssam2, if it does not work out of the box without interaction (in a subprocess not connected to terminal), then it's a bug in BuildStream | 14:01 |
| ssam2 | ok | 14:01 |
| tristan | Either we need to use some extra option to ssh to --shut-the-hell-up-and-dont-ask | 14:01 |
| tristan | Or if thats not possible, we need to check that at startup, before running the scheduler | 14:01 |
| ssam2 | that's possible, but it's a possible of a security risk because you aren't protected against man-in-the-middle attacks | 14:01 |
| ssam2 | although since you're just sending an artifact that might not be an issue | 14:02 |
| ssam2 | if the artifact is something secret that an attacker might want to steal, then it would be a problem | 14:02 |
| ssam2 | I say check at startup | 14:02 |
| tristan | I see | 14:02 |
| tristan | yeah okay | 14:02 |
| tristan | Dont know how it's done | 14:02 |
| ssam2 | i'll see if I can fix it | 14:02 |
| tristan | ssam2, probably we want to document that you need ssh-agent to hold on to your unlocked key too | 14:02 |
| tristan | It seems that everybody just "has that" nowadays | 14:03 |
| ssam2 | ah yeah, if your key has a passphrase you'll want that | 14:03 |
| tristan | but I can recall in a not so distant past, that I needed to ensure that | 14:03 |
| tristan | my desktop does it automatically, gnome-keyring or smth | 14:03 |
| tristan | anyway, logging in unlocks it | 14:03 |
| tristan | pretty typical, but it's wrong to assume that people have that feature I guess | 14:04 |
| tlater | Most of my desktops don't, I'd get really confused x) | 14:04 |
| ssam2 | artifacts pushed \o/ | 14:06 |
| ssam2 | https://ostree.baserock.org/cache/refs/heads/baserock/ | 14:06 |
| tristan | ssam2, :) | 14:29 |
| ssam2 | and successfully pulling them on a different machine | 14:29 |
| ssam2 | nice! | 14:29 |
| tristan | hehe | 14:29 |
| tristan | yeah, holy shit it works ! | 14:29 |
| ssam2 | next stop: autobuilders! | 14:30 |
| ssam2 | well, and fixing the issues I hit | 14:30 |
| tristan | yeah | 14:30 |
| tristan | So I guess one annoying thing is we can have autobuilders with gitlab | 14:30 |
| tristan | afaics | 14:30 |
| ssam2 | we can't have them? I think we can ... | 14:30 |
| tristan | not that it's a huge deal, I dont think gitlab is the right place for that anyway | 14:30 |
| tristan | But, well... if we can, then I dont see how we can do it without exposing an acceptable private key for uploading artifacts | 14:31 |
| ssam2 | i think we can set variables privately in gitlab config | 14:31 |
| ssam2 | https://docs.gitlab.com/ce/ci/variables/README.html#secret-variables | 14:31 |
| tristan | So we would have the whole private key as that, and then cat it invisibly into /root/.ssh/id_rsa ? | 14:32 |
| ssam2 | yeah | 14:32 |
| ssam2 | although seems that's not totally secure ... if someone pushed a branch with a .gitlab-ci.yml file that did `cat /root/.ssh/id_rsa` then they would see it | 14:32 |
| tristan | true | 14:33 |
| ssam2 | only trusted people can push branches I suppose | 14:33 |
| tristan | Well | 14:33 |
| tristan | there are different levels of trust, but it seems the features on that page have limitations | 14:33 |
| ssam2 | oh, or we could make them available only for "protected branches" | 14:33 |
| tristan | so you can share the secret only with trusted branches | 14:33 |
| tristan | right | 14:33 |
| ssam2 | yeah | 14:33 |
| tristan | protected | 14:33 |
| ssam2 | i.e. just 'master', which nobody can push to | 14:33 |
| tristan | right, well in buildstream I have it so that you and a few others can | 14:34 |
| ssam2 | you can trust us :-) | 14:34 |
| tristan | because I think it's totally stupid to go through the whole pipeline thing and waste time for trivial patches | 14:34 |
| tristan | which I can okay on irc at least | 14:34 |
| tristan | but yeah I can :) | 14:34 |
| tristan | also I suppose the policy would be different for a repository dedicated to building operating systems | 14:35 |
| juergbi | tristan: strict as top-level bool option in userconfig or where should i put it? | 14:37 |
| tristan | hmmm | 14:39 |
| tristan | juergbi, I guess toplevel, right there at the top ? | 14:39 |
| tristan | I guess this deserves high visibility, and I dont know where we could group it | 14:40 |
| juergbi | same here, let's do that, then | 14:40 |
| tristan | Probably it would make sense to put the work directories in a group and rename them | 14:40 |
| tristan | directories: source, build, artifacts, logs | 14:40 |
| tristan | I'm sure nobody is using that (yet) | 14:41 |
| tristan | So I have a fork of gnome-modulesets which has a stack to build everything in org.gnome.Sdk.json, except for vala-bootstrap, dbus-glib, libdatrie, libthai, gtk+-2, opus, libcanberra, enchant and python-gstreamer | 14:43 |
| tristan | Basically those are the only parts of the GNOME SDK that dont have declarations in the jhbuild modulesets | 14:44 |
| tristan | The pipeline resolves, but those need to be added | 14:44 |
| tristan | Also the pipeline resolves with variant conditionals, so that some things which we do build in GNOME, we dont in the flatpak runtime | 14:45 |
| tristan | So one half of the deps is figured out, now I have to fill in the blanks of the other side of those variant conditionals, to depend on and build the other elements | 14:45 |
| tristan | Then, I have to figure out how I'm going to get jhbuild2bst to continuously churn out a converted modulesets with the added elements and variant conditionals; I suppose that will be a hard coded overlay of the conversion | 14:46 |
| tristan | Final touches build-wise, will be to take some org.gnome.Sdk flavored configure options which dont match the jhbuild options, and variant conditionalize those | 14:46 |
| tristan | And after that; deploy a flatpak runtime, and test it in flatpak proper | 14:47 |
| tristan | Goal is; I should be able to take flatpak-builder, and tell it to build a gedit flatpak, against my buildstream-built SDK instead of the ones published on sdk.gnome.org | 14:47 |
| tristan | and of course run the result in flatpak | 14:48 |
| tristan | and that should be a good enough demo to win. | 14:48 |
| tristan | ssam2, so do you like the updated UI and summary ? | 14:51 |
| tristan | :) | 14:51 |
| tristan | seen the utf8 arrows ? | 14:51 |
| *** xjuan has quit IRC | 14:51 | |
| tlater | So I assume the neatest way to prevent pushing artifacts made with a workspaced source is just to include a "tainted" file in its metadir, and looking for that when loading the element data? | 14:53 |
| ssam2 | oh yeah, nice arrows... it's quite a lot of info now | 14:54 |
| tristan | tlater, juergbi so I was hoping you guys would hash that out together, but I would prefer everything be recorded in yaml honestly, not flag files | 14:54 |
| tlater | juergbi: Opinion on adding this to public.yaml? | 14:54 |
| tristan | So right now there is the public yaml, and soon there is going to be the keys | 14:54 |
| tristan | not public.yaml, thats public data | 14:55 |
| tristan | i.e. straight up encoded public: | 14:55 |
| tristan | Element.set_public_data() | 14:55 |
| juergbi | yes, either we add it to keys.yaml | 14:55 |
| juergbi | or create a new one | 14:55 |
| tristan | juergbi, maybe there should be a general artifact.yaml, and add the general stuff in there, including cache keys ? | 14:56 |
| tlater | What would this new one contain? I feel like it's too little info to really create a new file for | 14:56 |
| tristan | exactly | 14:56 |
| juergbi | would be fine with me | 14:56 |
| tlater | An artifact.yaml with a node `tainted: yes|no`, so that there can be other reasons to exclude an element? | 14:57 |
| juergbi | not sure whether it's better to use a boolean here or tainted: workspace | 14:58 |
| tristan | tlater, So stick with True/False I think for booleans, just consistent with config files | 14:58 |
| tristan | Or, 'workspaced: True' ? | 14:58 |
| tristan | and resolve the tainted state based on that ? | 14:58 |
| tlater | That's moreso my question | 14:58 |
| tlater | I suppose `workspaced: True` gives more flexibility | 14:59 |
| juergbi | in a way it would be useful for the push blacklisting code not to have to know that detail | 14:59 |
| juergbi | it depends from what angle you're looking at it | 14:59 |
| tristan | Right | 14:59 |
| tristan | I think I prefer 'workspaced: True' | 14:59 |
| tristan | From the angle of forward compatibility of the yaml files | 15:00 |
| tristan | So we could have an Element internal API that decides tainted state based on that | 15:00 |
| tristan | And buildstream can also be able to tell the user why (even if we dont right away) | 15:00 |
| tristan | if we dont lose the information | 15:00 |
| tlater | Yeah, I like that. | 15:00 |
| tristan | oh cute | 15:03 |
| tristan | I had suspended push task | 15:03 |
| tristan | and resumes beautifully haha | 15:04 |
| *** tristan has quit IRC | 15:09 | |
| *** tristan has joined #buildstream | 15:26 | |
| *** ChanServ sets mode: +o tristan | 15:26 | |
| gitlab-br-bot | push on buildstream@master (by Sam Thursfield): 1 commit (last: _artifactcache/pushreceive.py: Fix obvious typo) https://gitlab.com/BuildStream/buildstream/commit/eb67b93f14fe0b8b7882650f4e3ab5868c984013 | 15:36 |
| *** xjuan has joined #buildstream | 16:28 | |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 16 commits (last: _pipeline.py: Handle exceptions when checking out elements) https://gitlab.com/BuildStream/buildstream/commit/b9cfc6399d71db9764f1e167709dfe423b972b66 | 16:36 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: WIP: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 16:36 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 1 commit (last: WIP: Add workspace artifact cache metadata) https://gitlab.com/BuildStream/buildstream/commit/f57a9da9a79aa2dcb769f3599ec25d1c567216fd | 17:04 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: WIP: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 17:05 |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 1 commit (last: WIP: Add workspace artifact cache metadata) https://gitlab.com/BuildStream/buildstream/commit/31610a50b6c95050b611d6ba49121f62bbe9a93d | 17:05 |
| gitlab-br-bot | buildstream: merge request (workspaces->master: WIP: Workspaces) #50 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/50 | 17:05 |
| *** tlater has quit IRC | 17:06 | |
| *** jonathanmaw has quit IRC | 17:40 | |
| *** ssam2 has quit IRC | 18:03 | |
| *** tristan has quit IRC | 23:36 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!