IRC logs for #buildstream for Friday, 2020-05-08

*** narispo has quit IRC01:39
*** narispo has joined #buildstream01:39
*** tristan has quit IRC04:44
*** tristan has joined #buildstream04:54
*** narispo has quit IRC06:03
*** narispo has joined #buildstream06:04
*** pointswaves has joined #buildstream06:55
*** ChanServ sets mode: +o tristan07:55
tristanjuergbi, I've posted an elaborate email on the problems of the junction jungle (aka 'the juncle')07:55
tristanI think some people are having a day off today for some holiday reason07:55
juergbiyes, UK people are on bank holiday07:56
juergbiwill take a look07:56
benschubertHey, could someone have a look at https://gitlab.com/BuildStream/bst-plugins-experimental/-/merge_requests/107 ?08:01
* tristan takes a look08:08
benschubertthanks :)08:10
* tristan notices that bst-plugins-experimental is doing `from buildstream.testing.something_else import foo` rather than `from buildstream.testing`, and wonders if we want to cleanup around there08:11
tristanbut I guess we want to cleanup more than just those nitpicks08:11
benschubertyeah, guess there is quite a bit of things to do there08:12
tristanAt first I was confused that formatting changes seemingly intended for one commit ended up in the other, but I see the second commit adds setup.py and also fixes it's format08:16
tristanlooks clean to me08:16
benschubertthanks :) let's get that in before more bleeding happens08:16
*** tristan has quit IRC08:24
pointswavestristan they move the early may bankholiday to today because its 75 years since the end of the second world war in Europe.08:45
*** phoenix has joined #buildstream09:06
benschubertpointswaves: concerning what oyu asked yesterday. are both versions of buildstream you are using on the same commit?09:58
coldtompointswaves, your issue is you're using the 1.4 bst-artifact-server image, and a master client10:01
coldtomthere is no corresponding image for master, in celduin-infra we use buildstream/buildstream:dev for this, as it still provides the `bst-artifact-server` command10:02
pointswavescoldtom, i presumed it must be soemthing like that, master-140892353 tag seems rather missleading10:15
pointswavesthanks10:15
*** phildawson has quit IRC10:38
*** rdale has joined #buildstream11:25
jjardonHi would it be possible to have a new bst-1.x release? IT has some issues that is affecting some projects12:44
jjardon1.4.3 I think it would be12:45
*** toscalix has joined #buildstream13:46
*** slaf_ has joined #buildstream14:04
*** slaf has quit IRC14:06
pointswavesWhen i build https://gitlab.com/pointswaves/keytest on x86 i get a cache key of 67b38a5cf9461b1c896412a789af9b26a338546cf900f969ceaa1dca8eb1090d but on aarch64 51a1df4f9e3a2184deda7f621368562e2a2081a96ed3ccd841acf14d17ffd07f why do includ elements like this not have the same key on all architectures? i am running both at 1.93.3 and i updated both today14:37
pointswavesi wish to pull and checkout remotely build artifacts, how todo ask bst to make the artifacts mach so i can pull the other arches artifact?14:39
juergbipointswaves: there is no special casing for `import`14:41
juergbimaybe we could skip the sandbox parts of the cache key for all elements with BST_RUN_COMMANDS = False14:41
abderrahim[m]+114:42
abderrahim[m]I've stumbled upon this too and didn't know there was already something in plugins that tells they don't run any command14:42
abderrahim[m]pointswaves: for your second question, define an option for the arch (and assign it to a variable), and define sandbox:build-arch to the value of the option14:44
abderrahim[m]then you can do things like `bst -o arch aarch64 artifact pull some/element.bst`14:44
pointswavesthat sounds like what i want14:45
pointswavesis sandbox:build-arch a config option or a var?14:45
abderrahim[m]it's actually14:45
abderrahim[m]sandbox.14:45
abderrahim[m]sandbox:14:45
abderrahim[m]  build-arch: '%{arch}'14:45
pointswaves:)14:46
abderrahim[m]in project.conf14:46
pointswaveslike https://gitlab.com/pointswaves/keytest/-/blob/willsalmon/sandboxarch/project.conf ?14:52
pointswavesbecause bst dose not seem to expand the target_arch..14:52
abderrahim[m]pointswaves: you named your variable arch https://gitlab.com/pointswaves/keytest/-/blob/willsalmon/sandboxarch/project.conf#L13 and are trying to expand the name of the option rather than that of the variable14:54
abderrahim[m](you can have the same name if you want)14:55
abderrahim[m]and it's probably a good idea14:55
pointswavesare you recomending setting the var to target_arch ?14:56
pointswavesthat dose not seem to work..14:57
abderrahim[m]I think it's less confusing if the option and the variable containing it are the same name, yes14:58
abderrahim[m]let me try14:59
pointswavesi have bumped https://gitlab.com/pointswaves/keytest/-/blob/willsalmon/sandboxarch/project.conf14:59
pointswaveshttps://pastebin.com/GYwLwsX215:00
abderrahim[m]I believe it did work at some point, may be a regression15:02
juergbisounds like we're missing variable substitution. if so, we definitely need to fix that, and add a test15:04
*** tristan has joined #buildstream15:08
pointswavesjuergbi, were should i look? i never knew much about the loader even when i worked on the code regually15:13
juergbipointswaves: it's outside/after the loader. we need something like Elelment.__expand_environment() also for the sandbox15:19
pointswavesjuergbi, like https://gitlab.com/BuildStream/buildstream/-/commit/a4f504dce4cf3dba74837f4d64b58a9003463157 ? these new nodes confuse me, can i turn a dict back in to one? im gessin this means im doing it wrong?15:57
pointswavesoh i got it the worng way round..15:59
pointswavesi have something that runs...16:07
pointswavesSAME CACHE KEY :O16:09
pointswavesjuergbi, how do i test for this?16:10
*** slaf_ has quit IRC16:21
*** slaf has joined #buildstream16:21
pointswavesI gues we could just have a project conf were its a var and make sure that it atleast expands..16:22
juergbiyep16:25
juergbias BuildStream validates arch names, unexpanded variable will immediately bail out, as you've noticed16:26
benschubertjuergbi: I know it's been a long while but should we still get https://gitlab.com/BuildStream/website/-/merge_requests/121 in ? If so, mind merging it I don't have permissions there16:51
pointswavesi need to verify but something like https://gitlab.com/BuildStream/buildstream/-/commit/e0890f7bb3fde705038c0e40c9efd43a1d42dcfb16:51
juergbibenschubert: yes, I think so. pip3 install --user -e . doesn't even work anymore, afaik16:53
juergbiI think we simply forgot to merge this back then, right?16:53
juergbiand cs-shadow has added venv instructions in the meantime16:54
benschubertah, should I just remove it entirely then?16:54
benschubertyeaah we did indeed forget that16:54
*** pointswaves has quit IRC16:55
juergbibenschubert: as long as we have instructions to install from tarball without venv, instructions to install from git without venv are not less useful, imo16:58
benschubertOk I'll clsoe this one then :)16:58
juergbiI wrote _not_ less useful16:58
juergbimaybe focusing on fewer installation variants would make sense16:59
benschubertah woops missed that one16:59
juergbithat MR is definitely better than the status quo, though. I don't have a strong opinion on it right now16:59
benschubertOk, then let's get it in and we can make it better later?17:00
juergbiok17:00
benschubertthanks17:03
*** pointswaves has joined #buildstream17:27
pointswavesjuergbi, i have added https://gitlab.com/BuildStream/buildstream/-/merge_requests/190918:12
juergbipointswaves: thanks. I think this might not be working for element-specific sandbox config18:15
juergbii.e., it only expands project sandbox config18:15
juergbishould probably expand after composite18:15
*** phoenix has quit IRC18:42
*** toscalix has quit IRC18:42
*** toscalix has joined #buildstream18:43
*** toscalix has quit IRC18:52
*** toscalix has joined #buildstream18:54
benschubertCould someone have a look at https://gitlab.com/BuildStream/buildstream/-/merge_requests/1910 ? We'll need to rebase all our PRs on this afterwards because of https://docs.gitlab.com/ee/user/compliance/license_compliance/#migration-from-license_management-to-license_scanning19:22
pointswavesbenschubert, could you add a description?19:23
benschubertpointswaves: done, I've copied the commit message there19:24
benschubertthanks!19:32
pointswavesjuergbi, do we really want element wise arch? seems a bit mental but i dont see why not19:35
* pointswaves tries to grok what `<juergbi> should probably expand after composite` means in code19:36
juergbipointswaves: it can be interesting for cross compilation with bootstrapping19:44
juergbialso, variable expansion needs to be handled correctly independent of whether anyone will use per-element build arch19:44
juergbiother parts of the `sandbox` config may be more commonly set in elements (e.g., future #38 knobs) and they may use variables as well19:45
pointswavesfair19:48
pointswavesIm just trying to understand, it needs to happen in the same place tho? as that info is not inside the cls so cant happen in the function but needs to happen before the funtion as thats were the bang happens19:49
pointswavesdo i just need a alternative function to __expand_environment?19:49
juergbipointswaves: yes, expand_environment is only meant for env. variables19:56
juergbisandbox config is a bit different. maybe the split between extract and expand helper functions doesn't make sense19:58
juergbii.e., we could also consider making extract an instance method and adding expansion after _composite() in there19:58
juergbibecause I don't think we want to build SandboxConfig first with unexpanded values and then expand them19:59
juergbiI think benschubert suggested we actually expand all variables early on instead of the current explicit node_subst_vars() steps, which can be missed20:00
juergbiso it might change a bit again in the future20:01
pointswavesI dont really understand the element to follow all of that..20:04
benschubertjuergbi: correct, haven't been able to build a nice design yet though :/20:04
benschubertand currently focused on plugins and related tests20:04
benschubertbut might get to it in the month (at least I hope so) though if someone else want to give it  a go, I'm happy to help20:04
pointswavesI dont think i am going to have time to attack anything that big in my own time.. do you feel the current MR is going to make further work harder? if not can we have it as a incremental imporement?20:11
juergbino, this shouldn't be a blocker for this MR at all. was just fyi20:19
benschubertyeah, I'm not entirely convinced with the MR though20:19
benschubertwonder if we could get something nicer (and an explicit test)20:19
*** rdale has quit IRC20:19
*** Paisley4198 has joined #buildstream20:44
*** Paisley4198 has quit IRC20:46
*** pointswaves has quit IRC22:07
*** slaf has quit IRC22:24
*** slaf has joined #buildstream22:24
*** toscalix has quit IRC22:58
*** benschubert has quit IRC23:11
*** mohan43u_ has joined #buildstream23:37
*** mohan43u has quit IRC23:39

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!