| *** juanalday has joined #buildstream | 00:35 | |
| *** nimish has joined #buildstream | 01:16 | |
| *** juanalday has quit IRC | 02:37 | |
| *** mlalkaka has joined #buildstream | 02:40 | |
| mlalkaka | Hi all, I'm trying to follow the Newcomers/BuildSystemComponent guide to build parts of GNOME using BuildStream. However, when BuildStream tries to build librsvg, it runs into an error: 'failed to select a version for the requirement `cairo-rs = "= 0.6.0"`. candidate versions found which didn't match: 0.5.0. location searched: directory source `/buildstream/gnome/sdk/librsvg.bst/crates` (which is replacing registry | 02:43 |
|---|---|---|
| mlalkaka | `https://github.com/rust-lang/crates.io-index`)' | 02:43 |
| mlalkaka | Does anyone know how to fix this? | 02:43 |
| mlalkaka | (Wasn't sure whether this belonged in the #newcomers group or here.) | 02:43 |
| *** mlalkaka has quit IRC | 03:12 | |
| *** mlalkaka has joined #buildstream | 03:13 | |
| *** nimish has quit IRC | 03:17 | |
| *** juanalday has joined #buildstream | 05:35 | |
| *** juanalday has quit IRC | 05:38 | |
| *** tristan has joined #buildstream | 06:26 | |
| *** mlalkaka has quit IRC | 07:48 | |
| gitlab-br-bot | tristanvb opened issue #929 (Spurious blockingio error) on buildstream https://gitlab.com/BuildStream/buildstream/issues/929 | 07:51 |
| *** tristan has quit IRC | 09:16 | |
| *** tristan has joined #buildstream | 09:25 | |
| juergbi | tristan: I'm considering replacing the file list parameter of copy_files/link_files/Directory.import_files with a filter function parameter | 10:52 |
| juergbi | i.e., instead of using list_relative_paths(), optionally creating a subset of that file list and then passing that to copy_files(), the code would directly pass a suitable filter function (e.g., doing the split match()) and copy_files() would iterate over the directory | 10:55 |
| juergbi | I think this would be more efficient, especially in the CAS-based directory case (allow recursive operation without having to scan the file list multiple times), but probably also in the regular filesystem case | 10:56 |
| juergbi | side-note, it may actually be possible to merge _process_list() with Directory.import_files(), having the replace logic etc. only in one place, but we should do this step by step | 10:58 |
| juergbi | any thoughts on this? | 11:04 |
| *** nanonyme has joined #buildstream | 11:46 | |
| *** nanonyme has left #buildstream | 11:46 | |
| doras[m] | I'm getting different cache keys on my x86_64 machine than the ones that are in the cache server, even though the build products of both the cache server and my own machine are expected to be the same; I set the same target_arch and bootstrap_build_arch options as the ones set in the aarch64 CI that pushed them to the cache server. | 11:47 |
| doras[m] | Is there a way for me to figure out why those keys are different, or exactly how they are calculated? | 11:47 |
| juergbi | doras[m]: same buildstream version? (1.2.4, I presume) | 11:48 |
| juergbi | doras[m]: ah, 1.2 doesn't support the `build-arch` config key yet | 11:50 |
| juergbi | due to that, buildstream 1.2 generates different cache keys depending on the host architecture | 11:51 |
| juergbi | this is fixed in buildstream master by allowing a project to set `build-arch` in `sandbox`, which may depend on a project option, of course | 11:52 |
| doras[m] | juergbi: yes, 1.2.4. Why does BuildStream 1.2 assume that the host arch is the arch of the build products as well? Sounds like a wrong assumption to make. | 11:53 |
| juergbi | in which case, buildstream will use that architecture for cache key generation instead of the host's | 11:53 |
| juergbi | it doesn't exactly assume that | 11:53 |
| juergbi | it assumes that the execution environment (i.e., the build architecture) is the same as the host | 11:53 |
| juergbi | which is typically true, if you use local execution and don't use qemu-user | 11:54 |
| juergbi | as mentioned, this limitation is already lifted, but only in master | 11:54 |
| doras[m] | juergbi: does BuildStream 1.2 expose the host arch in some way to elements? | 11:55 |
| juergbi | there is no configuration or variable for it in 1.2, if that's what you mean | 11:56 |
| doras[m] | Then wouldn't it be correct to assume that the host arch doesn't affect the execution environment in any way? | 11:57 |
| doras[m] | In which case, taking into account the host arch in cache key calculation is done for no actual benefit. | 11:58 |
| doras[m] | juergbi: if the above is correct, I'd like to submit a MR for 1.2 to remove the host arch from cache key calculation. | 12:03 |
| juergbi | doras[m]: the host arch can affect the execution | 12:11 |
| juergbi | e.g., `uname -m` | 12:11 |
| juergbi | and, of course, if the staged binaries are not supported by the host arch (and no qemu-user is installed), the build would fail | 12:12 |
| juergbi | (and we support caching failures) | 12:12 |
| doras[m] | uname is just an executable, isn't it? It comes from the execution environment. If my execution environment is aarch64, it would be aarch64. If it's x86_64, it would be x86_64. | 12:13 |
| juergbi | it's about the output that differs | 12:13 |
| juergbi | an element may do things differently depending on what `uname -m` says | 12:14 |
| juergbi | e.g., autotools does that | 12:14 |
| doras[m] | I meant that the output is aarch64 in my aarch64 execution environment and x86_64 in my x86_64 execution environment. | 12:14 |
| juergbi | yes, exactly | 12:14 |
| doras[m] | At least this is what my testing shows. | 12:14 |
| doras[m] | So it doesn't have anything to do with the host. | 12:15 |
| doras[m] | Or did I miss anything? | 12:15 |
| juergbi | if the underlying syscall is called, the kernel will return the host arch | 12:15 |
| juergbi | qemu-user intercepts that, of course | 12:16 |
| juergbi | however, with mixed arch environments you can get issues | 12:16 |
| juergbi | e.g., you could have a x86-32 bootstrap | 12:17 |
| juergbi | if you run that on a x86-64 system, the x86-32 `uname -m` would still return x86_64 because that's what the kernel returns | 12:17 |
| juergbi | i.e., if you run ./configure using the x86-32 bootstrap, you get a different build depending on whether you run that on a x86-64 or a x86-32 system | 12:18 |
| juergbi | (the latter can be simulated using `linux32`) | 12:18 |
| doras[m] | So x86-64 and x86-32 are a special case because the kernel can support both execution environments at the same time. | 12:19 |
| juergbi | you could also have Itanium and x86-32 | 12:19 |
| juergbi | or aarch64 and aarch32 | 12:19 |
| juergbi | and with qemu-user you can have similar issues, depending on how you mix binaries | 12:20 |
| doras[m] | aarch64 can run aarch32 machine code? | 12:20 |
| juergbi | aarch64 kernels can be configured to support aarch32 userspace | 12:20 |
| juergbi | if the hardware supports it | 12:20 |
| juergbi | e.g., typical smartphone SoCs support that | 12:20 |
| juergbi | some server CPUs don't | 12:20 |
| doras[m] | How did you avoid these issues in BuildStream master? | 12:21 |
| juergbi | we allow the project/element to set `build-arch` (in `sandbox` config group) | 12:21 |
| juergbi | and use that value instead of the host arch | 12:21 |
| juergbi | (default is the host arch) | 12:22 |
| doras[m] | Do you intercept syscalls to avoid uname issues? | 12:22 |
| juergbi | if you specify build-arch: x86-32, we also automatically run the sandbox via `linux32` | 12:22 |
| juergbi | and if you specify an incompatible build-arch, you currently get an error, iirc | 12:23 |
| juergbi | this would have to be extended to allow operation with qemu-user | 12:24 |
| doras[m] | But what about uname? Will I still get x86-64 on a x86-64 host even if I set "build-arch" to x86-32? | 12:25 |
| juergbi | no, you get i686 thanks to linux32 | 12:25 |
| juergbi | unfortunately, there is no such approach to override it across incompatible architecture, afaik | 12:26 |
| juergbi | we might have to use seccomp/ptrace to fully support this | 12:27 |
| doras[m] | So can 1.2 be made to configure its sandbox similarly to master and avoid these issues? | 12:27 |
| doras[m] | Or are there significant dependencies that are missing? | 12:27 |
| juergbi | we normally don't backport format enhancements | 12:28 |
| juergbi | freedesktop-sdk would also have to use it, and thus depend on that new version | 12:28 |
| doras[m] | We already depend on 1.2.4 :) | 12:29 |
| juergbi | one option could be to add an unsupported environment variable that can be used to override the host arch | 12:33 |
| *** nimish has joined #buildstream | 13:40 | |
| *** nimish has quit IRC | 15:12 | |
| *** juanalday has joined #buildstream | 17:31 | |
| *** juanalday has quit IRC | 17:45 | |
| *** juanalday has joined #buildstream | 17:46 | |
| *** juanalday has quit IRC | 17:50 | |
| *** juanalday has joined #buildstream | 17:53 | |
| *** juanalday has quit IRC | 20:07 | |
| *** tristan has quit IRC | 20:12 | |
| *** juanalday has joined #buildstream | 20:12 | |
| *** juanalday has quit IRC | 21:13 | |
| *** mlalkaka has joined #buildstream | 21:42 | |
| doras[m] | juergbi: I've added the environment variable, and it seems to work well. I've been able to pull from the cache server and build using the pulled artifacts. Pretty much covers my cross-building use case. Should submit a MR for bst-1.2? | 23:09 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!