| *** toscalix has joined #baserock | 06:30 | |
| *** toscalix has quit IRC | 06:31 | |
| *** toscalix has joined #baserock | 06:31 | |
| *** rdale has joined #baserock | 08:06 | |
| *** jonathanmaw has joined #baserock | 08:36 | |
| *** paulwaters_ has joined #baserock | 09:43 | |
| *** paulwaters_ has left #baserock | 09:43 | |
| noisecell | https://gitlab.com/baserock/definitions/pipelines/24823524 <--- ALL IN GREEN!! \o/ | 10:32 |
|---|---|---|
| ironfoot | "in 605 minutes 14 seconds" xD | 10:32 |
| ironfoot | anyway, yaaaaaaaay! | 10:33 |
| benbrown_ | ;0 | 10:33 |
| benbrown_ | :)* | 10:34 |
| gitlab-br-bot | definitions: merge request (franred/use-1.1.3-BuildStream-release->master: Update buildStream to 1.1.3) #79 changed state ("opened"): https://gitlab.com/baserock/definitions/merge_requests/79 | 10:35 |
| noisecell | ironfoot, some more minutes .... | 10:36 |
| noisecell | ironfoot, and probably we should tell buildstream that downloading multiples times the same repository is causing big issues in our definitions pipeline | 10:37 |
| noisecell | or at least remind them that there is a bug filled 1 year ago about this :/ | 10:38 |
| ironfoot | that bug would be a big win for us | 10:39 |
| ironfoot | https://gitlab.com/BuildStream/buildstream/issues/5 | 10:42 |
| ironfoot | last comment there is a fair comment, and what we should fix is our CI cache | 10:42 |
| ironfoot | i looked into fixing this issue myself, and couldn't think of an easy solution | 10:42 |
| noisecell | ironfoot, I think is tangential, we both are right. If for some reason someone doesn't want to use the cache, then they are screw with this bug | 10:44 |
| ironfoot | only if you clean up the environment for every build though (CI) | 10:45 |
| ironfoot | you didn't have this experience building locally | 10:45 |
| noisecell | true | 10:45 |
| persia | I don't understand how this is a bug after reading ths issue. Am I correct that it only happens when two elements require the same source? | 10:48 |
| noisecell | persia, yes, you are correct | 10:51 |
| persia | In that case, isn't the easier solution to only build each element once, and split the artifacts if one needs multiple things? | 10:52 |
| noisecell | jonathanmaw, https://gitlab.com/baserock/definitions/blob/d7f306350e1b7f8dc3977fe637c8fc3f3cc67655/elements/gnu-toolchain/base.bst -- I can not see the element pushed for the armv8l64 into https://ostree.baserock.org/cache/refs/heads/baserock/bootstrap-stage3-sysroot/ . Did they got into the ostree cache anytime? | 10:52 |
| noisecell | persia, the repository is cloned for different reasons depending on the element. The most common issue is when clonning delta/linux for building linux and getting the linux headers, for example | 10:53 |
| noisecell | the issue is when cloning the repository not when building the elements | 10:53 |
| persia | I've expressed myself poorly. Apologies. | 10:54 |
| persia | In a classic environment, there was often an effort to maintain separate "kernel" and "headers" packages, in the interest of avoiding rebuilding things that depended on headers when the kernel changed. | 10:55 |
| persia | In an environment where things are built upon other things with careful effort taken to ensure reliable checksums of sources, etc., is this split still needed? | 10:56 |
| persia | Also, are there sources other than linux that are used more than once? | 10:56 |
| ironfoot | i think the main and most important case is linux.git, because of its size and how many times is used | 10:57 |
| ironfoot | it's used a couple of times in the base, to build the headers, once in the stage2 and once in the stage3 | 10:57 |
| ironfoot | (iirc) | 10:57 |
| ironfoot | in that case we need to build it twice | 10:57 |
| ironfoot | regarding using the same build to build the kernels in the bsp... maybe? but it would be a definitions rework | 10:58 |
| persia | Ah, yes, for bootstrapping it probably also applies to things like the core C library and the compiler. | 10:58 |
| persia | So bootstrapping will always be affected. | 10:58 |
| persia | My thought was that *some* of the pain could be removed with the definitions rework, and reading the gitlab issue against BuildStream, it wasn't clear to me how BuildStream *could* fix this reasonably. | 10:59 |
| ironfoot | yes, gcc and others suffer from this too, but their repos don't take that long to clone (probably because we use tarball imports) | 10:59 |
| noisecell | jonathanmaw, Im asking because I am planning to do something similar, if this was done before I can use it and carry on with my real task which is building one system in aarch64. | 11:00 |
| ironfoot | No, it's not clear how to solve this issue easily | 11:00 |
| ironfoot | I guess a less easy solution would be to change the fetching scheduler to check for identical jobs in the queue, and make them dependant or something | 11:00 |
| persia | For the kernel/header split, I think the right answer is definitions rework (although that isn't necessarily easier than sorting BuildStream). | 11:02 |
| persia | For the bootstrap case, if we're seeing parallel fetches, I think something else is wrong with the scheduling. | 11:02 |
| ironfoot | what do you mean by "something else" other than the scheduler allowing it | 11:03 |
| persia | In that, since we don't have the build-dependencies to build the thing being fetched (because we haven't fetched the prior version), we probably shouldn't have scheduled the job. | 11:03 |
| ironfoot | aha, I think that BST PoV is to fetch everything before it can be even considered to build | 11:04 |
| ironfoot | s/to build/to be built/ | 11:04 |
| persia | Oh my. That definitely matches my "something else is wrong with the scheduling" :) | 11:06 |
| jonathanmaw | noisecell: hrm, iirc I didn't manage to get an armv8 sysroot working | 11:25 |
| jonathanmaw | I'm checking to see if I have any stray files to indicate what state I left it in | 11:25 |
| noisecell | jonathanmaw, oh, I see. Yeah, any file that indicates what I'm facing would be great | 11:27 |
| noisecell | and appreciated | 11:28 |
| jonathanmaw | noisecell: looks like I don't have any work to share, if you've already found https://gitlab.com/baserock/definitions | 12:19 |
| jonathanmaw | I don't think I added anything to that while trying to get armv8 working | 12:20 |
| jonathanmaw | aha, I apparently managed to generate a bootstrap tarball with that, but didn't get as far as running it on an aarch64 system | 12:21 |
| noisecell | jonathanmaw, ok, I am following the instructions in: http://wiki.baserock.org/guides/how-to-cross-bootstrap/ -- let's see if I manage something | 12:21 |
| noisecell | jonathanmaw, I've got the stage2 tarball, now attempting to build the stage3 in aarch64 | 12:22 |
| *** gtristan has joined #baserock | 14:32 | |
| *** gtristan has quit IRC | 14:38 | |
| *** gtristan has joined #baserock | 14:41 | |
| *** toscalix has quit IRC | 16:51 | |
| *** gtristan has quit IRC | 17:08 | |
| *** jonathanmaw has quit IRC | 17:15 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!