| *** juergbi has quit IRC | 01:19 | |
| *** juergbi has joined #baserock | 03:45 | |
| *** gtristan has joined #baserock | 04:37 | |
| *** locallycompact has joined #baserock | 06:26 | |
| *** gtristan has quit IRC | 06:53 | |
| *** jude_ has joined #baserock | 06:55 | |
| *** jude_ has quit IRC | 07:18 | |
| *** jude_ has joined #baserock | 07:20 | |
| *** gtristan has joined #baserock | 07:30 | |
| *** locallycompact has quit IRC | 07:54 | |
| *** tiagogomes has joined #baserock | 08:07 | |
| *** locallycompact has joined #baserock | 08:44 | |
| *** tiagogomes has quit IRC | 09:09 | |
| *** ssam2 has joined #baserock | 09:10 | |
| *** ChanServ sets mode: +v ssam2 | 09:10 | |
| ssam2 | it didn't work, of course : https://gitlab.com/baserock/definitions/-/jobs/37306927 | 09:10 |
|---|---|---|
| ssam2 | something must have broken recently in the conversion process... no way that should take 8 hours | 09:10 |
| *** tiagogomes has joined #baserock | 09:17 | |
| *** tiagogomes has quit IRC | 09:20 | |
| *** tiagogomes has joined #baserock | 09:20 | |
| ssam2 | ouch, i've spotted at least one issue | 09:38 |
| ssam2 | which is that ybd stats every file in the current working directory | 09:38 |
| ssam2 | and since we have to have the cache in the build directory, that means it'll stat every file in the cache | 09:38 |
| paulsherwood | eek | 09:40 |
| ssam2 | paulsherwood: could we make ybd ignore hidden directories when looking for .morph files | 09:44 |
| ssam2 | ? | 09:44 |
| ssam2 | then we could call the cache dir `.cache` and solve that | 09:44 |
| ssam2 | i guess ybd is also pointlessly traversing the `.git` directory at the moment | 09:44 |
| paulsherwood | makes sense | 09:44 |
| ssam2 | ok, i'll do a patch | 09:44 |
| paulsherwood | tvm | 09:44 |
| benbrown_ | ssam2: that'd be os.walk | 09:48 |
| ssam2 | yeah | 09:49 |
| ssam2 | it seems like the code checks for hidden dirs, but does so *after* os.walk has touched every subdir | 09:49 |
| paulsherwood | hah | 10:01 |
| *** tiagogomes has quit IRC | 10:05 | |
| *** tiagogomes has joined #baserock | 10:19 | |
| ssam2 | morph-cache-server was broken on git.baserock.org after the migration, i've fixed that now. YBD CI is a lot faster :-) | 10:36 |
| paulsherwood | :) | 10:42 |
| ssam2 | could someone review the patch please ? https://gitlab.com/baserock/ybd/merge_requests/396 | 10:45 |
| gtristan | ssam2, added comment, paulsherwood would probably know best how safe it is to revert that | 10:49 |
| paulsherwood | ssam2: what if directory != './' ? is that no longer possible in any code path? | 10:52 |
| ssam2 | paulsherwood, it's inside a `chdir(directory)` context manager | 11:16 |
| ssam2 | so that should still work fine (not sure how I test that though, is there a flag I can pass to YBD to change the directory?) | 11:16 |
| ssam2 | i've updated the commit message to make it clearer that we now ignore all hidden dirs | 11:20 |
| benbrown_ | ssam2: lgtm | 11:26 |
| benbrown_ | paulsherwood, ssam2: Yeah, the chdir should ensure that | 11:28 |
| benbrown_ | ssam2: the .def suffix did pick up a Makefile.def in a git repo in definitions, which confused me momentarily | 11:31 |
| benbrown_ | but I renamed it so it was hidden and it was ignored :) | 11:32 |
| ssam2 | oh hang on, i've reintroduced the .def suffix | 11:32 |
| ssam2 | that wasn't deliberate | 11:32 |
| benbrown_ | ah, wasn't sure if it was | 11:32 |
| ssam2 | if it was, i should have said so in the commit message anyway :-) | 11:33 |
| benbrown_ | :) | 11:33 |
| ssam2 | doesn't seem to have fixed https://gitlab.com/baserock/definitions/-/jobs/37420887 | 12:13 |
| ssam2 | it seems like YBD is stuck waiting for a zombie child process | 12:14 |
| ssam2 | oh, it's moved on now | 12:14 |
| ssam2 | i do wonder what's going on though | 12:15 |
| paulsherwood | ssam2: none of that output is ybd? | 12:21 |
| ssam2 | its defs2bst | 12:21 |
| ssam2 | which calls YBD internally | 12:22 |
| *** tiagogomes has quit IRC | 12:22 | |
| paulsherwood | ah | 12:30 |
| paulsherwood | can you paste the ybd output somewhere? | 12:30 |
| ssam2 | i don't know if it's recorded anywhere | 12:30 |
| ssam2 | that's probably something that should be fixed in defs2bst | 12:30 |
| ssam2 | https://gitlab.com/baserock/definitions/-/jobs/37420892 is finally building with buildstream and pushing to the cache ... | 12:36 |
| ssam2 | it's still way too slow to get started because there's 13GB of sources in the cache, so it takes like 15 minutes to fetch and unpack everything | 12:36 |
| benbrown_ | that is rather large | 12:37 |
| ssam2 | it includes 2.4GB worth of failed linux clones in tmpdirs | 12:38 |
| ssam2 | that can at least be removed | 12:38 |
| ssam2 | other than that... very little we can do, we are building the world after all | 12:39 |
| benbrown_ | :) | 12:39 |
| *** gtristan has quit IRC | 13:03 | |
| *** laurenceurhegyi is now known as ltu | 13:06 | |
| *** tiagogomes has joined #baserock | 13:08 | |
| ssam2 | a new breakage in the CI! https://gitlab.com/baserock/definitions/-/jobs/37420892 | 14:01 |
| ssam2 | thankfully i can reproduce this one locally | 14:01 |
| ssam2 | must be a buildstream regression; nothing has changed in definitions.git | 14:02 |
| ssam2 | could be that there's nothing at /bin/sh | 14:05 |
| ssam2 | `bst shell` manages to find /tools/bin/sh though | 14:05 |
| ssam2 | maybe the issue is PATH not being set when integration commands run | 14:05 |
| *** CTtpollard has quit IRC | 14:05 | |
| paulsherwood | looking at that log... am i right that there's no way of distinguishing the various SUCCESS messages? | 14:05 |
| * paulsherwood expressly put unique words in ybd logs containing 'Elapsed' to make it easy to grep for build times, for example | 14:06 | |
| ssam2 | to the left of the word 'SUCCESS' is the name of the task | 14:07 |
| paulsherwood | right, but there are several build: SUCCESS messages for (say) stage2-fake-bash.bst] | 14:09 |
| benbrown_ | grep for "SUCCESS *-build.*.log"? | 14:10 |
| benbrown_ | though, the tasks are prefixed by 'build: ' anyway | 14:11 |
| paulsherwood | ack | 14:12 |
| *** gtristan has joined #baserock | 14:23 | |
| ssam2 | gtristan, are you about to quickly explain something in the new world order of YAML composition ? | 14:25 |
| ssam2 | in gnu-toolchain/fhs-dirs.bst we do something like: `environment: { PATH: /usr/bin:/tools/bin}` | 14:25 |
| ssam2 | that fails to override the default PATH set in the project defaults | 14:26 |
| ssam2 | seems like a bug actually, the docs say default behaviour is to replace the list | 14:27 |
| * ssam2 will try and make a test case | 14:27 | |
| gtristan | that aint a list | 14:56 |
| gtristan | ssam2, but it can be a bug indeed, this popped up today too: https://gitlab.com/BuildStream/buildstream/issues/127 | 14:57 |
| gtristan | I'm quite surprised though, if that is a bug; I tried something similar just this weekend and I thought it worked | 14:58 |
| ssam2 | hmmm true that it's not a list | 14:59 |
| gtristan | still the string should overwrite the other for that element | 14:59 |
| ssam2 | it seems like the test cases already cover this kind of thing, and it works in those | 14:59 |
| ssam2 | i tried to create a new test case with the exact setup, but that also passes | 14:59 |
| gtristan | hrrmmmm | 14:59 |
| gtristan | Really ?! | 15:00 |
| gtristan | wtf ? | 15:00 |
| ssam2 | so, possibly some pecularity in Element.__extract_env() | 15:00 |
| ssam2 | or i've missed something, of course -- i've not really looked at this code before | 15:00 |
| gtristan | Ah, you constructed a test case without involving the project / element compositing mechanics | 15:00 |
| ssam2 | yeah, i put it in tests/yaml/yaml.py | 15:00 |
| ssam2 | i'll try putting one in tests/variables/ somewhere that's a bit more realistic | 15:00 |
| gtristan | yeah, I want to keep the yaml tests around, but otherwise, most/all new tests should use testutils/runcli.py stuff | 15:01 |
| gtristan | and test real things | 15:01 |
| gtristan | ssam2, do it in tests/format I think | 15:01 |
| gtristan | ssam2, there we mostly use `bst show` to test ... right now thats where the option tests live, but eventually the loader tests should be migrated to that too | 15:01 |
| gtristan | always test from the whole cli, dont bother bone picking with internals, not a good approach | 15:02 |
| ssam2 | i agree in general | 15:02 |
| gtristan | (still, I wanna keep tests/yaml/yaml.py, and maaaybe the plugin loader stuff) | 15:02 |
| * gtristan notes, it should be very easy to add a test to tests/format too, just `cp -a a_test_directory new_test_directory`, do this override in the element.bst and `bst show --format %{env}` | 15:05 | |
| gtristan | (most of the test directories around there I think, just have a project.conf and an element.bst) | 15:07 |
| * gtristan has a feeling the bug is somewhere in element.py indeed, it smells like the same test for issue 127 would probably not be reproducible in tests/yaml/yaml.py either) | 15:08 | |
| ssam2 | interestingly, `bst show --format '%{env}' --deps none gnu-toolchain/fhs-dirs.bst` shows the correct behaviour | 15:10 |
| ssam2 | it seems to only be wrong when running integration commands | 15:10 |
| gtristan | eww | 15:11 |
| gtristan | weird | 15:11 |
| gtristan | maybe it's not picking up the integration commands from the right place | 15:11 |
| ssam2 | that's using element._Element__environment | 15:11 |
| gtristan | although, not sure where else it could come from | 15:12 |
| gtristan | I mean, the env | 15:12 |
| ssam2 | where element.get_environment() uses self.__environment | 15:12 |
| gtristan | Ah, you found the bug ! | 15:12 |
| ssam2 | did i? good :-) | 15:12 |
| gtristan | sounds like - so wait, which one is using which ?? | 15:12 |
| gtristan | Nah, that is the same thing | 15:13 |
| ssam2 | yeah, i thought it was | 15:13 |
| gtristan | element._Element__environment should be replaced with the accessor, just adding noise | 15:13 |
| ssam2 | I added print() statements around the composition, and that seems to be done wrong | 15:13 |
| ssam2 | but given that, how does the format codepath work ?? | 15:13 |
| gtristan | node_sanitize ? | 15:14 |
| gtristan | oh strange, it's called by both places | 15:15 |
| ssam2 | yeah | 15:15 |
| ssam2 | i just traced through the format codepath, and final_env actually ends up with the correct value | 15:16 |
| gtristan | but not when running bst build, specifically for integration commands | 15:16 |
| ssam2 | same with the build codepath ... | 15:16 |
| ssam2 | but that must be being lost somewhere | 15:16 |
| gtristan | so `bst build`... has the right environment when running the sandbox in ->assemble() ? | 15:17 |
| gtristan | But not at integration command time ? | 15:17 |
| ssam2 | ohhh | 15:17 |
| ssam2 | this could be something much more obvious | 15:17 |
| ssam2 | such as the integration command is on a different element | 15:17 |
| gtristan | In which case, there's no bug :) | 15:18 |
| ssam2 | indeed | 15:18 |
| ssam2 | it did use to work though | 15:18 |
| ssam2 | so maybe there was a bug before :-) | 15:18 |
| gtristan | Heh | 15:18 |
| ssam2 | perhaps stack elements would previously inherit environment from their children, or something weird | 15:18 |
| gtristan | Question is, do you think it works the way it should ? | 15:18 |
| ssam2 | if I can set environment: in the gnu-toolchain/stage2.bst element, that seems fine | 15:19 |
| ssam2 | and correct | 15:19 |
| gtristan | Nah, that sounds very unlikely | 15:19 |
| gtristan | and stack elements dont run integration anyway, but yeah they might provide commands for depending elements | 15:19 |
| ssam2 | it does seem to work | 15:19 |
| gtristan | so yeah, elements run the stack integration commands | 15:20 |
| gtristan | but, the question remains: Is the environment used for building something, the same as the environment used for integrating *against* that thing, using the commands it provides ? | 15:20 |
| gtristan | this seems unclear to me, and worth thinking on | 15:20 |
| ssam2 | well, it seems like the behaviour is now that the stack runs the integration commands | 15:21 |
| gtristan | ssam2, i.e. in your specific case, you are working around this with a stack | 15:21 |
| gtristan | normally what happens, is something like; you build GTK+, which installs gtk-query-immodules, then things which depend on GTK+, have the input method modules queried before they build | 15:22 |
| gtristan | (or gdk-pixbuf loaders, etc, that kinda thing) | 15:22 |
| ssam2 | yeah | 15:22 |
| ssam2 | this is a strange case i guess, where the stack decides that ldconfig needs to be run | 15:22 |
| ssam2 | actually i guess this is working around the fact that we need stage2 to be cross-buildable | 15:23 |
| ssam2 | otherwise we could have ldconfig as an integration command of stage2-glibc | 15:23 |
| gtristan | I also added stage1, stage2 stacks because they made the thing more comprehensive | 15:23 |
| ssam2 | yeah, i like them | 15:23 |
| gtristan | before you changed it, and also used similar stacks | 15:23 |
| gtristan | It allows one to easily drop them behind with build type dependencies | 15:24 |
| gtristan | I think was the original point | 15:24 |
| gtristan | (without verbosely build depending on dozens of elements) | 15:24 |
| ssam2 | yeah | 15:25 |
| gtristan | well - I kind of feel like; the element which built something (like gdk-pixbuf for instance), is the one who knows the environment which should be used | 15:25 |
| gtristan | for integrating itself on behalf of a reverse dependency | 15:25 |
| gtristan | But, feel free to raise the issue on the ML if you like, it might be worth exploring | 15:25 |
| * gtristan thinks it's behaving right - BUT - I cant explain how this behavior could have *changed* in buildstream | 15:26 | |
| ssam2 | i think this behaviour makes sense | 15:26 |
| gtristan | this is highly unlikely, too | 15:26 |
| ssam2 | it's definitely new, though | 15:27 |
| gtristan | that said, I think we're doing things better now - I dont understand how the behavior could have changed, *however*, it's possible we fixed some edge cases in dictionary compositing in general | 15:27 |
| gtristan | e.g. the new ChainMap derived copy-on-write dictionary fixes some things; and the code is a bit less spaghetti around _yaml.py now | 15:28 |
| ssam2 | there was a successful build with bst a580a4a032c2e492c5287a976ced1db178467549 | 15:28 |
| ssam2 | which is before 'Implement array composition directives' landed | 15:29 |
| ssam2 | maybe that fixed some edge case, indeed | 15:29 |
| ssam2 | it was also before the new ChainMap | 15:30 |
| *** locallycompact has quit IRC | 16:00 | |
| * paulsherwood wonders if buildstream on converted definitions is 'ready for primetime' yet? | 16:07 | |
| ssam2 | no | 16:07 |
| paulsherwood | ok | 16:07 |
| ssam2 | first we need to fix this CI | 16:07 |
| paulsherwood | ack | 16:08 |
| ssam2 | then there are some outstanding branches to merge from the cross-bootstrap work myself and jonathanmaw did | 16:08 |
| ssam2 | then we need to update for some changes in buildstream | 16:08 |
| ssam2 | then we need to figure out how to host binary sysroots ('staging fillers' :-) | 16:09 |
| ssam2 | then it's ready! | 16:09 |
| ssam2 | hopefully that'll coincide with adds68 incorportating the important bits into the freedesktop base runtime | 16:09 |
| ssam2 | all hopes pinned on this build! https://gitlab.com/baserock/definitions/-/jobs/37465169 | 16:10 |
| * benbrown_ crosses fingers | 16:10 | |
| * paulsherwood vaguely remembers that he first got into gitlab ci as it was clocking through 10M builds | 16:10 | |
| adds68 | ssam2, i should hopefully have some bst files produced tomorrow morning i can push into the freedesktop-sdk repo :) | 16:11 |
| adds68 | Then it's just fine-tuning and bug fixing in order to get the CI stood up! | 16:11 |
| ssam2 | nice! | 16:15 |
| ssam2 | CI failure again :-( | 16:15 |
| ssam2 | i guess i'll just build locally first to find out how much more stuff has bitrotted | 16:15 |
| ssam2 | HMMM... another chunk failing because /bin/sh is missing... which can't ever have worked | 16:19 |
| ssam2 | but clearly did | 16:19 |
| ssam2 | oh... maybe overlaps behaviour has changed | 16:22 |
| ssam2 | and ruined that horrible thing we do with /bin symlinks | 16:22 |
| ssam2 | building with buildstream commit a580a4a032c2e492c5287a976ced works, so *something* has changed in buildstream ... i wonder what | 16:28 |
| *** jude_ has quit IRC | 16:29 | |
| ssam2 | ahhhhh | 16:36 |
| ssam2 | fhs-dirs requires use of the (>) operator now :-) | 16:36 |
| ssam2 | as does stage2-fhs-dirs... that no doubt explains the issue before as well | 16:38 |
| ssam2 | as usual i spent hours barking up the wrong tree | 16:39 |
| ironfoot | woof woof | 16:39 |
| ironfoot | well, is good to find the fix for an issue just before beer time :) | 16:39 |
| ssam2 | climbing time! | 16:42 |
| gtristan | funny, we didnt *remove* post-/pre- commands yet | 17:06 |
| gtristan | but anyway, should be using (<) and (>) instead yes | 17:06 |
| ssam2 | it's not pre-commands, but an arch conditional that adds extra install-commands | 17:06 |
| ssam2 | but of course, now it just overwrites the install-commands :-) | 17:06 |
| ssam2 | simple to fix anyway | 17:07 |
| *** ssam2 has quit IRC | 17:07 | |
| gtristan | ahh | 17:13 |
| gtristan | right, that behavior did change | 17:14 |
| *** gtristan has quit IRC | 17:23 | |
| *** gitlab-br-bot has joined #baserock | 17:27 | |
| *** gitlab-br-bot has joined #baserock | 17:29 | |
| *** jude_ has joined #baserock | 18:34 | |
| *** gtristan has joined #baserock | 18:35 | |
| *** jude_ has quit IRC | 18:57 | |
| gitlab-br-bot | definitions: merge request (pedro/pbr-fix->master: Fix pbr builds for python3-core) #1 changed state ("closed"): undefined | 22:21 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!