| *** benschubert has joined #buildstream | 07:22 | |
| *** tristan has quit IRC | 07:36 | |
| *** tristan has joined #buildstream | 07:41 | |
| *** ChanServ sets mode: +o tristan | 07:41 | |
| douglaswinship | Has anyone seen build failures like this? https://hastebin.com/aqovorafup.md | 07:48 |
|---|---|---|
| douglaswinship | (See the failure at the end) | 07:49 |
| douglaswinship | Normally, when something fails to build, buildstream gives an interactive menu, and you get the option to view logs / drop into a shell / etc. But this time, nothing. It just finishes cleanly. I didn't even notice it was failing, at first. | 07:49 |
| tristan | douglaswinship, That appears to be a cached failure | 07:51 |
| tristan | I.e. BuildStream did not fail, it encountered a failed build element in the cache | 07:51 |
| tristan | I'm not sure if this is a bug or not | 07:51 |
| douglaswinship | I'm a little confused. Does "a failed build element in the cache" just mean that it has a record in the cache of a build that didn't build successfully? Because that happens all the time, and doesn't look like this. | 07:52 |
| tristan | It looks like this to me | 07:54 |
| tristan | failed c7184f3b8c6612b9e23c99c8d16b6e21ee79b10563c23c345e067f5578e61431 bsp/raspi/linux.bst | 07:54 |
| tristan | Last line of the element summary, printing elements before the session starts | 07:54 |
| tristan | douglaswinship, I'm not completely up to speed on what is supposed to happen with failed elements, some of this happened a while ago and was redefined over time | 07:55 |
| tristan | douglaswinship, I have a suspicion though | 07:55 |
| tristan | Since I'm not seeing a "--retry-failed" or similar option, I'm guessing that the expected UX is for the build to not happen, but you end up in an interactive shell for that element | 07:56 |
| tristan | Yes, abderrahim[m] indeed has a patch which will fix the horrible hacks which make this happen | 07:56 |
| tristan | I.e. it's still supposed to happen, but it was implemented in such a way that makes your skin crawl, with access violations all over the place | 07:56 |
| tristan | douglaswinship, So here is my suspicion: That is broken when the failed element is the ultimate target | 07:57 |
| tristan | i.e. *perhaps*, the expected UX only kicks in when the failed element is a dependency, not the target | 07:57 |
| abderrahim[m] | douglaswinship: please give it a spin https://gitlab.com/BuildStream/buildstream/-/merge_requests/1970 I think it should fix your issue | 07:58 |
| abderrahim[m] | Unless you're running in non interactive mode | 07:59 |
| abderrahim[m] | Which is #1335 | 07:59 |
| abderrahim[m] | tristan: I originally set out to fix this issue, and the horrible hacks got in my way | 08:03 |
| abderrahim[m] | So I got rid of them | 08:04 |
| douglaswinship | Will do, but I think I'm still lost on what you mean by a failed element. In normal day to day operations, if I put a bad line of code into a bst file (say.. in the build-commands), then the element won't build, and I get the list of menu options. Is that all you mean by 'failed element'? | 08:08 |
| douglaswinship | because that gives me a different result. I get the interactive list. | 08:09 |
| tristan | abderrahim[m], :) | 08:12 |
| benschubert | douglaswinship: if your build fails, and is pushed to the cache, BuildStream will not retry it. Since in theory everything is repeatable and you would get the same result anyways. There are two things: | 08:12 |
| benschubert | 1) We want to allow forcing the retry, since there are cases where an operation could fail for a random reason (Like fetching from the internet might fail more than 3 times), that's abderrahim[m] MR | 08:12 |
| benschubert | 2) If you don't have a remote cache and delete your element locally, you can then rebuild it easily | 08:12 |
| tristan | douglaswinship, I'll just add that the reasoning behind caching the failed build is that it could happen remotely | 08:14 |
| tristan | douglaswinship, So when your build fails on some CI service like gitlab or whatever, you should be able to debug the build locally without having to build anything, inspect the logs, get the `bst shell` experience of the state of the failed build | 08:14 |
| tristan | benschubert, Is there a benefit to having functions not on classes with cython ? | 08:15 |
| benschubert | tristan: not AFAIK | 08:16 |
| tristan | So if _expand_var() https://gitlab.com/BuildStream/buildstream/-/blob/master/src/buildstream/_variables.pyx#L275 were a Variables method, that in itself would not cause a performance hit | 08:17 |
| douglaswinship | Oh... wow. I never noticed that before. I guess I never tried rebuilding the exact same element twice, before. I suppose I always tweaked something before I tried again. | 08:17 |
| tristan | Ok, was just curious why things were layed out that way, passing around the `content` dict when it's readily available on the Variables instance | 08:17 |
| benschubert | I don't *think* so, but it might be subtly different, though I don't expect a real different | 08:18 |
| douglaswinship | In that case, I guess the thing I pointed out was just normal behaviour. I just need to dig in and research the build failure | 08:18 |
| tristan | douglaswinship, no, it's the issue I said, and abderrahim[m] confirmed | 08:18 |
| benschubert | tristan: the reason is: it was like that before I converted it to Cython, and I did a 'straight' port to avoid too many problems. We can for sure refactor and make it better | 08:18 |
| tristan | douglaswinship, if the failed element was not the target (and one of the dependencies), then the imperative of building the target would have caused you to get a failed build UX (without actually rebuilding) | 08:19 |
| tristan | benschubert, thanks for confirming :) | 08:19 |
| tristan | I was worried about changing that | 08:19 |
| tristan | I'm wondering if it's worth doing the whole _resolve() thing at instantiation also | 08:21 |
| tristan | If we're going to resolve/expand variables lazily instead of all in __init__(), we could be equally lazy about populating PARSE_CACHE | 08:22 |
| benschubert | that's also true, but would we need it? | 08:22 |
| tristan | I don't know if it would hurt mostly | 08:22 |
| tristan | Maybe an iteration over the initial dict is overall cheaper than multiple hash lookups | 08:23 |
| benschubert | that's what I would think | 08:23 |
| benschubert | since in theory we would access everything | 08:24 |
| benschubert | (since I don't expect users to have hundreds of unused variables) | 08:24 |
| tristan | Well | 08:29 |
| tristan | I do | 08:29 |
| tristan | benschubert, The Variables instance is one per element, I would guess that each element uses a small percentage of the amount of variables in it's context | 08:30 |
| benschubert | ah good point | 08:30 |
| tristan | That said, if we do want to perform validation on unused variables, we're going to traverse them anyway | 08:31 |
| benschubert | Yep | 08:31 |
| tristan | And of course, there is a lot of reuse in the PARSE_CACHE | 08:31 |
| benschubert | If we ensure variables are resolved, there is no way we can't check all of them | 08:31 |
| tristan | still hash lookups, but still a lot of reuse | 08:31 |
| *** santi has joined #buildstream | 08:43 | |
| WSalmon | I notice that people are still using the old ML list, are we not encouraging people still using the old one to use the new one? tristan Frazer | 08:49 |
| tristan | WSalmon, I'm going to cut it off but I was hoping for traffic to die off first, it's in the works | 09:00 |
| tristan | Better not to shut it down in mid-conversation | 09:00 |
| WSalmon | i meant more a gental nug than a shut off | 09:03 |
| tristan | Feel free to nug if you wish | 09:12 |
| tristan | I announced that there would be a shutdown, the plan is to have a custom bounce message which indicates the new list | 09:12 |
| tristan | it should have happened by early this week but I'll let it slide into the weekend | 09:13 |
| douglaswinship | Can't remember if I've raised this before, but does anyone know how to get around the problem that cmake can't find gRPCConfig.cmake, when installing buildbox-common? | 09:17 |
| douglaswinship | I've been using a docker container for buildstream 1.93.4, to get around worrying about things like this. | 09:17 |
| douglaswinship | Now i'm trying to go through the actual installation process | 09:18 |
| douglaswinship | When I run "cmake -DBUILD_TESTING=OFF -Bbuild", i get the complaints about gRPCConfig.cmake not being found. | 09:19 |
| douglaswinship | https://hastebin.com/alawaxikex.rb | 09:19 |
| coldtom | douglaswinship, my first guess would be missing some sort of grpc-dev package from your distribution | 09:21 |
| WSalmon | douglaswinship, why not look at something like https://gitlab.com/celduin/crash/docker-images/-/blob/master/dockerfiles/dockerfile.buildstream2 | 09:21 |
| douglaswinship | WSalmon: if you mean the list of commands that's contained there, then I think i've run all of those already. (Or at least the updated versions from https://buildstream.build/buildbox_install.html). | 09:23 |
| douglaswinship | I'll inspect thoroughly though to see if I've missed anything | 09:23 |
| douglaswinship | coldtom: I wouldn't be surprised. But I don't know what i'd be missing. | 09:24 |
| WSalmon | also if you need a custom docker image for testing you could just tweak and build that docker image.. | 09:24 |
| WSalmon | anyone can clone and build that locally | 09:25 |
| douglaswinship | I shall definitely look into that | 09:26 |
| *** tomaz has joined #buildstream | 09:28 | |
| tristan | Hmmm, we don't even have provenance in unresolved variables ?? | 09:40 |
| tristan | Odd | 09:41 |
| tristan | I was sure we had that | 09:41 |
| Frazer | WSalmon tristan I don't mind moving to a new ML. But in the CONTRIBUTING docs, it will need updating to whatever the new one is, and the docs seem to have broken :ref: links https://gitlab.com/BuildStream/buildstream/-/blob/master/CONTRIBUTING.rst | 09:41 |
| tristan | Yes, it's on the TODO list, as well as wiki, bla, bla, bla... all the stuff I have to do earlier this week which is still pending | 09:41 |
| tristan | Part of the announcement I made last week points out that in the (short as possible, already delayed) interim one week period, I would have to go and update all the references to the list | 09:42 |
| tristan | Ok we do have provenance, not everywhere though | 09:44 |
| tristan | Some errors fail to provide provenance | 09:44 |
| tristan | _check_variables() does it though | 09:44 |
| *** tristan has quit IRC | 09:55 | |
| *** phildawson has joined #buildstream | 10:05 | |
| *** tristan has joined #buildstream | 11:38 | |
| *** ChanServ sets mode: +o tristan | 11:38 | |
| tristan | Mhmmm, no yield/generators in cython | 12:38 |
| *** tristan has quit IRC | 12:52 | |
| benschubert | tristan: no :) And we should generally not use those in performance-critical paths, the change of context is quite costly | 13:10 |
| *** Frazer has quit IRC | 13:18 | |
| *** cphang has quit IRC | 13:18 | |
| *** Frazer has joined #buildstream | 13:19 | |
| douglaswinship | what's the bst1 version of "bst artifact delete"? | 14:47 |
| WSalmon | im not sure it was ever back ported | 14:48 |
| douglaswinship | Damn | 14:48 |
| coldtom | manually deleting the reference from the cas | 14:48 |
| douglaswinship | won't it then try to fetch from the remote though? | 14:48 |
| douglaswinship | I'm trying to write some updates for plugins | 14:49 |
| douglaswinship | if I don't change the key, it won't rebuild | 14:49 |
| douglaswinship | I guess i just answered my own question. I can change the key while testing. Then change it back. | 14:49 |
| coldtom | ah, whenever i've done plugin dev i keep incrementing the format version to get rebuilds until i'm done | 14:49 |
| douglaswinship | yeah, I think that was the advice you gave me, way back when I was first working on release_url_manifest | 14:50 |
| douglaswinship | I better undertand why it works, now :) | 14:50 |
| douglaswinship | *worked | 14:50 |
| douglaswinship | because the format version was included as part of the unique cache key | 14:50 |
| douglaswinship | Come to think of it... I should be incrementing the version number anyway. | 14:54 |
| WSalmon | how do i ask bst not to pull? --fetchers 0 seems not to effect pulling? | 14:59 |
| WSalmon | how do i ask bst not to pull? `--fetchers 0` seems not to effect pulling? | 15:00 |
| coldtom | WSalmon, for master use `-r blah` | 15:05 |
| coldtom | where "blah" is anything that won't resolve as a url | 15:05 |
| douglaswinship | in the bst 1 plugin API: get_public_data seems to return a dict, rather than a node. | 15:36 |
| douglaswinship | I can't call "node_subst_vars" on a dict element. | 15:36 |
| douglaswinship | Does variable substitution happen automatically, before I get the dict element? | 15:36 |
| douglaswinship | Or do I have to take care of substitutions beforehand? | 15:38 |
| coldtom | douglaswinship: https://gitlab.com/BuildStream/bst-external/-/blob/master/bst_external/elements/collect_integration.py#L92 is an example of doing something similar to this | 15:42 |
| douglaswinship | thanks | 15:42 |
| coldtom | you can treat the dict as a node of yaml, and you can call node_subst_member for members of the dict | 15:42 |
| juergbi | in bst master variables are expanded implicitly beforehand | 15:43 |
| douglaswinship | thanks | 15:48 |
| *** tristan has joined #buildstream | 16:00 | |
| *** ChanServ sets mode: +o tristan | 16:00 | |
| *** santi has quit IRC | 17:46 | |
| *** xjuan_ has joined #buildstream | 17:57 | |
| *** tomaz has quit IRC | 18:59 | |
| *** phildawson has quit IRC | 23:00 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!