| *** aminbegood has joined #buildstream | 02:02 | |
| *** aminbegood has quit IRC | 02:22 | |
| *** narispo has quit IRC | 04:27 | |
| *** aminbegood has joined #buildstream | 05:12 | |
| *** persia has quit IRC | 06:42 | |
| *** persia has joined #buildstream | 06:44 | |
| *** jude has joined #buildstream | 07:29 | |
| *** hasebastian has joined #buildstream | 07:30 | |
| *** tpreston has joined #buildstream | 07:42 | |
| *** benschubert has joined #buildstream | 07:51 | |
| *** aminbegood has quit IRC | 07:58 | |
| *** Stella8252 has joined #buildstream | 08:25 | |
| *** Stella8252 has quit IRC | 08:26 | |
| *** ikerperez has quit IRC | 08:27 | |
| *** hasebastian has quit IRC | 08:28 | |
| *** narispo has joined #buildstream | 08:40 | |
| *** santi has joined #buildstream | 08:51 | |
| *** tpollard has joined #buildstream | 08:52 | |
| *** tristan has joined #buildstream | 09:03 | |
| *** phildawson has joined #buildstream | 09:06 | |
| *** ChanServ sets mode: +o tristan | 09:09 | |
| tristan | is it known that we're getting spurious errors from artifactservice.py::test_update_artifact() ? | 09:38 |
|---|---|---|
| tristan | https://gitlab.com/BuildStream/buildstream/-/jobs/558536470 | 09:38 |
| tristan | benschubert, Do we have some convenience function to load a string which is supposed to be an element, and does some basic validation on that ? | 09:44 |
| tristan | Or, should we ? | 09:44 |
| tristan | hmmm | 09:44 |
| benschubert | tristan: what kind of validation? AFAIK we can just yaml.load_data it then try loading it as an element? Probably not the best though. Or are you thinking about something else? | 09:45 |
| tristan | benschubert, well, we have something for ensuring project paths and that they exist, when we have strings which are expected to be paths in the project | 09:47 |
| tristan | I thought something like that but which takes the element path into consideration | 09:47 |
| tristan | I'm not sure it makes sense to have that though honestly | 09:48 |
| benschubert | If we need it, probably, but not sure right now where there is a need | 09:48 |
| tristan | maybe not, for elements which need configuration of specified elements, we're only really allowed to know about dependencies, and for that I believe elements like script and x86image will have to validate on `self.search()` | 09:49 |
| tristan | I was thinking of plugin origins specifying a junction, but in that case it's also fine to just trust the string and complain that "the specified junction element '../@ - @/;;' doesn't exist" (or we failed to load it or smth) | 09:51 |
| benschubert | tristan: for plugin origins with junctions, we'd try to load the junction first right? So the validation would happen then | 09:54 |
| juergbi | tristan: test_update_artifact: hm, that's an INTERNALERROR, i.e., before a test is actually running. pytest/xdist bug? | 09:55 |
| tristan | benschubert, right, I'm seeing that basically we would call the current loader's .get_loader() method for the specified junction, and delegate validation to that | 09:58 |
| * tristan is noticing that .get_loader() still needs to be made public on the Loader() object (it's currently _get_loader()) | 09:59 | |
| benschubert | tristan: that's what I think is simplest, as we make sure that all our junction loading code goes through the same place :) | 09:59 |
| tristan | juergbi, I guess yeah | 10:00 |
| tristan | wish that would magically get fixed | 10:00 |
| tristan | Also looking at Loader() public API, I'm seeing get_state_for_child_job_pickling(), this is confusing... is this part of Angelos's win32 support ? | 10:02 |
| tristan | The confusing part about that, is I don't really understand why we would ever send a "Loader" from one thread/process to another | 10:02 |
| juergbi | tristan: yes, child job pickling is from Angelos's win32 effort. the code path should be exercised in the 'spawn' CI job we have (running on Linux as win32 port is incomplete) | 10:21 |
| *** devcurmudgeon has quit IRC | 10:21 | |
| *** cphang has quit IRC | 10:21 | |
| juergbi | sounds odd to me as well OTOH, however, I haven't taken a closer look | 10:21 |
| juergbi | maybe just some small bits of data are needed from the loader object | 10:22 |
| tristan | Yeah, I feel like it would be more worth while finding out what those are and eliminating them, rather than supporting pickled loaders | 10:24 |
| tristan | (or loaded pickles) | 10:24 |
| * tristan tries to stay on track with what he's doing, but is getting distracted by some warts heh | 10:24 | |
| tristan | might be time for some minor refactors soon | 10:24 |
| tpollard | There should be a catchall issue for the win32 work | 10:26 |
| tpollard | can probably link back to why it's needed through that | 10:26 |
| tristan | Strange, it appears from pluginfactory.py that local plugins don't support the default .yaml files | 10:34 |
| tristan | That's more in line with the code I'm writing so I can ensure there is a test for that | 10:35 |
| tristan | tpollard, Yeah I recall Angelos was verbose and we should be able to track the reason for that pretty easily | 10:36 |
| *** cphang has joined #buildstream | 10:37 | |
| *** hasebastian has joined #buildstream | 10:55 | |
| *** narispo has quit IRC | 11:03 | |
| *** narispo has joined #buildstream | 11:03 | |
| *** Aubree2429 has joined #buildstream | 11:11 | |
| tristan | It also looks like no more than one plugin loaded from a given pip origin can have it's defaults loaded | 11:12 |
| *** Aubree2429 has quit IRC | 11:12 | |
| * tristan expands the scope of his pluginfactory refactor before moving on to junctions | 11:13 | |
| tristan | needs untangling | 11:14 |
| *** tristan has quit IRC | 11:20 | |
| *** phoenix has joined #buildstream | 11:31 | |
| *** tristan has joined #buildstream | 11:51 | |
| *** phoenix has quit IRC | 12:17 | |
| *** flatmush has quit IRC | 12:26 | |
| *** Adeline4916 has joined #buildstream | 12:41 | |
| *** Adeline4916 has quit IRC | 12:42 | |
| *** aminbegood has joined #buildstream | 12:49 | |
| *** aminbegood has quit IRC | 12:54 | |
| WSalmon | juergbi, now that benschubert's variable fix has landed can we rebase juerg/dynamic-plan there isnt a gui button while its wip, are you ok with me pushing the rebase to it? it was a clean rebase | 13:24 |
| juergbi | WSalmon: yes, go ahead | 13:37 |
| juergbi | should still add a test before we can merge it | 13:37 |
| WSalmon | +1 for tests | 13:42 |
| *** Melanie6395 has joined #buildstream | 14:31 | |
| *** Melanie6395 has quit IRC | 14:33 | |
| *** phoenix has joined #buildstream | 14:50 | |
| *** phildawson has quit IRC | 15:22 | |
| *** phildawson has joined #buildstream | 15:22 | |
| *** jude has quit IRC | 16:26 | |
| tpreston | Is it possible to affect the "variables" mapping in a stack element? | 16:32 |
| tpreston | Is it possible to affect the "variables" mapping in one of a stack element's dependencies? | 16:32 |
| tpreston | In particular, I'm interested in changing the value of `variables: rootfs-size: 2G` in an element which is in another project (included via junction) | 16:34 |
| tpreston | I believe I can do this with includes, but its a bit hacky | 16:34 |
| tpreston | (include the element first, then overwrite the value) | 16:34 |
| juergbi | tpreston: you can't affect variables of dependencies as each element exists only once in a pipeline (unless in the special case where the same project is used multiple times via junctions) | 16:35 |
| tpreston | Ok that makes sense | 16:37 |
| juergbi | if you want multiple elements in a single pipeline that differ only slightly, includes are probably the right approach indeed | 16:37 |
| juergbi | and in some cases junctions (e.g., for bootstrapping stages or multi-arch builds) | 16:38 |
| tpreston | juergbi: what if you have an element that likely needs changing, eg linux kernel (config). What is the canonical approach for doing this in two projects? | 16:44 |
| tpreston | For example: gnome-build-meta "consumes" freedesktop-sdk. Can gnome-build-meta consume and extend the linux kernel in freedesktop-sdk? | 16:45 |
| tpreston | Or do we have to re-implement the kernel element in every project? | 16:45 |
| juergbi | that's quite a different question as you typically don't want to build multiple different kernels in a single pipeline | 16:45 |
| juergbi | (if you do, junctions probably make sense for this, but they don't solve the sharing of the common part) | 16:46 |
| juergbi | linux kernel config is tricky as it needs to meet the requirements of hardware (platform as well as peripherals) as well as userspace and project/product policy | 16:48 |
| juergbi | for good reuse you probably want a modular configuration where you separate e.g. hardware-specific config fragments from other aspects | 16:49 |
| benschubert | tpreston: what I usually do is I have a second source on my junction, which applies a quilt patch. It's not great as that voids most of the keys in the junction's project, so no cache hit, but it allows me greater control of my dependency | 16:50 |
| juergbi | for a kernel with different config you anyway need a rebuild but userspace components shouldn't build-depend on the kernel, so that aspect should not be an issue | 16:51 |
| tpreston | juergbi: sure but how do I split the module kernel config across two projects? | 16:51 |
| tpreston | For example, the project I'm working on collects together all the bits for a raspberry pi | 16:52 |
| tpreston | It builds a certain kernel, with a certain config | 16:52 |
| tpreston | And my other project - which consumes the raspberry pi project - needs some extra kernel config | 16:52 |
| tpreston | how do I use the "rasbperry pi kernel element" but add in my own kernel config | 16:53 |
| juergbi | theoretically, one option would be to define an element for each kernel config fragment | 16:53 |
| tpreston | Right, so the | 16:54 |
| juergbi | these would be simple import elements that place the config in a particular location | 16:54 |
| tpreston | whoops | 16:54 |
| juergbi | and then you depend on those config elements that you want | 16:54 |
| juergbi | and the common part of the kernel element itself (build/install) would either be defined in an include, or maybe it would even make sense to create an element plugin for the linux kernel build system | 16:55 |
| tpreston | Sure an element plugin would make sense. I want to build for a few different boards to get a feel for specifics of that | 16:55 |
| tpreston | So, using this example, the "raspberry pi support project". Now it only provides a config fragment - to be consumed by the kernel element in my "main app" project | 16:56 |
| tpreston | But what if the kernel source is somewhere weird | 16:56 |
| tpreston | how do I "provide" the source URL | 16:57 |
| tpreston | another include?: | 16:57 |
| juergbi | yes | 16:58 |
| tpreston | ok | 16:58 |
| tpreston | that's useful, I'll go away and experiment with some of those ideas | 16:58 |
| tpreston | thank you! | 16:58 |
| juergbi | tpreston: kernel source is often not straight forward, of course. hardware and userspace might even have conflicting constraints on kernel version/patches | 16:59 |
| juergbi | or may need manual reconciliation | 16:59 |
| tpreston | juergbi: sure, and ultimately if something really complex needs to happy then the "main app" project needs to have full control over how everything is built | 16:59 |
| juergbi | yep | 16:59 |
| tpreston | Our goal atm is really just to provide a grab-bag "BSP" project, to make life easy for someone who just wants to put X on a raspberry pi | 17:00 |
| juergbi | sure, that would definitely be great as starting point | 17:01 |
| tpreston | benschubert: thanks also, but I think we're trying to avoid tracking patches! | 17:01 |
| benschubert | that makes a lot of sense :) no worries! | 17:02 |
| juergbi | tpreston, benschubert: I'm wondering whether it could make sense to generalize/extend the `origin: junction` approach for element sources as well | 17:02 |
| juergbi | to get 'local' files from a junctioned project | 17:03 |
| juergbi | but there might be better ways to approach that | 17:03 |
| benschubert | wouldn't an import element work from the base project? | 17:03 |
| tpreston | juergbi: I'm not sure what you mean? | 17:03 |
| tpreston | Like, accessing local files via a "self" junction? | 17:04 |
| juergbi | benschubert: the issue is that you can't put an element into the source list | 17:04 |
| benschubert | right | 17:04 |
| juergbi | tpreston: no, adding e.g. a kernel config fragment from a bsp junction to the source list in the kernel element | 17:05 |
| benschubert | I'm not sure I like the idea but if there's a need, it woul dmake sense unless we have a nicer solution | 17:05 |
| juergbi | benschubert: it is related to the question of whether to support specifying the destination path for dependencies | 17:05 |
| juergbi | from some point of view, a source is also just a dependency | 17:05 |
| juergbi | they have different constraints. and sometimes it would be useful to be able to use one as the other | 17:06 |
| juergbi | (just a thought related to the above discussion. don't have a concrete proposal at this time) | 17:07 |
| tpreston | benschubert, juergbi: do you mind if I copy this discussion somewhere? | 17:18 |
| tpreston | (issue tracker, public) | 17:18 |
| benschubert | tpreston: it's archived and public anyways, no problem with that for me :) | 17:18 |
| tpreston | thanks! | 17:18 |
| benschubert | juergbi: good point, I sometimes would wish to be able to use an element as a source too (imagine building deb files. You could have sources -> package -> install in the pipeline) | 17:19 |
| benschubert | juergbi: would that be something we should consider? | 17:19 |
| juergbi | tpreston: sure | 17:21 |
| philn | i'd need to add a "ruby gem" in my project, that doesn't supported yet? | 17:22 |
| philn | https://paste.debian.net/1147666/ | 17:23 |
| juergbi | benschubert: it's not in our bst 2.0 plan. we probably don't want any major core changes related to this. however, it wouldn't hurt to give it some thought whether bst could become a bit more flexible in this regard | 17:24 |
| benschubert | juergbi: right, that might be a good thing to open for discussion on the ML | 17:25 |
| benschubert | I can start that in a few days | 17:25 |
| benschubert | if you think it might be useful | 17:25 |
| pointswaves | philn: the issue is how the tar source expects your tar to have a folder inside its self and to extract that | 17:25 |
| pointswaves | you need to tell it not to worry and just extract everything in the tar | 17:26 |
| philn | could i just download the file and not extract it? | 17:26 |
| philn | i suspect `gem install` expects a .gem file | 17:27 |
| philn | ah maybe with the `remote` source kind | 17:28 |
| pointswaves | philn: if you change https://docs.buildstream.build/1.4.3/sources/tar.html base_dir:'*' to base_dir:'' it will just extract the tar | 17:28 |
| pointswaves | oh sorry right you want the gem as is | 17:28 |
| pointswaves | yep remote may be a better bet | 17:28 |
| philn | yay that works, now need to fix the gem install call | 17:29 |
| pointswaves | philn i would expect that you need to tell it to use a prefix | 17:30 |
| pointswaves | so it installs in the install folder not the base of the sandbox | 17:30 |
| philn | pointswaves: ERROR: While executing gem ... (Gem::FilePermissionError) | 17:30 |
| philn | You don't have write permissions for the /usr/lib/x86_64-linux-gnu/ruby/gems/2.6.0 directory. | 17:30 |
| pointswaves | yep because `/` is read only | 17:31 |
| juergbi | benschubert: I'm not sure whether it makes sense to open this potential can of worms as general discussion on the ML. I would probably only post if we have a concrete proposal. or if you want to ask for suggestions for a concrete use case | 17:33 |
| philn | pointswaves: --install-dir %{install-root} got me further, need to fix some compile error now :) thanks! | 17:33 |
| pointswaves | philn: no problem | 17:33 |
| pointswaves | philn: i would gues you need some more build dependencies? | 17:34 |
| philn | pointswaves: i'm not sure yet, https://paste.debian.net/1147668/ | 17:35 |
| pointswaves | also projects like FD get you to install in something like %{install-root}/%{prefix} as lots of things like you to install in /usr/bin etc | 17:35 |
| philn | ah, yeah right, i'll need to double-check that | 17:36 |
| pointswaves | yep that looks a lot like a missing dependency | 17:37 |
| pointswaves | but im not promising | 17:37 |
| *** tpreston has quit IRC | 17:38 | |
| philn | pointswaves: i was using an old version of them gem, it installed now after updating to 1.8.5 | 17:38 |
| pointswaves | Brill | 17:39 |
| *** santi has quit IRC | 17:40 | |
| *** santi has joined #buildstream | 17:42 | |
| philn | pointswaves: seems to work! thanks again! https://paste.debian.net/1147670/ | 17:42 |
| pointswaves | Great, I have never used ruby in buildstream so let use know how it's going we can have a plugin to reduce the boiler plate if needed | 17:44 |
| philn | first time ruby user here too :) | 17:44 |
| *** santi has quit IRC | 17:44 | |
| philn | https://paste.debian.net/1147671/ is my final version | 17:44 |
| benschubert | juergbi: makes sense I might postpone | 17:46 |
| philn | pointswaves: ah, now packaging the highline gem, seems like it depends on more gems... i'd need to download them as sources, similarly to the cargo/crate plugin | 17:52 |
| philn | gem install highline-2.0.3.gem --install-dir /buildstream-install//usr | 17:52 |
| philn | WARNING: Unable to pull data from 'https://rubygems.org/': no such name (https://rubygems.org/specs.4.8.gz) | 17:52 |
| pointswaves | Phil yep, that way you have trust worthy cache keys | 18:02 |
| pointswaves | philn even | 18:02 |
| pointswaves | No network in the sandbox | 18:02 |
| pointswaves | For building | 18:02 |
| philn | https://help.rubygems.org/kb/rubygems/installing-gems-with-no-network | 18:03 |
| philn | i'll have a look at writing a plugin for this | 18:05 |
| pointswaves | Looks like if you know all the deps then you could just install them one at a time and dep on the bst files but a plugin would be a much snazzier way to do it | 18:07 |
| *** tpollard has quit IRC | 18:15 | |
| *** hasebastian has quit IRC | 18:26 | |
| *** xjuan has joined #buildstream | 18:56 | |
| *** xjuan has quit IRC | 19:36 | |
| *** xjuan has joined #buildstream | 19:37 | |
| *** philn has quit IRC | 19:49 | |
| *** philn has joined #buildstream | 19:49 | |
| *** benschubert has quit IRC | 19:58 | |
| *** phoenix has quit IRC | 20:32 | |
| *** Luna7407 has joined #buildstream | 20:43 | |
| *** Luna7407 has quit IRC | 20:45 | |
| *** xjuan has quit IRC | 21:50 | |
| *** Kennedy8735 has joined #buildstream | 22:20 | |
| *** Kennedy8735 has quit IRC | 22:20 | |
| *** Aurora6901 has joined #buildstream | 22:26 | |
| *** Aurora6901 has quit IRC | 22:27 | |
| *** narispo has quit IRC | 22:45 | |
| *** narispo has joined #buildstream | 22:51 | |
| *** Nora7531 has joined #buildstream | 23:39 | |
| *** Nora7531 has quit IRC | 23:41 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!