| *** rdale has joined #buildstream | 00:08 | |
| *** rdale_ct has quit IRC | 00:11 | |
| *** rdale_ct has joined #buildstream | 00:37 | |
| *** rdale has quit IRC | 00:38 | |
| *** rdale has joined #buildstream | 00:45 | |
| *** rdale_ct has quit IRC | 00:46 | |
| *** alatiera has quit IRC | 01:09 | |
| gitlab-br-bot | tristanvb merged MR !1102 (tristan/insufficient-storage-error->master: Tristan/insufficient storage error) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1102 | 01:59 |
|---|---|---|
| *** rdale_ct has joined #buildstream | 03:11 | |
| *** rdale has quit IRC | 03:13 | |
| *** rdale_ct has quit IRC | 03:15 | |
| *** rdale has joined #buildstream | 03:16 | |
| *** phil has quit IRC | 03:32 | |
| *** tpollard has quit IRC | 03:36 | |
| *** nimish has joined #buildstream | 04:20 | |
| *** rdale has quit IRC | 04:27 | |
| *** rdale has joined #buildstream | 04:28 | |
| *** nimish has quit IRC | 04:30 | |
| *** nimish has joined #buildstream | 04:31 | |
| gitlab-br-bot | tristanvb opened MR !1103 (tristan/test-element-states->master: Reduce number of calls to `bst show` in tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1103 | 05:07 |
| *** rdale_ct has joined #buildstream | 05:14 | |
| *** rdale has quit IRC | 05:17 | |
| gitlab-br-bot | tristanvb opened MR !1104 (tristan/track-test-reduce->master: test_track_error_cannot_write_file() fixup) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1104 | 05:27 |
| *** rdale has joined #buildstream | 05:55 | |
| *** rdale_ct has quit IRC | 05:57 | |
| *** rdale_ct has joined #buildstream | 05:58 | |
| *** rdale has quit IRC | 05:59 | |
| gitlab-br-bot | tristanvb merged MR !1103 (tristan/test-element-states->master: Reduce number of calls to `bst show` in tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1103 | 06:13 |
| *** nimish has quit IRC | 06:40 | |
| *** nimish has joined #buildstream | 06:42 | |
| *** nimish has quit IRC | 06:52 | |
| *** nimish has joined #buildstream | 06:52 | |
| *** jesica has joined #buildstream | 06:59 | |
| gitlab-br-bot | tristanvb merged MR !1104 (tristan/track-test-reduce->master: test_track_error_cannot_write_file() fixup) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1104 | 07:00 |
| *** jesica has quit IRC | 07:18 | |
| *** nimish has quit IRC | 07:22 | |
| *** nimish has joined #buildstream | 07:22 | |
| *** nimish has quit IRC | 07:37 | |
| *** nimish has joined #buildstream | 07:38 | |
| *** nimish has quit IRC | 07:42 | |
| *** nimish has joined #buildstream | 07:43 | |
| *** tpollard has joined #buildstream | 08:08 | |
| *** nimish has quit IRC | 08:13 | |
| *** nimish has joined #buildstream | 08:13 | |
| *** nimish has quit IRC | 08:18 | |
| *** nimish has joined #buildstream | 08:19 | |
| *** cs-shadow has quit IRC | 08:28 | |
| *** nimish has quit IRC | 08:39 | |
| *** nimish has joined #buildstream | 08:39 | |
| *** bochecha has joined #buildstream | 08:49 | |
| *** toscalix has joined #buildstream | 09:01 | |
| *** nimish has quit IRC | 09:04 | |
| *** nimish has joined #buildstream | 09:05 | |
| *** nimish has joined #buildstream | 09:05 | |
| *** nimish has quit IRC | 09:20 | |
| *** nimish has joined #buildstream | 09:20 | |
| *** nimish has quit IRC | 09:25 | |
| *** nimish has joined #buildstream | 09:26 | |
| *** nimish has quit IRC | 09:41 | |
| *** nimish has joined #buildstream | 09:41 | |
| *** phildawson has joined #buildstream | 09:45 | |
| *** nimish has quit IRC | 09:46 | |
| *** nimish has joined #buildstream | 09:46 | |
| *** raoul has joined #buildstream | 09:49 | |
| *** nimish has quit IRC | 09:51 | |
| *** nimish has joined #buildstream | 09:52 | |
| gitlab-br-bot | raoul.hidalgocharman opened (was WIP) MR !1100 (raoul/870-root-cache-dir->master: root cache directory) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1100 | 09:58 |
| WSalmon | for other pices of code in the buildstream eco system that want to use bst as part of thier own CI (in my case bstgen and maybe a logging tool) is there a standard docker image that i can use that has bst installed?, I gues there are a few issues with this, eg, the standard images used by bst's own ci just have the dependancies not bst its self. al | 10:08 |
| WSalmon | so which version would i want to use.. is 1.2 too old, is tracking master too much of a moving target for CI that should only fail if the patch is bad not if some dependancy has changed.. ummm, i might take this to the ML but thought i would bring it hear first | 10:08 |
| *** jonathanmaw has joined #buildstream | 10:18 | |
| *** lachlan has joined #buildstream | 10:37 | |
| *** nimish has quit IRC | 10:37 | |
| *** nimish has joined #buildstream | 10:37 | |
| *** nimish has joined #buildstream | 10:38 | |
| valentind | jjardon, do you know if there is a way to log into the droplets created by gitlab-ci-runner? | 10:54 |
| benschubert | WSalmon: we don't have one yet :/ It's in an issue in buildstream-docker-images | 10:54 |
| valentind | We are running out of disk on buildstream builds. And I would like to know why. | 10:54 |
| jjardon | WSalmon: buildstream-fedora image is what was used in freedesktop-sdk until we created our own | 10:56 |
| tpollard | juergbi: what is the reason to always ignore junctioned elements in !925? | 10:56 |
| gitlab-br-bot | MR !925: Add support for default targets https://gitlab.com/BuildStream/buildstream/merge_requests/925 | 10:56 |
| jjardon | https://hub.docker.com/r/buildstream/buildstream-fedora | 10:57 |
| jjardon | WSalmon: 1.2.x is the only officially release version, and It's what is being used in freedesktop-sdk and GNOME | 10:58 |
| juergbi | tpollard: junctions themselves (not to be mixed up with elements that are specified via a junction) refer to another project and cannot be built/pushed/pulled (there is never an artifact for a junction itself) | 10:58 |
| juergbi | tpollard: normally when you attempt bst build/push/pull with a junction as target you get an error message. however, a junction may end up in the list of default targets because it's also a .bst file. and this works, e.g., for bst show/fetch/track | 10:59 |
| *** lachlan has quit IRC | 10:59 | |
| tpollard | ahh, that makes sense | 10:59 |
| juergbi | if we don't ignore them for bst build/push/pull, we'd get an error message, but that wouldn't be useful | 11:00 |
| WSalmon | oh cool, i was just reading though the ticket benschubert described and found buildstream/buildstream-fedora which seems like the closest thing we have, i gues in the future we would have buildstream/buildstream:dev that would be ruffly what i am after :D thanks for all the info guys, jjardon, benschubert et al. | 11:00 |
| tpollard | I was misinterpreting the comments | 11:00 |
| *** lachlan has joined #buildstream | 11:00 | |
| tpollard | juergbi: thanks for the clarification | 11:01 |
| juergbi | tpollard: feel free to suggest clearer comments. unfortunately, we can't filter out junctions before loading, so had to modify stream a bit more than I wanted | 11:01 |
| *** nimish has quit IRC | 11:03 | |
| *** nimish has joined #buildstream | 11:03 | |
| valentind | jjardon, found. From the bastion it is "docker-machine ssh <nameofdroplet>" | 11:06 |
| jjardon | valentind: sorry, I missed you message; good to know, I didn't know about that command | 11:07 |
| valentind | A build of buildstream now takes 14GB on the builders | 11:08 |
| valentind | 8G of integration cache. 6G of temporary files for tests | 11:09 |
| valentind | test_autotools_build2 takes 1.4Gb | 11:09 |
| juergbi | 8G of integration cache. this should only be sources, right? or do we cache any artifacts across CI runs (maybe accidentally)? | 11:10 |
| tpollard | the integration tests share by default share the artifact cache too afaik | 11:11 |
| valentind | juergbi, 406M of sources | 11:11 |
| valentind | 7.5Gb of artifacts | 11:11 |
| juergbi | tpollard: across CI runs or do you just mean across tests in a single CI job? | 11:11 |
| juergbi | valentind: I think that's wrong | 11:12 |
| juergbi | I mean, we shouldn't cache artifacts | 11:12 |
| *** solid_black has joined #buildstream | 11:12 | |
| tpollard | across a job | 11:12 |
| valentind | I am not sure if it keeps it across runs. | 11:12 |
| raoul | sources are kept across runs, artifacts should be deleted after the test suite | 11:13 |
| raoul | it's in the integration_cache fixture | 11:14 |
| valentind | It is cleaned up at the beginning of the next job. It shows what is removed at the beginning of the build log on gitlab. | 11:14 |
| valentind | Yes, but one run of the test takes 14GB | 11:14 |
| valentind | Our builders are 25Gb. | 11:15 |
| valentind | It is not enough. | 11:15 |
| raoul | Is that just artifacts for integration tests? | 11:21 |
| valentind | YEs | 11:21 |
| raoul | We probably want a cleverer way of only keeping ones we want to reuse then | 11:22 |
| raoul | With any other ones being deleted after the test | 11:22 |
| valentind | jjardon, do you know where to get the list of droplet size identifiers without having to use the http api? | 11:23 |
| *** alatiera has joined #buildstream | 11:24 | |
| *** alatiera has quit IRC | 11:26 | |
| jjardon | You can check the API Doc's but I normally go to the DO UI, create a droplet I want and then I check the name automatically generated, wich contain the droplet size slug | 11:27 |
| *** nimish has quit IRC | 11:28 | |
| *** nimish has joined #buildstream | 11:28 | |
| valentind | jjardon, The current size is c-2 which I think is the CPU optimized. | 11:29 |
| valentind | Do you know why we need CPU optimized? | 11:29 |
| valentind | I would go for a standard with same cpu and memory with much more disk, it is half the price. | 11:30 |
| jjardon | valentind: yeah, that's CPU optimized | 11:32 |
| jjardon | Sure go for it | 11:33 |
| *** alatiera has joined #buildstream | 11:33 | |
| valentind | jjardon, I am not getting CPU optimized. | 11:33 |
| valentind | It was CPU optimized. | 11:33 |
| valentind | I just wonder what we gained from it. | 11:34 |
| *** alatiera has quit IRC | 11:34 | |
| *** alatiera has joined #buildstream | 11:35 | |
| gitlab-br-bot | tpollard approved MR !925 (issue-638-validate-all-files->master: Add support for default targets) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/925 | 11:36 |
| jjardon | valentind: CPUs are mean to be much faster; but that was chosen by Tiago so maybe there are another reasons | 11:37 |
| valentind | OK, I will not use CPU optimized. And we will see. | 11:37 |
| valentind | We just need space now. | 11:37 |
| *** cs-shadow has joined #buildstream | 11:38 | |
| *** toscalix has quit IRC | 11:43 | |
| *** lachlan has quit IRC | 11:49 | |
| *** nimish has quit IRC | 11:53 | |
| *** nimish has joined #buildstream | 11:54 | |
| phildawson | For some tests, I'm attempting to patch a class attribute of a plugin, but I don't understand the behaviour I'm seeing. I've put together a simple example here: https://paste.gnome.org/pnp7axx4g. Is anyone able to explain to me what's happening? | 11:55 |
| *** lachlan has joined #buildstream | 11:59 | |
| jennis | phildawson, because no object of that class has been instantiated? | 12:01 |
| jennis | You're not asserting that an object of that class has that attribute? | 12:01 |
| phildawson | jennis, That's deliberate, I'm attaching the attribute to the class object, not an instance. | 12:02 |
| *** lachlan has quit IRC | 12:02 | |
| jennis | ahh, fair enough | 12:02 |
| phildawson | at least that's what I think I'm doing :P | 12:03 |
| *** bochecha has quit IRC | 12:09 | |
| *** nimish has quit IRC | 12:14 | |
| *** nimish has joined #buildstream | 12:14 | |
| *** lachlan has joined #buildstream | 12:16 | |
| *** nimish_ has joined #buildstream | 12:20 | |
| *** lachlan has quit IRC | 12:22 | |
| *** nimish has quit IRC | 12:22 | |
| *** nimish_ is now known as nimish | 12:22 | |
| *** lachlan has joined #buildstream | 12:25 | |
| *** lachlan has quit IRC | 12:28 | |
| *** lachlan has joined #buildstream | 12:36 | |
| *** raoul has quit IRC | 12:39 | |
| *** lachlan has quit IRC | 12:40 | |
| *** nimish has quit IRC | 12:50 | |
| *** nimish has joined #buildstream | 12:50 | |
| *** lachlan has joined #buildstream | 13:01 | |
| *** nimish has quit IRC | 13:05 | |
| *** nimish has joined #buildstream | 13:05 | |
| *** raoul has joined #buildstream | 13:08 | |
| juergbi | tpollard: thanks for the review. I forgot to add a section to the project configuration documentation. maybe you could take a quick read whether this is explained clearly enough: https://gitlab.com/BuildStream/buildstream/commit/d1b6b5cb1059cafee10b9af82834cd7c80e5b8f7?merge_request_iid=925 | 13:13 |
| *** lachlan has quit IRC | 13:16 | |
| *** sambishop has joined #buildstream | 13:22 | |
| tpollard | juergbi: np! | 13:28 |
| *** nimish has quit IRC | 13:30 | |
| *** nimish has joined #buildstream | 13:31 | |
| gitlab-br-bot | juergbi merged MR !1042 (jonathan/test-missing-workspace-guessing->master: Add tests to cover reinstated support for guessing targets) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1042 | 13:36 |
| *** nimish has quit IRC | 13:51 | |
| *** nimish has joined #buildstream | 13:52 | |
| *** nimish has quit IRC | 14:01 | |
| *** nimish has joined #buildstream | 14:02 | |
| phildawson | I'm trying and failing to register a local plugin for a small test project (as per https://docs.buildstream.build/format_project.html#local-plugins). Can anyone point out my mistake in https://paste.gnome.org/porejcha0 ? It's undoubtedly something simple, but I can't for the life of me see it. | 14:05 |
| *** alatiera has quit IRC | 14:07 | |
| *** alatiera has joined #buildstream | 14:10 | |
| *** lachlan has joined #buildstream | 14:21 | |
| phildawson | jonathanmaw, any chance you could help me out with the above? | 14:32 |
| * jonathanmaw has a look | 14:33 | |
| phildawson | Thanks :) | 14:33 |
| *** nimish has quit IRC | 14:42 | |
| *** nimish has joined #buildstream | 14:42 | |
| jonathanmaw | phildawson: that looks like normal config to me | 14:44 |
| cs-shadow | phildawson: you have listed it as a source | 14:45 |
| cs-shadow | i think you need it declared as an element | 14:45 |
| jonathanmaw | aha | 14:45 |
| cs-shadow | in project.conf | 14:45 |
| * jonathanmaw was about to ask what deprecated.bst looked like | 14:45 | |
| phildawson | ah /o\ thanks both :) | 14:46 |
| jonathanmaw | though tbh I was barking up the wrong tree and wondering whether calling `bst` from somewhere other than the root of the project had caused it | 14:46 |
| jonathanmaw | since I couldn't see anywhere that the plugin search path was normalised to the root of the project | 14:47 |
| *** coldtom has quit IRC | 14:58 | |
| *** coldtom has joined #buildstream | 15:00 | |
| *** nimish has quit IRC | 15:01 | |
| gitlab-br-bot | juergbi merged MR !925 (issue-638-validate-all-files->master: Add support for default targets) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/925 | 15:10 |
| *** kapil___ has joined #buildstream | 15:11 | |
| *** nimish has joined #buildstream | 16:08 | |
| *** nimish has quit IRC | 16:09 | |
| *** lachlan has quit IRC | 16:26 | |
| *** lachlan has joined #buildstream | 16:37 | |
| *** solid_black has quit IRC | 16:48 | |
| *** tristan has joined #buildstream | 16:51 | |
| *** alatiera has quit IRC | 16:56 | |
| *** ChanServ sets mode: +o tristan | 16:59 | |
| *** alatiera has joined #buildstream | 17:04 | |
| tristan | benschubert, jjardon... in my opinion #869 and #733 are the same issue... would you agree ? | 17:04 |
| gitlab-br-bot | Issue #869: Local cache size calculation is wrong https://gitlab.com/BuildStream/buildstream/issues/869 | 17:04 |
| gitlab-br-bot | Issue #733: quota config parameter should be the maximum amount of cache allowed, not the minimum https://gitlab.com/BuildStream/buildstream/issues/733 | 17:04 |
| benschubert | tristan: they definitely overlap yes. We can close mine in favor of the other | 17:08 |
| *** brlogger has joined #buildstream | 17:12 | |
| tristan | benschubert, that's what I was hoping indeed | 17:13 |
| tristan | benschubert, do you have an idea for a fix ? | 17:14 |
| tristan | benschubert, even if BuildStream should behave well when hitting the wall in out of disk scenarios, I don't believe that running until the disk is full is the right answer | 17:15 |
| *** brlogger has joined #buildstream | 17:17 | |
| benschubert | tristan: I'm not sure I have a definitive answer there. However, something I think sensible would be: | 17:19 |
| benschubert | 1) Define a maximum size usable by BuildStream. BuildStream should not go over this size even if there is free space. | 17:20 |
| tristan | Like the current quota | 17:20 |
| benschubert | Yes, but it should not reserve it so it should not be a min space | 17:21 |
| tristan | Ok | 17:21 |
| benschubert | 2) Define a "reserved space", tweakable, that BuildStream substracts to it's maximum usable space (or the current free space, whichever is smaller), and do not use if for things other than cleaning things up. Similar to the (by default) 5% of inodes/disk space ext4 reserves for root on disks | 17:22 |
| tristan | currently the algorithm works with a hardcoded "headroom" | 17:22 |
| tristan | Maybe that is the only hard limit for bailing out | 17:22 |
| tristan | And should be user configurable ? | 17:22 |
| benschubert | I think so, I personally don't care sacryfing 50Go of space if that means my build never fails :) | 17:23 |
| benschubert | And we should also have better heuristics on this (maybe start with a small headroom and show the amount of space that would have been needed in addition when failing?) | 17:23 |
| benschubert | I'm not sure if all of this makes sense :) | 17:23 |
| *** sambishop has quit IRC | 17:30 | |
| tristan | benschubert, Keeping a statistic on how big a build directory is for a given user/project might make for a better automatically derived headroom yes | 17:37 |
| tristan | but then, if it becomes big because of a rarely built element... it is annoying to halt the build because you only have 20GB of remaining space | 17:38 |
| tristan | benschubert, maybe we can't solve that one today then | 17:38 |
| tristan | I was hoping ;-) | 17:38 |
| benschubert | tristan: I wouldn't be to automatically decide it, but to give more information to the user so they might take a better decision :) | 17:38 |
| tristan | I'm ready to hammer out a fix for it but... what would it be ? | 17:39 |
| benschubert | Not reserving the space? :) That would fix our most immediate problem | 17:39 |
| tristan | benschubert, good point, but ... it should *still* have a default though right ? | 17:39 |
| tristan | benschubert, that could result in hitting lots of stack traces instead of cleanups, if we're not careful, though | 17:39 |
| benschubert | tristan: Sorry, when I said "not reserving space", I mean let's not check at the start that the space is indeed available and if it's too big, let's use infinity (does that make more sense?) | 17:40 |
| tristan | Hmmm | 17:41 |
| tristan | benschubert, maybe :) | 17:41 |
| tristan | Let me experiment with that I think | 17:42 |
| *** kapil___ has quit IRC | 17:49 | |
| gitlab-br-bot | tristanvb merged MR !1091 (tristan/cache-management->master: Cache management fixes) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1091 | 18:01 |
| gitlab-br-bot | tristanvb opened MR !1105 (tristan/cache-management-logging->master: Cache management logging enhancements) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1105 | 18:05 |
| tristan | benschubert, perhaps "whether the quota currently fits in the available disk space", is a warning that can be configurable as fatal ? | 18:07 |
| benschubert | tristan: We could do that, yup! | 18:09 |
| tristan | Not 100% sure, it's tempting but configuration of warning fatality is considered in cache keys :-S | 18:10 |
| *** tristan has quit IRC | 18:17 | |
| *** jonathanmaw has quit IRC | 18:21 | |
| *** dtf has joined #buildstream | 18:28 | |
| *** tristan has joined #buildstream | 18:31 | |
| *** nimish has joined #buildstream | 18:36 | |
| *** raoul has quit IRC | 18:37 | |
| *** nimish_ has joined #buildstream | 18:40 | |
| *** nimish_ has joined #buildstream | 18:41 | |
| *** nimish has quit IRC | 18:42 | |
| *** nimish_ is now known as nimish | 18:42 | |
| *** alatiera has quit IRC | 18:45 | |
| *** lachlan has quit IRC | 18:49 | |
| *** tristan has quit IRC | 18:51 | |
| *** nimish has quit IRC | 18:51 | |
| *** nimish has joined #buildstream | 18:51 | |
| *** tristan has joined #buildstream | 19:11 | |
| *** nimish has quit IRC | 19:11 | |
| *** ChanServ sets mode: +o tristan | 19:12 | |
| *** nimish has joined #buildstream | 19:12 | |
| tristan | jennis, juergbi ... around ? | 19:12 |
| juergbi | yes | 19:12 |
| tristan | I maybe didn't give it as much thought as you guys, but I think it would be sad to abandon loading elements from artifacts | 19:12 |
| tristan | like real ones | 19:13 |
| tristan | I mean, maybe you guys figured out some limitations meaning that would not be possible ? I am thinking of trying my hand at it | 19:14 |
| juergbi | I'm wondering whether it wouldn't be cleaner to have a separate Artifact class | 19:15 |
| tristan | for `bst artifact checkout --deps <run> <artifact-name>` for instance, that doesnt require *all* element capabilities | 19:15 |
| juergbi | and use that also when the real Elements are available | 19:15 |
| tristan | And also, it would require quite the same thing that is required to reconstruct a resolved project state from a loaded dependency graph of artifacts | 19:15 |
| juergbi | do we have a use case for reconstructing the full project state from artifacts? I can't remember | 19:16 |
| tristan | Which seems like a nice goal | 19:16 |
| tristan | Well, we have some overlapping cases for sure | 19:16 |
| tristan | It's been requested to be able to have the sources encoded into artifact metadata | 19:17 |
| tristan | mostly matter of full provenance tracking | 19:17 |
| tristan | "this output contains evidence of how it was built" | 19:17 |
| tristan | juergbi, I believe the thread about `bst artifact` subgroup did have some mention about ability to display the sources and refs which went into an artifact | 19:19 |
| juergbi | yes, having that information around certainly sounds sensible | 19:19 |
| tristan | I think that if we as a rule, encoded all metadata needed to reconstruct an element; that any of this automatically becomes easy | 19:20 |
| tristan | It demands however, that the user have the plugins for the given artifacts installed | 19:20 |
| tristan | that as far as I can see is the tricky thing | 19:20 |
| juergbi | well, do we want to allow actions that would actually require the plugins? | 19:21 |
| juergbi | e.g., rebuilding an element from an artifact without other information? | 19:22 |
| juergbi | not sure whether that's useful | 19:22 |
| tristan | A failed element ? | 19:22 |
| juergbi | I don't see why you would want to rebuild using the failed artifact as base instead of the regular project | 19:22 |
| tristan | Right now it is totally on the user to be able to reproduce the project state which caused a build to fail | 19:23 |
| tristan | If I give you an artifact and say it failed, without any other information, you can't do anything with it | 19:23 |
| tristan | Now loading the dependency graph is certainly more interesting in general, I concede | 19:23 |
| tristan | For instance, `bst artifact pull --deps run <artifact name>` is something good to have | 19:24 |
| juergbi | not sure whether there is a real use case for passing around failed artifact on its own | 19:24 |
| tristan | obviously | 19:24 |
| juergbi | yes, that can be useful | 19:24 |
| juergbi | (also implicitly, e.g., for bst artifact checkout) | 19:25 |
| juergbi | however, we should be able to get the log of an artifact without actually having all dependencies around | 19:25 |
| juergbi | i.e., the Element would need to support absent dependencies (and it should also probably not even attempt to load them if not needed, even if they are here/pullable) | 19:26 |
| tristan | I don't think the two are related | 19:26 |
| tristan | Oh you mean the element needs it's deps to even exist | 19:26 |
| tristan | There is a point | 19:26 |
| juergbi | it stretches the Element API a bit | 19:26 |
| juergbi | not impossible to handle, just adds a bit more of complexity | 19:26 |
| *** nimish has quit IRC | 19:27 | |
| tristan | but that is a bit horrible | 19:27 |
| tristan | you remove the guarantees about reading reverse dependencies, or have a ruleset for when what can be accessed | 19:27 |
| tristan | that's ugly | 19:27 |
| juergbi | so one possibility would be to have a separate Artifact class | 19:27 |
| juergbi | and use that for the pure single artifact operations | 19:27 |
| juergbi | and reconstruct Element (with dependencies) only when needed | 19:28 |
| tristan | Well | 19:28 |
| juergbi | and then use ArtifactJob in the scheduler for pull/push queue, e.g.,. always | 19:28 |
| tristan | Another possibility is to have an intermediate element interface in the core | 19:28 |
| tristan | and have Element derive from that | 19:28 |
| tristan | Handle things which cannot be done using ImplError handling | 19:29 |
| tristan | Also I don't think that we need to tie together dependencies with element instantiation | 19:29 |
| tristan | like, we should be able to construct dependency graphs out of these shallow virtual elements | 19:30 |
| tristan | if we have the backing artifact data and if we want to, without necessarily instantiating | 19:30 |
| tristan | then we can leave instantiating to a future pie in the sky universe where artifacts build themselves | 19:31 |
| tristan | juergbi, ? | 19:31 |
| juergbi | tristan: so some operations would then only require that base class while some would require a full Element derived instance? | 19:33 |
| juergbi | may not be trivial to separate on the API level especially without static typing | 19:34 |
| tristan | Right, so the build graph, regardless of how it is resolved, is either an interface or a common abstract class of Elements and these shallow virtual things, which raise a lot more ImplErrors | 19:34 |
| tristan | The alternative is to have dual code paths bubble outwards from the data model | 19:35 |
| tristan | seems | 19:35 |
| *** xjuan has joined #buildstream | 19:35 | |
| juergbi | so that wouldn't be far off from ArtifactElement except to clearly separate the Element API parts that are supported by that (moving that into the base class) and those that require a full Element | 19:37 |
| juergbi | but Python won't enforce this separation due to dynamic typing. maybe we should revisit type hinting | 19:38 |
| tristan | Right, it would only be caught by runtime assertions if you call something unsupported | 19:39 |
| tristan | you get an unhandled ImplError() if the core didnt expect it | 19:40 |
| tristan | (when calling an element API) | 19:40 |
| tristan | For `bst artifact pull --deps run`, it would be interesting to have this element dynamically fill in it's dependencies, and re-examine dynamic injection of elements in the scheduler | 19:43 |
| tristan | I also wonder now if all `bst` commands really require being launched in a project directory anymore | 19:43 |
| gitlab-br-bot | tristanvb merged MR !1105 (tristan/cache-management-logging->master: Cache management logging enhancements) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1105 | 19:44 |
| *** bochecha has joined #buildstream | 19:53 | |
| tristan | How come the CI is taking so long ? | 20:08 |
| tristan | So yesterday... the jobs were intermittently failing with out of disk conditions (hard to believe, the cache partition is 25G on these runners, but we still use an astounding 10G or more of cache) | 20:09 |
| tristan | Today they are taking over an hour | 20:10 |
| tristan | last week we were still at around 30min | 20:10 |
| tristan | In the .gitlab-ci.yml, nothing has changed to provoke this, so it would seem to be infra related | 20:11 |
| tristan | Well, the closest I can see is now we put the logs of the nightlies into an artifact | 20:12 |
| tristan | but those should not be related to the cache of a different job (and it's an artifact, not the cache) | 20:12 |
| *** rdale has joined #buildstream | 20:57 | |
| *** rdale_ct has quit IRC | 20:58 | |
| gitlab-br-bot | tristanvb opened MR !1106 (tristan/cache-quota-max-only->master: _artifactcache.py: Don't require the quota to be available on disk.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1106 | 21:10 |
| tristan | benschubert, how about that, !1106 ? | 21:10 |
| *** tristan has quit IRC | 21:19 | |
| *** tristan has joined #buildstream | 21:19 | |
| gitlab-br-bot | cs-shadow opened MR !1107 (chandan/toxic-man->master: Generate man pages using tox) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1107 | 21:23 |
| *** alatiera has joined #buildstream | 21:25 | |
| *** alatiera has quit IRC | 21:26 | |
| *** alatiera has joined #buildstream | 21:26 | |
| *** tristan has quit IRC | 21:34 | |
| *** alatiera has quit IRC | 21:40 | |
| *** alatiera has joined #buildstream | 21:41 | |
| gitlab-br-bot | cs-shadow opened (was WIP) MR !1107 (chandan/toxic-man->master: Generate man pages using tox & update them) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1107 | 21:58 |
| *** bochecha has quit IRC | 22:06 | |
| *** alatiera_ has joined #buildstream | 22:27 | |
| *** alatiera has quit IRC | 22:28 | |
| *** alatiera_ is now known as alatiera | 22:28 | |
| *** alatiera_ has joined #buildstream | 22:32 | |
| *** alatiera has quit IRC | 22:32 | |
| *** alatiera_ is now known as alatiera | 22:32 | |
| *** alatiera_ has joined #buildstream | 22:50 | |
| *** alatiera has quit IRC | 22:50 | |
| *** alatiera_ is now known as alatiera | 22:51 | |
| *** alatiera has quit IRC | 23:34 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!