| *** rdale has quit IRC | 01:35 | |
| *** toscalix has joined #buildstream | 05:07 | |
| *** tristan has quit IRC | 05:38 | |
| *** tristan has joined #buildstream | 06:00 | |
| *** tristan has quit IRC | 07:47 | |
| *** tristan has joined #buildstream | 08:04 | |
| gitlab-br-bot | BenjaminSchubert approved MR !1257 (shared/yaml-rework->master: YAML New World Order) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1257 | 09:22 |
|---|---|---|
| *** raoul has joined #buildstream | 09:31 | |
| *** rdale has joined #buildstream | 09:39 | |
| *** jonathanmaw has joined #buildstream | 09:51 | |
| jennis | juergbi, are you happy for me to assign !1257 to marge? | 09:53 |
| juergbi | jennis: yes, I think all of my comments have been addressed - and I hope I haven't missed anything substantial | 09:56 |
| jennis | Hopefully none of us have! :p | 09:56 |
| jennis | thanks | 09:56 |
| gitlab-br-bot | marge-bot123 closed issue #591 (Missing required key on project.conf causes wrong provenance file name on error message) on buildstream https://gitlab.com/BuildStream/buildstream/issues/591 | 10:04 |
| gitlab-br-bot | marge-bot123 merged MR !1257 (shared/yaml-rework->master: YAML New World Order) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1257 | 10:04 |
| *** phildawson_ has joined #buildstream | 10:08 | |
| jennis | \o/ | 10:12 |
| *** nimish2711 has joined #buildstream | 10:33 | |
| *** lachlan has joined #buildstream | 10:37 | |
| *** phil has joined #buildstream | 10:40 | |
| *** phildawson_ has quit IRC | 10:40 | |
| *** ChanServ sets mode: +o tristan | 10:42 | |
| tristan | new world YAML order, exciting | 10:42 |
| jennis | tristan, sure is, there are some benchmarks on the MR that shows the difference | 10:43 |
| tristan | jennis, I'm curious why the tests all change the regular dictionary accesses for `_yaml.node_get()` | 10:44 |
| tristan | Are regular dictionary accesses now invalid ? | 10:44 |
| jennis | Now nodes are namedtuples | 10:44 |
| jennis | Of (value=whateva, file_index=int, line=int, col=int) | 10:45 |
| jennis | So all loaded values have their provenance upon load | 10:45 |
| tristan | I mean, that would be an interesting way to force/ensure that plugins *cannot* access configuration node members without going through an API which will raise an appropriate LoadError() | 10:45 |
| tristan | So probably we want to make sure that the structure of "nodes" is not something that a plugin is allowed to know, right ? | 10:45 |
| jennis | That's the idea, we *must* be using _yaml functions now | 10:46 |
| tristan | right, well plugins will be going through self.node_get_member() etc, but it amounts to the same :) | 10:46 |
| tristan | jennis, nice :) | 10:46 |
| jennis | Yes, they go via Plugin.whateva() to _yaml functions | 10:46 |
| tristan | Is the yamlcache now gone completely ? or is it replaced by something internal ? | 10:47 |
| jennis | It didn't provide any benefit, so completely gone | 10:47 |
| tristan | Also nice | 10:48 |
| jennis | Even for 75k elements | 10:48 |
| tristan | I still see that loading a project the first time is slower than later times | 10:48 |
| tristan | And then I notice that when it is slower, there is network activity | 10:48 |
| tristan | I.e. we are not running a scheduler and messaging to the user properly when autofetching subprojects | 10:48 |
| tristan | completely orthogonal to your branch of course :) | 10:49 |
| tristan | just an observation I've been making last week | 10:49 |
| jennis | Ahh interesting | 10:49 |
| jennis | Also what's interesting, is how much more memory consumption there is when we 'show' elements with cached artifacts | 10:49 |
| jennis | It's significantly more] | 10:49 |
| tristan | It also means that like, if I want to force BuildStream to never access the network, say by setting my fetchers to 0, it will fetch *anyway* | 10:49 |
| tristan | Would be nice to streamline things more so that any activities go through the same policed codepaths | 10:50 |
| tristan | (but not a huge concern really) | 10:50 |
| jennis | So the junction is implicitly fetched in project loading? | 10:50 |
| tristan | Yes, that has always been the case | 10:51 |
| jennis | Makes sense | 10:51 |
| tristan | But it would be nice if say, the Stream object which causes the loader to operate, were to have a callback called when the loader says "Hey, I cant complete without fetching this thing !" | 10:51 |
| tristan | Then the stream could go through the normal codepath to do the fetch on behalf of the loader | 10:51 |
| tristan | Then I'd have nice messaging I guess is the biggest win... I'd see the status bar for a moment while the thing fetches and some corresponding log lines | 10:53 |
| tristan | (right now it just takes a long time in "Loading elements") | 10:53 |
| tristan | meh | 10:53 |
| jennis | Ok, so I'm doing some cleanup of bst-external's test suite. To run the tests, we've always been using a literal copy and paste of buildstream's testutils (at whatever commit...), now, in master buildstream.testutils exists as something we can import | 10:58 |
| jennis | However, bst-external is being tested with bst-1.2 | 10:59 |
| jennis | Does anyone have any ideas how I might be able to import this module *and* run the tests with bst-1.2? | 10:59 |
| tristan | jennis, at some point we're going to have to fork | 10:59 |
| tristan | As of last IRC team meeting, jjardon raised that distros have packaged bst-external, so it is becoming more urgent | 11:00 |
| tristan | I still have to make a writeup to our list on what changes we have to make so that the environments can coexist without stepping on eachothers toes | 11:01 |
| jennis | So a fork which is bst-external that is compatible with master, fair enough | 11:01 |
| tristan | jennis, but first step I think is going to be to fork bst-external and create bst-experimental (better name to not risk anyone packaging it) | 11:01 |
| jennis | but for the bst-external that is compatible with bst-1.2, is there anyway I can (probably hackily) import this module from master and then use bst-1.2's binary | 11:02 |
| jennis | I can't think of a way | 11:02 |
| jennis | Just wondered if anyone else had an idea | 11:02 |
| tristan | Not without being very messy I think | 11:02 |
| tristan | cant think of anything that wouldnt be a horrible hack, an option would be to backport the testutils but I don't see much point in that | 11:03 |
| jennis | It would literally just be for the tests | 11:03 |
| tristan | jennis, before I forget, have you documented this YAML new world order as an API break ? | 11:03 |
| jennis | It's slightly annoying that the majority of plugins in bst-external are untested too, maybe we should start mandating a test | 11:04 |
| tristan | jennis, I think currently we do that in NEWS, until a porting guide starts getting written | 11:04 |
| jennis | tristan, whoops, no. Will create an MR for this now | 11:04 |
| tristan | I.e. we need to remember that in BuildStream 2, plugins cannot access the config passed in Plugin.configure() in the same way, and now *must* go through the Plugin.node_get_*() family of functions | 11:04 |
| tristan | so we can have that in a porting guide | 11:05 |
| tristan | jennis, thanks ! | 11:05 |
| jennis | We did mention doing this... obviously lost track | 11:05 |
| tristan | yeah of course | 11:05 |
| gitlab-br-bot | jmacarthur approved MR !1235 (jennis/track_is_overworking->master: Write test to ensure that track does not needlessly rewrite) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1235 | 11:05 |
| jennis | Yeah, there are some plugins in bst-external now (e.g. flatpak-image) that will not 'just work' with the new world YAML order | 11:06 |
| jennis | That plugin does all sorts of crazy things with dicts | 11:06 |
| jjardon | should we create a bst-1.2 branch of bst-external? | 11:09 |
| jennis | jjardon, and have bst-external consume BuildStream master? | 11:15 |
| jennis | Thought we recently agreed to keep bst-external for bst-1.2? https://gitlab.com/BuildStream/bst-external/merge_requests/72#note_152343823 | 11:16 |
| jjardon | no, create a new repo and buildstream master will use that. like buildtream-plugins-good or something like that | 11:16 |
| jjardon | then we can remove everything from bst-external master | 11:16 |
| jjardon | to clearly state bst-external is for bst-1.2.x only | 11:17 |
| jennis | So, a bst-external 1.2 branch *and* a new repo for the same plugins, but working with master? | 11:17 |
| jennis | Where the bst-1.2 branch (of bst-external) is the default? | 11:18 |
| jjardon | yes | 11:18 |
| tristan | jjardon, if we create a new repo, I'm not sure we need to branch in bst-external | 11:22 |
| jjardon | we do not need, It's more for clarity | 11:23 |
| tristan | Sure, I'm not fussed either way - I think less branches is easier to follow, though | 11:23 |
| tristan | I think jonathanmaw maintains bst-external, I would leave that up to him if he still maintains bst-external | 11:24 |
| *** tristan has quit IRC | 12:06 | |
| *** tristan has joined #buildstream | 12:32 | |
| *** nimish2711 has quit IRC | 12:37 | |
| *** lachlan has quit IRC | 12:37 | |
| gitlab-br-bot | jennis opened MR !1264 (jennis/add_NEWS_entry->master: FOLLOW UP: Add NEWS entry for YAML New World Order) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1264 | 12:39 |
| jennis | tristan, I've added the NEWS entry ^^ | 12:39 |
| *** nimish2711 has joined #buildstream | 12:43 | |
| *** alatiera has joined #buildstream | 12:54 | |
| *** alatiera_ has joined #buildstream | 13:02 | |
| *** alatiera has quit IRC | 13:02 | |
| *** alatiera_ is now known as alatiera | 13:03 | |
| benschubert | I'm hitting problems with the local cache again. Starting bst build fails because some fetches can't stage source, as no space is left (expected). However, the cache cleanup never kicked in... Am I missing something? | 13:04 |
| tpollard | anyone seen a ci job stuck like this? https://gitlab.com/BuildStream/buildstream/-/jobs/185954070 | 13:04 |
| * tpollard kills it | 13:10 | |
| benschubert | tpollard: runners tend to get stuck sometimes :/ | 13:11 |
| jjardon | jonathanmaw: hi, would it be possible to have another bst-external release? Fixes there are needed to fix https://gitlab.com/BuildStream/buildstream/issues/976 | 13:23 |
| jonathanmaw | jjardon: okie doke | 13:24 |
| jjardon | thanks! | 13:25 |
| jennis | jjardon, have you tried building fdsdk since !1257 landed? | 13:25 |
| gitlab-br-bot | MR !1257: YAML New World Order https://gitlab.com/BuildStream/buildstream/merge_requests/1257 | 13:25 |
| jennis | I.e. at somepoint today, as this MR will require that some of the bst-external plugins (which I'm pretty sure are used in fdsdk) use the public API | 13:26 |
| jjardon | jennis: not myself | 13:26 |
| benschubert | jennis: fdsdk is already broken by !1254 | 13:27 |
| gitlab-br-bot | MR !1254: Rework Element.dependencies to be more efficient https://gitlab.com/BuildStream/buildstream/merge_requests/1254 | 13:27 |
| jennis | benschubert, yeah I just read that, but it's going to be more broken by !1257 due to the bst-external plugins | 13:28 |
| jjardon | jennis: benschubert if breaking changes are being pushed to master, how come the cache key is not begin invalidated? https://gitlab.com/BuildStream/buildstream/-/jobs/185517860 | 13:32 |
| jjardon | being* | 13:32 |
| benschubert | jjardon: no breaking changes have been made on the cache key | 13:32 |
| benschubert | jjardon: the only changes were reworking the yaml API, which is a breaking change | 13:33 |
| benschubert | The other one, responsible for the earlier fsdsk failure was due to bst-external using a private API in BuildStream, which it shouldn't have | 13:33 |
| jjardon | mmm, so how is possible the cached build is ok but the uncached one no? https://gitlab.com/BuildStream/buildstream/-/jobs/185517861 | 13:33 |
| jjardon | for the same pipeline: https://gitlab.com/BuildStream/buildstream/pipelines/53938494 | 13:34 |
| benschubert | jjardon: because the uncached build actually exercise the part of the code that is using the private API | 13:34 |
| benschubert | whereas the cached one doesn't since it's already cached | 13:34 |
| jjardon | cool, that will get fixed when jonathanmaw release a new bst-external tag | 13:35 |
| benschubert | exact | 13:35 |
| jennis | jonathanmaw, benschubert, I've addressed your comments: https://gitlab.com/BuildStream/bst-external/merge_requests/77 please let me know if you're happy with the MR | 13:36 |
| jonathanmaw | 👍 | 13:37 |
| jonathanmaw | new version pushed https://gitlab.com/BuildStream/bst-external/tree/0.12.0 | 13:51 |
| gitlab-br-bot | marge-bot123 merged MR !1235 (jennis/track_is_overworking->master: Write test to ensure that track does not needlessly rewrite) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1235 | 14:04 |
| gitlab-br-bot | jjardon opened MR !1265 (jjardon/overnigth_tests_fix->master: .gitlab-ci.yml: Use bst-external 0.12.0 for overnigth tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1265 | 14:13 |
| *** lachlan has joined #buildstream | 14:25 | |
| tpollard | CI seems very flaky today | 14:28 |
| *** lachlan has quit IRC | 14:35 | |
| raoul | tpollard, is there any reason the methods in `_artifact.py` for `get_files` and similar, return a virtual directory and a key? it seems like the key is ignored everywhere it's used | 14:37 |
| *** lachlan has joined #buildstream | 14:41 | |
| laurence | marge bot has (finally) been amended server side to now have CI time out set to 120 minutes | 14:43 |
| *** lachlan has quit IRC | 14:45 | |
| *** raoul has quit IRC | 14:48 | |
| *** sney has joined #buildstream | 14:59 | |
| *** Dragonkeeper has joined #buildstream | 14:59 | |
| *** kraft has joined #buildstream | 14:59 | |
| *** Bantalon has joined #buildstream | 14:59 | |
| *** pinaraf has joined #buildstream | 14:59 | |
| *** Guest69142 has joined #buildstream | 14:59 | |
| *** taaperotassu has joined #buildstream | 14:59 | |
| *** Guest69142 has quit IRC | 14:59 | |
| *** Dragonkeeper has quit IRC | 14:59 | |
| *** taaperotassu has quit IRC | 14:59 | |
| *** kraft has quit IRC | 14:59 | |
| *** sney has quit IRC | 14:59 | |
| *** pinaraf has quit IRC | 14:59 | |
| *** jadew has joined #buildstream | 15:00 | |
| *** Bantalon has quit IRC | 15:00 | |
| *** davidebeatrici has joined #buildstream | 15:00 | |
| *** Slade has joined #buildstream | 15:00 | |
| *** tinoco has joined #buildstream | 15:00 | |
| *** Supersonic has joined #buildstream | 15:00 | |
| *** SimonVT has joined #buildstream | 15:00 | |
| *** orderchaos has joined #buildstream | 15:00 | |
| *** jadew has quit IRC | 15:00 | |
| *** orderchaos has quit IRC | 15:00 | |
| *** davidebeatrici has quit IRC | 15:00 | |
| *** Slade has quit IRC | 15:00 | |
| *** Sargun has joined #buildstream | 15:00 | |
| *** Supersonic has quit IRC | 15:01 | |
| *** SimonVT has quit IRC | 15:01 | |
| *** nemish has joined #buildstream | 15:01 | |
| *** LuMint has joined #buildstream | 15:01 | |
| *** ntt has joined #buildstream | 15:01 | |
| *** nemish has quit IRC | 15:01 | |
| *** ntt has quit IRC | 15:01 | |
| *** LuMint has quit IRC | 15:02 | |
| *** KramerC has joined #buildstream | 15:02 | |
| *** FRidh has joined #buildstream | 15:02 | |
| *** KramerC has quit IRC | 15:02 | |
| *** FRidh has quit IRC | 15:02 | |
| *** raoul has joined #buildstream | 15:03 | |
| *** rektide_ has joined #buildstream | 15:03 | |
| *** tinoco has quit IRC | 15:03 | |
| tpollard | raoul: probably just to keep it inline from when the places where they were used were calling get_directory directly which was returning the tuple | 15:05 |
| *** lachlan has joined #buildstream | 15:06 | |
| tpollard | get_subdirectory definitely needs to return the used key, but if it's redundant in to the specific 'get' methods accessing it then probably no need to propagate it through | 15:08 |
| *** jackc has joined #buildstream | 15:10 | |
| raoul | do you mean get_directory? get_subdirectory is only used for the get_files and get_buildtree | 15:10 |
| tpollard | raoul: sorry was looking at my branch, I'm planning to use it on all the methods that want the meta subdir | 15:11 |
| raoul | tpollard, Ah I should probably have a look at what your followup issue is ... sounds like it's gonna clash with teh reworking to use the artifact protos | 15:13 |
| tpollard | raoul: it's not really an issue, more of a tidyup really | 15:13 |
| *** CryingCyclops has joined #buildstream | 15:14 | |
| raoul | Well regardless there'll probably be some clashing with the stuff I've made a start on | 15:15 |
| tpollard | oh for sure | 15:16 |
| *** jjacky11 has joined #buildstream | 15:18 | |
| *** jjacky11 has quit IRC | 15:18 | |
| *** bsaggy has joined #buildstream | 15:19 | |
| *** Ainsey has joined #buildstream | 15:19 | |
| *** lmartin92 has joined #buildstream | 15:20 | |
| *** ikkuranus has joined #buildstream | 15:20 | |
| tpollard | raoul: hoping to have a WIP that hits most of #955 for the end of tomorrow | 15:32 |
| gitlab-br-bot | Issue #955: Follow-up from "Artifact 'abstraction' class" https://gitlab.com/BuildStream/buildstream/issues/955 | 15:32 |
| raoul | cool cool, I'm just gonna push ahead at the moment and worry about the clash later :P | 15:33 |
| gitlab-br-bot | jjardon closed issue #976 (Regression: buildstream master cannot build fdsdk anymore) on buildstream https://gitlab.com/BuildStream/buildstream/issues/976 | 15:39 |
| gitlab-br-bot | jjardon merged MR !1265 (jjardon/overnigth_tests_fix->master: .gitlab-ci.yml: Use bst-external 0.12.0 for overnigth tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1265 | 15:39 |
| *** lachlan has quit IRC | 15:40 | |
| *** mart has joined #buildstream | 16:11 | |
| *** nimish2711 has quit IRC | 16:13 | |
| *** nimish2711 has joined #buildstream | 16:13 | |
| *** _StratoSwitch_ has joined #buildstream | 16:15 | |
| *** lachlan has joined #buildstream | 16:17 | |
| *** NostawRm has joined #buildstream | 16:18 | |
| *** cythrawll has joined #buildstream | 16:20 | |
| *** lujan has joined #buildstream | 16:21 | |
| *** mdoep has joined #buildstream | 16:21 | |
| *** Kadet has joined #buildstream | 16:23 | |
| gitlab-br-bot | raoul.hidalgocharman approved MR !1252 (tpollard/945->master: Add initial TestArtifact() abstraction class to testutils) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1252 | 16:30 |
| *** gokcennurlu has joined #buildstream | 16:32 | |
| *** natorious has joined #buildstream | 16:44 | |
| *** octorian has joined #buildstream | 16:46 | |
| *** xkonni has joined #buildstream | 16:48 | |
| *** alreece45 has joined #buildstream | 16:48 | |
| *** lachlan has quit IRC | 16:50 | |
| *** shbst has joined #buildstream | 16:53 | |
| *** shayr has joined #buildstream | 16:56 | |
| *** Henry1512 has joined #buildstream | 17:09 | |
| gitlab-br-bot | tristanvb approved MR !1264 (jennis/add_NEWS_entry->master: FOLLOW UP: Add NEWS entry for YAML New World Order) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1264 | 17:10 |
| jennis | ta tristan | 17:17 |
| *** lachlan has joined #buildstream | 17:22 | |
| *** dkp has joined #buildstream | 17:22 | |
| *** olus has joined #buildstream | 17:23 | |
| *** lachlan has quit IRC | 17:28 | |
| *** fishwithapipe has joined #buildstream | 17:33 | |
| *** lachlan has joined #buildstream | 17:38 | |
| *** toothe has joined #buildstream | 17:54 | |
| *** jonathanmaw has quit IRC | 18:01 | |
| *** lidel has joined #buildstream | 18:02 | |
| *** raoul has quit IRC | 18:07 | |
| tpollard | I've got this twice in a row https://gitlab.com/BuildStream/buildstream/-/jobs/186244112 | 18:09 |
| tpollard | 'python3 -m sphinx.apidoc --force --separate --module-first --no-headings --no-toc -o source /builds/BuildStream/buildstream/doc/../buildstream *_pb2*.py | 18:09 |
| tpollard | '/builds/BuildStream/buildstream/.tox/docs/bin/python3: No module named sphinx.apidoc' | 18:09 |
| *** HairAndBeardGuy has joined #buildstream | 18:17 | |
| *** SFaulken has joined #buildstream | 18:22 | |
| *** lachlan has quit IRC | 18:30 | |
| *** lyxia has joined #buildstream | 18:30 | |
| *** bbee has joined #buildstream | 18:31 | |
| *** mntmn has joined #buildstream | 18:43 | |
| *** macdice has joined #buildstream | 18:45 | |
| *** nBurn has joined #buildstream | 18:48 | |
| *** kapil___ has joined #buildstream | 18:48 | |
| *** Dave_R has joined #buildstream | 18:53 | |
| *** deadcyclo-cloud has joined #buildstream | 19:07 | |
| *** eth2 has joined #buildstream | 19:11 | |
| *** KnownUnown has joined #buildstream | 19:15 | |
| *** Naeblis has joined #buildstream | 19:20 | |
| *** Sammitch has joined #buildstream | 19:26 | |
| *** cinnamntig3r_ has joined #buildstream | 19:28 | |
| *** redskull has joined #buildstream | 19:39 | |
| *** tw has joined #buildstream | 19:42 | |
| *** Habbie has joined #buildstream | 19:50 | |
| *** galstrom_zzz has joined #buildstream | 19:55 | |
| *** sxe has joined #buildstream | 19:56 | |
| *** rdale has quit IRC | 19:57 | |
| *** insanitea has joined #buildstream | 19:57 | |
| *** keks has joined #buildstream | 20:02 | |
| *** Paul-S has joined #buildstream | 20:07 | |
| *** moymoy has joined #buildstream | 20:41 | |
| *** PatrickC_ has joined #buildstream | 20:49 | |
| *** Daiyousei has joined #buildstream | 20:51 | |
| *** RangerRick has joined #buildstream | 20:55 | |
| *** andyn has joined #buildstream | 21:15 | |
| *** pjz has joined #buildstream | 21:16 | |
| *** tatanka_br has joined #buildstream | 21:20 | |
| *** rivyn has joined #buildstream | 21:34 | |
| *** drygol has joined #buildstream | 21:46 | |
| *** HeathHayle has joined #buildstream | 22:06 | |
| *** Blazed has joined #buildstream | 22:15 | |
| *** Nekit1234007 has joined #buildstream | 22:19 | |
| *** gargantua has joined #buildstream | 22:21 | |
| *** ix007 has joined #buildstream | 22:44 | |
| *** alatiera has quit IRC | 22:54 | |
| *** alatiera has joined #buildstream | 22:55 | |
| *** alatiera has quit IRC | 22:57 | |
| *** lanceball has joined #buildstream | 23:06 | |
| *** TML has joined #buildstream | 23:27 | |
| *** zkxs has joined #buildstream | 23:38 | |
| *** kstar has joined #buildstream | 23:44 | |
| *** svillemot has joined #buildstream | 23:51 | |
| *** megaxlr has joined #buildstream | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!