| *** tristan has joined #buildstream | 05:05 | |
| *** givascu has joined #buildstream | 06:55 | |
| *** adds68 has joined #buildstream | 07:34 | |
| gitlab-br-bot | buildstream: issue #111 ("Unix platform cannot interact with artifact shares") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/111 | 07:41 |
|---|---|---|
| gitlab-br-bot | buildstream: issue #112 ("Artifact configuration is confusing and fragile, need canonical push/pull urls.") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/112 | 07:48 |
| *** anahuelamo has joined #buildstream | 08:17 | |
| *** anahuelamo has joined #buildstream | 08:18 | |
| *** givascu has quit IRC | 08:25 | |
| *** givascu has joined #buildstream | 08:25 | |
| *** jude has joined #buildstream | 08:40 | |
| *** jonathanmaw has joined #buildstream | 08:46 | |
| juergbi | tristan: regarding https://gitlab.com/BuildStream/buildstream/issues/5 i don't think it makes sense to solve this by requiring the user to specify shared sources in a separate place | 08:53 |
| juergbi | for one, this would still allow accidents to happen (unless we detect this and simply error out) and isn't user friendly | 08:54 |
| juergbi | also there might be cases where a user wants to use two different refs of the same source repo (e.g., gtk+ 2 and 3). and shared source doesn't help in that case | 08:54 |
| juergbi | i.e., i think we should fix this internally and thus, completely independent of the recursive pipeline approach | 08:55 |
| juergbi | regarding definition of subprojects for recursive pipelines. while it would be possible to put this all in project.conf, i think it would be better to put those in separate files | 09:02 |
| juergbi | to not let project.conf grow too big/complex, and the file-based approach is easier for things like bst track | 09:02 |
| juergbi | while it would be possible that a project would be referenced twice, this would also be possible when declared in project.conf, and the conceptual issue even exists for two independent projects that have .bst files for the same module | 09:03 |
| juergbi | also, i don't see any implementation issues dealing with it (elements in different subproject definitions would be treated as completely separate elements) | 09:03 |
| juergbi | does this make sense to you so far? | 09:03 |
| tristan | juergbi, yeah I raised issue 5 because I thought it might very well fit into this scenario, but dont let that be a blocker for you | 09:12 |
| * tristan reads the rest... | 09:12 | |
| tristan | ... so far... agree that I would prefer also the details of what Source/configuration options, live in a separate file... | 09:13 |
| tristan | Hopefully it would also be an Element | 09:14 |
| juergbi | that's the big question now | 09:14 |
| juergbi | if we treat it as Element, it would still be pretty special | 09:14 |
| juergbi | not all Element actions would make sense on it | 09:14 |
| *** ssam2 has joined #buildstream | 09:15 | |
| juergbi | i don't think treating it as a stack would be useful | 09:15 |
| tristan | So... sorry juggling conversation... one sec... | 09:15 |
| tristan | juergbi, so... I think that having the junction of one project depending on another being "special" is very, very acceptable; even expectable; for a core feature like this | 09:17 |
| juergbi | formulated like this, i agree | 09:17 |
| juergbi | the question for me is whether it would be confusing to have this in the same directory with the same file extension | 09:18 |
| tristan | I would go so far as to say; when depending on a "junction" (for conversation sake, and cause I like the name)... it would be totally acceptable to extend the core dependency declaration format for depending on such an element | 09:18 |
| juergbi | i'd be ok with it if you are | 09:18 |
| tristan | So for instance... you can have an element depend on a junction element.bst, withe type: build or run... and with element: foo.bst | 09:19 |
| tristan | meaning you depend on element foo.bst from that project with that configuration | 09:19 |
| juergbi | i was planning on either junction.bst:foo.bst or junction.bst/foo.bst | 09:19 |
| tristan | if that also makes sense with your plan, that sounds interesting | 09:19 |
| tristan | I see | 09:19 |
| juergbi | but conceptually i guess that's pretty close | 09:19 |
| tristan | Yeah... for the YAML format, I feel I prefer addressing things with an extension to the dictionary | 09:20 |
| juergbi | wouldn't matter much implementation wise, i suppose | 09:20 |
| juergbi | just different syntax | 09:21 |
| tristan | You will, however need a new display name for the elements which the project builds | 09:21 |
| tristan | i.e. for the logs and stuff | 09:21 |
| juergbi | i like the concise path-like syntax as we can use it everywhere as a single thing | 09:21 |
| juergbi | but i'm flexible on the syntax | 09:21 |
| tristan | Or, that can be solved by just saying "If there is more than one project... elements are displayed as 'project:element.bst' " | 09:21 |
| tristan | full element name vs short-hand, local element name | 09:22 |
| *** tlater has joined #buildstream | 09:22 | |
| juergbi | wouldn't it be nice for the display name to match the dependency name in the .bst files | 09:22 |
| juergbi | ? | 09:22 |
| tristan | It's unneeded in this approach I think, no ? | 09:22 |
| juergbi | either way works fine | 09:22 |
| juergbi | i generally like consistency with how things are represented | 09:23 |
| tristan | Or, we give people a rope to hang themselves with, and dont force a single junction:project association ? | 09:23 |
| juergbi | but dict extension works for me as well | 09:23 |
| juergbi | multiple projects in a single junction? i don't understand what you mean with that | 09:24 |
| tristan | in which case, if you can have multiple junctions to the same project at different versions / configurations (which *is* useful), then you need to distinguish using the junction element name | 09:24 |
| juergbi | ah, the other way round, ok | 09:24 |
| tristan | No, multiple junctions to the same project, depended on for different reasons | 09:24 |
| tristan | right | 09:24 |
| juergbi | this should conceptually be allowed but i don't think it should frequently be used/needed | 09:25 |
| juergbi | (except for the import case but that will be slightly different) | 09:25 |
| juergbi | *import-like case | 09:25 |
| tristan | Right | 09:26 |
| juergbi | ok, i'll assume dict extension for now but syntax is anyway a detail in the end implementation-wise | 09:27 |
| tristan | juergbi, if having multiple elements from the same project, but with differing cache keys; does not present huge obstacles to our core (separate element instances I suppose, should be fine)... then lets keep the rope people can hang themselves with existing, hiding somewhere out of sight | 09:27 |
| tristan | And if it causes trouble, we'll later need to add some strong error handling around that | 09:28 |
| tristan | but I dont see it as a huge concern for initially landing it | 09:28 |
| juergbi | yes, i don't foresee any real obstacles in supporting that | 09:28 |
| tristan | sounds like an all around nice way forward to me ! | 09:29 |
| juergbi | and i understand it right that you're ok with using a special 'kind' of .bst files that may not conform to regular Element? | 09:30 |
| juergbi | can hopefully provide a more complete description soon | 09:31 |
| tristan | juergbi, yes; i.e. it's a core element 'kind' (class)... derives from Element... and has sneaky special sauce | 09:34 |
| juergbi | ok, that might work well. thanks for the feedback | 09:34 |
| tristan | And in the format documentation about dependencies, it's documented that the additional attribute *only* applies to this special element | 09:35 |
| juergbi | right | 09:35 |
| *** semanticdesign has joined #buildstream | 09:50 | |
| tlater | jonathanmaw: The reason this doesn't work when installed with the editable flag is that it is then *not* installed as a zip file | 10:28 |
| tlater | Which means that setuptools won't have to extract the file, and just keeps it in the original moduke | 10:28 |
| tlater | *module | 10:28 |
| tlater | Apparently python does not handle setting sys.path to a path inside a module | 10:29 |
| tlater | And since pluginbase does just that I think this is causing our issue | 10:29 |
| tlater | To fix this, I think the only possible solution is to copy the file to a tempdir | 10:29 |
| tlater | This also means that plugins always have to be exactly one file | 10:29 |
| tlater | But I think they had to already... | 10:29 |
| * tlater attempts this change and sees if he's correct | 10:31 | |
| tlater | I can't believe I wrote a plugin system that works only exactly when python decides to zip its modules, as opposed to the opposite... | 10:32 |
| gitlab-br-bot | buildstream: issue #113 ("Distinguish between tracking and saving in `bst build --track`") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/113 | 10:33 |
| tristan | Any opinions on issue #113 ? | 10:33 |
| * tristan thinks this is easy to implement; and if we do - it should block 1.0 | 10:34 | |
| tristan | because it would otherwise be a semantic behavior change of `bst build --track` | 10:34 |
| ssam2 | i've always avoided `bst build --track` because i don't like the idea that building stuff can effectively update my source code | 10:34 |
| tristan | So that is a +1 I guess | 10:35 |
| ssam2 | pretty much | 10:35 |
| tristan | I'm thinking the typical GNOME case here | 10:35 |
| tristan | people are used to always building latest and desiring that | 10:35 |
| ssam2 | i find `bst track` a bit weird but also super useful | 10:35 |
| tristan | and git stash or such will get in the way | 10:35 |
| juergbi | could it make sense to use sources without ref for such cases? | 10:37 |
| tristan | yes, I think I intend to for GNOME modulesets of devel release, too | 10:38 |
| juergbi | (that would implicitly track but not save) | 10:38 |
| tristan | (and that already works fwiw) | 10:38 |
| tristan | I think that we want gnome-modulesets master to always refer to master branches in GNOME with no ref | 10:39 |
| tristan | And release branches to ideally only refer to release branches of GNOME modules (where possible) | 10:39 |
| tristan | And only have tags for tracked things | 10:39 |
| tristan | that is something to explore and needs more thought, but I think it's about right | 10:40 |
| juergbi | do we already implicitly track without save if there is no ref? | 10:40 |
| juergbi | or error out? | 10:40 |
| tristan | error out | 10:40 |
| tristan | That I want to keep | 10:40 |
| tristan | ... because we still focus on deterministic, predictable stuff ... it should be explicit | 10:41 |
| tristan | ltu, did you start looking at the wiki for monthly meeting minutes yet ? | 10:42 |
| tristan | ltu, also my proposed tentative first meeting date, some *cough* people might be interested in attending we might want to chase after and see if the date is suitable | 10:42 |
| tlater | jonathanmaw: That wasn't it either :( | 10:53 |
| jonathanmaw | :( | 10:53 |
| tlater | jonathanmaw: Have you tried running the test directly, outside of the test cases? When I do it seems to work - perhaps it's caused by specific CLI flags. | 10:58 |
| jonathanmaw | tlater: I get a different error, so that's hopeful :) | 10:59 |
| ltu | tristan, heh...my view is that the core folk are necessary, i hadn't thought of those interested parties as part of that | 10:59 |
| tlater | Oooh, what's the error? | 10:59 |
| ltu | i can ask of course but i suspect they will just want to read the minutes after the meeting | 10:59 |
| ltu | right so i'm creating a new page on the wiki now | 10:59 |
| jonathanmaw | tlater: error was https://pastebin.com/kyuhcZkW. making some changes to see if it persists. | 10:59 |
| jonathanmaw | I made some changes and it went back to __import__ not found :/ fingers crossed I can freely move between those two fail cases :P | 11:00 |
| jonathanmaw | I get the pasted error when I install the package with "--egg" | 11:01 |
| tristan | ltu, fair enough, I've sent private emails around | 11:05 |
| tristan | ltu, "participation in the project" can take on other forms than just writing code; I think it's fair to encourage stake holders to participate | 11:06 |
| * tlater has no idea what this issue is anymore | 11:26 | |
| tlater | It turns out that building multiple elements makes the issue go away | 11:27 |
| tlater | It only occurs when building exactly one element | 11:27 |
| tlater | Which has to be the external plugin one | 11:27 |
| tlater | It happens in the test case, because that one just so happens to have its dependencies cached already | 11:27 |
| tlater | jonathanmaw: Can you confirm that behavior? | 11:28 |
| jonathanmaw | tlater: hmm, I can wipe the cache, so it has to pull in base-configure again | 11:29 |
| tlater | jonathanmaw: You can also wipe dpkg-build-test's cache and then build dpkg-configure(?)-test | 11:29 |
| tlater | deploy, sorry | 11:30 |
| jonathanmaw | tlater: hrm, seems to be the same problem. I even went as far as to change the config to a completely separate cachedir | 11:49 |
| jonathanmaw | but I'm still getting 'NoneType' object is not subscriptable. | 11:50 |
| tlater | Not sure why that solves the issue for me then :/ | 11:50 |
| * tlater found an interesting function in pluginbase that might allow importing from python packages, maybe that solves some of this weirdness | 11:50 | |
| jonathanmaw | tlater: what's the function? I'd like to manually hack around to see if I can load it that way | 11:52 |
| tlater | It's http://pluginbase.pocoo.org/#pluginbase.PluginSource -> open_resource | 11:53 |
| * tristan wonders if jonathanmaw and tlater are in the same country... maybe taking a look at jonathanmaw's console and debugging a bit might yield interesting results | 11:54 | |
| jonathanmaw | tlater: yep, just across the hall from each other. tlater had a look earlier this morning. | 11:54 |
| tristan | :) | 11:54 |
| tlater | Unfortunately we're equally dumbfounded, and I have managed to reproduce almost exactly what he has. | 11:54 |
| * tristan is not jealous | 11:55 | |
| tristan | not about the country thing, the bug, heh | 11:55 |
| tristan | (although not jealous of that either heh) | 11:55 |
| tlater | ;P | 11:56 |
| *** givascu has quit IRC | 12:24 | |
| *** tristan has quit IRC | 12:34 | |
| *** tristan has joined #buildstream | 13:05 | |
| tlater | I can load files from plugins now, but can't load plugins from files. Something tells me this open_resource idea wasn't great. | 13:49 |
| tristan | yeah, anything pip is, 90% pain | 13:52 |
| * tristan usually throws in a provocative troll in #python on freenode to get people riled up and subtly extract the info he needs | 13:52 | |
| tlater | In this case the issue appears to be pluginbase, actually. | 13:53 |
| tlater | But I'm not sure, I still can't find the cause | 13:53 |
| tristan | remember that we _only_ care about pip as a means of *discovery* of plugins | 13:57 |
| tristan | which are python files in a directory | 13:57 |
| tristan | plugin base works fine for loading files from a directory. | 13:57 |
| tlater | Yup. We can manage to find python files in a directory, but loading them somehow gets rid of all builtin python modules. | 13:57 |
| tristan | tlater, note that we distribute our plugins as data files | 13:58 |
| tlater | Which causes all kinds of neat errors. | 13:58 |
| tristan | and we make damn sure there is a gap, with no intermediate __init__.py mucking things up | 13:58 |
| gitlab-br-bot | buildstream: issue #114 ("Give better errors on overlaps") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/114 | 14:10 |
| jonathanmaw | tlater: looking at the code, should I be providing a "plugins" field to find the dpkg_build element? I'm guessing not, because the outcome would be more likely to be unable to find the plugin, not it behaving weirdly. | 14:18 |
| tlater | jonathanmaw: A plugins field where? | 14:19 |
| jonathanmaw | tlater: in project conf | 14:20 |
| tlater | No, that's not required for these plugins. | 14:20 |
| jonathanmaw | project uses _extract_plugin_paths | 14:20 |
| tlater | The code relevant to these plugins is almost entirely in _plugincontext.py :) | 14:21 |
| tlater | IT'S GARBAGE COLLECTION | 14:22 |
| * tlater is sure of this one | 14:22 | |
| tlater | Oh damnit | 14:23 |
| tlater | pluginsources have a 'persistent' attribute that I didn't notice - apparently they remove any plugins they imported when they are garbage collected | 14:24 |
| tlater | This is fine as long as the plugins are being loaded, but since I create a new pluginsource with a different searchpath to load the external plugins everything breaks down once those functions end | 14:24 |
| tlater | This is why internal plugins work, and why external plugins break - and why it's entirely inconsistent, because sometimes garbage collection kicks in and sometimes it doesn't. | 14:25 |
| tristan | That seems strange, we should implicitly have a reference to the loaded namespaced plugin - or... | 14:25 |
| tristan | Maybe *I* regressed your thing... when I made a change for on-demand loading ? | 14:25 |
| tlater | tristan: We do, but not to the plugin source, which removes the object that still has a reference | 14:26 |
| tristan | Or rather not, cause I think I made that change in advance of your branch | 14:26 |
| * tlater already tested this, unless I'm having odd issues again it solves it :) | 14:26 | |
| tlater | jonathanmaw: I'll clean it up a bit and push, would you mind trying to confirm the fix? | 14:27 |
| jonathanmaw | tlater: I'd love to | 14:27 |
| tristan | we should have jonathanmaw test every new feature ;-) | 14:27 |
| tristan | hehe | 14:27 |
| * tristan ducks | 14:27 | |
| gitlab-br-bot | buildstream: Tristan Maat created branch external_plugin_errors | 14:30 |
| gitlab-br-bot | push on buildstream@external_plugin_errors (by Tristan Maat): 1 commit (last: _plugincontext.py: Fix third party plugin loading) https://gitlab.com/BuildStream/buildstream/commit/4fbbad19602717a1a39fbdbd94dc50c4a72cfcf6 | 14:31 |
| tlater | jonathanmaw: ^^^ :) | 14:31 |
| *** tristan has quit IRC | 14:32 | |
| *** tristan has joined #buildstream | 14:39 | |
| jonathanmaw | ᕕ( ᐛ )ᕗ things are happening! | 14:41 |
| jonathanmaw | fingers crossed! | 14:41 |
| jonathanmaw | tlater: seems to have solved that particular problem! | 14:48 |
| tlater | \o/ | 14:48 |
| tlater | But what do you mean by 'that particular'? | 14:49 |
| jonathanmaw | I'm currently seeing "Directory '/buildstream/install' was not found inside the sandbox, unable to collect artifact contents", but that looks like it'll be specific to the dpkg_build element in some way. | 14:49 |
| tlater | Hm, yeah, probably. I'm not seeing this with my old plugin repo, but it is pretty old, so things may have changed. | 14:50 |
| tristan | that doesnt sound related to loading plugins at all no | 14:51 |
| gitlab-br-bot | buildstream: merge request (external_plugin_errors->master: _plugincontext.py: Fix third party plugin loading) #107 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/107 | 14:55 |
| gitlab-br-bot | push on buildstream@non-sandbox-builds (by Tristan Maat): 20 commits (last: _frontend/widget.py: Report selected project options, if any.) https://gitlab.com/BuildStream/buildstream/commit/ce5a47973bb639c1f61cd65cce212bb0308f0134 | 15:11 |
| gitlab-br-bot | buildstream: merge request (non-sandbox-builds->master: Add %{script} format to `buildstream show`) #102 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/102 | 15:11 |
| gitlab-br-bot | push on buildstream@external_plugin_errors (by Tristan Maat): 1 commit (last: _plugincontext.py: Fix third party plugin loading) https://gitlab.com/BuildStream/buildstream/commit/85c1a8bcfab764d780b21069f656061a2b2faf28 | 15:16 |
| gitlab-br-bot | buildstream: merge request (external_plugin_errors->master: _plugincontext.py: Fix third party plugin loading) #107 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/107 | 15:16 |
| gitlab-br-bot | push on buildstream@81-non-empty-read-only-directories-not-handled-during-bst-build-and-others (by Tristan Maat): 20 commits (last: _frontend/main.py: Added -o/--option main CLI params) https://gitlab.com/BuildStream/buildstream/commit/9dbeffa291f3eb3a951ae785f7b42b211f708470 | 16:05 |
| gitlab-br-bot | buildstream: merge request (81-non-empty-read-only-directories-not-handled-during-bst-build-and-others->master: Ensure that artifact file permissions are set in the right order) #100 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/100 | 16:05 |
| gitlab-br-bot | push on buildstream@81-non-empty-read-only-directories-not-handled-during-bst-build-and-others (by Tristan Maat): 1 commit (last: Ensure that artifact file permissions are set in the right order) https://gitlab.com/BuildStream/buildstream/commit/59c14af228a3cf3e5155e932cff01ebca4a4a44b | 16:57 |
| gitlab-br-bot | buildstream: merge request (81-non-empty-read-only-directories-not-handled-during-bst-build-and-others->master: Ensure that artifact file permissions are set in the right order) #100 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/100 | 16:57 |
| *** jude has quit IRC | 17:06 | |
| *** jude has joined #buildstream | 17:08 | |
| jonathanmaw | tlater: it looks like it was packaging-related after all. | 17:12 |
| jonathanmaw | I think the defaults from the .yaml files aren't getting merged in, since I was able to get my build to succeed by copying the defaults yaml into the element. | 17:13 |
| tlater | The /buildstream/install issue? | 17:13 |
| jonathanmaw | tlater: that one | 17:13 |
| jonathanmaw | /buildstream/install was empty because it never ran any commands. | 17:13 |
| tlater | Hrm, I'll have a look tomorrow - need to hurry up right now :/ | 17:13 |
| jonathanmaw | yep | 17:13 |
| jonathanmaw | see you later, tlater | 17:13 |
| tlater | o/ | 17:13 |
| *** tlater has quit IRC | 17:15 | |
| *** jonathanmaw has quit IRC | 17:16 | |
| *** ssam2 has quit IRC | 17:20 | |
| *** semanticdesign has quit IRC | 20:37 | |
| *** semanticdesign has joined #buildstream | 20:37 | |
| *** semanticdesign has quit IRC | 21:23 | |
| *** semanticdesign has joined #buildstream | 21:23 | |
| *** semanticdesign has quit IRC | 22:48 | |
| *** semanticdesign has joined #buildstream | 22:48 | |
| *** semanticdesign has quit IRC | 23:51 | |
| *** semanticdesign has joined #buildstream | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!