| *** leopi has joined #buildstream | 04:25 | |
| *** tristan has joined #buildstream | 04:54 | |
| *** leopi has quit IRC | 05:31 | |
| *** leopi has joined #buildstream | 06:05 | |
| *** tristan has quit IRC | 06:54 | |
| * laurence wonders if anyone is interested in responding to the mail on Artifact cache authentication/metrics - https://mail.gnome.org/archives/buildstream-list/2018-August/msg00016.html | 07:21 | |
| * laurence also wonders why Edmund didn't post to the list himself, but that's a separate point | 07:21 | |
| paulsherwood | laurence: i think in previous attempts to mail the list, edmund didn't get the outcomes he hoped for | 07:25 |
|---|---|---|
| paulsherwood | not just this list :) | 07:25 |
| *** paulsherwood has quit IRC | 07:34 | |
| *** paulsherwood has joined #buildstream | 07:34 | |
| *** paulsherwood has quit IRC | 07:35 | |
| *** paulsherwood has joined #buildstream | 07:36 | |
| *** bochecha has joined #buildstream | 07:42 | |
| *** toscalix has joined #buildstream | 07:57 | |
| Nexus | Is it possible to run buildstream without fuse? | 08:29 |
| * Kinnison would have expected the fallback chroot sandbox to work without fuse (albeit with no hermetic seal guarantees) | 08:33 | |
| * Kinnison is new here though | 08:33 | |
| Kinnison | so could be wrong | 08:33 |
| * phildawson is curious why Nexus wants to | 08:34 | |
| *** jonathanmaw has joined #buildstream | 08:39 | |
| *** tristan has joined #buildstream | 08:39 | |
| juergbi | Nexus: the chroot backend still depends on SafeHardlinks FUSE, if I'm not mistaken | 08:41 |
| juergbi | so the answer is no for unmodified BuildStream | 08:41 |
| juergbi | SafeHardlinks could easily be disabled but it would risk corrupting the artifact cache | 08:42 |
| juergbi | (which could be avoided by replacing hardlinks with full copies but that would be slow) | 08:43 |
| Kinnison | Interesting | 08:43 |
| Kinnison | Couldn't just bind-mount read-only ? | 08:43 |
| Kinnison | Then again, that won't work on all platforms | 08:44 |
| Kinnison | Hrm | 08:44 |
| tristan | Kinnison, FUSE provides the safe hardlinks experience, ensuring that when we run integration commands, we do not corrupt the artifact cache | 08:44 |
| Kinnison | tristan: Aah, yeah, that'd be a problem otherwise, I see | 08:45 |
| Kinnison | Otherwise a full copy would be needed, and then presumably a full compare | 08:46 |
| juergbi | with modern filesystems there may be other options (e.g., reflinks) but there isn't a fast non-FUSE cross-platform option, afaik | 08:48 |
| * Kinnison nods | 08:48 | |
| * juergbi would really like a permanent immutable file flag in Linux (not like the already existing impractical immutable flag), but that wouldn't solve the cross-platform issue either | 08:51 | |
| Kinnison | heh | 08:52 |
| *** bochecha has quit IRC | 09:01 | |
| WSalmon | tiagogomes, are you happy with my implementation of your feed back to https://gitlab.com/BuildStream/buildstream/merge_requests/595 ? | 09:03 |
| *** flatmush has quit IRC | 09:04 | |
| *** flatmush has joined #buildstream | 09:05 | |
| *** tiagogomes_ has joined #buildstream | 09:10 | |
| *** bochecha has joined #buildstream | 09:15 | |
| CTtpollard | hi, I have a small project that I've been using for testing last week. I've updated my local buildstream since then now I get this error when trying to junction 'alpine-junction.bst [line 8 column 8]: Specified path '../autotools' first component must not be '..'' | 09:25 |
| CTtpollard | is that expected? | 09:25 |
| jmac | Yes, it's intentional - tiagogomes_ might be able to explain more | 09:29 |
| juergbi | CTtpollard: see https://gitlab.com/BuildStream/buildstream/issues/195 | 09:29 |
| CTtpollard | cheers | 09:32 |
| *** CTtpollard is now known as tpollard | 09:32 | |
| valentind | tristan, I have found a bug in the include feature and fixed it. https://gitlab.com/BuildStream/buildstream/issues/565 | 09:48 |
| *** jonathanmaw_ has joined #buildstream | 09:49 | |
| *** jonathanmaw has quit IRC | 09:50 | |
| tpollard | so if I'm reading the comments correctly it's now not possible to junction to other projects? | 10:36 |
| tpollard | (outside of the current projects directory) | 10:36 |
| juergbi | tpollard: not via local junction, but you can use e.g. a git junction | 10:43 |
| phildawson | If that's now the case, I think the junctions tutorial I wrote will be broken :/ | 10:52 |
| phildawson | though the test suite should have flagged that | 10:52 |
| * phildawson goes to check | 10:52 | |
| jennis | juergbi, jonathanmaw_, on a related note to changing the default to dependency type to build, is it intentional behaviour that if none of an element's runtime dependencies contain sh we can't shell into the element? | 10:53 |
| Nexus | Does anyone recognise this error? http://hastebin.com/ojewowubad.sql | 10:53 |
| Nexus | also, best url ever | 10:53 |
| Nexus | Oh Gee, wow you bad | 10:53 |
| juergbi | jennis: including extra elements for bst shell has come up in the path, but yes, this would be expected behavior | 10:54 |
| jennis | e.g. I built vim with dpkg a few days ago, and was unable to shell into the element, as the platform element (which contained sh) is a build dep, the only way I could 'use vim in the shell' was by creating a stack element of the platform and vim elements | 10:54 |
| juergbi | Nexus: no, seems that macOS reports a higher hard limit than can actually be set, odd. for iniital testing you can simply disable the setrlimit() line | 10:55 |
| jennis | juergbi do you know what the conclusion was of that discussion? | 10:56 |
| Nexus | juergbi: kk ty | 10:56 |
| jennis | It is wrong to assume that: "once i've built something I should be able to test it with bst shell", because that was my assumption | 10:56 |
| jennis | ? | 10:56 |
| juergbi | jennis: I don't think we ever discussed this to conclusion. was on the side with tlater, maybe others | 10:56 |
| jennis | Because obviously my vim element should NOT include the platform as a runtime | 10:57 |
| jennis | ok I guess this is something I could write to the ML about | 10:57 |
| juergbi | jennis: you can directly execute vim, though, sh is just the default command | 10:58 |
| juergbi | vim might run-depend on sh, though | 10:58 |
| juergbi | even if it doesn't depend on the whole platform | 10:58 |
| jennis | juergbi, is this `bst shell -- vim`? | 10:58 |
| jennis | *`bst shell vim.bst -- vim`? | 10:59 |
| juergbi | yes, I think so | 10:59 |
| juergbi | maybe you need to specify the whole path | 10:59 |
| juergbi | I hope it doesn't internally call sh anyway | 10:59 |
| jennis | I will get back to you :) | 10:59 |
| jennis | need to rebuild it | 11:00 |
| qinusty | Can we build docs from ./setup.py? | 11:01 |
| * qinusty reads HACKING.rst | 11:02 | |
| jennis | qinusty, I don't think so, `make -C doc` | 11:02 |
| jennis | from the top level of the buildstream repo | 11:02 |
| *** cs_shadow has joined #buildstream | 11:04 | |
| tiagogomes_ | Can a maintainer increase the CI timeout to say 10h | 11:26 |
| juergbi | tiagogomes_: that's for the nightlies? | 11:33 |
| juergbi | it's currently 7h, that's not enough? can increase it to 10h | 11:34 |
| tiagogomes_ | yup, 7h turns out to not be enough: https://gitlab.com/BuildStream/buildstream/-/jobs/87451938 | 11:34 |
| juergbi | ok | 11:34 |
| juergbi | done | 11:34 |
| tiagogomes_ | thanks :) | 11:34 |
| *** jonathanmaw_ is now known as jonathanmaw | 12:26 | |
| jonathanmaw | tristan: I have an implementation for fixing the git source when upstream is absent. Do you agree with the following technical decisions? a) If the source's URL doesn't contain an alias, (i.e. translate_url's output is the same as its input) it should use the old behaviour of reading the list of submodules from the repo. b) When it does use an alias, the list of source fetchers is the list of submodules defined in the source config, with the | 12:34 |
| jonathanmaw | SourceFetcher's path and ref being read from .gitmodules when they're needed. | 12:34 |
| Nexus | for instances where i won't have bubblewrap available (macOSX) can i simple run as root or is that not acceptable? | 12:49 |
| jmac | Buildstream won't magically work if you just run as root, no | 12:53 |
| jmac | I suppose you could make another Sandbox-type class that just runs the commands and requires root but it sounds dangerous | 12:54 |
| *** johnward has joined #buildstream | 12:58 | |
| * Kinnison imagines that using the chroot sandbox, as root, may get you there, but as evidenced earlier, I'm low on domain knowledge in this area | 12:59 | |
| *** johnward has quit IRC | 13:00 | |
| Nexus | if i do run as root, i stop getting bubblewrap errors (those tests start passing) and i startgetting hardlink errors | 13:00 |
| jmac | Hmm, interesting | 13:01 |
| *** johnward has joined #buildstream | 13:01 | |
| jmac | I wonder if hard links work the same way on HFS+/APFS as they do on ext4 | 13:02 |
| tpollard | should it be possible to override elements 'depends' from project.conf? I have an .bst with not explicit dependencies listed, but I'd like it to have a dependency provided via project.conf | 13:04 |
| tpollard | via 'elements: depends: - ncurses.bst', however it doesn't seem to work | 13:05 |
| noisecell | why would you like to do that? | 13:07 |
| tpollard | noisecell: I'm working on this https://gitlab.com/BuildStream/buildstream/issues/467 | 13:08 |
| * noisecell would prefer to have the dependencies written down in elements, and you can make these elements as big as you want and then depends on it (buildtime or runtime) | 13:09 | |
| tpollard | but as it stands I can't seem to populate depends from project.conf either directly or via an include | 13:09 |
| noisecell | I can not see the point to have that in project.conf, tbh, but this is my opinion. | 13:09 |
| Nexus | so this is meant to be a way of avoiding putting the dep in every element? | 13:10 |
| tpollard | Nexus: yes, appending or overriding in elements where appropriate | 13:10 |
| noisecell | which is useless when you are trying to have a implicit descriptive system, though | 13:10 |
| noisecell | s/implicit/explicit/ | 13:11 |
| noisecell | Nexus, tpollard ^^ | 13:11 |
| Nexus | who's our current test guru? | 13:15 |
| tpollard | noisecell: thanks for the input | 13:16 |
| tristan | jonathanmaw, iiuc, I don't really agree - rather I think flow should be pretty much the following: (A) Instantiate the GitMirror() objects at startup based on what was in the configuration... (B) Return only SourceFetchers for the modules which have explicit configuration... (C) Fill in the gaps with the .gitmodules parsing; while accepting that that list might have changed due to `bst track` causing the new ref to bring in new submodules; which | 13:25 |
| tristan | the project author has not yet bothered to configure | 13:25 |
| tristan | jonathanmaw, essentially; it should still work when (i) The main repo did not specify an alias, but did specify an alias for one or more submodules... (ii) The main repo did specify an alias, but either none, or some of the relevant submodules are not explicitly aliased | 13:27 |
| tristan | jonathanmaw, When I say "still work", I mean to say that source mirroring need not be tried for repos/submodules which did not specify an alias; but those should still be fetched | 13:28 |
| tristan | jonathanmaw, implementation wise, it might make sense to just tie in the fetching of unaliased repos with the main Source fetch() routine, or with the first SourceFetcher() | 13:28 |
| tristan | jmac, as I understand, the unix backend is only intended to be supported as root; it is possible that on systems configured to not allow regular users to create namespaces, and no setuid bit on bwrap, that root will allow it to work properly where BuildStream would otherwise use a fallback | 13:33 |
| tristan | jmac, not sure of all the corner cases, but we do test the unix backend as root in CI | 13:33 |
| * jmac nods | 13:34 | |
| jmac | I couldn't remember the rules for selecting the chroot sandbox; if it's just "not linux" then it should work | 13:35 |
| jmac | Assuming chroot works on OSX, which I've no idea about | 13:35 |
| Kinnison | OSX pretends to be mostly POSIXy | 13:36 |
| tristan | The issues to support OSX I believe are closer to the issues to support BSD | 13:36 |
| tristan | I think there is an issue open somewhere about that, too. While probably not too difficult to support, the binary format's which can be run on either BSD/OSX will differ from linux | 13:38 |
| tristan | *formats | 13:38 |
| qinusty | Do sources have any way of checking their assosciated project? | 13:38 |
| tristan | self.get_project() should get that | 13:38 |
| tristan | on a Source | 13:38 |
| qinusty | part of plugin.py I assume? | 13:39 |
| tristan | ohh, that is not exposed ? | 13:39 |
| tristan | maybe it's not | 13:39 |
| qinusty | it's _ | 13:39 |
| tristan | ah right, it's the same | 13:39 |
| tristan | qinusty, the project is private, itself; that's why | 13:39 |
| qinusty | ah. | 13:39 |
| qinusty | re https://gitlab.com/BuildStream/buildstream/issues/483 | 13:40 |
| tristan | So yes, the core can always consult the project of any source/element, but if a plugin needs something specific; it needs specific API | 13:40 |
| qinusty | The git source would need access to project to see whether that warning was considered fatal? | 13:40 |
| qinusty | Okay | 13:40 |
| tristan | Without looking at that; I think the git source would need to be provided with a Source or Plugin level API, which might involve consulting the project internally | 13:40 |
| qinusty | Thought so | 13:41 |
| * tristan gets confused about Project sometimes, we made Project & Context private in the last minute before 1.0, as there was alot of stuff plugins didn't need, and the few things they do need, we gave them Plugin API for | 13:42 | |
| qinusty | tpollard: What is the current state of !564? | 13:45 |
| tpollard | qinusty: issue 564? | 13:48 |
| qinusty | MR 564, sorry should've linked | 13:48 |
| qinusty | https://gitlab.com/BuildStream/buildstream/merge_requests/564 | 13:48 |
| tpollard | np | 13:48 |
| qinusty | I've implemented the fatal warnings suggested in https://gitlab.com/BuildStream/buildstream/issues/526, which may be of assistance to your MR. I'll try and tidy up an MR for my work this afternoon | 13:49 |
| *** noisecell has quit IRC | 13:49 | |
| tpollard | qinusty: as it stands my opinion is that it's not to be merged until 'fail on warnings' can be set in project.conf | 13:49 |
| tpollard | at which point I was planning to switch the errors to warnings | 13:50 |
| qinusty | Fair enough, I'll ping you when it's in master then | 13:50 |
| tpollard | :) ta | 13:50 |
| qinusty | There will also need to be some work into an API for accessing these fatal warnings within plugins. Should I throw a commit into my MR to make that possible tristan? | 13:51 |
| tristan | qinusty, sounds like it all belongs together as part of the same MR no ? | 13:53 |
| tristan | qinusty, I think that we probably want somewhere where the list of public facing strings are declared, at least for docs purposes too | 13:54 |
| tristan | qinusty, i.e. these are not just strings, these are defined by the core and constants should be used to address them... like Warning.WARNING_NAME | 13:55 |
| qinusty | Okay, currently I have _project.py with _ALL_WARNINGS. But I assume we'll want it all publicly facing | 13:55 |
| qinusty | Perhaps a warning.py? | 13:55 |
| tristan | qinusty, it would be nice to have the warning constants declared in plugin.py; if only as a matter of making plugin author facing documentation nicer | 14:04 |
| tristan | qinusty, otherwise we need to add a whole new page to the reference for it (add it to a ToC), and things seem disconnected from the authoring perspective | 14:04 |
| qinusty | Makes sense | 14:05 |
| valentind | qinusty, I have found a fix for the track bug. We need to pass "copy_tree=True" to _yaml.load in _includes.py. | 14:07 |
| qinusty | nice! | 14:07 |
| tristan | tpollard, I sort of expected we'd start with a warning, and make it optionally fatal with that feature; however I have no objection to your approach of blocking it until then either :) | 14:07 |
| valentind | qinusty, will run tests and push. | 14:08 |
| qinusty | nice, ping me after it's pushed, happy to review | 14:08 |
| tristan | valentind, I had been suspecting that in the back of my mind, but didn't believe that could have regressed | 14:08 |
| valentind | tristan, Is there a use for copy_tree being false? | 14:08 |
| tristan | valentind, qinusty indeed; the loading of the YAML is faster without enabling proper round trip support | 14:08 |
| qinusty | But tristan, when did includes get merged? | 14:08 |
| qinusty | I do believe this is a bigger issue, but I think we were still seeing indentation issues with ruamel prior to the merge | 14:09 |
| tristan | So we only ever set it true, when we're going to rewrite yaml files | 14:09 |
| valentind | tristan, Is it an optimization thing? | 14:09 |
| tristan | qinusty, I don't believe the indentation issues can be fixed on our side; I have a feeling that you are just seeing the limitations of ruamel.yaml | 14:09 |
| tristan | valentind, yes; it is quite significantly slower to load | 14:10 |
| valentind | tristan, So I suppose I should do copy_tree=True only when including in elements, not in project.conf. | 14:10 |
| qinusty | probably, as I mentioned ruamel struggles with round tripping multiline `|` when I tried it | 14:10 |
| tristan | valentind, indeed, and project.refs; however I think before, we've been doing it as an all-or-nothing thing | 14:11 |
| valentind | tristan, I do not remember. Did we have include in project.refs? | 14:12 |
| tristan | valentind, it *might* be dangerous to not do all-or-nothing, but not doing it for project.conf indeed sounds like it should be sensible | 14:12 |
| tristan | valentind, I think that was explicitly unsupported by design, but I don't recall if you explicitly unsupported it in implementation | 14:13 |
| tristan | I seem to recall mentioning why includes should not be supported in project.refs in the email thread | 14:14 |
| valentind | It is not. So it is fine. | 14:15 |
| tristan | In any case, people are free to add comments and explicitly order their project.refs | 14:15 |
| tristan | which means round-tripping should be supported there (orthogonal to includes, of course) | 14:15 |
| rdale | does the 1.1.5 release of buildstream have the cache purging feature? | 14:20 |
| tristan | rdale, it does | 14:23 |
| tristan | rdale, apparently, tiagogomes_ found a hot issue with it, though :-S | 14:23 |
| tristan | It may not be purging the extract directories while purging the artifacts themselves, rendering it rather futile | 14:24 |
| rdale | ah ok - how do you invoke it - i have a full disk and it would be nice to clear out some space? | 14:24 |
| tristan | (should be easy enough to fix, though) | 14:24 |
| tristan | rdale, http://buildstream.gitlab.io/buildstream/using_config.html#default-configuration ... see the 'cache:' settings | 14:25 |
| tiagogomes_ | I'd provide a MR to cleanup extract at startup, but I don't think you would approved if made running multiple instances of bst simultaneously more racy | 14:25 |
| tristan | rdale, I've set my 'quota' to 20GB | 14:25 |
| rdale | tristan: thanks i'll have a look | 14:26 |
| tristan | tiagogomes_, extract anyway should never be purged, either the ArtifactCache.remove() codepath should take care of it, or the code which calls that should | 14:26 |
| tristan | tiagogomes_, either case is racy wrt parallel instances, but lets treat that separately | 14:28 |
| tiagogomes_ | In abnormal conditions there might be some debris left | 14:28 |
| tristan | tiagogomes_, the cleanup has to happen as the build progresses; if debris is left for some abnormal reason, that sounds like a class of isolated bugs to fix | 14:28 |
| tristan | in any case, deleting an artifact within a single instance is supposed to be guaranteed to be isolated from any other cache accesses (multi instance being a separate thing to think about, which is regressed by cleanup but not a huge priority right now) | 14:30 |
| tristan | tiagogomes_, seeing as the ArtifactCache offers the extract() method and basically dictates/owns the specific artifact extract path, it makes sense that ArtifactCache.remove() also remove any extract | 14:32 |
| * tiagogomes_ nods | 14:32 | |
| tristan | looking at that code, it seems it must have been tested under conditions where the artifacts were created but never extracted | 14:33 |
| * tristan throws a toaster at tlater ;-) | 14:33 | |
| skullman | Greetings comrades. I'm looking to create a new GitLab project, so it's bike-shedding time! | 14:33 |
| skullman | The purpose is to have a project with .bst files so we can use BuildStream to develop BuildStream. | 14:33 |
| skullman | I'm thinking buildstream-integration since I'd rather have a boring name, the convention is to name boring projects by the purpose of their contents, buildstream-dogfood is an unpleasant name and the project is needed to facilitate dogfooding rather than being the direct purpose of the project, and buildstream-app while it describes the contents is too close to the purpose of buildstream.git | 14:33 |
| *** noisecell has joined #buildstream | 14:35 | |
| * phildawson votes for buildstreamception :P | 14:36 | |
| * Kinnison thinks 'buildstream-integration' is a pretty good name | 14:36 | |
| Kinnison | Repositories can be renamed if need-be | 14:37 |
| phildawson | more seriously, buildstream-integration sounds sensible | 14:37 |
| toscalix | skullman: please do not start the name with buildstream | 14:38 |
| toscalix | tabbing would be more expensive | 14:38 |
| phildawson | toscalix, why not? | 14:38 |
| toscalix | I would also avoid dash | 14:38 |
| toscalix | we have a buildstream repo already | 14:38 |
| toscalix | integration-buildstream if you want to go that route, would be my suggestion | 14:39 |
| phildawson | toscalix, no more expensive than it already is. we already have buildstream-docker-images for example | 14:39 |
| toscalix | phildawson: which is why I say it. You can insist on it.... or not | 14:40 |
| skullman | toscalix: we have multiple projects in the BuildStream/ project namespace that are not directly related to the BuildStream tool, and there is plenty of precedent i.e. buildstream-{sysroots,docker-images,examples} | 14:41 |
| toscalix | think about the urls also.... | 14:41 |
| toscalix | skullman: bad precedents, we have plenty of bad precedents.... you can insist on them or try to make things... better | 14:41 |
| toscalix | the names of the repos is something that we can improve. I am not propossing changing them... yet. But if we can have unique simple names for them.... better | 14:42 |
| tristan | I certainly prefer buildstream-integration over integration-buildstream | 14:43 |
| toscalix | I have two: alignment and communication | 14:43 |
| phildawson | personally I like the buildstream-* naming scheme. I find it very intuative to interpret. | 14:43 |
| toscalix | Buildstream/buildstream-integration | 14:43 |
| tristan | I think everything so far is nicely consistent and named this way, where related things start with "buildstream-" (and we have been preferring dashes over underscores, no need to change that) | 14:44 |
| toscalix | and when we have more and we start creating subgroups you will have..... Buildstream/buildstre-xx/buildstream-yy | 14:44 |
| valentind | What about dropping the "buildstream-" prefix for the repository name, but call it with the prefix in other contexts? | 14:44 |
| toscalix | if you of that as a good naming squeme... I will not be able to convince you, obviously | 14:44 |
| tristan | that is possible also, but this will dictate what is the default directory name from `git clone`, so it's very nice that it starts with "buildstream-" | 14:45 |
| phildawson | toscalix, that's a good point. But it does make the default git clone directory names nice this way. | 14:45 |
| tristan | i.e. regarding valentind's comment; I also thought, we could have gitlab.com/BuildStream/docker-images, gitlab.com/BuildStream/integration, etc | 14:45 |
| phildawson | oops | 14:46 |
| phildawson | snap | 14:46 |
| tristan | but then it would not be named "buildstream-foo" when you clone it by default | 14:46 |
| toscalix | we have the same name for the project, the tool, the group, the repo.... with different capitalization.... I think this is all a big mess that we will pay as we scale up | 14:47 |
| tristan | I have a feeling that what is incubated in this repository may eventually get upstreamed, too; e.g. if we build official flatpaks of BuildStream (somehow), we'd want the BuildStream project that creates it, in the same repo | 14:47 |
| tristan | toscalix, if anything is named "Buildstream", that needs fixing | 14:47 |
| tristan | if it's not capitalized at all, that seems alright, a repo name, a checkout | 14:47 |
| tristan | if there is capitalization, it looks like a brand, it should be "BuildStream" | 14:48 |
| tristan | I recently corrected the mailing list to not have this problem | 14:48 |
| toscalix | groups is BuildStream...my bad....anyway, the point remains | 14:48 |
| toscalix | stands | 14:48 |
| skullman | are we suggesting the new repo should be BuildStream/BuildStream.git? | 14:49 |
| tristan | skullman, I don't think so no | 14:49 |
| toscalix | BuildStream will be a different thing for different target audience because we are using it as brand, but also in other places for other things. We are shooting ourself in our own feet | 14:49 |
| * Nexus votes for buildfooding | 14:50 | |
| toscalix | Is top here. Sorry, I think the whole branding thing has not been thought through and it will have consequences | 14:50 |
| toscalix | the project, the tool, the group, the repo... should have different names | 14:50 |
| tristan | skullman, I'm talking about when we write things, branding etc, emails, public communications, we should say "BuildStream", or not capitalize at all, which looks casual, I'm only opposed to "Buildstream", or "buildstreaM", or other arbitrary capitalizations | 14:50 |
| Nexus | BuIlDsTrEaM | 14:51 |
| toscalix | in any case... I think everybody will be using bst in as we get popular | 14:51 |
| tristan | Nexus, exactly, that reads to me, the same way as "Buildstream" | 14:51 |
| skullman | fair, I thought we were suggesting that the flatpak would be branded as BuildStream so the project that may end up building it should be | 14:52 |
| Nexus | who worked on artifact cache expiry? | 14:52 |
| skullman | Nexus: I think it was tlater | 14:52 |
| Nexus | damn | 14:53 |
| tristan | Also, as in many cases; consistency is often more important than the arbitrary choice; if we were to create "integration" in the BuildStream group... we should change all the related "buildstream-" repos to match | 14:53 |
| tristan | but still, these are git repository names, it's arbitrary that gitlab.com puts them into a "Group" of the same name | 14:53 |
| tristan | when we check them out, clone them, use them, it's nice that they have "buildstream-" in the name | 14:53 |
| valentind | qinusty, I have pushed the fix: https://gitlab.com/BuildStream/buildstream/merge_requests/622 | 14:58 |
| tristan | cs_shadow, I'm sorry I haven't had time to really give pip a full proper review; haven't looked at test cases at all either; but I went through a straightforward reading of the plugin itself and it certainly looks promising and straight forward | 15:00 |
| tristan | cs_shadow, there are some limitations and XXX comments indicating those, but I doubt there is much sensible we can do about it; most importantly the ref produced by `pip freeze` needs to be something we can trust | 15:00 |
| tristan | cs_shadow, I will give a more thorough look through tomorrow but I think it's looking good... and thanks for following up with the plugin !! | 15:01 |
| qinusty | tristan, do we want fatal-warnings to be extendable by plugins? E.g. we are currently looking to add `ref-not-in-track`, but third party plugins won't have that ability. | 15:02 |
| tristan | cs_shadow, one thing comes to mind... I will write a comment for it... | 15:02 |
| tristan | qinusty, that is a very good question; I have a background process running that says we might leverage the BstError machine readable "reason" strings | 15:03 |
| tristan | qinusty, feel free to draw up (reasonably detailed) counter proposal to consider :) | 15:03 |
| qinusty | I have a basic implementation, but possible warnings are restricted to those defined within plugin.py as constants | 15:04 |
| qinusty | Which isn't ideal | 15:04 |
| tristan | qinusty, I'm concerned with being able to document the API clearly, and if extensibility is allowed; a way to ensure namespacing | 15:08 |
| qinusty | Looks good valentind, definitely fixes the issues I was seeing | 15:08 |
| tristan | qinusty, I can see how extensibility is desirable, so is informing the project author of all the possible warnings which can be turned into errors (or at least informing the project author how to find out) | 15:09 |
| qinusty | I see, Perhaps each plugin can document any fatal-warnings it provides. Adding them to some form of configuration when the plugin is set up | 15:10 |
| tristan | qinusty, I guess that BstError doesnt help with warnings either, which means a new parameter to Plugin.warn() | 15:10 |
| qinusty | What parameter will it need? | 15:10 |
| tristan | qinusty, dont think of them as fatal warnings, they are just warnings; it's up to the author to decide if they are fatal :) | 15:10 |
| qinusty | AH, I see where you're going | 15:11 |
| cs_shadow | tristan: thanks for taking a look! `pip freeze` guarantees to produce the package list in a case-sensitive sorted order, which I think can be trusted. We can split the output and parse the output ourselves but I'm not sure if we'd do anything different than what it does by default | 15:11 |
| qinusty | You're suggesting that plugin.warn handles the checks against the fatal-warnings configuration and throws an error if the warning is configured as fatal | 15:11 |
| cs_shadow | I'll follow up with your other comment on GitLab itself | 15:11 |
| tristan | qinusty, so essentially yeah, it's better if the core just decides to make a warning fatal, and a plugin only raises a warning | 15:11 |
| qinusty | meaning that the plugin itself, just calls plugin.warn("text goes here, info and such", warning_tag="overlaps" | 15:12 |
| qinusty | etc | 15:12 |
| tristan | qinusty, plugins deciding themselves whether to raise an error, and adding branch statements to individual plugins, would probably be bad | 15:12 |
| tristan | qinusty, right, we're on the same page :) | 15:12 |
| qinusty | Yes | 15:12 |
| tristan | qinusty, in all cases, we want to control as much as we can from the core, and leave the plugin with least responsibility as possible | 15:12 |
| qinusty | Now the main issue is coming up with a way for a plugin to extend the fatal-warning options and provide documentation for such additions | 15:13 |
| tristan | qinusty, I think that can solve extensibility, and back compat is addressed with a statement that "Any plugin which does not provide the context of a warning, or any warning issued without that added context, cannot be marked fatal" | 15:13 |
| qinusty | indeed | 15:14 |
| tristan | qinusty, the last issue is about ensuring project authors can have a sensible way of knowing what those warnings contexts can be | 15:14 |
| qinusty | E.g. read the documentation for their plugins ;) | 15:14 |
| tristan | Well, yes indeed | 15:14 |
| jjardon | heads up: cache artifact server deployed for GNOME builds | 15:14 |
| tristan | qinusty, It would be good though at least for core plugins, that we have some index where those warnings are aggregated in one place | 15:15 |
| tristan | jjardon, yyyyeaaaaaahhhhhhhhhhh \o/ | 15:15 |
| * tristan jumps up and down | 15:15 | |
| tristan | party time ! | 15:15 |
| skullman | seems like the majority like buildstream-integration, and while there's extra friction on tab-completion and redundancy in the url, consistency, distinguishing it from projects in BuildStream/ which aren't related to the buildstream tool and the nice default clone name have swung it for me. | 15:15 |
| skullman | jjardon: ooh | 15:16 |
| tristan | jjardon, now all that's left is to ensure downloading an llvm artifact doesnt take 3 hours ! | 15:16 |
| tristan | hehehe | 15:16 |
| jjardon | If someone ask, It's a 3 cores machine, with 1GB of RAM | 15:16 |
| qinusty | I was thinking https://buildstream.gitlab.io/buildstream/format_project.html#fail-on-overlaps tristan | 15:16 |
| qinusty | But it did just take me 2-3 minutes to find that branch of the docs | 15:17 |
| jjardon | tristan: I have created a new label for cache server related stuff, hoe you dont mind: https://gitlab.com/BuildStream/buildstream/issues?label_name[]=cache_server | 15:17 |
| tristan | qinusty, maybe we also want an Object() class with constants, for the errors which are core | 15:18 |
| qinusty | Yeah for namespacing purposes | 15:18 |
| tristan | qinusty, for instance the ref-not-in-track can be raised by multiple plugins, they also need to know which core warnings should be used | 15:18 |
| tristan | err, s/errors/warnings | 15:18 |
| qinusty | Indeed | 15:18 |
| laurence | jjardon, pls propose a new label on the list | 15:19 |
| tristan | qinusty, good point ! we might solve namespacing issues by saying... if it's not a core warning, it will be automatically prefixed with Plugin.get_kind() | 15:19 |
| laurence | we should do it before we make them | 15:19 |
| jjardon | laurence: it was discussed on Monday meeting | 15:19 |
| laurence | I would also like a new label at the moment, so thanks for reminding me of that :) | 15:20 |
| * laurence posts to list | 15:20 | |
| tristan | qinusty, so plugin specific warnings automatically become something like... "local:some-warning", when they call self.warn("Some warning is being issued", context="some-warning") | 15:20 |
| tristan | qinusty, if for instance, it was issued by the local source plugin | 15:20 |
| qinusty | Meaning that your project.conf has to have local:some-warning etc | 15:20 |
| qinusty | as a fatal-warning | 15:21 |
| tristan | exactly, ensuring separate namespace for any warning that is plugin specific | 15:21 |
| qinusty | Alright, sounds good. When are plugins loaded in and setup() called? | 15:21 |
| WSalmon | jjardon, you asked about having the git track back ported, i have set up the MR https://gitlab.com/BuildStream/buildstream/merge_requests/621 are we just merging them if they work and are excepted to 1.2 or are we waiting for tristan to ok them or are we doing a fresh review cycle? | 15:22 |
| laurence | jjardon, well, ok, but that is not giving everyone a chance to respond. you are leaving out many users of the tool | 15:22 |
| tristan | qinusty, later than project.conf, by the end of Stream._load() | 15:22 |
| qinusty | :/ Meaning we can't limit fatal-warnings to those provided by loaded plugins | 15:22 |
| tristan | qinusty, probably asserting plausible correct values for warning names has to be done late | 15:22 |
| qinusty | yes | 15:22 |
| tristan | well, we need to do it after loading the project | 15:22 |
| tristan | before running a scheduler or anything | 15:22 |
| jjardon | WSalmon: I'd wait for tristan to ack it | 15:23 |
| qinusty | Sounds good, I'll get started with some of that and see how it feels on the plugin end | 15:23 |
| *** leopi has quit IRC | 15:23 | |
| tristan | WSalmon, jjardon, technically, we can backport the warning, but we can't technically add the feature to mark those as fatal in 1.2.x | 15:23 |
| skullman | tristan: looks like I'll need an admin to create BuildStream/buildstream-integration project. | 15:24 |
| skullman | I'll get on with a local repository for now. | 15:24 |
| skullman | so no rush | 15:25 |
| WSalmon | in that case it might be worth making it a (more) protected repo in gitlab? | 15:28 |
| WSalmon | so are you happy with the _pipeline bit but the git.py bit wants to be a warning? | 15:28 |
| tristan | uhhh, me ? ... link again ? | 15:29 |
| WSalmon | https://gitlab.com/BuildStream/buildstream/merge_requests/621/diffs | 15:29 |
| tristan | WSalmon, it must be a warning and not an error, and I seem to recall having commented before on this... lemme look | 15:29 |
| tristan | waaait a sec, this is different | 15:29 |
| tristan | this is not warning if ref is not in the specified track | 15:30 |
| WSalmon | nope this is if you dont give a ref or a track then you get a never ending loop of track then build then error saying go do track | 15:30 |
| tristan | WSalmon, no sorry; that is not right, the core is not allowed to know anything about plugin specific configuration, or attributes a plugin might have used | 15:31 |
| tristan | WSalmon, this looks like it has some overlap with other things, e.g. the proposal which came up about manifests | 15:31 |
| tristan | WSalmon, so for instance, we might need to come up with some real API for plugins to report additional things, like the 0-N URLs which might be associated, etc | 15:32 |
| tristan | WSalmon, a core-level warning would have to depend on that having been implemented by the plugins (and the API having been defined for it) | 15:32 |
| *** xjuan has joined #buildstream | 15:32 | |
| tristan | WSalmon, jjardon, I think that the quick fix is to improve the "Try running bst track" error message, to also inform the user that they must ensure that they have provided tracking information for the mentioned ref-less elements | 15:33 |
| tristan | I.e. the current error message just says "try running bst track" on the elements which didnt have a ref, failing to tell the user/project author, that it is also their responsibility to provide tracking information | 15:34 |
| tristan | Alternatively, one could reword it to say "If you have provided tracking information in these elements, then you can try running bst track, otherwise please specify the specific refs yourself" | 15:35 |
| tristan | tracking is after all, just a productivity feature, not a requirement | 15:35 |
| * tristan pasted the relevant irc log in the merge request... and heads out... | 15:38 | |
| WSalmon | sorry i think i may have over paraphrased my summary, dose the commit not do that? | 15:42 |
| *** noisecell has quit IRC | 15:47 | |
| skullman | hmm, seeing build failure from caching build trees, since some projects may leave sockets in their build tree | 16:00 |
| jennis | jonathanmaw if you get a chance, do you mind taking a look at: https://gitlab.com/BuildStream/bst-external/merge_requests/41 | 16:04 |
| Nexus | skullman: what do you mean? | 16:04 |
| skullman | Nexus: freedesktop-sdk's gpgme.bst fails when it tries to cache the build tree, since as part of its test suite it creates unix sockets and leaves them around in the build tree. This causes problems because the code to copy the build tree into the artifact can't handle unix sockets | 16:06 |
| jonathanmaw | jennis: I'll add it to my list. I'll probably have time tomorrow. | 16:07 |
| skullman | I've added the following to get on with things: | 16:08 |
| skullman | config: | 16:08 |
| skullman | install-commands: | 16:08 |
| skullman | (>): | 16:08 |
| skullman | - | | 16:08 |
| skullman | find . '(' -type s -o -type p ')' -delete | 16:08 |
| Kinnison | I'd be tempted to say that it should ignore sockets entirely since they're 100% unuseful once the processes behind them have gone away | 16:11 |
| jmac | laurence: I don't understand the 'paper-cuts' reference... am I missing something? | 16:11 |
| jennis | ty jonathanmaw :) | 16:12 |
| Kinnison | jmac: in reference to issues, papercuts are small issues which, on their own, don't necessarily prevent anything, but when you have hundreds of them, you die. | 16:12 |
| jmac | OK | 16:12 |
| jmac | Never heard the term before but it makes sense | 16:12 |
| laurence | hhhm i should have explained better | 16:12 |
| laurence | s/better/more/ | 16:13 |
| laurence | :) | 16:13 |
| *** tlaxkit has joined #buildstream | 16:13 | |
| *** leopi has joined #buildstream | 16:22 | |
| tiagogomes_ | skullman https://gitlab.com/BuildStream/buildstream/issues/500 | 16:31 |
| skullman | tiagogomes_: cheers | 16:42 |
| *** jonathanmaw has quit IRC | 17:30 | |
| *** toscalix has quit IRC | 17:39 | |
| *** tlaxkit has quit IRC | 17:44 | |
| *** milloni has quit IRC | 17:46 | |
| *** milloni has joined #buildstream | 17:52 | |
| *** xjuan has quit IRC | 20:20 | |
| *** leopi has quit IRC | 20:38 | |
| *** tristan has quit IRC | 21:31 | |
| *** bochecha has quit IRC | 22:45 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!