| *** persia has quit IRC | 06:34 | |
| *** persia has joined #buildstream | 06:36 | |
| *** mohan43u has joined #buildstream | 06:54 | |
| *** tristan has joined #buildstream | 06:56 | |
| *** mohan43u has quit IRC | 06:59 | |
| *** mohan43u has joined #buildstream | 07:00 | |
| *** mohan43u has quit IRC | 07:59 | |
| *** mohan43u has joined #buildstream | 08:37 | |
| tlater | Hm, looks like gitlab MR pages aren't working atm? | 09:18 |
|---|---|---|
| tristan | I've been viewing and editing one in the last 30min... | 09:25 |
| tlater | Odd, I'm getting 503's | 09:26 |
| jennis | working for me | 09:27 |
| tlater | Could someone try to open https://gitlab.com/BuildStream/buildstream/merge_requests/315 ? | 09:27 |
| tlater | Perhaps it's just a few isolated ones | 09:27 |
| jennis | That's not working ^^ | 09:27 |
| jennis | 503 | 09:27 |
| tlater | Well that sucks | 09:27 |
| * tlater wonders what he can do to unbreak the MR | 09:28 | |
| tlater | Looks to be any MR that "can't be merged automatically"... ? | 09:29 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: WIP: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 09:34 |
| tlater | Yep | 09:34 |
| tlater | If anyone runs into an MR that 503s - the solution is to rebase against master | 09:34 |
| tristan | tlater, that is probably a gitlab bug and not particularly related to rebase; you just hit a nerve, kicked it in the nuts so that it finally woke up | 09:35 |
| tlater | tristan: The 503 happens on any MR that "can't be merged automatically" - rebasing against master means it can now be merged automatically ;) | 09:36 |
| * tlater checks if there is a gitlab issue already | 09:36 | |
| tristan | What ?! | 09:39 |
| tristan | That sounds new | 09:40 |
| tlater | I can't find an issue on their tracker, writing one now | 09:41 |
| tlater | Aaand the issues are working again | 09:43 |
| tlater | Hm | 09:43 |
| jjardon[m] | Hi, has 1.1.2 been released already? | 09:43 |
| tlater | Gremlins everywhere | 09:44 |
| tristan | jjardon[m], I'm thinking of rolling one very soon because of project.refs having landed | 09:44 |
| jjardon[m] | I can not see a tag in the git repo but https://buildstream.gitlab.io/buildstream/ shows 1.1.2* | 09:44 |
| tristan | right now, trying to fix a critical bug first | 09:44 |
| jjardon[m] | ah, rigth | 09:45 |
| tristan | almost there actually | 09:45 |
| jjardon[m] | cheers | 09:45 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: WIP: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 09:48 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: WIP: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 09:49 |
| *** jonathanmaw has joined #buildstream | 09:51 | |
| *** dominic has joined #buildstream | 09:54 | |
| *** aday has joined #buildstream | 09:56 | |
| *** valentind has joined #buildstream | 09:59 | |
| juergbi | tristan: are you looking into #305 or a different issue? | 10:03 |
| tlater | juergbi: About !313 | 10:06 |
| * tlater is annoyed that he can't reply to normal comments on gitlab | 10:06 | |
| juergbi | maybe i should have started a discussion. it's a bit odd having to choose this | 10:07 |
| tlater | Are you proposing we use format-version in workspaces or version in project.conf? | 10:07 |
| juergbi | but you know about 'r' to do a quoted reply, right? | 10:07 |
| juergbi | the former | 10:07 |
| tlater | No, checking gitlab markdown then :) | 10:07 |
| juergbi | it's not special markdown, it's a key shortcut | 10:08 |
| juergbi | select text in some existing comment and simply press 'r' | 10:08 |
| tlater | Ah, nice | 10:08 |
| tlater | tyvm, that's entirely new to me | 10:08 |
| tlater | Hm, well, either way, probably easier to discuss here anyway | 10:08 |
| tlater | I'm also annoyed by the name == 'version' checks | 10:09 |
| tlater | juergbi: I take it you're thinking about making _yaml.py error out if it comes accross anything named 'version'? | 10:09 |
| juergbi | no, that would be too low level. i'm not proposing of moving the version handling to yaml.py | 10:10 |
| juergbi | i was just thinking of adding some central element name validation, maybe in loader.py or in element.py, and then disallowing version (or format-version) there | 10:10 |
| juergbi | additional 'keywords' could be added later in the same place, if necessary | 10:10 |
| tlater | Hm, yeah, I think that's better | 10:11 |
| juergbi | i still think it's not ideal as we don't need it in any other file, at least for now | 10:11 |
| juergbi | an alternative would be to add an extra level to the workspace file | 10:11 |
| tlater | Yeah, that was my initial idea | 10:11 |
| tlater | I'd much prefer that tbh | 10:12 |
| * skullman too | 10:12 | |
| juergbi | was there a reason it wasn't done that way or just because the other format was already proposed/implemented? | 10:12 |
| tlater | The latter, it was part of awacheux[m] MR | 10:12 |
| * skullman stuck with the proposed format since he didn't know if it was the result of a wider discussion already | 10:12 | |
| juergbi | right, makes sense | 10:13 |
| juergbi | but as this is not actually used yet and we all seem to agree on wanting to avoid the 'version' checks, i think we should change the format | 10:13 |
| tlater | Yup, that makes sense | 10:14 |
| juergbi | especially as the format is not even targeted for human readability. odd to add special cases for machine readable file | 10:14 |
| tlater | juergbi: Do you have any further comments on !315 btw? | 10:15 |
| * tlater hopes to get these mergeable before 1.1.2 lands | 10:15 | |
| juergbi | not yet but haven't fully reviewed yet. just added a comment for the small thing i noticed | 10:16 |
| tlater | Right, ta, wasn't sure if that was it :) | 10:16 |
| juergbi | it's at the top of my list, though | 10:16 |
| juergbi | i looked at !315 for context of !313 | 10:16 |
| gitlab-br-bot | buildstream: merge request (tristan/main-process-errors->master: Handle errors in main process gracefully) #333 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/333 | 10:40 |
| jennis | Ok juergbi, here is the issue: https://gitlab.com/BuildStream/buildstream/issues/136 | 10:40 |
| jennis | I think tristan may have some ideas for this too | 10:40 |
| tristan | juergbi, MR !333 above is part 1 of todays firefighting | 10:41 |
| tristan | juergbi, I think it's ready to merge, you can take a look; mostly - I think you will find it interesting | 10:41 |
| jennis | So, first step is: we want to effectively implement an `rm -rf` of the REMOTE cache once a quota is reached | 10:41 |
| juergbi | jennis: yes, big hammer stop gap solution but it's considered better than the status quo | 10:42 |
| juergbi | although, i'm wondering whether we can't implement something slightly smarter even for the initial solution | 10:42 |
| jennis | One idea i've had is to have a quota: / size: node in the project.conf and then use tlater's function which checks the local artifact cache size to check the size of the remote cache | 10:43 |
| juergbi | automatically deleting whole artifact cache is excessive | 10:43 |
| jennis | But i'm getting confused about where the remote cache is dealt with in the two mentioned scripts | 10:43 |
| tristan | juergbi, also note: I added an unusual test case to tests/frontend/fetch.py, it doesnt fail in an ideal way - but I thought it better to have a test with FIXME comment so that when #197 is improved, the test is already there and only needs some modification | 10:43 |
| tristan | sorry to interrupt your conversation :) | 10:43 |
| jennis | ATM they're completely reconfiguring the server each time | 10:43 |
| juergbi | tristan: ok, will take a look later | 10:43 |
| jennis | they're being the freedesktop team | 10:43 |
| jennis | and I'm sure others have a similar problem | 10:44 |
| juergbi | jennis: yes but e.g., we could do something like delete only half (or 20% or whatever) of the refs (the oldest ones based on mtime) and then run ostree prune | 10:45 |
| jennis | So juergbi, now i guess the discussion is, firstly: do I try and implement this stop gap or shall we start discussing this other idea you have? | 10:45 |
| juergbi | this sounds like it wouldn't be a very large effort and should already be massively better. don't you think? | 10:45 |
| jennis | Ok let's talk about that | 10:45 |
| jennis | I agree | 10:45 |
| tristan | gah | 10:45 |
| tristan | I need to update that MR, forgot some local changes | 10:46 |
| jennis | So my current issue is that I'm not totally clued up on how the pushrecieve and artifactcache scripts work together | 10:46 |
| juergbi | bst-artifact-receive is invoked (via ssh) on the remote artifact server | 10:47 |
| juergbi | that's receive_main() in pushreceive.py | 10:47 |
| gitlab-br-bot | buildstream: merge request (tristan/main-process-errors->master: Handle errors in main process gracefully) #333 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/333 | 10:47 |
| juergbi | the code in ostreecache.py only runs on the client side | 10:47 |
| juergbi | and the remote server doesn't know anything about projects | 10:48 |
| jennis | right ok | 10:48 |
| juergbi | so it also can't use project.conf | 10:48 |
| jennis | So the remote server is only concerned with pushreceive? | 10:48 |
| juergbi | yes | 10:48 |
| juergbi | the quota should possibly be a CLI option to bst-artifact-receive/pushreceive | 10:49 |
| jennis | right... ok because artifactcache.py also has some remote-server related code | 10:49 |
| juergbi | only the client side, right? | 10:49 |
| jennis | But this is the other side | 10:49 |
| jennis | ahh beat me to it | 10:49 |
| juergbi | i'm wondering whether we should delete based on available disk space or based on size of repo | 10:50 |
| juergbi | the issue with the latter is that it's not that cheap to calculate | 10:50 |
| juergbi | so as a stop gap i would recommend to use available disk space | 10:50 |
| Nexus | tristan, regarding: `opening a workspace with a cached build` how should we handle it if an element has multiple sources? | 10:50 |
| jennis | I think it's wise to use tlater's work on local cache's which I'm pretty sure does on available disk space | 10:50 |
| jennis | At least for the sake of consistency | 10:51 |
| juergbi | Nexus: the build tree includes all sources, as does the workspace (we changed this a while ago) | 10:51 |
| Nexus | ah kk | 10:51 |
| tlater | jennis: My branch does this based on size of repo, as it will be user-configurable | 10:51 |
| tlater | There will be some stuff to figure out potential clashes with disk size, but that | 10:52 |
| tlater | 's mostly not the goal | 10:52 |
| jennis | ahh right ok | 10:52 |
| tlater | We will certainly have enough overlap to share a few utils though :) | 10:52 |
| juergbi | from the configuration point of view, size of repo sounds nicer but doing that on the server on every push sounds excessive | 10:52 |
| jennis | So let's be consisten | 10:52 |
| jennis | t | 10:52 |
| juergbi | a possibility would be to do it in a cron job instead | 10:53 |
| jennis | cron job...? | 10:53 |
| juergbi | we anyway already have one for updating the summary file | 10:53 |
| tlater | juergbi: Does the server happen to receive artifact sizes as part of the protocol? | 10:53 |
| juergbi | i.e., check the quota every 5 minutes instead of on every push | 10:53 |
| jennis | I see | 10:53 |
| tlater | If not, we could make that part of the protocol, because clients will have to calculate it anyway | 10:53 |
| juergbi | we send a tar stream | 10:54 |
| jennis | Maybe first we do every push, and once it's working properly we implement a cron job | 10:55 |
| jennis | in another MR | 10:56 |
| juergbi | this will make pushes very slow, i suspect | 10:56 |
| juergbi | although, if the server has sufficient RAM, the kernel dentry cache might make this fast enough | 10:57 |
| juergbi | not sure how this would be in practice | 10:57 |
| * tlater figures the difference in code is small enough that testing both wouldn't be a major time sink | 10:57 | |
| juergbi | and that's something where the Meltdown patch causes significant slowdown :/ | 10:57 |
| jennis | That's what I'm hoping tlater | 10:58 |
| juergbi | yes. the actual expiration part should be the same | 10:58 |
| jennis | in terms of actually setting a quota | 10:58 |
| jennis | This should be done client side? | 10:58 |
| juergbi | not for the remote, no | 10:58 |
| jennis | One idea was to have a quota: or size: node in project.conf | 10:58 |
| jennis | mhmm | 10:58 |
| juergbi | artifact cache can be shared across projects | 10:59 |
| jennis | ahh ofc | 10:59 |
| tlater | jennis: The problem with that is that there are lots of people pushing to the same remote. If everybody configures a different value, that would break | 10:59 |
| juergbi | and we generally don't want to trust the client with something like this | 10:59 |
| juergbi | we don't have a server config file so far, iirc | 10:59 |
| juergbi | so for now this would simply be a bst-artifact-receive CLI option | 10:59 |
| juergbi | that would be added to ForceCommand in sshd config | 10:59 |
| jennis | So whoever configures the server sets the quota | 10:59 |
| juergbi | we could introduce a server config file at some point but i wouldn't block on this | 10:59 |
| juergbi | yes | 10:59 |
| gitlab-br-bot | buildstream: merge request (issue-243_dpkg_import_source_plugin->master: Created DPKG Source plugin for Issue #10) #305 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/305 | 11:00 |
| tristan | Nexus, where is the issue report, or other relevant detailed information regarding `opening a workspace with a cached build` ? | 11:00 |
| juergbi | i'm wondering whether we should still also support expiration based on available disk space | 11:00 |
| Nexus | tristan: it doesn't exist yet, i was going to make one once i had a plan written up for it | 11:00 |
| jennis | So in terms of a client wanting to use a remote server, they use the documentation and do it themselves? | 11:00 |
| jennis | atm ^? | 11:01 |
| juergbi | as that's something we could enable by default (or even always) | 11:01 |
| juergbi | jennis: do you mean to push to a remote server or to configure/setup a remote server? | 11:01 |
| tristan | Nexus, that is unfortunate, with a detailed writeup of what we wanted to achieve for that, I could answer your question much more easily :-S | 11:01 |
| jennis | configure/setup | 11:01 |
| jennis | we have this: https://buildstream.gitlab.io/buildstream/artifacts.html | 11:02 |
| juergbi | tristan: it's mentioned in the description of #21 but we don't have a separate issue for it, afaik | 11:02 |
| gitlab-br-bot | buildstream: merge request (issue-243_dpkg_import_source_plugin->master: Created DPKG Source plugin for Issue #10) #305 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/305 | 11:02 |
| juergbi | jennis: yes, should be added here https://buildstream.gitlab.io/buildstream/artifacts.html#configure-and-run-sshd | 11:02 |
| * tristan takes a look... | 11:03 | |
| Nexus | the deb source plugin script is tiny now o.o | 11:03 |
| juergbi | leveraging existing code is good :) | 11:03 |
| tristan | mhm, ok I see | 11:04 |
| jennis | ok so juergbi, this /etc/ssh/sshd_config is where we want to set a quota? | 11:04 |
| tristan | Nexus, as far as I know, we have already changed, or at least planned to change, workspaces to be element-wide | 11:04 |
| jennis | or something like that ^ | 11:04 |
| tristan | Nexus, in which case, I dont see any issue with your question; you just create the workspace based on that instead, right ? | 11:05 |
| juergbi | jennis: that's where the remote cache is currently configured, yes. a config file definitely sounds nicer but that would be a separate change. could be done as first part of the same MR, though, if you like | 11:05 |
| jennis | Let's do that as a separate change after this one | 11:05 |
| jennis | Is there an issue for a config script? | 11:06 |
| juergbi | i don't think so | 11:06 |
| jennis | Ok, I think it'll be better for me if I write up an issue for that and deal with that in another MR after this one | 11:08 |
| jennis | or do you disagree? | 11:08 |
| Nexus | tristan: i was going to, whe nthe workspace is opened, search for any cached build trees for the element and if found, stage those instead of sources | 11:08 |
| gitlab-br-bot | buildstream: merge request (tristan/main-process-errors->master: Handle errors in main process gracefully) #333 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/333 | 11:08 |
| juergbi | jennis: either way is fine with me | 11:09 |
| *** valentind has quit IRC | 11:09 | |
| *** aday has quit IRC | 11:10 | |
| *** aday has joined #buildstream | 11:12 | |
| jennis | ok juergbi, so you suspect in terms of actually implementing a solution to deal with #136, main_recieve() is where the magic needs to happen? | 11:12 |
| *** toscalix has joined #buildstream | 11:13 | |
| jennis | *receive_main() | 11:13 |
| juergbi | jennis: we trigger it there (unless we go the cron job route) | 11:13 |
| juergbi | at least part of the implementation could make sense in some generic ostree util class to support code sharing with local artifact cache expiry | 11:14 |
| jennis | ok so that functions is executed for every push? | 11:14 |
| juergbi | yes | 11:14 |
| *** toscalix has quit IRC | 11:14 | |
| juergbi | jennis: please note that we have to be careful with regards to concurrency | 11:14 |
| jennis | not sure what you mean with that last point | 11:14 |
| tristan | Nexus, I dont think "any cached build trees" makes sense in this context, rather; if the artifact is cached, that is the only one which is appropriate to use | 11:15 |
| jennis | Oh wrt sharing tlater's function? | 11:15 |
| juergbi | yes | 11:15 |
| tristan | Nexus, just clarifying, the plurality in your sentence was a bit worrying | 11:15 |
| Nexus | tristan: ah no, i was generalising more than pluralising | 11:15 |
| juergbi | jennis: regarding concurrency, i.e., multiple pushes can be happening at the same time. we must ensure not to corrupt the repo etc | 11:15 |
| * Nexus may have just invented a word | 11:16 | |
| *** toscalix has joined #buildstream | 11:16 | |
| tristan | Nexus, sure, just making sure we're not confusing this with incremental builds, where there is some plausible plurality :) | 11:16 |
| tristan | Nexus, also; one problem you're going to encounter is the special sauce which we delegate to Source implementations when opening a workspace | 11:17 |
| tristan | Nexus, I suppose you will propose some plan of action for that... | 11:17 |
| *** toscalix has quit IRC | 11:17 | |
| *** toscalix has joined #buildstream | 11:18 | |
| gitlab-br-bot | buildstream: merge request (issue-243_dpkg_import_source_plugin->master: Created DPKG Source plugin for Issue #10) #305 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/305 | 11:19 |
| jennis | juergbi, right ok. That's something I'll worry about once I have *something* | 11:20 |
| jennis | but thank you for making me aware | 11:20 |
| jennis | I stil feel unsure where to begin with this though. From what we've discussed, it sounds like I either go for an implementation of a "check" within receive_main() or implement some kind of check in an ostree util class elsewhere? | 11:22 |
| juergbi | you may have to coordinate with tlater | 11:26 |
| juergbi | for the part that is not shareable, you can put it in pushreceive.py for now. could be moved later if necessary | 11:26 |
| * Nexus is getting really sick of this error: ` The requested URL returned error: 502 Bad Gateway` | 11:27 | |
| Nexus | now 500 :p | 11:45 |
| tristan | jmac, added my comments to MR !248 ... in general it looks good :) | 12:14 |
| jmac | Cool, thanks tristan. | 12:15 |
| *** mohan43u has quit IRC | 12:15 | |
| tlater | jennis: Sorry, wasn't around for a little while back there | 12:18 |
| tlater | Do tell if you get to implementing cache expiry | 12:19 |
| tlater | I'm already adding a lot of helper functions to _ostree.py, so we should make sure we're not duplicating too much work. | 12:19 |
| juergbi | tristan: !333 looks good to me. the FIXME comment appears to stop in the middle of a sentence | 12:30 |
| juergbi | already adding this test case and improving the output later is certainly fine with me | 12:30 |
| tristan | hmm | 12:31 |
| tristan | ah, good catch | 12:32 |
| jmac | What was the upshot of our YAML type conversion work? juergbi mentioned ' the common routine for enforcing type constraints ' in a review but I haven't found it yet | 12:34 |
| juergbi | jmac: i essentially meant the same as tristan in his comment later, use node_get_member() | 12:36 |
| juergbi | where you pass the expected type as argument | 12:36 |
| juergbi | further down the stack this will call yaml.node_get(), if i'm not mistaken | 12:36 |
| jmac | Right | 12:37 |
| tristan | jmac, juergbi: Note that, similarly to how I have changed project.py; these calls to self.node_get_member() (or _yaml.node_get()) should *not* be providing any default arguments | 12:41 |
| tristan | Instead, the defaults for sandbox: should be added to data/projectconfig.py | 12:41 |
| juergbi | right | 12:42 |
| juergbi | don't want to duplicate defaults | 12:42 |
| tristan | This is a general rule, for anything which has a default; it should show up there (not true for options, or things which dont really have defaults) | 12:42 |
| tristan | right, no duplication of defaults and more to the point: Any default value should show up there for documentation purposes | 12:42 |
| juergbi | yes, that's already the case in this branch | 12:43 |
| tristan | :) | 12:43 |
| *** mohan43u has joined #buildstream | 12:44 | |
| gitlab-br-bot | buildstream: merge request (214-filter-workspacing->master: Make workspace commands on a filter element transparently pass through to their build-dependency) #317 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/317 | 12:44 |
| gitlab-br-bot | buildstream: merge request (214-filter-workspacing->master: Make workspace commands on a filter element transparently pass through to their build-dependency) #317 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/317 | 12:44 |
| jmac | node_get_member only exists for plugins, doesn't it? | 12:45 |
| jmac | Oh, Element is one | 12:45 |
| juergbi | Element is a subclass of Plugin | 12:45 |
| jmac | n/m | 12:45 |
| gitlab-br-bot | buildstream: merge request (jjardon/contributing->master: Add HACKING document to official docs) #332 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/332 | 12:48 |
| gitlab-br-bot | buildstream: issue #303 ("Document how to contribute to the official documentation") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/303 | 12:49 |
| gitlab-br-bot | buildstream: issue #303 ("Document how to contribute to the official documentation") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/303 | 12:49 |
| gitlab-br-bot | buildstream: merge request (jjardon/contributing->master: Add HACKING document to official docs) #332 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/332 | 12:49 |
| jmac | tristan: You have to provide a default argument to the first _yaml.node_get(); that's just reading the user-supplied config and will fault if the key is not there and there is no default | 12:56 |
| tristan | jmac, I dont think so, you only call that *after* all the composition has occurred, right ? | 12:57 |
| tristan | in Element.__extract_sandbox() or such | 12:58 |
| jmac | But __extract_* does the composition | 12:58 |
| jmac | node_get is the second thing done in all those functions | 12:58 |
| jmac | Try removing the default from __extract_environment, for example | 12:58 |
| tristan | Ah *that* line | 12:59 |
| jmac | node_get_member should be fine without it | 13:00 |
| tristan | jmac, right, for *that* line, which is getting the defaults declared by the element type | 13:00 |
| tristan | you will need the default value | 13:00 |
| tristan | jmac, after the compositions, no `default=` is needed | 13:01 |
| jmac | Yes | 13:01 |
| *** mohan43u has quit IRC | 13:05 | |
| gitlab-br-bot | buildstream: merge request (tristan/main-process-errors->master: Handle errors in main process gracefully) #333 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/333 | 13:08 |
| tristan | Ok problem... https://gitlab.com/BuildStream/buildstream/-/jobs/58643211 | 13:26 |
| tristan | jennis, has anything changed in documentation ? that is now failing for the autogenerated buildstream.sandbox.rst, which should not be an orphan | 13:27 |
| tristan | last pipeline for master however; passes: https://gitlab.com/BuildStream/buildstream/pipelines/19211390 | 13:28 |
| jennis | tristan, not afaik | 13:58 |
| jennis | The only changes to docs were the theme change + toc tree alterations + title/typo fixes | 13:58 |
| *** tristan has quit IRC | 14:03 | |
| gitlab-br-bot | buildstream: issue #311 ("Opening a workspace with a cached build") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/311 | 14:04 |
| jennis | tristan, I will review the toc tree changes as that's likely caused it | 14:05 |
| Nexus | juergbi: hows this look to you? https://gitlab.com/BuildStream/buildstream/issues/311 | 14:05 |
| Nexus | in terms of ideas | 14:05 |
| *** tristan has joined #buildstream | 14:08 | |
| gitlab-br-bot | buildstream: merge request (tristan/main-process-errors->master: Handle errors in main process gracefully) #333 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/333 | 14:15 |
| tristan | jennis, ok - so what is happening is sphinx got an update, last pipeline to master I pushed also failed | 14:18 |
| tristan | I have to hunt it down and pin the version for CI as a first step | 14:18 |
| jennis | Ok so the sphinx update changed the way toctree is defined/used? | 14:24 |
| tristan | I dont know yet what it is | 14:28 |
| tristan | have to fix CI hours ago | 14:28 |
| tristan | chasing clock | 14:28 |
| tristan | first fix CI, then open bug | 14:28 |
| Nexus | tristan: https://gitlab.com/BuildStream/buildstream/issues/311 | 14:29 |
| Nexus | i made an issue for the task i'm on | 14:29 |
| Nexus | anything to add/thoughts? | 14:29 |
| tlater | Nexus: A flag for this could be `bst checkout --clean` | 14:31 |
| Nexus | for which? | 14:32 |
| tlater | And have it automatically stage a build tree, except when --clean is set | 14:32 |
| Nexus | i see | 14:32 |
| tlater | Basically your second option but inverted | 14:32 |
| tristan | Ok that worked: https://gitlab.com/BuildStream/buildstream/pipelines/19262761 | 14:34 |
| Nexus | tlater: ok added that option | 14:36 |
| Nexus | Should it fail if it doesn't find a build tree or try to stage the sources? | 14:36 |
| Nexus | tlater: your option seems a bit tempramental to me | 14:37 |
| gitlab-br-bot | buildstream: merge request (tristan/main-process-errors->master: Handle errors in main process gracefully) #333 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/333 | 14:38 |
| jennis | tristan, may I ask why/how you picked 1.7.1? | 14:43 |
| Nexus | last known good? | 14:43 |
| gitlab-br-bot | buildstream: issue #312 ("Documentation does not build with latest version of sphinx") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/312 | 14:43 |
| jennis | tristan, ignore that | 14:43 |
| tristan | docs bug filed here: https://gitlab.com/BuildStream/buildstream/issues/312 | 14:44 |
| gitlab-br-bot | buildstream: merge request (tristan/git-inconsistent-submodules->master: Ignore inconsistent git submodules) #334 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/334 | 14:47 |
| gitlab-br-bot | buildstream: merge request (tristan/main-process-errors->master: Handle errors in main process gracefully) #333 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/333 | 14:53 |
| *** Prince781 has joined #buildstream | 14:54 | |
| jmac | I can't figure out why https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/element.py#L40 is not 'from .sandbox import SandboxFlags' | 14:55 |
| jmac | I can't import other classes from sandbox.py unless I say `from .sandbox`, but this seems to be able to import it from '.'. | 14:55 |
| tlater | Nexus: I think it should always stage the sources if it can't find a build tree - I don't think the user expects an exception when he just wants to see what's in an element | 14:55 |
| tlater | The only case in which there would be no build tree is if the element has never been built; I don't think the user cares whether they've ever built the element before. | 14:56 |
| tristan | juergbi, this fixes the issue I encountered this morning which set the world on fire: https://gitlab.com/BuildStream/buildstream/merge_requests/334 | 14:58 |
| tristan | juergbi, I need to land that pronto, would you like to take a quick look ? | 14:58 |
| jennis | jmac, possibly the way its declared in __init__.py? | 14:59 |
| jennis | https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/__init__.py#L26 | 14:59 |
| tristan | jmac, as jennis says, we do some generalization in the __init__.py of these sub-modules | 15:00 |
| tristan | juergbi, found one error in my message .format() string, re-pushing 334 | 15:03 |
| gitlab-br-bot | buildstream: merge request (tristan/git-inconsistent-submodules->master: Ignore inconsistent git submodules) #334 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/334 | 15:09 |
| *** mohan43u has joined #buildstream | 15:13 | |
| *** brlogger has joined #buildstream | 15:18 | |
| abderrahim[m] | Hi. I'm trying to build a module that uses a different build system but exposes a configure and Makefile that are mostly compatible with autotools, except they don't ignore unknown arguments that I have in conf-global. What is the best way to deal with this. | 15:20 |
| * abderrahim[m] almost typed bst instead of best | 15:20 | |
| abderrahim[m] | (which would have been correct) | 15:20 |
| abderrahim[m] | should I just use kind: manual? | 15:21 |
| *** mohan43u has joined #buildstream | 15:25 | |
| *** mohan43u has quit IRC | 15:27 | |
| * jonathanmaw 's knowledge of how autotools works is a bit spotty, but if autotools doesn't work for this particular module, I'd suggest manual, yeah. | 15:30 | |
| *** mohan43u has joined #buildstream | 15:35 | |
| jmac | jennis: Yeah, I spotted that - but I couldn't get that to work for another class I added | 15:35 |
| jmac | I was trying to add 'SandboxConfig' alongside 'SandboxFlags' and import it, but couldn't get it to import in the same way | 15:36 |
| *** ernestask has joined #buildstream | 15:36 | |
| jennis | jmac, is SandboxConfig a class you're writing? Can't find it | 15:39 |
| juergbi | tristan: hm, i thought git submodule add creates/updates the .gitmodules and nothing else | 15:40 |
| juergbi | does it do anything else? | 15:40 |
| jmac | jennis: Yes, I'm adding it | 15:42 |
| jmac | I'll push my new version to demonstrate shortly | 15:42 |
| jennis | jmac, ok sure | 15:43 |
| tristan | jennis, I've commented on *everything* now... THANKS for taking control here ! | 15:44 |
| tristan | juergbi, only if you call `git submodule add` | 15:44 |
| jennis | uhh nice, thanks tristan | 15:45 |
| tristan | juergbi, when you are an idiot that just manually updated the .gitmodules file, hell breaks lose and buildstream doesnt handle it | 15:45 |
| juergbi | tristan: i'm still confused. what does 'git submodule add' do beyond updating the .gitmodules file? | 15:45 |
| tristan | juergbi, OR it can also happen when you explicitly set submodules URLs ... and you fast forward to a commit that doesnt happen, again inconsistent | 15:45 |
| tristan | Ok... lemme explain | 15:45 |
| tristan | juergbi, when you already have a submodule in play... and then say, you want to update your project to use a new HEAD for the submodule... you have to do `git commit` in the parent project | 15:46 |
| tristan | That commit doesnt modify any files | 15:46 |
| tristan | it modifies internal "tree" thingamagigie | 15:47 |
| juergbi | ah, the commit sha is not in .gitmodules, right | 15:47 |
| tristan | that state is required for it to really be a submodule | 15:47 |
| juergbi | so that state is missing | 15:47 |
| tristan | which means, we cannot check it out or stage it | 15:47 |
| juergbi | right | 15:48 |
| tristan | *IF* we were to do a `git submodule add` ourselves, it would result in git trying to play smart and download it (bad for us)... and worse, it will modify the local tree, so it's no longer the correct ref | 15:48 |
| tristan | the only correct thing I can think of doing for this, is emit a nice warning, and ignore the submodule | 15:49 |
| juergbi | or fail completely, yes | 15:49 |
| tristan | It would *anyway* be ignored by `git submodule init && git submodule update` | 15:49 |
| juergbi | if they ignore it, it sounds sensible to ignore it as well (with the warning) | 15:49 |
| tristan | So I presume, ignoring matches most closely what would happen if a user tried to build it on their host | 15:49 |
| juergbi | ok, looks good to me | 15:52 |
| juergbi | fixup should obviously be squashed | 15:52 |
| tristan | wooops | 15:52 |
| tristan | done. | 15:53 |
| * tristan was a ninja today | 15:54 | |
| *** gitlab-br-bot has joined #buildstream | 15:55 | |
| tristan | oops dont merge... | 15:58 |
| tristan | Fixed commit message | 15:59 |
| gitlab-br-bot | buildstream: merge request (tristan/git-inconsistent-submodules->master: Ignore inconsistent git submodules) #334 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/334 | 15:59 |
| tristan | juergbi, this fixes #299 :) | 15:59 |
| juergbi | great | 15:59 |
| juergbi | #299 was abused a bit for two issues, though | 16:00 |
| tristan | yeah, I personally tend to want to be a hard ass about that stuff | 16:02 |
| tristan | but I need to learn to be lenient towards users who file reports :) | 16:02 |
| tristan | even if I expect them to be developers :) | 16:02 |
| juergbi | ;) | 16:08 |
| gitlab-br-bot | buildstream: issue #299 ("Build sometimes hangs forever after "Unknown exception in SIGCHLD handler"") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/299 | 16:12 |
| gitlab-br-bot | buildstream: merge request (tristan/git-inconsistent-submodules->master: Ignore inconsistent git submodules) #334 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/334 | 16:12 |
| jonathanmaw | abderrahim[m]: after having a look at conf-global, if you have your bst file redefine conf-global, then you can make sure that it doesn't have the args that it's failing to ignore | 16:15 |
| jonathanmaw | that way you'd be able to re-use more of autotools' functionality | 16:15 |
| abderrahim[m] | jonathanmaw: thanks. I'm just not sure whether it's okay to redefine conf-global in a module. | 16:16 |
| tristan | abderrahim[m], sec... | 16:17 |
| tristan | abderrahim[m], it's not desirable to do so no, it can be done, though | 16:18 |
| tristan | abderrahim[m], Basically, conf-global is *intended* to be used in project.conf, to say something like "--enable-gtk-doc" or not, for the *whole project* at once | 16:19 |
| tristan | abderrahim[m], conf-local adds to these configure args | 16:19 |
| abderrahim[m] | it's for gnome-build-meta | 16:19 |
| tristan | I just re-read what you said above... | 16:20 |
| * tristan had missed that part | 16:20 | |
| abderrahim[m] | I wouldn't want it to be rejected at review time ;-) | 16:20 |
| tristan | Well - that is a bit tricky, which build system is this anyway ? | 16:20 |
| abderrahim[m] | waf | 16:20 |
| abderrahim[m] | for samba | 16:21 |
| tristan | Aha, that is not entirely uncommon, I've heard of it before at least | 16:21 |
| tristan | abderrahim[m], You could upstream a waf plugin to buildstream, modeled after autotools, it is really quite easy to do | 16:22 |
| tristan | It's just some boilerplate python, and a .yaml file (and a one line patch to include it in the docs) | 16:23 |
| abderrahim[m] | except I have to be careful, the standard autoconf arguments (--libdir et al.) were a plugin that the author can enable in the build description | 16:24 |
| abderrahim[m] | so they aren't always available | 16:24 |
| abderrahim[m] | (at least it was like this last time I touched waf) | 16:25 |
| tristan | Hmm, that could be handled with boolean variables, and conditional shell checks | 16:26 |
| tristan | abderrahim[m], if it's a lot of work though, I think I would prefer a manual element in gnome-build-meta | 16:26 |
| tristan | that is reliable, straight forward, only downside is it's a bit verbose | 16:27 |
| abderrahim[m] | tristan: yes, that's what I think | 16:28 |
| * abderrahim[m] is now debugging a build failure (anyone knows where ID_EFFECTIVE might be defined?) | 16:30 | |
| tristan | abderrahim[m], not at all :) | 16:49 |
| tristan | hehe | 16:50 |
| abderrahim[m] | Thanks anyway :D | 16:50 |
| abderrahim[m] | it looks that it is in some unix, and the build system couldn't find the linux specific function because of the sandbox | 16:51 |
| juergbi | abderrahim[m]: because it runs as unprivileged root? | 16:52 |
| abderrahim[m] | "Failed to set gid privileges to (-1,1) now set to (0,0) uid=(0,0)" | 16:52 |
| abderrahim[m] | this is the message I found in the logs | 16:52 |
| juergbi | i.e., as normal user it doesn't attempt setuid() at all but because it's uid 0 it tries but fails | 16:53 |
| abderrahim[m] | It seemes it is using SYS_setresuid | 16:53 |
| juergbi | which module is this? | 16:54 |
| *** gitlab-br-bot has quit IRC | 16:54 | |
| abderrahim[m] | samba | 16:55 |
| juergbi | abderrahim[m]: right, i removed the getuid() != 0 check in source3/lib/util_sec.c to work around this | 16:56 |
| juergbi | i.e., never test it at configure time instead of the upstream inconsistency between uid zero and uid non-zero | 16:56 |
| abderrahim[m] | juergbi: thanks, will try | 16:57 |
| *** gitlab-br-bot has joined #buildstream | 16:59 | |
| jmac | If I'm adding a NEWS item but not increasing the Buildstream version, do I add it to the current version (1.1.2)? | 17:00 |
| juergbi | jmac: as 1.1.2 is not released yet, yes | 17:04 |
| jmac | Ah, I see | 17:04 |
| *** valentind has joined #buildstream | 17:09 | |
| *** gitlab-br-bot has quit IRC | 17:15 | |
| *** gitlab-br-bot has joined #buildstream | 17:20 | |
| jmac | !318 (the current build-uid MR) has been updated - I've addressed everything on there but there were a few not-quite-resolved conversations on it. | 17:22 |
| juergbi | ok, will take a look, but possibly tomorrow morning | 17:28 |
| juergbi | if you no longer consider it WIP, please update the title accordingly | 17:30 |
| tristan | juergbi, has his claws in every codebase... samba !? | 17:33 |
| juergbi | noticed this building my distro with buildstream | 17:33 |
| tristan | hahaha | 17:35 |
| tristan | here comes a new distro ! | 17:35 |
| juergbi | i've completed the buildstream build for the default desktop package set, incl. e.g. libreoffice | 17:36 |
| jmac | Yeah, no rush | 17:36 |
| juergbi | but i'm not actually using it yet. will require some more work. on a future weekend ;) | 17:37 |
| gitlab-br-bot | buildstream: merge request (jmac/build-uid-2->master: WIP: Sandbox configuration key to specify uid/gid for sandbox) #318 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/318 | 17:38 |
| jmac | I'll mark it non-WIP when the tests pass | 17:38 |
| juergbi | makes sense | 17:38 |
| *** Prince781 has quit IRC | 17:56 | |
| *** dominic has quit IRC | 17:58 | |
| *** jonathanmaw has quit IRC | 18:03 | |
| *** toscalix has quit IRC | 18:09 | |
| *** toscalix has joined #buildstream | 18:10 | |
| gitlab-br-bot | buildstream: merge request (jmac/build-uid-2->master: Sandbox configuration key to specify uid/gid for sandbox) #318 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/318 | 18:14 |
| *** toscalix has quit IRC | 18:18 | |
| *** gitlab-br-bot has quit IRC | 18:28 | |
| *** ernestask has quit IRC | 20:00 | |
| *** valentind has quit IRC | 22:14 | |
| *** aday has quit IRC | 22:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!