| *** Prince781 has joined #buildstream | 00:40 | |
| gitlab-br-bot | buildstream: issue #383 ("AssertionError when caching artifact") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/383 | 01:51 |
|---|---|---|
| *** bochecha_ has joined #buildstream | 04:57 | |
| *** Prince781 has quit IRC | 05:59 | |
| *** bochecha_ has quit IRC | 07:16 | |
| *** tristan has joined #buildstream | 07:36 | |
| *** toscalix has joined #buildstream | 08:01 | |
| *** tristan has quit IRC | 08:08 | |
| *** tristan has joined #buildstream | 08:59 | |
| *** bethw has joined #buildstream | 09:12 | |
| *** dominic has joined #buildstream | 09:12 | |
| *** aday has joined #buildstream | 09:15 | |
| *** tristan has quit IRC | 09:38 | |
| *** tristan has joined #buildstream | 09:51 | |
| *** jsgrant has quit IRC | 10:05 | |
| toscalix | Hi, how many times my mail made it to the mailing list? I have 3 copies | 10:07 |
| jmac | One so far for me toscalix | 10:08 |
| toscalix | Subject - Gitlab segmentation: subgroups | 10:08 |
| toscalix | thanks | 10:08 |
| paulsherwood | one for me also | 10:15 |
| toscalix | thanks | 10:18 |
| tlater | skullman: I'm aware of the issues you pointed out in !347 - I had a discussion with tristan a little while ago about that | 10:33 |
| tlater | tristan is very opposed to introducing lockfiles anywhere | 10:33 |
| tlater | I suspect it is *very* unlikely for anyone to run into actual issues here though, since you'd essentially need to be working on a single project that by itself is large enough to fill the entire cache | 10:34 |
| tlater | *And* try to build two pipelines at the same time | 10:34 |
| tlater | We might definitely delete the wrong artifact in a pretty unlucky scenario though. | 10:35 |
| tlater | In that case one of the builds will fail, which is quite bad. | 10:35 |
| tlater | We can try to add a lockfile, but those make us vulnerable to breaking cache cleanup entirely if buildstream crashes. | 10:36 |
| paulsherwood | what's the problem with lockfiles? | 10:38 |
| valentind | How important is it to keep the syntax of "bst workspace open" like it is? I think the proposal of introducing "--multiple" is not really great using click. It would make also bash completion difficult. | 10:38 |
| tlater | paulsherwood: We can't guarantee that we'll remove lock files, because unexpected crashes can happen at any time. | 10:39 |
| paulsherwood | so what? be idempotent | 10:39 |
| paulsherwood | ybd does this... the code is admittedly ugly, but the lockfile approach is bulletproof | 10:40 |
| * tlater isn't sure how he'd go about this | 10:41 | |
| paulsherwood | tlater: fwiw folks should always test by forcing crashes imo :-) | 10:41 |
| *** tristan has quit IRC | 10:41 | |
| tlater | Can't we always end up with a scenario in which we don't know whether another thread is running or if the application crashed? | 10:42 |
| * tlater googles lockfile handling strategies | 10:43 | |
| paulsherwood | iirc, if we try to delete a lockfile that's held open by another thread, we fail | 10:43 |
| paulsherwood | so we can only clean up lockfiles that are debris | 10:43 |
| paulsherwood | (provided we're handling permissions correctly) | 10:43 |
| tlater | Ah, that's a cute approach | 10:44 |
| * tlater wasn't aware | 10:44 | |
| paulsherwood | this was explained to me by an expert colleague by the way... i wouldn't have known to do it myself | 10:44 |
| Nexus | is _artifactcache/pushreceive.py, the `encode message` method validates the message. Isn't this already done in the `validate` method? | 10:53 |
| Nexus | in* | 10:53 |
| Nexus | it just seems a bit wasteful | 10:53 |
| jmac | Nexus: Which validation are you talking about in encode_message? | 10:56 |
| Nexus | 125 if not isinstance(command, PushCommand): | 10:57 |
| Nexus | 126 raise PushException('Command must by GLib.Variant') | 10:57 |
| jmac | That validates that the passed command is a PushCommand; PushCommand.validate validates the contents of it | 10:58 |
| jmac | What's confusing is that both use 'command' as a variable, but they mean different things | 10:59 |
| *** tristan has joined #buildstream | 10:59 | |
| jmac | IMO it should be PushCommand.validate(command_type, args) | 10:59 |
| Nexus | there's also a typo in the error, and shouldn't it be "Command must be a PushCommand"? | 11:00 |
| jmac | No, that's correct | 11:01 |
| Nexus | which part, the "typo" or the message? | 11:01 |
| jmac | The message | 11:02 |
| jmac | 'Message command must be PushCommandType' is the correct error | 11:02 |
| Nexus | no i mean raise PushException('Command must by GLib.Variant') | 11:02 |
| Nexus | on line 126 | 11:02 |
| jmac | aha. Right, yes, that error message is wrong and your suggestion is better | 11:03 |
| Nexus | can a quick fix to that be pushed onto master? | 11:04 |
| Nexus | juergbi maybe? | 11:04 |
| juergbi | sure | 11:06 |
| Nexus | juergbi: want me to write it or will you? | 11:06 |
| juergbi | Nexus: I don't mind either way, your choice | 11:08 |
| Nexus | kk, making a quick mr | 11:11 |
| Nexus | gitlab is being so slow | 11:12 |
| jmac | "https://buildstream.gitlab.io/buildstream" is returning a 404 for me atm | 11:12 |
| *** tristan has quit IRC | 11:16 | |
| gitlab-br-bot | buildstream: merge request (pushreceive_message_fix->master: Modified incorrect error message) #443 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/443 | 11:27 |
| Nexus | there we go | 11:27 |
| Nexus | that was a lot harder than it should have been | 11:27 |
| Nexus | juergbi: ^ | 11:31 |
| juergbi | Nexus: please follow the commit message rules, i.e., prefix the commit summary with the filename for such a localized change | 11:33 |
| juergbi | can help a lot when reading the history | 11:33 |
| gitlab-br-bot | buildstream: merge request (pushreceive_message_fix->master: Modified incorrect error message) #443 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/443 | 11:36 |
| Nexus | juergbi: better? | 11:36 |
| juergbi | ta, marked for merge | 11:37 |
| Nexus | ty :) | 11:37 |
| *** bethw has quit IRC | 11:43 | |
| tlater | Nexus: I think you're the first to make it onto the 1.4 contributor list? | 11:55 |
| *** jonathanmaw has joined #buildstream | 11:56 | |
| gitlab-br-bot | buildstream: merge request (pushreceive_message_fix->master: Modified incorrect error message) #443 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/443 | 11:57 |
| Nexus | tlater: woo \o/ | 12:17 |
| * tlater finally found a way to ask ostree the size of a commit :) | 12:17 | |
| Nexus | nicely? | 12:17 |
| Nexus | sarcastically? | 12:18 |
| tlater | Nicely. | 12:18 |
| tlater | Looks like it isn't uncommon for all space to sort of go into one commit though | 12:19 |
| tlater | I wonder if this is because ostree deduplicates data or if it doesn't quite list the commits the way I want it to... | 12:19 |
| tlater | Do we have some large repository that I could try my script on? | 12:20 |
| oknf[m] | docs are broken? | 12:24 |
| tlater | oknf[m]: It does indeed look lik eit | 12:25 |
| tlater | *like it | 12:25 |
| tlater | Seems like, at least, gitlab is having issues. | 12:26 |
| * tlater checks the pages config | 12:26 | |
| tlater | Yeah, our configuration looks fine, deploy is working as usual, and our CI puts files in public/ as it should | 12:29 |
| oknf[m] | just fyi: I was reading those docs just moments ago | 12:30 |
| tlater | oknf[m]: Looks like it's not just us: https://pages.gitlab.io/jekyll | 12:31 |
| tlater | ^ One of their sample projects | 12:31 |
| oknf[m] | ok, thanks for looking at this. I will switch to source then, or is there some alternative location with build docs? | 12:32 |
| tlater | Unfortunately not, just the gitlab pages. You can build the docs locally and read them as usual though. | 12:32 |
| *** bethw has joined #buildstream | 12:52 | |
| jmac | Docs appear to be back online. | 12:56 |
| *** finn has joined #buildstream | 13:01 | |
| Nexus | who's the funny guy that made the hello method return 'a{ss}'? | 13:12 |
| tlater | Heh, I think that's a valid GLib.Variant string | 13:19 |
| tlater | So perhaps that's entirely unintentional, Nexus | 13:19 |
| * Nexus questions that these things ever happen by accident | 13:21 | |
| Nexus | Are remote cache artifacts stored as tars? | 13:33 |
| jmac | Nope | 13:33 |
| jmac | They're uploaded as tars, but extracted by the receive process and stored as normal files | 13:34 |
| * Nexus tosses that theory out of the window | 13:34 | |
| jennis | Nexus, see send_putobjects and receive_putobjects() in pushreceive | 13:35 |
| Nexus | wait. so, it leaves our system as a tar? | 13:35 |
| Nexus | yaeh thats what i'm looking at | 13:35 |
| jmac | Yes. | 13:35 |
| Nexus | ahh | 13:35 |
| Nexus | is if i want to add something new to the remote cash, i need to add it to that tar? | 13:35 |
| Nexus | so* | 13:35 |
| jmac | Yes, or send a new tar | 13:35 |
| Nexus | how would that work? if i added a new tar to an existing artifact? | 13:36 |
| jennis | So the script works out a list of blobs (objects) that are present in the local cache but NOT in the remote cache | 13:39 |
| jennis | We then create a TarInfo object for each blob | 13:39 |
| jennis | Work out the size and and mtime, and add it to the tar file (which is open for writing) | 13:40 |
| jennis | May be useful: https://docs.python.org/3.5/library/tarfile.html | 13:45 |
| Nexus | thanks :) | 13:47 |
| jennis | So juergbi, if my understanding is correct, CAS is going to function as an artifacts cache AND an "ingredients for a build" cache? | 13:49 |
| jennis | Will phase 1 include the latter? | 13:49 |
| juergbi | jennis: what exactly do you mean with the latter? sources? | 13:50 |
| *** tristan has joined #buildstream | 13:50 | |
| jennis | To remotely execute the build command, we have to upload all of the inputs to the CAS, right? So yeah sources would be one, I was thinking maybe also elements? | 13:51 |
| juergbi | the dependencies are artifacts as well, so they are already covered by the artifact cache function. however, yes, besides sources we'll also need to add the result of 'virtual staging' (generating a single directory tree from all dependencies) | 13:53 |
| juergbi | these parts are _not_ planned for the initial CAS MR | 13:53 |
| juergbi | the initial CAS MR will only cover the artifact cache function | 13:53 |
| jennis | right, perfect, thanks | 13:54 |
| jennis | Remember #buildstream-meetings everyone (: | 13:58 |
| dominic | oh, cheers for the reminder jennis :) | 14:00 |
| Nexus | i remembered but spend 5 minutes guessing the channel server and name xD | 14:01 |
| *** finn has quit IRC | 14:27 | |
| *** Prince781 has joined #buildstream | 14:27 | |
| *** tristan has quit IRC | 14:49 | |
| *** aday has quit IRC | 14:54 | |
| *** aday has joined #buildstream | 14:55 | |
| *** aday has quit IRC | 14:59 | |
| *** aday has joined #buildstream | 14:59 | |
| *** bochecha_ has joined #buildstream | 15:01 | |
| *** tiago has quit IRC | 15:15 | |
| *** aday has quit IRC | 15:23 | |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 15:32 |
| *** finn has joined #buildstream | 15:50 | |
| *** aday has joined #buildstream | 15:58 | |
| *** dominic has quit IRC | 16:03 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #420 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/420 | 16:04 |
| *** finn has quit IRC | 16:06 | |
| *** bochecha_ has quit IRC | 16:06 | |
| *** aday has quit IRC | 16:06 | |
| *** zalupik has quit IRC | 16:06 | |
| *** csoriano has quit IRC | 16:06 | |
| *** m_22[m] has quit IRC | 16:06 | |
| *** segfault3[m] has quit IRC | 16:06 | |
| *** pro[m] has quit IRC | 16:06 | |
| *** jjardon[m] has quit IRC | 16:06 | |
| *** theawless[m] has quit IRC | 16:06 | |
| *** cgmcintyre[m] has quit IRC | 16:06 | |
| *** asingh_[m] has quit IRC | 16:06 | |
| *** oknf[m] has quit IRC | 16:06 | |
| *** alatiera has quit IRC | 16:06 | |
| *** ssssam[m] has quit IRC | 16:06 | |
| *** abderrahim[m] has quit IRC | 16:06 | |
| *** awacheux[m] has quit IRC | 16:06 | |
| *** albfan[m] has quit IRC | 16:06 | |
| *** kailueke[m] has quit IRC | 16:06 | |
| *** rafaelff[m] has quit IRC | 16:06 | |
| *** waltervargas[m] has quit IRC | 16:06 | |
| *** connorshea[m] has quit IRC | 16:06 | |
| *** ernestask[m] has quit IRC | 16:06 | |
| *** mattiasb has quit IRC | 16:06 | |
| *** ptomato[m] has quit IRC | 16:06 | |
| *** inigomartinez has quit IRC | 16:06 | |
| *** aday has joined #buildstream | 16:07 | |
| *** finn has joined #buildstream | 16:07 | |
| *** bochecha_ has joined #buildstream | 16:07 | |
| *** oknf[m] has joined #buildstream | 16:07 | |
| *** albfan[m] has joined #buildstream | 16:07 | |
| *** zalupik has joined #buildstream | 16:07 | |
| *** csoriano has joined #buildstream | 16:07 | |
| *** m_22[m] has joined #buildstream | 16:07 | |
| *** segfault3[m] has joined #buildstream | 16:07 | |
| *** pro[m] has joined #buildstream | 16:07 | |
| *** jjardon[m] has joined #buildstream | 16:07 | |
| *** theawless[m] has joined #buildstream | 16:07 | |
| *** cgmcintyre[m] has joined #buildstream | 16:07 | |
| *** asingh_[m] has joined #buildstream | 16:07 | |
| *** alatiera has joined #buildstream | 16:07 | |
| *** ssssam[m] has joined #buildstream | 16:07 | |
| *** abderrahim[m] has joined #buildstream | 16:07 | |
| *** awacheux[m] has joined #buildstream | 16:07 | |
| *** kailueke[m] has joined #buildstream | 16:07 | |
| *** rafaelff[m] has joined #buildstream | 16:07 | |
| *** waltervargas[m] has joined #buildstream | 16:07 | |
| *** connorshea[m] has joined #buildstream | 16:07 | |
| *** ernestask[m] has joined #buildstream | 16:07 | |
| *** mattiasb has joined #buildstream | 16:07 | |
| *** ptomato[m] has joined #buildstream | 16:07 | |
| *** inigomartinez has joined #buildstream | 16:07 | |
| *** slaf has quit IRC | 16:07 | |
| *** slaf has joined #buildstream | 16:07 | |
| *** slaf has joined #buildstream | 16:08 | |
| *** slaf has joined #buildstream | 16:08 | |
| *** slaf has joined #buildstream | 16:08 | |
| *** slaf has joined #buildstream | 16:09 | |
| *** slaf has joined #buildstream | 16:09 | |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 16:16 |
| tlater | I *think* the expiry stuff is running properly on a separate thread now :) | 16:19 |
| * tlater will need to carefully review once or twice and make sure everything's set for unix, but hopes to finally land this soon | 16:19 | |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 16:36 |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 16:38 |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 16:39 |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 16:39 |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 16:39 |
| laurence | tlater, great news ! | 16:41 |
| *** bochecha_ has quit IRC | 16:44 | |
| *** jonathanmaw has quit IRC | 17:00 | |
| *** toscalix has quit IRC | 17:09 | |
| *** bethw has quit IRC | 17:10 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #420 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/420 | 17:34 |
| jmac | More 404s from gitlab | 17:42 |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #444 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/444 | 17:49 |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #420 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/420 | 17:50 |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #444 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/444 | 18:05 |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #444 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/444 | 18:07 |
| gitlab-br-bot | buildstream: issue #384 ("AssertionError when building element with workspace after clearing cache") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/384 | 18:40 |
| *** Prince781 has quit IRC | 19:07 | |
| *** Prince781 has joined #buildstream | 19:38 | |
| *** Prince781 has quit IRC | 19:41 | |
| *** aday has quit IRC | 20:01 | |
| *** tristan has joined #buildstream | 20:07 | |
| *** cs_shadow has quit IRC | 20:14 | |
| gitlab-br-bot | buildstream: issue #385 ("Should notify when build completes") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/385 | 20:27 |
| *** tristan has quit IRC | 21:12 | |
| gitlab-br-bot | buildstream: issue #386 ("Don't print failure summary when there are no failures") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/386 | 22:18 |
| *** slaf_ has joined #buildstream | 22:26 | |
| *** slaf_ has joined #buildstream | 22:26 | |
| *** slaf_ has joined #buildstream | 22:26 | |
| *** slaf_ has joined #buildstream | 22:26 | |
| *** slaf_ has joined #buildstream | 22:27 | |
| *** slaf_ has joined #buildstream | 22:27 | |
| *** slaf has quit IRC | 22:28 | |
| *** slaf_ is now known as slaf | 22:28 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!