| *** tristan has quit IRC | 04:03 | |
| *** tristan has joined #buildstream | 04:29 | |
| *** ernestask has joined #buildstream | 06:25 | |
| *** dominic has joined #buildstream | 07:28 | |
| * Nexus wishes there was an option to "just show last line of stderr" when a test fails :P 9 times out of 10, it's the only one that matters | 07:29 | |
| gitlab-br-bot | buildstream: merge request (tristan/private-refactor-2->master: More refactoring related to private symbol policy) #355 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/355 | 07:56 |
|---|---|---|
| *** toscalix has joined #buildstream | 07:56 | |
| gitlab-br-bot | buildstream: issue #334 ("unhandled exception when building gtk+-3.bst") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/334 | 08:03 |
| *** slaf has quit IRC | 08:07 | |
| *** slaf has joined #buildstream | 08:09 | |
| *** aday has joined #buildstream | 08:13 | |
| *** dominic has quit IRC | 08:16 | |
| *** noisecell has joined #buildstream | 08:27 | |
| *** jonathanmaw has joined #buildstream | 08:47 | |
| *** bethw has joined #buildstream | 08:52 | |
| *** dominic has joined #buildstream | 08:59 | |
| gitlab-br-bot | buildstream: merge request (tristan/private-refactor-2->master: More refactoring related to private symbol policy) #355 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/355 | 09:03 |
| gitlab-br-bot | buildstream: merge request (tristan/only-one-strict-mode->master: Fix strict setting to be consistent throughout the pipeline.) #356 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/356 | 09:19 |
| tristan | juergbi, can you please take a look at https://gitlab.com/BuildStream/buildstream/merge_requests/356 ? | 09:20 |
| juergbi | sure | 09:22 |
| tristan | juergbi, that's the thing I was a little unsure of this weekend, I think the fix is correct, though | 09:26 |
| tristan | I'm not sure why on earth we would allow strict & non-strict mode in the same pipeline/session | 09:26 |
| juergbi | tristan: some projects might not support non-strict builds at all (static linking) but that should likely be solved differently | 11:07 |
| tristan | juergbi, we have an issue for making strict-rebuild a dependency attribute for that, indeed | 11:08 |
| juergbi | right, besides that I can't think of a sensible use case why you'd want to mix | 11:08 |
| tristan | we have it by element type, but I dont think we want to think of that in terms of project granularity | 11:08 |
| tristan | juergbi, alright, I'll merge then, I was more concerned that we might have special code in state handling for this mixing of policies | 11:09 |
| juergbi | we have the STRICT_REBUILD support. I don't see any issues merging this | 11:10 |
| gitlab-br-bot | buildstream: merge request (tristan/only-one-strict-mode->master: Fix strict setting to be consistent throughout the pipeline.) #356 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/356 | 11:11 |
| gitlab-br-bot | buildstream: issue #337 ("Add option to close all workspaces") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/337 | 11:18 |
| *** bethw has quit IRC | 12:01 | |
| *** bethw has joined #buildstream | 12:01 | |
| tlater | Do we have some standard way of prompting the user yet? | 12:08 |
| tlater | There's app.interrupt_handler | 12:08 |
| tlater | Which seems to halt any ongoing jobs - I suspect that's necessary to be able to prompt | 12:09 |
| tlater | But it's hard to call anything in app from outside that class | 12:09 |
| tlater | And it looks like a few of those methods are required... | 12:09 |
| tristan | tlater, the frontend must remain the frontend; any interaction with the user should be done there | 12:20 |
| tristan | tlater, why specifically ? | 12:21 |
| tlater | tristan: Still working on local cache expiry, which should prompt the user about what to do if the cache is full. | 12:21 |
| tristan | Should it ? | 12:21 |
| *** Prince781 has joined #buildstream | 12:21 | |
| tlater | Though I am starting to wonder whether that's necessary | 12:21 |
| tlater | Yeah... | 12:21 |
| tlater | The issue mentions it | 12:22 |
| tristan | I didnt think it should... where is that... | 12:22 |
| tlater | tristan: Issue is this: https://gitlab.com/BuildStream/buildstream/issues/135 | 12:22 |
| * tlater thinks the intention is not to annoy the user with a failed webkit build after 5h of compilation | 12:22 | |
| tlater | Which is probably the most likely candidate for overflowing caches | 12:23 |
| tristan | ah, I wrote that in interactive mode | 12:23 |
| tlater | Yep, definitely not *always* prompt | 12:23 |
| tristan | Right, that seems like a reasonable feature, but not exactly easy to implement | 12:24 |
| tristan | tlater, I think that first off, we should consider it a separate feature | 12:24 |
| tristan | In order to implement that kind of interaction *from a task*, we're going to need some supporting code | 12:25 |
| tristan | it's certainly possible | 12:25 |
| tlater | tristan: Think you'd be happy landing an initial implementation without that feature? | 12:25 |
| tristan | Certainly | 12:25 |
| tristan | That should be a separate feature | 12:25 |
| tristan | But a very nice one | 12:26 |
| tlater | It could still have settings to configure the default, but I find it hard to see options beyond "Would you like to ignore your limit?" | 12:26 |
| tristan | tlater, we want to avoid blurring the line between frontend and core, I already unfortunately did that with recent workspace changes; but it looks like we could use some kind of "main" in the core | 12:27 |
| tristan | tlater, that would not be a reasonable setting, of course | 12:27 |
| tristan | it would simply ignore the cache quota :) | 12:27 |
| tristan | rendering the quota useless | 12:28 |
| tlater | Yeah, I think adding any sort of option beyond the quota is pointless | 12:28 |
| tristan | To do prompting from a task, we'd need cooperation with the scheduler for a task to ask a question with a well defined API, and have the scheduler suspend and run a callback, *if* the frontend has provided such a callback | 12:29 |
| tristan | all perfectly doable, and a refactor to make that happen would probably happily result in splitting up the ugly pipeline code into API and pipeline resolution stuff separately, ending up with a nicer initialization, and consuming the workspace operations which bled into App | 12:30 |
| tlater | tristan: Yeah, I think adding all of that as part of this branch will be too much | 12:32 |
| tlater | tristan: On another one of those "edge cases" actually, I think implementing a quota that takes disk space into account is not too far-fetched | 12:37 |
| tlater | The problem is just, what's a reasonable hard-coded quota? | 12:37 |
| tlater | In general, what should the default be? | 12:38 |
| * tlater deferred thinking about this, but is close to finishing up, so... | 12:39 | |
| tristan | tlater, I think default should be determined by current cache size + available space on the volume where the artifact cache is stored | 12:40 |
| tristan | i.e. "All the space" | 12:40 |
| tristan | Avoiding erroring out when you run out of space, and expiring LRU artifacts which are not in the build plan along the way | 12:41 |
| tristan | The only out of space condition which remains, is the one where the size of your volume is not big enough to hold the entire build results of the build plan | 12:41 |
| tlater | tristan: Do we want a ceiling on top of the artifact cache to make that less likely? | 12:42 |
| *** noisecell has quit IRC | 12:42 | |
| tlater | i.e., say we stop something like 20GB before we reach the actual disk limit? | 12:42 |
| tristan | rather than a ceiling, I would say a headroom | 12:42 |
| tristan | i.e. we need to allow some extra space for builds to be checked out... and we cannot know for certain how much additional space a build result will take in the cache | 12:43 |
| tristan | however, that second part should be meaningless; because that second part will always be less than the full size of the artifact | 12:44 |
| tristan | in the case that there are push queues enabled, we need headroom to be able to push artifacts (cause we create temporary ostree repos for pushing) | 12:44 |
| tristan | tlater, all of that would be considered as a hard coded headroom that we reserve I think | 12:45 |
| tristan | a couple of GB should be sufficient | 12:45 |
| * tlater unfortunately rarely pays attention to the actual size of builds | 12:45 | |
| tristan | tlater, in other words... default "quota" is "cache size + available space - hard coded headroom" | 12:45 |
| tlater | I'm guessing some large elements might grow up to tens of GB - how much headroom do we actually want? | 12:46 |
| tlater | Are there statistics for this? | 12:46 |
| tristan | We cannot know | 12:46 |
| tristan | assuming we know would be bad, but we have a quota which we need to try to stay under | 12:46 |
| tristan | and if it's unset, defaulting to "all" is sensible | 12:47 |
| tlater | It's always going to be a heuristic with these, because we can't predict the sizes of artifacts... | 12:47 |
| tristan | That is not entirely true | 12:47 |
| tristan | tlater, that is just a consequence of not having support from ostree to work out the exact numbers | 12:47 |
| tristan | well | 12:47 |
| tristan | for the size of a build, that will always be a heuristic | 12:48 |
| tristan | indeed, that is something we cannot predict | 12:48 |
| tristan | I am wondering if we should force the build directory to be on the same volume as the artifact cache | 12:49 |
| tristan | but meh, maybe not | 12:49 |
| tlater | Somebody might have an incredible disk driver that somehow makes performance not struggle too much ;) | 12:50 |
| tristan | Well, it's just not currently a requirement, and might make things more manageable if we removed the configurability of the build dir | 12:51 |
| tlater | Hm, I don't think its configurability really inhibits anything | 12:52 |
| tristan | needing headroom works partly under the assumption that builds will occur on the same volume, I think | 12:52 |
| tlater | Ah, right, you mean we technically might not need headroom if we don't actually do anything on the same volume? | 12:53 |
| * tlater wonders if it's not sensible to keep headroom anyway just to keep other applications on the machine happy | 12:53 | |
| tristan | Well, rather we would want to make sure it's always on the same volume | 12:53 |
| tristan | and never ever make any distinction, I dont think we want to make distinctions about that | 12:54 |
| tlater | Oh, using the headroom as a way to expire artifacts when we need more build space? | 12:54 |
| tristan | Right | 12:54 |
| tlater | I don't think that makes much sense | 12:54 |
| tlater | Hm | 12:54 |
| tristan | tlater, so... to make sure we're still on the same page... we are *still* only talking about the default value for the *artifact cache* quota | 12:54 |
| tlater | Yeah | 12:54 |
| tristan | In this case, it makes sense to default to something which reserves some extra space for builds | 12:55 |
| tristan | and take as much available space as possible | 12:55 |
| tlater | I understand that, but I feel that this should be handled by something else? | 12:55 |
| tristan | Once you've computed that value... it's all internally treated as "a quota of the artifact cache" | 12:55 |
| tristan | Doing something dynamic as the build progresses sounds like overkill | 12:56 |
| tlater | Well, we can't really predict the build size | 12:57 |
| tristan | if you want to build something large, and download 10 torrents of your favorite porn at the same time on the same volume, you could have errors | 12:57 |
| tristan | tlater, we cant predict the build size, but we can reasonably use a default number | 12:57 |
| tlater | Hm, yeah, I suppose this is the best way | 12:58 |
| tlater | It just feels ugly to have the artifact cache determine how much space we need for builds | 12:58 |
| tristan | I think 2GB is sensible, considering someone using buildstream should probably have a lot of free space, and considering the size of SSD on modern work stations | 12:58 |
| tristan | maybe 3GB or smth | 12:59 |
| tlater | tristan: Should that headroom be hard? | 12:59 |
| tristan | <tristan> tlater, so... to make sure we're still on the same page... we are *still* only talking about the default value for the *artifact cache* quota | 12:59 |
| tristan | the default value | 12:59 |
| tristan | of the quota | 12:59 |
| tlater | Absolutely, yes | 12:59 |
| tristan | the artifact cache probably need only ever care about what that quota *is* | 12:59 |
| tristan | i.e., this is probably computed as a default in the initialization of the Context() object | 13:00 |
| tlater | My question is whether we should refuse to add an artifact to the cache if everything is full and there's 2GB left *technically*, but the headroom prohibits it | 13:00 |
| tlater | I guess you're saying yes x) | 13:00 |
| tristan | tlater, I think that case *should* be very hard to hit | 13:01 |
| tristan | so yes :) | 13:01 |
| tristan | tlater, i.e., because when you need to add an artifact, you are going to be expiring LRU artifacts to make space for it, so long as those artifacts are not needed for the currently processed targets | 13:02 |
| tristan | this raises one more interesting point, though; which is more important to think about | 13:02 |
| tlater | Yeah, you'd need to be building essentially a giant mono-repo that's too large for your disk anyway | 13:02 |
| tristan | tlater, parallel builds on the same machine which share the same artifact cache | 13:02 |
| tristan | tlater, re the above, *exactly*, that's the point | 13:03 |
| tlater | ... That's a possibility? :( | 13:03 |
| tristan | parallel builds is a problem, though | 13:03 |
| tristan | We have been very careful about making sure operations never assume one build process for a given user, yes | 13:03 |
| tlater | Yeah, I suppose buildstream isn't a daemon, so just running twice... | 13:03 |
| tristan | that is a very supported thing which would be a shame to throw away | 13:03 |
| tristan | still, LRU approach is decent for it | 13:03 |
| tristan | no daemons please no | 13:04 |
| tristan | and no lock files | 13:04 |
| tlater | Hrm, we'd need to make sure we don't accidentally remove artifacts required for a parallel build | 13:04 |
| * tlater doubts that's possible without lock files | 13:04 | |
| tristan | tlater, probably we need some well communicated policy, it seems impossible to satisfy this case :'( | 13:04 |
| tlater | tristan: Maybe with a *tiny* lockfile? ;p | 13:06 |
| tristan | ugh | 13:06 |
| tristan | I dont think it's worth it | 13:07 |
| tristan | We definitely have to support that people will build multiple projects on the same machine | 13:07 |
| tristan | Building in parallel is less usual or likely | 13:07 |
| tlater | Multiple projects certainly isn't an issue, we could lock individual artifacts | 13:08 |
| tristan | tlater, rather, we want to make sure that we do things atomically and fail gracefully if parallel builds cause havoc like this | 13:08 |
| tristan | you are talking about locking again, which I would rather not get into, if we can avoid it, it's less fuss | 13:08 |
| tlater | Fair | 13:08 |
| tlater | We could check artifact mtimes | 13:08 |
| tlater | And ensure we don't delete anything that was recently modified | 13:09 |
| tristan | We can use lock files, if we are 150% that there is never a crash which leaves the lockfile locked hehe | 13:09 |
| tristan | having unclean state after exit is worse than a graceful failure for a reasonable reason | 13:09 |
| tristan | tlater, marking things as used at the beginning of a build is already a good mitigation | 13:10 |
| tristan | i.e. even if you have not used an artifact *yet*, but you know you will need it... mark it right away at startup | 13:10 |
| tlater | It's not perfect, but I suspect most builds will run longer than a few hours | 13:10 |
| tristan | that will make the case much less likely to hit | 13:10 |
| tlater | s/will/will not/ | 13:10 |
| tristan | most of my builds run for a few hours | 13:11 |
| tristan | but that's me, if you're not building ~400 modules, do you even call it a "build" ? | 13:11 |
| tristan | hehe | 13:11 |
| tlater | Yeah, that's what I meant, if we don't delete anything with an mtime of say 10 hours in the past we should be fine | 13:11 |
| tlater | Then again, someone might just constantly build the same thing | 13:11 |
| tristan | uhhhhh mtime ? | 13:11 |
| tristan | wait a sec, we're not considering mtime are we ? | 13:12 |
| tlater | Why not? | 13:12 |
| tristan | mtime has nothing to do with least recently used | 13:12 |
| tlater | We're currently using mtime on ostree refs to figure out when something was used | 13:12 |
| tristan | nope. | 13:12 |
| tlater | Ah, well, in my branch I manually set the mtime when I extract something | 13:12 |
| tristan | Are we able to *modify* the mtime of a ref ? | 13:12 |
| tlater | I wasn't sure atime was handled correctly | 13:12 |
| tristan | Aha | 13:12 |
| tlater | Yep, we are | 13:12 |
| tristan | Ok, in that case, mtime, but not a number of hours... rather what I am saying is... | 13:13 |
| tristan | At startup, see which artifacts we need to extract, and bump the mtime *right away* | 13:13 |
| tristan | so that when we extract it 2 hours later (and probably bump the mtime again), we've already preemptively made a reservation on that artifact | 13:14 |
| tristan | so a parallel build is much less likely to delete what we need | 13:14 |
| tlater | Ah | 13:14 |
| tlater | I see what you mean | 13:14 |
| tlater | Use the LRU cache as it is already implemented | 13:14 |
| tlater | Yeah, that would work | 13:14 |
| *** noisecell has joined #buildstream | 13:16 | |
| *** xjuan has joined #buildstream | 13:25 | |
| *** tristan has quit IRC | 13:25 | |
| *** tristan has joined #buildstream | 13:30 | |
| *** jonathanmaw has quit IRC | 14:21 | |
| *** jonathanmaw has joined #buildstream | 14:21 | |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 14:21 |
| *** bethw has quit IRC | 15:13 | |
| *** bethw has joined #buildstream | 15:38 | |
| gitlab-br-bot | buildstream: issue #338 ("buildstream fails with "EOFError: Compressed file ended before the end-of-stream marker was reached"") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/338 | 16:09 |
| *** valentind has joined #buildstream | 16:12 | |
| *** Prince781 has quit IRC | 16:16 | |
| *** toscalix has quit IRC | 16:24 | |
| *** ernestask has quit IRC | 16:32 | |
| *** bethw has quit IRC | 16:33 | |
| *** dominic has quit IRC | 16:41 | |
| *** dominic has joined #buildstream | 16:41 | |
| *** noisecell has quit IRC | 16:42 | |
| *** aday has quit IRC | 17:04 | |
| *** aday has joined #buildstream | 17:05 | |
| *** aday has quit IRC | 17:09 | |
| jmac | BuildStream commit 500f4f330fde8b6001a2f8d8921bd5b8acb79960 has broken benchmarks somehow; I'll investigate further later but be aware if you're doing anything with the benchmarks suite | 17:09 |
| *** aday has joined #buildstream | 17:09 | |
| *** jonathanmaw has quit IRC | 17:15 | |
| *** Prince781 has joined #buildstream | 17:57 | |
| *** Prince781 has quit IRC | 18:03 | |
| *** aday has quit IRC | 19:22 | |
| *** aday_ has joined #buildstream | 19:22 | |
| *** aday_ is now known as aday | 19:23 | |
| *** tristan has quit IRC | 20:05 | |
| *** xjuan has quit IRC | 20:06 | |
| *** aday has quit IRC | 21:07 | |
| gitlab-br-bot | buildstream: merge request (chandan/close-all-workspaces->master: _frontend/cli.py: Add option to close multiple workspaces) #357 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/357 | 23:44 |
| gitlab-br-bot | buildstream: merge request (chandan/close-all-workspaces->master: WIP: _frontend/cli.py: Add option to close multiple workspaces) #357 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/357 | 23:46 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!