| *** Prince781 has joined #buildstream | 02:16 | |
| *** Prince781 has quit IRC | 02:21 | |
| *** tristan has joined #buildstream | 03:34 | |
| *** noisecell has joined #buildstream | 06:51 | |
| *** dominic has joined #buildstream | 07:45 | |
| *** toscalix has joined #buildstream | 07:52 | |
| *** dominic has quit IRC | 08:07 | |
| *** jonathanmaw has joined #buildstream | 08:23 | |
| *** aday has joined #buildstream | 08:35 | |
| *** dominic has joined #buildstream | 09:04 | |
| *** bethw has joined #buildstream | 09:06 | |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 09:06 |
|---|---|---|
| *** toscalix has quit IRC | 09:13 | |
| *** toscalix has joined #buildstream | 09:14 | |
| jennis | tristan, if you have some spare time today, would you be able to review !411, !413 and !418. !411 is the only one that is potentially mergeable. For the other two, I would like your thoughts on whether/how we should proceed | 09:39 |
| tristan | jennis, I am aware there is a queue... | 10:00 |
| gitlab-br-bot | buildstream: merge request (tristan/scheduler-private-refactor->master: Scheduler private refactor) #419 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/419 | 11:03 |
| * tlater is annoyed by how much of our code is tangled in trying to meet frontend demands | 11:14 | |
| gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 11:22 |
| gitlab-br-bot | buildstream: issue #366 ("Add `bst config` command") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/366 | 11:45 |
| *** tristan has quit IRC | 11:46 | |
| *** xjuan has joined #buildstream | 11:46 | |
| gitlab-br-bot | buildstream: merge request (richardmaw/key-summary->master: List resolved element keys in job summary) #389 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/389 | 11:47 |
| *** tristan has joined #buildstream | 11:48 | |
| tristan | looks like our walkers are rather crawlers this week :-S | 11:57 |
| tristan | tlater, I've been untangling *alot* of things over the weeks, and API is starting to look much better defined after going through it with a fine tooth comb, under the guise of #285 | 11:58 |
| tristan | tlater, one thing that is important is that we have a clear separation between the frontend and core, that means we can't tollerate the core reaching into the guts of the codebase and doing whatever it likes | 11:59 |
| tlater | Yeah, I do understand that. But it feels like the frontend is reaching really deep into core | 12:00 |
| tristan | tlater, I want to land https://gitlab.com/BuildStream/buildstream/merge_requests/419, and then look into the long queue of docs patches from jennis | 12:00 |
| tristan | tlater, I agree, it counts the queue status things, etc; which sucks, and it has absorbed workspace commands, due to the lack of a proper "main" in the core that is separate from the pipeline | 12:01 |
| tristan | a refactor which needs to be done is to split out all the "command" executing code from Pipeline, and have Pipeline only used to resolve the... pipeline | 12:01 |
| tristan | then we can move workspace things to there, out of the app; and try to provide a cleaner interface for the frontend | 12:02 |
| tristan | tlater, but anyway before I move on to docs patches I wanted to discuss a bit the changes I made to the scheduler, which you will be working on | 12:02 |
| tlater | Sure, looking through it as we speak :) | 12:03 |
| tristan | shame that our "gitlab crawlers" are crawling so slowly | 12:03 |
| tristan | yeah, you can see it before it merges | 12:03 |
| tristan | A huge benefit to solving #285 means we separate what is local private from ... shall we say, local public | 12:03 |
| tristan | which means APIs are easier to understand | 12:03 |
| tristan | tlater, notice that in Job(), there are 3 public members | 12:04 |
| tristan | element, action_name, and workspace_dict; these are *all* specific to jobs which are related to a Queue and process an element | 12:05 |
| tristan | So the resulting Job base class should not have them anymore | 12:05 |
| tlater | Going to be a bit tricky looking at how logging is done | 12:05 |
| tristan | That will have to change to something more generic, indeed | 12:05 |
| tristan | tlater, it can be done for instance, by having an abstract method report the logging file handle | 12:06 |
| tristan | which ElementJob can implement the way it's currently implemented | 12:06 |
| tristan | that would be a start | 12:06 |
| tlater | I don't like it much, feels like I'm littering with very specific helpers | 12:07 |
| tristan | Or, it could be that we move the element.py code for opening the log handle into Job, and pass the Job a logdir relative path for doing it itself | 12:07 |
| tlater | But I haven't been able to think of anything else, so... | 12:07 |
| tristan | the latter sounds better to me, actually | 12:07 |
| tlater | Hang, from where are we passing that path? | 12:08 |
| tlater | Currently the element decides, but it doesn't have much control over the job, does it? | 12:08 |
| tristan | Right | 12:08 |
| tristan | tlater, if we remove all of that from the element, we can make the Queue (which is totally element processing specific) decide on the logdir relative logfile to use | 12:09 |
| tristan | tlater, then we can have the base Job class handle that file | 12:09 |
| tlater | Sounds like a large change, but it definitely is neater | 12:09 |
| tristan | tlater, other mechanisms for launching a job, for artifact cache cleanup jobs, or for event tasks, will pass a different file | 12:09 |
| tristan | tlater, *no* | 12:09 |
| tristan | tlater, *stop thinking that way* | 12:09 |
| tristan | tlater, be ready to churn *alot* | 12:10 |
| tlater | Alright... I won't be lazy this time x) | 12:10 |
| tristan | tlater, we have a pretty lean codebase, which I recently had to untangle many messes | 12:10 |
| tristan | A "large change" is not the dangerous part, the dangerous part is leaving the codebase tangled afterwards | 12:10 |
| tristan | First, think of how it *should* be, then *make it that way* | 12:11 |
| tristan | tlater, another thing I want you to keep in mind... | 12:12 |
| tristan | tlater, is that JobElement tasks are guaranteed by the Queue mechanism to be mutually exclusive per element | 12:12 |
| tristan | this makes a subtle difference, and is an assumption that we can make | 12:12 |
| tristan | The Queue side of things ensures that Elements pass through Queues in order | 12:13 |
| tristan | tlater, probably you want to make a new file in the _scheduler package, which will be responsible for doing the stuff that Scheduler.sched() currently does | 12:14 |
| tristan | i.e. moving elements through queues, would no longer be in the direct domain of the Scheduler | 12:14 |
| tlater | Ah, that's what I've been trying to get working since last Thursday | 12:14 |
| tristan | Right, but you probably want to work from the ground up | 12:14 |
| * tlater just isn't sure why that would be required anymore at this point | 12:14 | |
| tristan | tlater, you need to abstract how the scheduler receives tasks, there are probably various approaches | 12:15 |
| tristan | tlater, One idea would be to make an abstract JobSource... and have this JobSourceQueue thing do that to provide jobs to the scheduler | 12:15 |
| tristan | tlater, I dont care if we create very small files, look at the present Queue implementations, they are tiny but by this virtue, things are very comprehensive | 12:16 |
| tristan | tlater, note that I have not thought this part through, I'm rambling into other topics | 12:17 |
| tristan | or rather, into possible solutions for makeing the scheduler do what we want | 12:17 |
| tlater | Hm, alright. I *thought* it would be possible to get queues to manage multiple jobs. | 12:17 |
| tristan | Nah, Queues are about passing elements through queues | 12:18 |
| tristan | Lets not convolute that with other side effects | 12:18 |
| tlater | Ok | 12:18 |
| *** xjuan has quit IRC | 12:18 | |
| tristan | rather, we need to have something which is able to react on the side effects of elements completing in queues | 12:18 |
| tristan | so that we can launch the right jobs | 12:18 |
| tlater | Hm, right | 12:18 |
| tlater | So the scheduler should become more of a classical worker pool | 12:18 |
| tlater | And we'll need something to figure out what to make this pool do | 12:19 |
| tristan | tlater, it's best if you think of it, and paint a picture and show it to me | 12:19 |
| tlater | Sure, I'll start on it | 12:19 |
| tristan | tlater, I am *sure* about what I have said about splitting up Job and JobElement, that part is straight forward and going to be needed | 12:19 |
| tristan | How the scheduler works this out with queues and all, is a creative task | 12:20 |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #420 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/420 | 12:30 |
| *** xjuan has joined #buildstream | 12:31 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/workspacedir_config->master: Add 'workspacedir' configuration.) #420 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/420 | 12:36 |
| *** bethw has quit IRC | 13:10 | |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_sandbox->master: WIP: docs: Fix build with latest version of sphinx) #414 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/414 | 13:13 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_readme->master: WIP: source/index.rst: Use the README as our intro) #405 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/405 | 13:13 |
| *** bethw has joined #buildstream | 13:26 | |
| cs_shadow | I was thinking about changing the merge request settings to "Fast-forward merge" as that would give us a linear commit history. Any objectons? | 13:29 |
| cs_shadow | @tristan @jjardon ^ | 13:30 |
| tristan | cs_shadow, that's what we use in BuildStream, you mean buildstream-docker-images does not have that setting ? | 13:30 |
| tristan | cs_shadow, are you able to configure it ? | 13:30 |
| tristan | Please do | 13:30 |
| cs_shadow | oh sorry, i forgot to mention the repo. Yes, i meant buildstream-docker-images | 13:31 |
| tristan | cs_shadow, it's the only sane default option, I have no clue why gitlab doesnt prefer it | 13:31 |
| cs_shadow | I agree, changing it now | 13:31 |
| cs_shadow | tristan: done. thanks! | 13:31 |
| jjardon | cs_shadow: that the case already | 13:34 |
| jjardon | I think? | 13:34 |
| tristan | jjardon, it was not for buildstream-docker-images repo | 13:34 |
| cs_shadow | jjardon: no, it used to default to "merge commit it semi-linear history" | 13:35 |
| tristan | jjardon, gitlab defaults to something else, and we never changed the default for that repo I guess | 13:35 |
| jjardon | yeah, I have changed it to merge commit it semi-linear history | 13:36 |
| jjardon | default is simply "merge", I think | 13:36 |
| jjardon | "merge commit it semi-linear history" is still linear but make it easier to separate what commits come from what MR | 13:37 |
| * tristan wants linear everywhere | 13:37 | |
| jjardon | so you can use the "Revert" buttin in the UI and revert all the commit in that MR if something goes wrong | 13:37 |
| tristan | If gitlab wants to litter my history with MR commits because it thinks our contributors are not responsible enough to mention issue numbers in their commit messages, that's a separate issue | 13:38 |
| cs_shadow | jjardon: operator clash! I had just changed it to "fast-forward merge". "Merge commit with semi-linear history" still creates a merge commit. | 13:38 |
| * jjardon prefers "merge commit it semi-linear history" | 13:38 | |
| jjardon | tristan: it's not about that at all | 13:38 |
| tristan | jjardon, :) | 13:39 |
| tristan | Yeah, I prefer strict fast forward policy honestly | 13:39 |
| tristan | regardless of the annoying merge commits | 13:39 |
| tristan | although gitlab could have that as a separate toggle | 13:39 |
| jjardon | I didnt like the MR commits in the beginning, but I see how useful can be sometimes to group changes per MR | 13:39 |
| jjardon | tristan: yeah, the history is completely linear with "merge commit with semi-linear history" | 13:40 |
| * cs_shadow prefers linear commit history and thinks commit messages should point to issues, which in turn should be related to MRs | 13:40 | |
| tristan | interestingly last time I checked, there is no option to generate the merge commits when linear history is strictly enforced | 13:40 |
| jjardon | tristan: that's the "merge commit with semi-linear history" option? | 13:40 |
| tristan | jjardon, how come it's called semi-linear then ? | 13:41 |
| tristan | in any case; I'm not about to enable merge commits :) | 13:41 |
| jjardon | cs_shadow: then you have to be very carefull to put in each of your commits what issue you are trying to solve | 13:41 |
| tristan | I'd rather be strict about mentioning issues in the commits | 13:41 |
| jjardon | tristan: maybe they called that way because you are putting merge commits in the middle? not sure | 13:41 |
| tristan | jjardon, that has been the standard since... forever | 13:41 |
| tristan | I know gitlab allows us to be a bit more careless, but we can still uphold the tradition :) | 13:42 |
| paulsherwood | i believe there's some intention to update https://buildstream.gitlab.io/buildstream/ to include the readme text, with a diagram. is that going to happen soon, do we think? | 13:42 |
| tristan | yeah maybe they called it that way because you need to pull the merge commit yourself | 13:42 |
| paulsherwood | or should i offer a patch? | 13:42 |
| tristan | paulsherwood, So the discussion about the graph makes it more complicated | 13:43 |
| tristan | the graph can only go into the docs | 13:43 |
| paulsherwood | graph? you mean diagram? | 13:43 |
| tristan | right the diagram | 13:43 |
| paulsherwood | ack | 13:43 |
| tristan | the image can only go in the docs, while jjardon has a patch to include the README part in an introduction section | 13:43 |
| paulsherwood | let's merge jjardon's patch, then :) | 13:43 |
| paulsherwood | perfect is the enemy of the good | 13:44 |
| paulsherwood | one thing at a time :) | 13:44 |
| tristan | which is nice to reuse that in both places, but means the introduction in the docs probably needs a split into two parts (second part having the image) | 13:44 |
| tristan | right, I think it's a good step | 13:44 |
| tristan | but I want to first look at jennis's 5 patches | 13:44 |
| * paulsherwood has to point someone at bst documentation soon | 13:45 | |
| tristan | and gitlab is *damn slow* this week :'( | 13:45 |
| tristan | https://gitlab.com/BuildStream/buildstream/merge_requests/419 | 13:45 |
| tristan | "in the process of being merged", for like 10 minutes | 13:45 |
| * tristan hazards a guess that... the CPU is not on the same machine as the SSD | 13:45 | |
| jjardon | tristan: :) | 13:46 |
| jjardon | can we merge the trivial-non controversial https://gitlab.com/BuildStream/buildstream/merge_requests/407 ? | 13:47 |
| * paulsherwood does not have the powers, but +1 | 13:47 | |
| jjardon | paulsherwood: do you have a link to the image, so I can adapt it to https://gitlab.com/BuildStream/buildstream/merge_requests/405 ? | 13:47 |
| paulsherwood | jjardon: i can email the odg to you, would that help? | 13:48 |
| jjardon | cs_shadow: tristan change back to linear history: I will try to convince you other day to linear with merge commit :) | 13:49 |
| jjardon | paulsherwood: sure | 13:49 |
| jjardon | cs_shadow: now that you are around; maybe you have some time to review https://gitlab.com/BuildStream/buildstream-docker-images/merge_requests/37 ? I'd like to add Debian stable to bst CI | 13:50 |
| tristan | Oh, we're sticking with linear in BuildStream, certainly | 13:50 |
| tristan | what cs_shadow wants to do in buildstream-docker-images, is up to him :) | 13:51 |
| jjardon | tristan: would you be interested on having CI in aarch64 as well? | 13:52 |
| paulsherwood | jjardon: sent | 13:52 |
| paulsherwood | jjardon: yes, we would | 13:52 |
| jjardon | paulsherwood: thanks | 13:53 |
| jjardon | ok, I will try to add it | 13:53 |
| paulsherwood | tristan: why no merge-commit? | 13:53 |
| tristan | I think merge commits are basically litter; everyone has the sense to mention the issue number in the commit message (except jjardon sometimes, but I remind him, he is the outlier), and it's in our policy | 13:54 |
| tristan | things are perfectly traceable without this | 13:55 |
| jmac | There's tonnes of commits without issue numbers in them | 13:56 |
| cs_shadow | jjardon: yeah, having a look now :) | 13:56 |
| tristan | jmac, not all commits are related to issues, and having at least the relevant one in a merge request mention the issue is enough for traceability | 13:56 |
| jmac | Yeah, but how do you know which other commits are part of the merge request? | 13:57 |
| tristan | once you are in a git log, you can see the related commits, and find the pointer back to the issue | 13:57 |
| tristan | much less important, you can index it by the issue if you really must, but I've never had a need | 13:57 |
| paulsherwood | tristan: i'm pretty sure that merge commits are useful | 13:57 |
| tristan | paulsherwood, grrrr | 13:58 |
| tristan | I'm pretty sure they just encourage carelessness | 13:58 |
| paulsherwood | heh | 13:58 |
| * tristan reads git logs a lot, wants to see well formatted commits, not MR pollution | 13:59 | |
| jmac | Sorry, I don't know how to see the related commits in the git log | 13:59 |
| paulsherwood | i don't think you are discouraging carelessness just by saying 'it's in our policy' etc | 13:59 |
| * tristan gotta swap location now :-S | 13:59 | |
| tristan | paulsherwood, honestly in GNOME this has always been a policy, for since like; forever in every org I've worked with, this is policy | 13:59 |
| tristan | people should have the sense to do so, or be scolded should they forget | 14:00 |
| jmac | This also isn't in our policy | 14:00 |
| dominic | I've not seen this mentioned anywhere | 14:00 |
| paulsherwood | tristan: you can be wrong, sometimes - give folks a chance to help :-) | 14:01 |
| jmac | "Commits in the branch which address specific issues must specify the | 14:01 |
| jmac | issue number in the commit message" | 14:01 |
| tristan | jmac, "Branches must be submitted as merge requests in gitlab and should usually be associated to an issue report on gitlab." | 14:01 |
| jjardon | tristan: I was thinking similar to you, until I have to revert some specific MR's: with no merge commits was almost impossible to be absolute sure what commits belong to which MR | 14:01 |
| tristan | jmac, But we can reword that, it's also in the example on the hacking page | 14:01 |
| jmac | tristan: That's different to what you just said | 14:01 |
| * paulsherwood is confident tristan is wrong this time | 14:01 | |
| tristan | they really wanna close the shop now | 14:01 |
| jjardon | go tristan, run! XD | 14:02 |
| *** tristan has quit IRC | 14:04 | |
| gitlab-br-bot | buildstream: issue #367 ("bst track refuses to track only one element") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/367 | 14:05 |
| *** tristan has joined #buildstream | 14:07 | |
| gitlab-br-bot | buildstream: merge request (jennis/136-expire-remote-cache-artifacts->master: WIP:136 expire remote cache artifacts) #364 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/364 | 14:12 |
| gitlab-br-bot | buildstream: merge request (tristan/scheduler-private-refactor->master: Scheduler private refactor) #419 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/419 | 14:13 |
| gitlab-br-bot | buildstream: merge request (jjardon/build_badge->master: README.rst: Add pipeline status badge) #407 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/407 | 14:16 |
| gitlab-br-bot | buildstream: merge request (documentation-formatting-in-HACKING->master: Documentation formatting in hacking) #411 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/411 | 14:18 |
| gitlab-br-bot | buildstream: merge request (jennis/136-clean-remote-cache->master: WIP: jennis/136 clean remote cache) #421 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/421 | 14:18 |
| gitlab-br-bot | buildstream: merge request (jjardon/build_badge->master: README.rst: Add pipeline status badge) #407 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/407 | 14:19 |
| jjardon | mmm, any idea why https://gitlab.com/BuildStream/buildstream/merge_requests/407 appear as closed, instead merged ? | 14:23 |
| dominic | appears to be merged | 14:25 |
| dominic | (only going off the committed 6 mins ago) | 14:25 |
| tlater | jjardon: Tristan has a habit of merging from CLI, which closes them instead - he usually comments if he does so though | 14:26 |
| * tlater thinks he actually closes manually | 14:26 | |
| jjardon | mmm, I'd say that is a bug in gitlab, as it confusing and it will destroy all the repo statistics | 14:28 |
| tlater | Well, gitlab isn't really meant to be used like that | 14:32 |
| tristan | tlater, jjardon actually, that is the second time it happened this week | 14:32 |
| tristan | gitlab is *damn slow* | 14:32 |
| tristan | I hit "Comment and close issue", the issue gets close, the comment doesnt get committed to the db :-S | 14:32 |
| tristan | jjardon, it is a bug in gitlab this week, you even reminded me earlier this week on another issue or MR, where I *clearly* had the text in that edit buffer | 14:33 |
| jjardon | tlater: seems it is; see the "You can merge this merge request manually using the command line" link in every MR | 14:33 |
| tristan | this time it confirms it | 14:33 |
| jjardon | tristan: rigth, so bad gitlab then :) | 14:33 |
| tristan | now I am sure, at least when gitlab.com is under load or crawling like it has been these couple days, they are not getting through | 14:34 |
| * jjardon forgives gitlab for all the nice things it has | 14:34 | |
| tristan | I'll try to now hammer them in a second time | 14:34 |
| tlater | tristan: On these logfiles - apparently part of their pathname is the pid of the process creating them | 14:34 |
| tristan | https://gitlab.com/BuildStream/buildstream/merge_requests/411 this one also missed it. | 14:35 |
| tlater | Probably necessary if we want to ensure exclusivity | 14:35 |
| tlater | The issue is that we need access to the element for everything else in that path name | 14:35 |
| tristan | tlater, then maybe it needs to be a "log format string" with the "{pid}" text in it ? | 14:35 |
| tlater | Yeah, I just realized | 14:35 |
| * tlater feels stupid for asking | 14:35 | |
| tristan | we can even `assert "{pid}" in log_format` there | 14:35 |
| tristan | haha | 14:36 |
| tristan | tlater, yeah, use your own brain why dont you ! | 14:36 |
| tristan | hehehe | 14:36 |
| tlater | ;p | 14:36 |
| tristan | jjardon, paulsherwood; so I've tried the branch which includes the readme in the docs home page, but I dont think I like it as much now that I'm seeing it | 14:40 |
| tristan | it's going in the opposite direction than I'd like, mostly because README.rst has gotten so big | 14:41 |
| gitlab-br-bot | buildstream: issue #368 ("Allow running bst commands from project sub-directories") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/368 | 14:41 |
| jjardon | no, it needs reworking of the README so it can go in both places | 14:41 |
| tristan | Rather, I'd like the ToC on the *main page* (not on the other pages, just on the main page)... to only have one level | 14:41 |
| tristan | and not show so much depth | 14:41 |
| tristan | and have only a short text on that page, so that the user is not too overwhelmed, and is presented with only a few options "blue pill or red pill ?" | 14:42 |
| tristan | jjardon, maybe we can have it so we include the README.rst as a separate, leading "About BuildStream" subpage, instead of blowing up the main page with it ? | 14:42 |
| tristan | and in that subsection, we could add another subsection with pretty graphs and more bla bla bla | 14:43 |
| tristan | currently if we expand it there, the user has to drag all the way through README to get to the links | 14:43 |
| * tristan thinks it *was* nice with a shorter README, though | 14:44 | |
| tristan | one which only contains the top paragraph, maybe | 14:44 |
| tristan | right, anyway I'll leave that patch alone for now | 14:44 |
| jjardon | yeah, I see what you mean; let me experiment a little bit | 14:45 |
| * tlater likes big readmes, means I don't have to go to documentation pages while figuring out if I want to use a tool but can stay on github/lab | 14:45 | |
| tristan | tlater, agree that a larger readme is desirable | 14:45 |
| tristan | blowing it up on the main docs page is less so | 14:45 |
| tlater | Ah, I understand, nevermind | 14:45 |
| tristan | maybe the option of including it in the toctree is better | 14:45 |
| tristan | (but then, it's a bit weird that the badges show up in the introductory About BuildStream "pill choice" | 14:46 |
| tristan | ) | 14:46 |
| tlater | Well, they *are* called pills in github land | 14:47 |
| tlater | It's "red pill" "blue pill" or "code coverage pill" | 14:47 |
| tristan | Uhhh | 14:51 |
| tristan | tlater, crossed topics, I think you've bruised your funny bone | 14:52 |
| tristan | tlater, I prefer hatching fried chickens | 14:52 |
| tristan | err eggs | 14:52 |
| * paulsherwood thinks having the whole readme on the docs page is probably wrong given it has so many contents links | 14:53 | |
| tristan | I wonder if the README.rst itself is allowed to include content from relative file paths | 14:54 |
| paulsherwood | so i suggest: 1) just take the new 2 paras from the readme and put them in place of the (worse imo) paras in the docs page | 14:54 |
| paulsherwood | 2) then have a separate FAQ page with the whole of the readme, or just the questions/answers | 14:55 |
| paulsherwood | 3) have a further Overview (or About) with some pictures on it | 14:55 |
| gitlab-br-bot | buildstream: merge request (jennis/136-clean-remote-cache->master: WIP: jennis/136 clean remote cache) #421 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/421 | 14:56 |
| paulsherwood | and let's not bother about trying to re-use the content between the two systems... gitl rendering of rst is too limited imo | 14:56 |
| tristan | I wouldnt mind (1) right away, but I would let jjardon play with docs sharing possibilities, because I think he wants to (and it would be nice to be able to share things) | 14:57 |
| tristan | if he decides it's not worth spending his time fiddling, I'd go for what you suggest | 14:58 |
| paulsherwood | ack | 14:58 |
| paulsherwood | jjardon: can we do (1) immediately? | 14:59 |
| gitlab-br-bot | buildstream: merge request (richardmaw/key-summary->master: List resolved element keys in job summary) #389 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/389 | 14:59 |
| * tristan notes that a *very brief* "What is BuildStream" paragraph, with a maximum of 3 sentences, is desirable and can be reused in release notes and just about everywhere | 15:01 | |
| tristan | but is very hard to achieve (not just for us) | 15:01 |
| tristan | I think the current text is good, still feel the references to other tooling in related areas is a bit of a stretch, but doesnt care enough to complain at the moment :) | 15:02 |
| tristan | s/doesnt/don't | 15:02 |
| paulsherwood | you wrote that sentence originally :) | 15:02 |
| tristan | Not as a "What is BuildStream" in my release notes or in the readme | 15:03 |
| tristan | possibly in a blog | 15:03 |
| paulsherwood | :-) | 15:03 |
| tristan | I mean; boiling it down into a condensed very small phrase | 15:03 |
| tristan | is difficult | 15:03 |
| gitlab-br-bot | buildstream: merge request (jennis/136-clean-remote-cache->master: WIP: jennis/136 clean remote cache) #421 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/421 | 15:04 |
| paulsherwood | agreed. we can drop the 'inspired by'. i'll attempt a different sentence | 15:04 |
| jjardon | what about https://paste.pics/eb5a77f0de53c51a44b8fc7e01b37a9a and https://paste.pics/c88bbbbc09c0363c9b99c41974fba00b ? | 15:04 |
| gitlab-br-bot | buildstream: issue #252 ("Print summary of discovered cache keys at end of session.") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/252 | 15:05 |
| paulsherwood | jjardon: ok by me. but i'd still suggest we have a sentence or two on the first page, rather than forcing folks to click | 15:06 |
| jjardon | ok, sec | 15:06 |
| paulsherwood | jjardon: i'm writing two sentences, please hold :) | 15:07 |
| tristan | cs_shadow, I think you mean #341 in your new #368, shall I fix it ? | 15:07 |
| cs_shadow | tristan: good catch! I've updated it now | 15:08 |
| * cs_shadow is a bit sad that GitLab keeps thinking that his issues are spam | 15:09 | |
| tristan | cs_shadow, I think it's tangentially related to the "running bst from a workspace", but #368 and #341 probably have the same patch to fix it (doing like git and searching for a directory with project.conf, with perhaps minimal validation on it) | 15:09 |
| tristan | or a similar one | 15:10 |
| * tristan should consider food at this point | 15:10 | |
| cs_shadow | tristan: i think #368 would fix #341 if the workspaces are in sub-directories but if they are outside, then we'd need more magic | 15:10 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_readme->master: WIP: source/index.rst: Use the README as our intro) #405 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/405 | 15:10 |
| tristan | cs_shadow, oh yeah, weird people put their workspaces *in* their projects | 15:11 |
| tristan | cs_shadow, I expect the workspace thing is only fixable with additional metadata in the checked out workspace | 15:11 |
| * tristan never expected that to happen, but never forbade it either | 15:12 | |
| cs_shadow | tristan: yes, and it's especially tricky because we can't assume that the sources are using git. For things like tar, i don't see an easy solution | 15:12 |
| cs_shadow | we can always add a magic file that opens the door for various conflicts | 15:13 |
| tristan | cs_shadow, I mean, we'd have to add a .bst/ directory to the workspace checkout, and have some local state there | 15:13 |
| tristan | cs_shadow, probably we can only support it on the condition that the workspace has been opened with a relative path | 15:13 |
| tristan | even then, it *can* break | 15:13 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_readme->master: WIP: source/index.rst: Use the README as our intro) #405 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/405 | 15:13 |
| cs_shadow | I'm bit skeptical about adding .bst directory because it assumes that the sources won't have that directory | 15:14 |
| tristan | but is much less likely, cs_shadow... maybe make relative path support at least *explicit* at workspace open time, so the user who forced it, gets what they deserve if they move their project directory and workspace directory around | 15:14 |
| tristan | cs_shadow, alternatively, we might track it in ~/.cache/buildstream/user or smth | 15:15 |
| tristan | still a bit yuck, but all state is yuck | 15:15 |
| cs_shadow | yeah, i don't see a very elegant solution for this | 15:16 |
| paulsherwood | jjardon: https://paste.baserock.org/oletamofor | 15:19 |
| jjardon | paulsherwood: do you want me to put that or https://paste.pics/956c3426b0e2fa02cd44ee7099b74ddd ? | 15:20 |
| paulsherwood | jjardon: yours is ok, but s/know to// | 15:21 |
| * paulsherwood is struggling to parse it the way jjardon wrote it | 15:22 | |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_readme->master: WIP: source/index.rst: Use the README as our intro) #405 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/405 | 15:23 |
| jjardon | corrected ^ | 15:23 |
| jjardon | paulsherwood: are you saying something about my flawlessly english ?? :) | 15:23 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_readme->master: source/index.rst: Use the README as our intro) #405 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/405 | 15:24 |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_readme->master: source/index.rst: Use the README as our intro) #405 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/405 | 15:25 |
| paulsherwood | jjardon: it was flawless, just hard to understand | 15:34 |
| jjardon | do we want to add something like this to the main? https://paste.pics/1f8e7bab12975e9eb7eed7188d008674 | 15:35 |
| jjardon | ot leave it as it is for now? | 15:35 |
| tristan | jjardon, lookup paulsherwood's MR which had done what I requested with the diagram | 15:36 |
| tristan | to make sure the MR includes the source odg, named the same as the cropped/rendered png, in doc/images, doc/image-sources; and has the diagram, wording updates to relevant, recent terminology | 15:37 |
| jjardon | tristan: ok, so leave it as is for now (no diagram); I will rearange the diagram later; If you can review !405 that would be great | 15:37 |
| tristan | I have docs appointment tomorrow morning and trying to tear myself away from the screen so that I end up getting a couple hours sleep | 15:37 |
| jjardon | sure, you or paulsherwood or anyone; the MR is ready | 15:38 |
| tristan | in other words, I'm sure it can wait a day ;-) | 15:38 |
| *** noisecell has quit IRC | 15:52 | |
| *** jonathanmaw has quit IRC | 15:59 | |
| *** dominic has quit IRC | 16:04 | |
| tristan | jjardon, I'm merging, under the assumption we can have a one-level ToC on the main page | 16:05 |
| * tristan checked it out and built docs, looks good | 16:06 | |
| jjardon | thanks tristan ; now go to sleep! | 16:07 |
| * jjardon wants to go out to the sun | 16:07 | |
| *** Prince781 has joined #buildstream | 16:11 | |
| *** toscalix has quit IRC | 16:13 | |
| gitlab-br-bot | buildstream: merge request (jjardon/doc_readme->master: source/index.rst: Use the README as our intro) #405 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/405 | 16:19 |
| juergbi | tlater: does the cache expiry work already have protection against removing very recently added objects that are not referenced yet by a ref? | 16:20 |
| tlater | juergbi: Certainly not for multiple instances of buildstream | 16:23 |
| juergbi | is there really a difference? | 16:23 |
| tlater | I'm think about that, but I don't think so, no | 16:23 |
| tlater | Yeah, we don't | 16:23 |
| tlater | I'm not sure how we would protect against that, frankly | 16:24 |
| tlater | Hm, ostree probably has a flag to prune() that prevents pruning artifacts younger than a certain age | 16:25 |
| tlater | Set that to a couple of seconds and we should be fine | 16:25 |
| * tlater hopes, at least | 16:25 | |
| juergbi | that's only for the commit date | 16:25 |
| tlater | Ah, darn | 16:25 |
| juergbi | the generalization of this using the object mtime should suffice | 16:26 |
| tlater | juergbi: What's the scenario in which this can happen, considering you dreamed one up? | 16:26 |
| tlater | juergbi: We use an mtime on the ref, not objects | 16:26 |
| juergbi | adding an artifact to the cache is not a single atomic operation | 16:26 |
| juergbi | files get added one at a time | 16:26 |
| juergbi | if we run prune while an artifact is added, there is a high risk of issues | 16:27 |
| tlater | We could try to guarantee we're never pruning while something is being added | 16:27 |
| tlater | Oh, yeah, we definitely can do so for a single instance | 16:27 |
| juergbi | that's tricky to impossible without lock files across buildstream sessions | 16:27 |
| tlater | Yeah, that's the only problem | 16:28 |
| juergbi | a generalization of ostree's prune option should be easy to implement in CAS | 16:28 |
| juergbi | maybe we can ignore the issue until then | 16:28 |
| tlater | Well, we will already be ensuring that we halt build processes while we're pruning | 16:29 |
| tlater | So at least we mitigate this for the single-instance case | 16:29 |
| juergbi | as in suspend? | 16:29 |
| tlater | Err, yeah, suspend | 16:29 |
| juergbi | or wait for all pending ones to finish? | 16:29 |
| juergbi | if you suspend during commit, it doesn't help | 16:29 |
| tlater | Hm, that will have to be waiting then | 16:30 |
| tlater | That's a bit annoying, but will have to do | 16:30 |
| tlater | juergbi: I assume you intend to land the CAS cache quite soon then? | 16:31 |
| gitlab-br-bot | buildstream: merge request (make-source-dir-resemble-ToC->master: Restructure directories under source/ to resemble ToC) #413 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/413 | 16:32 |
| *** xjuan_ has joined #buildstream | 16:39 | |
| juergbi | tlater: at some point in May, I expect | 16:39 |
| *** xjuan has quit IRC | 16:41 | |
| *** xjuan_ is now known as xjuan | 16:41 | |
| *** Prince781 has quit IRC | 16:50 | |
| *** Prince781 has joined #buildstream | 16:54 | |
| tristan | this sounds less than ideal; I was hoping for concurrent prune and only stop queueing more build tasks once prune is taking too long and we reach the quota | 17:00 |
| tristan | but it can be a temporary measure I guess | 17:00 |
| *** bethw has quit IRC | 17:03 | |
| juergbi | yes, I'm not sure we really gain anything by suspending build tasks | 17:04 |
| juergbi | and am still worried about prune overhead in general | 17:07 |
| *** xjuan_ has joined #buildstream | 17:21 | |
| *** xjuan has quit IRC | 17:24 | |
| tristan | juergbi, I'm hoping if the thresholds are far enough apart, they wont happen too often, maybe once every 10 builds of 500 modules or so | 17:24 |
| tristan | suspend = NO | 17:24 |
| tristan | wait for an opportunity is better, especially if this is a stop gap until CAS | 17:25 |
| tristan | finally, we could make prune consume a build token | 17:25 |
| tristan | so if you have 4 concurrent builds, you can only have 3 + one prune task | 17:25 |
| juergbi | yes, large enough threshold gap sounds good to me | 17:26 |
| *** aday has quit IRC | 17:53 | |
| *** Prince781 has quit IRC | 18:00 | |
| *** Prince781 has joined #buildstream | 18:35 | |
| *** xjuan_ has quit IRC | 19:06 | |
| *** xjuan_ has joined #buildstream | 19:06 | |
| *** xjuan_ has quit IRC | 19:09 | |
| *** Prince781 has quit IRC | 19:09 | |
| *** aday has joined #buildstream | 19:16 | |
| *** tristan has quit IRC | 19:29 | |
| *** Prince781 has joined #buildstream | 20:47 | |
| *** Prince781 has quit IRC | 21:01 | |
| cs_shadow | So, https://gitlab.com/BuildStream/buildstream-docker-images/merge_requests/38 is created from the branch " chandan/improve-tests ". The branch has 4 commits (https://gitlab.com/BuildStream/buildstream-docker-images/tree/chandan/improve-tests) but the MR only shows 2 | 21:20 |
| cs_shadow | what sorcery is this? Any ideas? | 21:20 |
| *** aday has quit IRC | 21:36 | |
| jjardon | Does anyone here have access to the https://hub.docker.com/u/buildstream/ account? I'd like to create a new repo there | 23:06 |
| jjardon | buildstream-debian | 23:07 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!