| *** trash has joined #buildstream | 05:30 | |
| *** mohan43u has quit IRC | 06:30 | |
| *** mohan43u has joined #buildstream | 06:30 | |
| *** tristan has joined #buildstream | 07:44 | |
| benschubert | Hey tristan, would you have time to look at !702 and (more importantly) !819 ? | 08:39 |
|---|---|---|
| gitlab-br-bot | MR !702: WIP: Mock storage space checks for tests. https://gitlab.com/BuildStream/buildstream/merge_requests/702 | 08:39 |
| gitlab-br-bot | MR !819: Add `--deps build` option to `bst checkout` https://gitlab.com/BuildStream/buildstream/merge_requests/819 | 08:39 |
| *** toscalix has joined #buildstream | 08:46 | |
| Kinnison | Does anyone here happen to know what the rules are about overlapping plugin names and junctions? | 09:05 |
| juergbi | Kinnison: each (sub)project has their own plugin context with a corresponding ElementFactory and SourceFactory | 09:10 |
| Kinnison | juergbi: Excellent, my assumption there holds, thank you | 09:10 |
| juergbi | so I think there should be no conflict potential | 09:10 |
| Kinnison | nod. | 09:11 |
| benschubert | juergbi: if you have time, could you quickly check that !949 is good for you and that I addressed your comments from !938 in !958 ? | 09:20 |
| gitlab-br-bot | MR !949: Add tests for fedora with no bubblewrap available https://gitlab.com/BuildStream/buildstream/merge_requests/949 | 09:20 |
| gitlab-br-bot | MR !938: Fix os.rename in git source element to correctly handle error codes https://gitlab.com/BuildStream/buildstream/merge_requests/938 | 09:20 |
| gitlab-br-bot | MR !958: Followup on MR 938, addressing additional comments https://gitlab.com/BuildStream/buildstream/merge_requests/958 | 09:20 |
| *** alatiera_ has joined #buildstream | 09:20 | |
| *** alatiera_ has joined #buildstream | 09:20 | |
| juergbi | will take a look, probably this afternoon | 09:21 |
| benschubert | thanks! | 09:23 |
| *** solid_black has joined #buildstream | 09:24 | |
| *** rdale has joined #buildstream | 09:25 | |
| *** phildawson has joined #buildstream | 09:25 | |
| *** jonathanmaw has joined #buildstream | 09:38 | |
| valentind | juergbi, I updated the Freedesktop cache server to be object-based clean up, yesterday. | 09:44 |
| valentind | The clean-up worked OK. | 09:44 |
| valentind | But this morning we had a spike of CPU usage. | 09:44 |
| valentind | Corresponding with a spike in network bandwitdh | 09:44 |
| valentind | But disk I/O is relatively small. | 09:45 |
| juergbi | hm, the new approach is supposed to avoid expensive operations causing such spikes | 09:45 |
| valentind | It is 669 Mbps of network. And 230 kB/s of disk I/O | 09:45 |
| juergbi | spike in network bandwidth: would this be due to a spike in client requests? | 09:45 |
| valentind | juergbi, yes, but not that many. Around 25 clients. We have seen more. | 09:46 |
| juergbi | maybe a few with really fast connection, hm | 09:46 |
| valentind | I do not understand why there is so much more network than disk I/O | 09:46 |
| juergbi | low disk I/O doesn't mean anything if tons is in the page cache. however, the ratio is a bit extreme... | 09:47 |
| valentind | And CPU gets 100% then connect()s timeout. | 09:47 |
| valentind | juergbi, The machine does not have much memory to cache what it needs. | 09:47 |
| valentind | The cache data is huge. And builds require quite a lot of objects. | 09:47 |
| juergbi | yes, it seems quite odd | 09:48 |
| valentind | I am looking the history. And it does not seem my changes changed much of the behavior | 09:49 |
| valentind | It is outbound traffic mostly. | 09:50 |
| valentind | inbound is negligable. | 09:50 |
| valentind | And it is the push enabled server. So it must be the builders. | 09:51 |
| juergbi | did CPU usage remain at 100%, i.e., did you have to forcefully terminate it? | 09:53 |
| valentind | juergbi, No. It went down by itself. | 09:53 |
| valentind | It was also going up and down. | 09:53 |
| valentind | I saw it going between 140% and 190% in top (with 2 cpus). | 09:53 |
| valentind | and now it is much less than 50%. So no loop. | 09:54 |
| juergbi | ok | 09:54 |
| valentind | juergbi, we will increase the size of the server. | 09:56 |
| valentind | The outbound network follows very well the CPU usage in the graphs. | 09:57 |
| juergbi | did you see any client issues in the CI except for the connect failure? | 09:59 |
| juergbi | i.e., the outbound network traffic was likely correct? | 10:00 |
| *** tpollard has joined #buildstream | 10:01 | |
| valentind | Only connect() timeout. | 10:05 |
| valentind | Yes, it might have been OK. | 10:05 |
| valentind | It might be that all the build have been started exactly at the same time, which does not usually happen. | 10:08 |
| *** raoul has joined #buildstream | 10:17 | |
| *** lachlan has joined #buildstream | 10:27 | |
| *** lachlan has quit IRC | 10:29 | |
| *** lachlan has joined #buildstream | 10:29 | |
| *** lachlan has quit IRC | 10:44 | |
| *** lachlan has joined #buildstream | 10:52 | |
| benschubert | Do we have somewhere documentation about how staging is done exactly? | 10:53 |
| tiagogomes | benschubert https://wiki.gnome.org/Projects/BuildStream/ImportRules perhaps? | 10:57 |
| jmac | That's not meant to be user-facing documentation, but should be up-to-date | 10:57 |
| benschubert | ok, perfect thanks! | 10:58 |
| benschubert | I'm trying to understand why it is so slow and where I could make improvements (7 minutes for 600MB archive, which is my base platform) | 10:59 |
| juergbi | benschubert: CAS-based source cache + buildbox will likely improve this a lot | 10:59 |
| benschubert | I'm looking more into local dev, so without buildbox access (and on windows WSL, which is extremely slow by default) | 11:00 |
| *** lachlan has quit IRC | 11:02 | |
| Kinnison | tristan: thanks for your input on the plugin thread I started. I'll give your ideas some thought and see if I can get a response written today | 11:05 |
| juergbi | benschubert: the plan is to use source cache and buildbox also locally | 11:09 |
| benschubert | Oh, I seem to have missed that, won't look into optimizing then thanks! | 11:10 |
| juergbi | WSL can't use buildbox (FUSE) but can still use source cache. and in that case you anyway either have to use remote execution or docker | 11:10 |
| juergbi | (or local execution without full sandboxing but FUSE in docker would likely be faster) | 11:10 |
| *** lachlan has joined #buildstream | 11:12 | |
| benschubert | Ok, thanks! | 11:12 |
| *** adds68 has quit IRC | 11:14 | |
| *** adds68 has joined #buildstream | 11:15 | |
| *** ChanServ sets mode: +o tristan | 11:15 | |
| tristan | Kinnison, I did read through your email and it is a lot of stuff to think about indeed, just wanted to add some of the thoughts I've been having on the same subject :) | 11:15 |
| Kinnison | Aye, they're helpful to have on one place so we can all stew :-) | 11:16 |
| juergbi | tristan: in my second mail regarding execution environments, I'm proposing an approach how to handle backward compatibility with format-version. not extremely urgent but I'd appreciate a comment on whether this sounds like an acceptable approach in general or not | 11:23 |
| juergbi | and thanks for the command batching comments, will fix this up tomorrow morning | 11:23 |
| juergbi | relevant execution environments mail: https://mail.gnome.org/archives/buildstream-list/2018-November/msg00056.html | 11:24 |
| *** lachlan has quit IRC | 11:30 | |
| Nexus | benschubert: Would it be possible for me to have a look at te kindof projcet you said isn't working with `build-world`? I'm having difficulty visualising it to debug exactly what's happening | 11:31 |
| benschubert | Nexus: the problem is we don't have an element-path specified, Didn't occur to me directly. So we have every element at the root of the project | 11:32 |
| Nexus | benschubert: i see. As i understand it, buildstream currently does not support elements boing outside of an element-path | 11:33 |
| Nexus | though if it's in the root, i would expect it to still get caught by the build-all logic | 11:34 |
| Nexus | as i believe ethat;s the element-path defaultr | 11:34 |
| * Nexus can't type today | 11:34 | |
| tristan | juergbi, lemme take a look thanks, that one probably slipped through the cracks | 11:35 |
| juergbi | thanks | 11:35 |
| tristan | ohhh that | 11:35 |
| tristan | yeah I remember :-S | 11:35 |
| *** adds68 has quit IRC | 11:36 | |
| benschubert | Nexus: yeah it should, but there was a bug in the PR previously that hardcoded "elements" if I remember well? | 11:36 |
| *** adds68 has joined #buildstream | 11:36 | |
| benschubert | Nexus: mainly ```if build_world: elements = get_files(app._main_options.get("directory"), "elements"``` | 11:37 |
| *** lachlan has joined #buildstream | 11:44 | |
| *** lachlan has quit IRC | 11:49 | |
| *** lachlan has joined #buildstream | 11:49 | |
| gitlab-br-bot | aevri merged MR !887 (aevri/prompt-config->master: Add 'prompt.*' config options to buildstream.conf) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/887 | 12:12 |
| *** tristan has quit IRC | 12:13 | |
| *** lachlan has quit IRC | 12:20 | |
| *** lachlan has joined #buildstream | 12:22 | |
| *** lachlan has quit IRC | 12:27 | |
| *** phildawson has quit IRC | 12:29 | |
| *** phildawson has joined #buildstream | 12:29 | |
| paulsherwood | what's with the strongarm icon in gitlab? | 12:31 |
| Nexus | benschubert: yes, that's been removed now :) | 12:32 |
| paulsherwood | (it's one of the many things that is currently annoying me about https://gitlab.com/BuildStream/buildstream/issues/711) | 12:32 |
| jmac | That's just toscalix's user name | 12:32 |
| jmac | Oh no, maybe not | 12:33 |
| benschubert | Everything should be good then I can try after | 12:33 |
| jmac | No idea what that's for, actually | 12:37 |
| jmac | Looks like you can set a current status which includes an emoji | 12:38 |
| toscalix | jmac: solved. I removed it | 12:38 |
| paulsherwood | tvm. it was sending entirely the wrong signals imo | 12:38 |
| *** lachlan has joined #buildstream | 12:45 | |
| *** cs-shadow has quit IRC | 13:03 | |
| *** tristan has joined #buildstream | 13:06 | |
| *** lachlan has quit IRC | 13:19 | |
| *** phildawson has quit IRC | 13:26 | |
| *** phildawson has joined #buildstream | 13:28 | |
| *** alatiera_ has quit IRC | 13:33 | |
| *** catonano has joined #buildstream | 13:38 | |
| *** alatiera_ has joined #buildstream | 14:09 | |
| *** lachlan has joined #buildstream | 14:15 | |
| *** phildawson has quit IRC | 14:19 | |
| *** phildawson_ has joined #buildstream | 14:22 | |
| *** phildawson_ has quit IRC | 14:24 | |
| *** phildawson has joined #buildstream | 14:28 | |
| jonathanmaw | Is there a way to override what click describes for its usage string? | 14:32 |
| skullman | not that I've seen | 14:33 |
| jonathanmaw | I can trick it into letting me make `bst checkout`'s ELEMENT as optional if I flag LOCATION as optional, too | 14:33 |
| jonathanmaw | but then it changes the usage string :/ | 14:33 |
| * skullman opted to check the options and raise an AppError | 14:33 | |
| *** phildawson has quit IRC | 14:35 | |
| *** phildawson has joined #buildstream | 14:36 | |
| *** phildawson_ has joined #buildstream | 14:58 | |
| *** raoul_ has joined #buildstream | 15:00 | |
| *** phildawson has quit IRC | 15:00 | |
| *** raoul has quit IRC | 15:01 | |
| *** phildawson_ has quit IRC | 15:02 | |
| *** phildawson has joined #buildstream | 15:02 | |
| tpollard | tests seem to be broken on master for me | 15:12 |
| tpollard | not sure if it's a local issue yet though, will push to a WIP to check | 15:13 |
| *** solid_black has quit IRC | 15:13 | |
| *** alatiera__ has joined #buildstream | 15:17 | |
| *** alatiera_ has quit IRC | 15:20 | |
| gitlab-br-bot | tpollard opened issue #774 (Provide the ability to pull missing buildtrees via the scheduler outside of user invoked bst push & build) on buildstream https://gitlab.com/BuildStream/buildstream/issues/774 | 15:29 |
| *** raoul_ has quit IRC | 15:36 | |
| *** raoul_ has joined #buildstream | 15:36 | |
| *** raoul_ has quit IRC | 15:37 | |
| *** raoul_ has joined #buildstream | 15:38 | |
| *** Jobava has joined #buildstream | 15:43 | |
| *** lachlan has quit IRC | 15:50 | |
| *** lachlan has joined #buildstream | 16:06 | |
| *** alatiera__ has quit IRC | 16:08 | |
| *** CTtpollard has joined #buildstream | 16:11 | |
| *** tpollard has quit IRC | 16:13 | |
| *** lachlan has quit IRC | 16:15 | |
| *** lachlan has joined #buildstream | 16:18 | |
| *** CTtpollard has quit IRC | 16:29 | |
| *** tpollard has joined #buildstream | 16:33 | |
| *** solid_black has joined #buildstream | 16:33 | |
| *** solid_black_ has joined #buildstream | 16:36 | |
| *** solid_black_ has quit IRC | 16:38 | |
| *** solid_black__ has joined #buildstream | 16:38 | |
| *** solid_black has quit IRC | 16:38 | |
| *** solid_black__ is now known as solid_black | 16:39 | |
| juergbi | jennis: regarding !956, is 'pathway' a commonly used term (instead of just path) in the context of filesystems? | 16:41 |
| gitlab-br-bot | MR !956: Fix warning when using an absolute path which exists inside the project https://gitlab.com/BuildStream/buildstream/merge_requests/956 | 16:41 |
| *** solid_black has quit IRC | 16:42 | |
| juergbi | I don't think I've seen pathway used before in this area, or at the very least not frequently | 16:42 |
| juergbi | so I would just go for path | 16:42 |
| jennis | juergbi, noted. Will amend the MR | 16:44 |
| * Kinnison doesn't think 'pathway' is right either :) | 16:44 | |
| juergbi | ta | 16:44 |
| *** phildawson has quit IRC | 16:44 | |
| jennis | juergbi, all done | 16:46 |
| jennis | Also rebased | 16:47 |
| gitlab-br-bot | LaurenceUrhegyi opened issue #775 (Execution Environment Requirements) on buildstream https://gitlab.com/BuildStream/buildstream/issues/775 | 16:48 |
| tpollard | I seemed to have found a few tests that are picking up the host systems userconfig, I'll try to narrow it down | 16:50 |
| tpollard | I purposely had an incompatible config option in my user conf, which leads to some of the tests failing | 16:50 |
| *** alatiera_ has joined #buildstream | 16:56 | |
| gitlab-br-bot | juergbi closed issue #725 (Job cancellation on remote builds) on buildstream https://gitlab.com/BuildStream/buildstream/issues/725 | 16:59 |
| gitlab-br-bot | juergbi merged MR !900 (725-job-cancellation-on-remote-builds->master: _sandboxremote.py: Add sigterm handler that sends CancelOperation) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/900 | 16:59 |
| *** rkukura has joined #buildstream | 17:04 | |
| *** lachlan has quit IRC | 17:04 | |
| *** catonano has quit IRC | 17:12 | |
| *** tpollard has quit IRC | 17:25 | |
| *** tpollard has joined #buildstream | 17:27 | |
| gitlab-br-bot | juergbi merged MR !956 (jennis/less_restrictive_pathways->master: Fix warning when using an absolute path which exists inside the project) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/956 | 17:32 |
| *** abderrahim has joined #buildstream | 17:34 | |
| *** abderrah1 has quit IRC | 17:36 | |
| *** lachlan has joined #buildstream | 17:37 | |
| *** boucman_work has joined #buildstream | 17:41 | |
| *** lachlan has quit IRC | 17:41 | |
| benschubert | thanks juergbi for the reviews! I'll address the comments and merge the prs now ! | 17:41 |
| *** lachlan has joined #buildstream | 17:49 | |
| benschubert | For those interested I have a POC about a docker sandbox here https://gitlab.com/BuildStream/buildstream/tree/bschubert/docker-sandbox | 17:58 |
| *** lachlan has quit IRC | 18:04 | |
| *** catonano has joined #buildstream | 18:05 | |
| *** jonathanmaw has quit IRC | 18:08 | |
| *** raoul_ has quit IRC | 18:18 | |
| *** tiagogomes_ has joined #buildstream | 18:18 | |
| *** CTtpollard has joined #buildstream | 18:18 | |
| *** raoul has joined #buildstream | 18:18 | |
| *** tiagogomes has quit IRC | 18:18 | |
| *** tpollard has quit IRC | 18:18 | |
| *** toscalix has quit IRC | 18:35 | |
| *** catonano has quit IRC | 18:37 | |
| *** alatiera_ has quit IRC | 18:39 | |
| gitlab-br-bot | BenjaminSchubert merged MR !949 (bschubert/add-fedora-nobwrap-tests->master: Add tests for fedora with no bubblewrap available) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/949 | 18:41 |
| *** raoul has quit IRC | 19:08 | |
| *** catonano has joined #buildstream | 19:10 | |
| *** alatiera_ has joined #buildstream | 19:57 | |
| *** Guest54789 has joined #buildstream | 20:05 | |
| *** phildawson has joined #buildstream | 20:07 | |
| *** catonano has quit IRC | 20:28 | |
| *** catonano has joined #buildstream | 20:32 | |
| *** catonano has quit IRC | 20:36 | |
| *** Tony99 has joined #buildstream | 22:04 | |
| *** alatiera__ has joined #buildstream | 22:16 | |
| *** alatiera_ has quit IRC | 22:18 | |
| *** benschubert has quit IRC | 23:04 | |
| *** laurence has quit IRC | 23:04 | |
| *** skullman has quit IRC | 23:04 | |
| *** Demos[m] has quit IRC | 23:04 | |
| *** awacheux[m] has quit IRC | 23:04 | |
| *** kailueke[m] has quit IRC | 23:04 | |
| *** alatiera has quit IRC | 23:04 | |
| *** ssssam[m] has quit IRC | 23:04 | |
| *** abderrahim[m] has quit IRC | 23:04 | |
| *** pro[m] has quit IRC | 23:04 | |
| *** theawless[m] has quit IRC | 23:04 | |
| *** persia has quit IRC | 23:04 | |
| *** CTtpollard has quit IRC | 23:04 | |
| *** tristan has quit IRC | 23:04 | |
| *** mohan43u has quit IRC | 23:04 | |
| *** ctolentino has quit IRC | 23:04 | |
| *** slaf has quit IRC | 23:04 | |
| *** gitlab-br-bot has quit IRC | 23:04 | |
| *** lchlan has quit IRC | 23:04 | |
| *** paulsherwood has quit IRC | 23:04 | |
| *** jmac has quit IRC | 23:04 | |
| *** dineshdb[m] has quit IRC | 23:04 | |
| *** waltervargas[m] has quit IRC | 23:04 | |
| *** doras[m] has quit IRC | 23:04 | |
| *** inigomartinez has quit IRC | 23:04 | |
| *** connorshea[m] has quit IRC | 23:04 | |
| *** oknf[m] has quit IRC | 23:04 | |
| *** segfault3[m] has quit IRC | 23:04 | |
| *** jjardon[m] has quit IRC | 23:04 | |
| *** asingh_[m] has quit IRC | 23:04 | |
| *** tlater[m] has quit IRC | 23:04 | |
| *** krichter[m] has quit IRC | 23:04 | |
| *** cgmcintyre[m] has quit IRC | 23:04 | |
| *** rafaelff[m] has quit IRC | 23:04 | |
| *** mattiasb has quit IRC | 23:04 | |
| *** albfan[m] has quit IRC | 23:04 | |
| *** m_22[m] has quit IRC | 23:04 | |
| *** juergbi has quit IRC | 23:04 | |
| *** jjardon has quit IRC | 23:04 | |
| *** hergertme has quit IRC | 23:04 | |
| *** ironfoot has quit IRC | 23:04 | |
| *** tintou has quit IRC | 23:04 | |
| *** mohan43u has joined #buildstream | 23:46 | |
| *** mohan43u has quit IRC | 23:49 | |
| *** mohan43u has joined #buildstream | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!