| *** kapil___ has quit IRC | 00:24 | |
| *** nimish has joined #buildstream | 01:18 | |
| *** alatiera has quit IRC | 01:26 | |
| *** nimish has quit IRC | 01:28 | |
| *** nimish has joined #buildstream | 01:28 | |
| *** nimish has quit IRC | 01:33 | |
| *** nimish has joined #buildstream | 01:33 | |
| *** nimish has quit IRC | 01:43 | |
| *** nimish has joined #buildstream | 01:44 | |
| *** nimish has quit IRC | 01:49 | |
| *** nimish has joined #buildstream | 01:49 | |
| *** nimish has quit IRC | 01:54 | |
| *** nimish has joined #buildstream | 01:54 | |
| *** nimish has quit IRC | 02:19 | |
| *** nimish has joined #buildstream | 02:20 | |
| *** nimish has quit IRC | 02:30 | |
| *** nimish has joined #buildstream | 02:30 | |
| *** nimish has quit IRC | 02:55 | |
| *** nimish has joined #buildstream | 02:55 | |
| *** nimish has quit IRC | 03:00 | |
| *** nimish has joined #buildstream | 03:01 | |
| *** nimish has quit IRC | 03:06 | |
| *** nimish has joined #buildstream | 03:06 | |
| *** nimish has quit IRC | 03:11 | |
| *** nimish has joined #buildstream | 03:11 | |
| *** tpollard has joined #buildstream | 03:14 | |
| *** tpollard has quit IRC | 03:18 | |
| *** nimish has quit IRC | 03:21 | |
| *** nimish has joined #buildstream | 03:22 | |
| *** nimish has quit IRC | 03:29 | |
| *** mohan43u has quit IRC | 04:08 | |
| *** mohan43u has joined #buildstream | 04:09 | |
| *** nimish has joined #buildstream | 05:10 | |
| *** nimish has quit IRC | 05:44 | |
| gitlab-br-bot | jjardon opened issue #956 (BuildStream master seems to be slower than bst-1.2 branch) on buildstream https://gitlab.com/BuildStream/buildstream/issues/956 | 05:52 |
|---|---|---|
| jjardon | Seems the build time is worse in bst master in comparison with bst 1.2 ^ | 05:56 |
| *** toscalix has joined #buildstream | 08:05 | |
| *** toscalix has quit IRC | 08:06 | |
| *** toscalix has joined #buildstream | 08:06 | |
| *** persia has quit IRC | 08:26 | |
| *** asingh_[m] has quit IRC | 08:27 | |
| *** persia has joined #buildstream | 08:28 | |
| *** Trevinho[m] has quit IRC | 08:28 | |
| *** asingh_[m] has joined #buildstream | 08:40 | |
| *** Trevinho[m] has joined #buildstream | 08:47 | |
| *** nimish has joined #buildstream | 08:54 | |
| *** nimish has quit IRC | 08:58 | |
| *** nimish has joined #buildstream | 09:10 | |
| *** nimish has quit IRC | 09:13 | |
| *** nimish has joined #buildstream | 09:14 | |
| juergbi | jjardon: I generally see wildly varying CI times, so that could be one aspect | 09:14 |
| juergbi | another aspect could be buildtree caching, but this will again be disabled for successful builds in master very soon | 09:15 |
| jjardon | juergbi: Its exactly same hardware, it should be similar | 09:15 |
| jjardon | But yeah maybe there is a timeout in some repo or something. Still worth investigating I think | 09:16 |
| jjardon | The difference is not small, Its around 2 Hours | 09:17 |
| benschubert | jjardon: do you also happen to have memory usage by any chance? | 09:17 |
| jjardon | sorry what do you mean? | 09:18 |
| benschubert | jjardon: maximum memory usage for the two runs you mention | 09:18 |
| jjardon | benschubert: no idea | 09:18 |
| juergbi | I think it would be good to compare again as soon as !1208 is merged | 09:18 |
| gitlab-br-bot | MR !1208: Tweak cache-buildtrees option https://gitlab.com/BuildStream/buildstream/merge_requests/1208 | 09:18 |
| jjardon | I do not think buildstream records that | 09:18 |
| jjardon | in the good news, the overnight test are successful now :) they have been broken for ages | 09:20 |
| benschubert | jjardon: ok no worries. There's a whole lot of effort around performance currently, so I expect master to become gently faster | 09:21 |
| benschubert | at least for our internal use case, we went from 1-2h builds to 30 minutes ones | 09:22 |
| jjardon | yeah, that is why it surprised me it was actually 2hours slower | 09:22 |
| jjardon | than 1.2 | 09:22 |
| benschubert | jjardon: the 1-2h builds were with something after 1.2 though | 09:24 |
| jjardon | I see | 09:24 |
| *** raoul has joined #buildstream | 09:40 | |
| *** nimish has quit IRC | 09:44 | |
| *** nimish has joined #buildstream | 09:44 | |
| jjardon | Can I have reviews of https://gitlab.com/BuildStream/buildstream/merge_requests/1218 , please? | 09:53 |
| gitlab-br-bot | phildawson opened (was WIP) MR !1224 (phil/fixup-external-plugin-tests->master: Move external plugin tests to seperate tox environment.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1224 | 09:57 |
| phildawson | ^benschubert | 09:57 |
| benschubert | phildawson: appart, how did we ever get a "pip -e" here? Oo, how did that pass a code review? | 09:58 |
| *** nimish has quit IRC | 09:59 | |
| *** nimish has joined #buildstream | 09:59 | |
| benschubert | phildawson: first review done. I'll have a second look once this is fixed and the pipeline passes :) | 10:00 |
| phildawson | Cheers benschubert, that was a stupid mistake, I've fixed it now. | 10:05 |
| benschubert | phildawson: also not sure about https://gitlab.com/BuildStream/buildstream/merge_requests/1224/diffs#61be067c7cf3bdbf8a6b021a2b5167eb30612d0c_19_17 | 10:05 |
| benschubert | shouldn't this be removed? | 10:06 |
| phildawson | You are correct. Appologies, this is what I get for trowing together a patch in a rush before leaving for the evening :/ | 10:09 |
| phildawson | throwing | 10:09 |
| benschubert | no worries :) | 10:10 |
| phildawson | Fixes pushed. I'll give you a shout when the pipeline has passed :) | 10:12 |
| aevri | juergbi: maybe "FileNotFoundError: [Errno 2] No such file or directory: '/home/bschubert/.cache/buildstream-cache/buildstream/cas/tmpfa6k35pq'" could this be because CacheSizeJob is not always exclusive, and BuildQueue does an 'add_artifact_size'? | 10:17 |
| aevri | Maybe this would be writing the cache, when the non-exclusive size job is reading? https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/_scheduler/queues/buildqueue.py#L98 | 10:18 |
| *** nimish has quit IRC | 10:19 | |
| *** nimish has joined #buildstream | 10:20 | |
| juergbi | aevri: ah, I thought CacheSizeJob is exclusive. that would certainly explain this | 10:22 |
| benschubert | phildawson: can you lastly check that the external stuff can be called with every python version? Afterwards I'll be happy :D | 10:24 |
| *** jonathanmaw has joined #buildstream | 10:24 | |
| *** nimish has quit IRC | 10:25 | |
| *** nimish has joined #buildstream | 10:25 | |
| benschubert | juergbi: aevri: https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/_scheduler/scheduler.py#L400 seems exclusive. Might be a bug if it is not | 10:25 |
| phildawson | benschubert, okay :) | 10:25 |
| aevri | benschubert: if I'm reading it right, that stops size jobs from running concurrently, but not build+size jobs. | 10:27 |
| benschubert | aevri: then that means we would have a build job running and writing temporary files in cache. Correct? That seems really bad | 10:28 |
| aevri | My understanding is that the BuildQueue is doing that every time a build job completes, which is less bad perhaps. | 10:29 |
| aevri | I think it shouldn't write a temp file though. | 10:29 |
| *** lachlan has joined #buildstream | 10:40 | |
| *** connorshea[m] has quit IRC | 10:42 | |
| juergbi | aevri: how else do you (more or less portably) atomically update a file? | 10:43 |
| aevri | I think all the writing of the cache size file happens in the main process, so there isn't a data race exactly. It's just uncertain as to whether the size job included the newly finished build jobs or not. I get the impression this is a deliberate trade-off of accuracy for better concurrency. | 10:43 |
| aevri | I was imprecise :) | 10:44 |
| aevri | I think we should use the swap trick, but not in the cas dir | 10:44 |
| *** connorshea[m] has joined #buildstream | 10:44 | |
| aevri | i.e. use the 'temp' dir and swap into the 'cas' dir as with the other calls to save_file_atomic() | 10:45 |
| aevri | in cascache.py | 10:45 |
| *** nimish has quit IRC | 10:45 | |
| juergbi | right, we can certainly do that | 10:45 |
| *** kailueke[m] has quit IRC | 10:45 | |
| *** nimish has joined #buildstream | 10:45 | |
| aevri | afaict that would fully resolve the issue, if I'm right that there isn't a data race. | 10:46 |
| juergbi | yes, makes sense to me | 10:46 |
| *** nimish has quit IRC | 10:47 | |
| aevri | cool, will make an MR, and think about how to effectively test it :) | 10:49 |
| *** nimish has joined #buildstream | 10:51 | |
| *** nimish has quit IRC | 10:53 | |
| *** kailueke[m] has joined #buildstream | 10:53 | |
| juergbi | thanks | 10:56 |
| *** nimish has joined #buildstream | 10:57 | |
| *** lachlan has quit IRC | 11:05 | |
| gitlab-br-bot | juergbi opened MR !1225 (juerg/partial-cas->master: Initial support for partial local CAS) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1225 | 11:10 |
| *** lachlan has joined #buildstream | 11:10 | |
| *** lachlan has quit IRC | 11:13 | |
| *** lachlan has joined #buildstream | 11:15 | |
| *** nimish has quit IRC | 11:15 | |
| *** nimish has joined #buildstream | 11:16 | |
| *** lachlan has quit IRC | 11:19 | |
| *** lachlan has joined #buildstream | 11:27 | |
| *** lachlan has quit IRC | 11:33 | |
| *** nimish has quit IRC | 11:36 | |
| *** nimish has joined #buildstream | 11:36 | |
| *** nimish has quit IRC | 11:55 | |
| *** lachlan has joined #buildstream | 11:58 | |
| *** nimish has joined #buildstream | 12:12 | |
| *** lachlan has quit IRC | 12:14 | |
| *** nimish has quit IRC | 12:19 | |
| *** nimish has joined #buildstream | 12:19 | |
| *** lachlan has joined #buildstream | 12:22 | |
| *** nimish has quit IRC | 12:24 | |
| *** nimish has joined #buildstream | 12:25 | |
| *** lachlan has quit IRC | 12:25 | |
| *** lachlan has joined #buildstream | 12:33 | |
| *** nimish has quit IRC | 12:34 | |
| *** nimish has joined #buildstream | 12:35 | |
| raoul | Looks like someone's trying to backport a fix to 1.2 in !1180, but the pipeline wont run, does the old gitlab ci need updating for 1.2 or something? | 12:35 |
| gitlab-br-bot | MR !1180: plugins/sources/git.py: Cope with rename returning error EEXIST https://gitlab.com/BuildStream/buildstream/merge_requests/1180 | 12:35 |
| *** lachlan has quit IRC | 12:38 | |
| juergbi | raoul: ah, I think that's because the branch is not in the main repo | 12:40 |
| juergbi | and that other repo doesn't have CI enabled | 12:40 |
| raoul | ah that makes sense, should we get them to make a local branch? or is there a way of getting it to run or the main repos CI? | 12:41 |
| juergbi | afaik, that's the only option for running on the main CI | 12:46 |
| juergbi | alternatively, they could enable CI on their repo but that can be cumbersome | 12:47 |
| *** lachlan has joined #buildstream | 12:54 | |
| *** nimish has quit IRC | 12:55 | |
| *** nimish has joined #buildstream | 12:56 | |
| *** alatiera has joined #buildstream | 13:05 | |
| *** lachlan has quit IRC | 13:15 | |
| laurence- | anyone around who's good with the buildstream website? | 13:23 |
| *** nimish has quit IRC | 13:24 | |
| laurence- | would like to add a note to the News section explaining what was decided re not doing 1.4 | 13:24 |
| *** nimish has joined #buildstream | 13:28 | |
| laurence- | looking here but i can't see the News pages - https://gitlab.com/BuildStream/website/tree/master/content/pages | 13:28 |
| SotK | the only news article currently is in https://gitlab.com/BuildStream/website/tree/master/content/2018 | 13:29 |
| SotK | you probably want to make a similar file but in content/2019 | 13:29 |
| SotK | pelican should do the rest of the magic needed for it to show up in the news page | 13:30 |
| laurence- | SotK, thanks for that - so just creating a new directory names 2019 within content will be enough? | 13:32 |
| SotK | once you also put a markdown file in it, yes | 13:34 |
| laurence- | wow, seems easy... :) | 13:34 |
| gitlab-br-bot | aevri opened issue #957 (Race on temporary file in .cache/buildstream/cas/) on buildstream https://gitlab.com/BuildStream/buildstream/issues/957 | 13:36 |
| *** nimish has quit IRC | 13:44 | |
| *** raoul_ has joined #buildstream | 14:02 | |
| *** raoul has quit IRC | 14:03 | |
| *** nimish has joined #buildstream | 14:06 | |
| *** lachlan has joined #buildstream | 14:09 | |
| *** nimish has quit IRC | 14:17 | |
| *** nimish has joined #buildstream | 14:21 | |
| gitlab-br-bot | aevri opened (was WIP) MR !1223 (aevri/doc_artifact_log->master: 'artifact log': document the 'artifacts' argument) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1223 | 14:25 |
| *** nimish has joined #buildstream | 14:27 | |
| gitlab-br-bot | jmacarthur approved MR !1210 (aevri/nodefaultsset->master: element.__init_default: treat `None` plugin_conf as if missing file + refactor) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1210 | 14:34 |
| *** laurence- is now known as laurence | 14:35 | |
| *** nimish has quit IRC | 14:41 | |
| *** lachlan has quit IRC | 14:43 | |
| *** nimish has joined #buildstream | 14:46 | |
| *** lachlan has joined #buildstream | 14:48 | |
| *** lachlan has quit IRC | 14:53 | |
| *** nimish has quit IRC | 14:56 | |
| *** lachlan has joined #buildstream | 14:57 | |
| *** raoul_ has quit IRC | 14:58 | |
| *** raoul_ has joined #buildstream | 14:58 | |
| *** raoul_ has quit IRC | 15:02 | |
| *** nimish has joined #buildstream | 15:03 | |
| *** raoul_ has joined #buildstream | 15:03 | |
| *** lachlan has quit IRC | 15:20 | |
| *** lachlan has joined #buildstream | 15:21 | |
| *** lachlan has quit IRC | 15:27 | |
| *** lachlan has joined #buildstream | 15:28 | |
| *** lachlan has quit IRC | 15:35 | |
| benschubert | will tristan be around before the end of the week? | 15:52 |
| *** lachlan has joined #buildstream | 15:55 | |
| juergbi | benschubert: I think he is going to a conference, so he might be around sporadically | 15:55 |
| laurence | FOSSASIA starts on Thursday, I think. | 15:56 |
| laurence | benschubert, I believe tristan is currently travelling | 15:57 |
| benschubert | juergbi, laurence ok thanks! | 16:00 |
| *** kapil___ has joined #buildstream | 16:02 | |
| *** lachlan has quit IRC | 16:11 | |
| *** lachlan has joined #buildstream | 16:12 | |
| *** lachlan has quit IRC | 16:25 | |
| gitlab-br-bot | jennis opened issue #958 (Autocompletion outside of a project produces stack trace) on buildstream https://gitlab.com/BuildStream/buildstream/issues/958 | 16:29 |
| jmac | ouch | 16:33 |
| jennis | heh | 16:34 |
| gitlab-br-bot | marge-bot123 merged MR !1224 (phil/fixup-external-plugin-tests->master: Move external plugin tests to seperate tox environment.) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1224 | 16:54 |
| juergbi | raoul_: I don't recall whether I've tested your source cache branch with remote execution at some point. have you? | 17:13 |
| juergbi | I don't expect any issues but would be good to verify | 17:14 |
| juergbi | or maybe mablanch could trigger a RE end-to-end pipeline on the source cache branch | 17:14 |
| *** raoul_ is now known as raoul | 17:15 | |
| raoul | If that's working that'd be good | 17:15 |
| mablanch | raoul, juergbi: That pipeline doesn't have support for arbitrary sha at the moment, it can only test master I'm afraid... | 17:17 |
| juergbi | ok, thanks, will do a test on my laptop | 17:17 |
| raoul | Ah well I shall faff around with buildgrid check if it works | 17:17 |
| mablanch | But setting up a grid should be very easy now using the instructions at https://gitlab.com/BuildGrid/buildgrid.hub.docker.com | 17:18 |
| juergbi | raoul: will you test it or shall I? | 17:18 |
| raoul | I'll have a go with the docker compose | 17:19 |
| juergbi | ok | 17:19 |
| juergbi | we should be able to merge the branch as soon as that's done | 17:19 |
| raoul | Successfully worked for the autotools example :) | 17:29 |
| raoul | the docker-compose image definitely makes life easier mablanch | 17:29 |
| juergbi | great | 17:37 |
| juergbi | jennis: would it be easy to trigger the weekly performance tests for a branch on the same hardware? or shall I compare master and branch on my laptop manually? | 17:38 |
| *** jonathanmaw has quit IRC | 18:03 | |
| gitlab-br-bot | aevri opened (was WIP) MR !1226 (aevri/tmpdir_for_cache_size->master: cascache: atomically save size via tmpdir instead) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1226 | 18:07 |
| gitlab-br-bot | raoul.hidalgocharman approved MR !1215 (phil/consolidate-repo-tests->master: Consolidate templated source tests) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1215 | 18:11 |
| *** raoul has quit IRC | 18:16 | |
| gitlab-br-bot | aevri opened issue #959 (Element ref spec might be confusing) on buildstream https://gitlab.com/BuildStream/buildstream/issues/959 | 18:26 |
| gitlab-br-bot | chiaratolentino opened issue #960 (Better error messages for `bst track`) on buildstream https://gitlab.com/BuildStream/buildstream/issues/960 | 18:35 |
| gitlab-br-bot | phildawson opened issue #961 (Expose abstract git plugin as public api.) on buildstream https://gitlab.com/BuildStream/buildstream/issues/961 | 18:48 |
| *** toscalix has quit IRC | 19:06 | |
| *** toscalix has joined #buildstream | 19:10 | |
| *** toscalix has quit IRC | 19:10 | |
| *** kapil___ has quit IRC | 19:23 | |
| *** nimish has quit IRC | 21:19 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!