| *** alatiera_ has joined #buildstream | 03:08 | |
| *** alatiera_ has quit IRC | 03:19 | |
| *** alatiera_ has joined #buildstream | 03:35 | |
| *** dtf has quit IRC | 04:40 | |
| *** tristan has joined #buildstream | 04:53 | |
| *** ChanServ sets mode: +o tristan | 04:53 | |
| gitlab-br-bot | buildstream: merge request (tristan/notifications->master: _frontend/linuxapp.py: Fix special casing around desktop notification escape sequence) #692 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/692 | 04:54 |
|---|---|---|
| gitlab-br-bot | buildstream: merge request (tristan/pin-pytest-pylint->master: WIP: dev-requirements.txt: Pinning required pytest and pylint versions) #684 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/684 | 04:56 |
| gitlab-br-bot | buildstream: merge request (chandan/pypi-prep->master: setup.py: Add fields required for publishing on PyPi) #695 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/695 | 05:23 |
| gitlab-br-bot | buildstream: merge request (tristan/notifications->master: _frontend/linuxapp.py: Fix special casing around desktop notification escape sequence) #692 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/692 | 05:25 |
| *** leopi has joined #buildstream | 05:29 | |
| gitlab-br-bot | buildstream: merge request (willsalmon/versionTagRegrex->master: WIP: Search for tags with the *.*.* patten for version) #601 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/601 | 05:52 |
| gitlab-br-bot | buildstream: merge request (willsalmon/versionTagRegrex->master: WIP: Search for tags with the *.*.* patten for version) #601 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/601 | 05:52 |
| gitlab-br-bot | buildstream: merge request (jonathan/faster-except-elements-1.2->bst-1.2: Pipeline: Skip except_elements logic if no elements to except) #691 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/691 | 05:54 |
| tristan | juergbi, do you have an alternate take on whether we should unconditionally remove `.la` files instead of using split rules https://gitlab.com/BuildStream/buildstream/merge_requests/683 ? | 05:56 |
| juergbi | I'm a bit conflicted here. on one hand I'm in favor of removing .la files as they are not needed but occasionally cause trouble on Linux. however, I also don't like imposing so much policy from buildstream core | 05:58 |
| juergbi | I already think the current defaults do too much | 05:58 |
| tristan | yeah, I also don't like imposing this from BuildStream, while I have suffered .la issues for years; I don't see how they can cause build time problems with BuildStream | 06:04 |
| tristan | and also agree, we probably need to rethink strip commands and move them out of core | 06:04 |
| tristan | juergbi, oh, and this patch from valentind looks correct to me, but I wonder is there a more efficient way than the loop employed here: https://gitlab.com/BuildStream/buildstream/merge_requests/685/diffs ? | 06:06 |
| tristan | it's about cleaning up CAS extracts | 06:06 |
| tristan | "how to find every ref which refers to this tree" | 06:06 |
| juergbi | could read meta/keys.yaml | 06:10 |
| juergbi | .la files: I don't remember the details but I think many distros remove them as well and I don't see why BuildStream would be less problematic in this regard than other distros (especially in non-strict mode) | 06:13 |
| *** coldtom has joined #buildstream | 06:13 | |
| juergbi | or rather, distros using other build systems | 06:13 |
| tristan | juergbi, I think it is less problematic because there is only ever one .la file for a given library installed at once | 06:16 |
| tristan | But, that is just the nature of the headaches I've had with .la files, there could be others I'm not aware of | 06:16 |
| juergbi | is the same in my distro and I also had .la file issues, dropped .la files many years ago | 06:17 |
| juergbi | also, it's possible to install multiple versions of a library with buildstream as well | 06:18 |
| tristan | hmmm | 06:19 |
| tristan | juergbi, regarding keys.yaml, I don't think that fixes the loop; the loop is to find out if there are any other artifact refs which have identical content | 06:19 |
| juergbi | as the content SHA includes the meta directory, there shouldn't be any other refs than those in keys.yaml, or am I missing something? | 06:20 |
| juergbi | i.e., other builds with identical content would still use deduplication for files but would use a different extract dir | 06:21 |
| tristan | You are probably not missing anything, I am :) | 06:21 |
| juergbi | I was thinking about this myself before, though, it is indeed confusing with so many different hashes | 06:22 |
| gitlab-br-bot | buildstream: issue #385 ("Should notify when build fails") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/385 | 06:24 |
| gitlab-br-bot | buildstream: merge request (tristan/notifications->master: _frontend/linuxapp.py: Fix special casing around desktop notification escape sequence) #692 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/692 | 06:24 |
| tristan | juergbi, ah I see what you mean; two artifacts can never be identical | 06:24 |
| tristan | indeed | 06:25 |
| tristan | as the cache keys are themselves encoded | 06:25 |
| tristan | juergbi, it would be nice if you could comment on the .la files thing... personally I would like to encourage/test/improve the codepaths we explicitly layed out for the purpose of splitting and reducing output sizes | 06:27 |
| juergbi | (and logs are also rarely bit-for-bit identical) | 06:27 |
| tristan | but it could still be a good idea to remove the .la files if we really know they dont help anyone at all | 06:27 |
| juergbi | ImageMagick actually needs them (or at least it used to) but it's trivial to patch | 06:29 |
| tristan | Hmmm | 06:30 |
| tristan | even if we read keys.yaml in that loop... | 06:30 |
| tristan | do we still need to discover if there is another weak key leading to this tree ? | 06:30 |
| tristan | Or, I wonder if we even need to read keys.yaml at all | 06:31 |
| juergbi | reading keys.yaml would be to replace the loop | 06:31 |
| tristan | another weak key cannot lead to this tree | 06:31 |
| juergbi | correct, the same weak key can lead to multiple artifacts | 06:31 |
| juergbi | but a single artifact only ever has one weak key | 06:31 |
| tristan | right | 06:32 |
| juergbi | the loop should work fine. keys.yaml would be a potential optimization to avoid iterating over all keys of an element | 06:32 |
| juergbi | unlikely to make a big difference in practice | 06:32 |
| juergbi | but depends on how many old keys stay around in the cache | 06:32 |
| tristan | ah | 06:33 |
| tristan | Right, I wasn't catching on to the os.listdir() being that limited | 06:33 |
| * tristan heads out to lunch | 06:36 | |
| *** tristan has quit IRC | 06:39 | |
| gitlab-br-bot | buildstream: merge request (jonathan/faster-except-elements-1.2->bst-1.2: Pipeline: Skip except_elements logic if no elements to except) #691 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/691 | 06:51 |
| *** tristan has joined #buildstream | 07:18 | |
| *** tristan has quit IRC | 07:19 | |
| *** tristan has joined #buildstream | 07:19 | |
| *** ChanServ sets mode: +o tristan | 07:19 | |
| *** toscalix has joined #buildstream | 07:35 | |
| *** WSalmon has joined #buildstream | 07:35 | |
| *** toscalix has quit IRC | 07:39 | |
| WSalmon | hi tristan, you have mentioned that you want https://gitlab.com/BuildStream/buildstream/merge_requests/601 merging, i had put it in wip while i work out how to test but the only code that is really "our's" is the exception, given your comment shall i just merge, i did think of some test but they all involved running git to add tags to the main code base which seems very silly to me, i was wondering about making a sample git repo like the git | 07:39 |
| WSalmon | tests do but then getting bst to run inside that might be a faff. | 07:39 |
| *** toscalix has joined #buildstream | 07:40 | |
| *** toscalix has quit IRC | 07:41 | |
| *** toscalix has joined #buildstream | 07:42 | |
| tristan | WSalmon, agreed... lets just merge... only one thing... | 07:53 |
| tristan | WSalmon, let's make sure that the commit message has a link to the upstream bug report/patch for versioneer | 07:54 |
| tristan | now that I think of it, in case the commit message doesnt already have it | 07:54 |
| WSalmon | i did think of that the other day and forget, will do | 07:55 |
| tristan | WSalmon, thanks :) | 07:56 |
| tristan | One more problem in the rear view | 07:56 |
| *** rdale has joined #buildstream | 08:04 | |
| gitlab-br-bot | buildstream: merge request (willsalmon/versionTagRegrex->master: WIP: Search for tags with the *.*.* patten for version) #601 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/601 | 08:04 |
| gitlab-br-bot | buildstream: merge request (willsalmon/versionTagRegrex->master: WIP: Search for tags with the *.*.* patten for version) #601 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/601 | 08:05 |
| *** qinusty has joined #buildstream | 08:06 | |
| *** sstriker has quit IRC | 08:06 | |
| gitlab-br-bot | buildstream: merge request (willsalmon/versionTagRegrex->master: Search for tags with the *.*.* patten for version) #601 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/601 | 08:08 |
| gitlab-br-bot | buildstream: merge request (willsalmon/MR601-backport->bst-1.2: Willsalmon/mr601 backport) #696 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/696 | 08:22 |
| *** jennis has joined #buildstream | 08:37 | |
| *** solid_black has joined #buildstream | 08:40 | |
| tristan | oh, juergbi; sorry to ping you a lot today but... any thoughts on how to fix #461 https://gitlab.com/BuildStream/buildstream/merge_requests/669#note_95498854 ? | 08:45 |
| juergbi | oh, I forgot about that, will take a look | 08:48 |
| *** WSalmon has quit IRC | 08:50 | |
| tristan | I think it's difficult :-S | 08:51 |
| *** WSalmon has joined #buildstream | 08:52 | |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location->master: Don't strip pathname from debug symbol files) #697 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/697 | 08:54 |
| *** jonathanmaw has joined #buildstream | 08:57 | |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location-1.2->bst-1.2: Don't strip pathname from debug symbol files) #698 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/698 | 09:00 |
| gitlab-br-bot | buildstream: merge request (sam/debug-symbols-location->master: Don't strip pathname from debug symbol files) #529 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/529 | 09:02 |
| gitlab-br-bot | buildstream: merge request (valentindavid/extract-expiry->master: Remove artifact extracts when artifact expires in cache) #685 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/685 | 09:11 |
| tristan | WSalmon, any reason why https://gitlab.com/BuildStream/buildstream/merge_requests/696 is not merged ? | 09:29 |
| WSalmon | oh | 09:29 |
| tristan | I've been waiting and watching the pipelines, ready to rebase Sam's fix for strip-commands after your two branches land | 09:29 |
| WSalmon | no | 09:29 |
| WSalmon | the main one was merge on pipeline complete | 09:29 |
| WSalmon | i was gona ask if you want it to merge straight away or do a mini review | 09:30 |
| tristan | WSalmon, I think we did that already before, you updated the commit message right ? | 09:30 |
| tristan | WSalmon, please merge them :) | 09:31 |
| gitlab-br-bot | buildstream: merge request (willsalmon/MR601-backport->bst-1.2: Willsalmon/mr601 backport) #696 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/696 | 09:31 |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location-1.2->bst-1.2: Don't strip pathname from debug symbol files) #698 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/698 | 09:31 |
| gitlab-br-bot | buildstream: merge request (willsalmon/versionTagRegrex->master: Search for tags with the *.*.* patten for version) #601 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/601 | 09:33 |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location->master: Don't strip pathname from debug symbol files) #697 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/697 | 09:34 |
| tristan | WSalmon, thanks | 09:35 |
| gitlab-br-bot | buildstream: merge request (chandan/pypi-prep-1.2->bst-1.2: setup.py: Add fields required for publishing on PyPi) #699 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/699 | 10:11 |
| gitlab-br-bot | buildstream: merge request (chandan/pypi-prep-1.2->bst-1.2: setup.py: Add fields required for publishing on PyPi) #699 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/699 | 10:11 |
| tristan | cs-shadow, nice, I was going to chase that down right now haha :) | 10:11 |
| cs-shadow | :) just in time | 10:12 |
| tristan | cs-shadow, so what do you think our chances are of getting this all through by release ? | 10:12 |
| tristan | We really need to clean up the install story (you'll notice I added the docs task to the related issue) | 10:13 |
| cs-shadow | tristan: this is definitely doable, we should be able to release it right now if we want to. The only thing to sort out is the automation but in the worst case, we can do publish it manually | 10:13 |
| tristan | I think it doesnt need to be automated by then, that's a bit more of a nice-to-have | 10:13 |
| tristan | yeah | 10:13 |
| cs-shadow | yeah, and thanks for adding the docs task. I'll pick that up next | 10:14 |
| tristan | cs-shadow, Ok... so let's do that as soon as we land it in bst-1.2; we will need a new RC for it | 10:14 |
| cs-shadow | I can do a developer release on test-pypi so that people can try | 10:14 |
| * cs-shadow needs to run to a meeting now but will catch up in ~30 mins | 10:15 | |
| tristan | So here's a game plan, we land that change in bst-1.2; and I'll try to release an RC today or tomorrow | 10:15 |
| tristan | And then we'll have a 1.1.x released on PyPI, and get the docs spinning based on that | 10:15 |
| tristan | so it should all come together well in advance of the release | 10:15 |
| tristan | cs-shadow, sure :) | 10:15 |
| cs-shadow | Sounds good | 10:16 |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location->master: Don't strip pathname from debug symbol files) #697 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/697 | 10:38 |
| jennis | Hi all, so with the virtual directories patch, which is yet to be backported to bst-1.2, I've come across a PermissionError when trying to move files from a temporary directory to a virtual directory | 10:39 |
| tristan | virtual directories | 10:40 |
| jennis | I assume it's because we don't have the rights to delete these files, which happens when we call `vdirectory.import_files()` | 10:40 |
| tristan | Are you sure that belongs in bst-1.2 ? | 10:40 |
| jennis | No its **not** in bst-1.2 | 10:40 |
| jennis | https://paste.gnome.org/pddlrpk6l here is a stack trace of my problem | 10:40 |
| tristan | But you said "yet to be backported" so... I'm wondering... I think this is related to remote execution isn't it ? | 10:40 |
| jennis | Oh, tristan because I've been using bst-1.2 up until now, but wanted to test an element that uses the pip source plugin, so I switched to master, and found this bug | 10:41 |
| tristan | I see | 10:41 |
| jennis | Which occurs immediately when trying to import my base image | 10:42 |
| tristan | Don't mind me then :) | 10:42 |
| qinusty | Can I get a review on https://gitlab.com/BuildStream/buildstream/merge_requests/670/ if anyone has some free time? | 10:43 |
| tristan | I will turn my attention to master much more in a week or two after release, I expect jmac and others working on vdirectory will be interested in your report, though :) | 10:43 |
| jennis | So at some point, this import_files is calling os.unlink, which is essentially os.remove(), and this doesn't have the permission to remove files from the temp dir | 10:43 |
| jennis | Here is the function call: https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/element.py#L1379 | 10:43 |
| gitlab-br-bot | buildstream: merge request (Qinusty/597-non-alias-url-fix->master: WIP: Qinusty/597 non alias url fix) #700 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/700 | 10:45 |
| jennis | Does anyone have any ideas how we could solve this? | 10:45 |
| jennis | Otherwise I'll write up an issue | 10:45 |
| toscalix | tristan: based on the BuildStream calendar, we release 1.2 RC2 tomorrow | 10:46 |
| toscalix | calendar link: https://calendar.google.com/calendar/embed?src=codethink.co.uk_mpgah0uj538hnbsf4l7b4rchts%40group.calendar.google.com&ctz=Atlantic%2FCanary | 10:47 |
| tristan | toscalix, good timing then | 10:48 |
| *** bochecha has joined #buildstream | 10:55 | |
| gitlab-br-bot | buildstream: issue #601 ("PermissionError when calling `vdirectory.import_files()`") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/601 | 11:00 |
| gitlab-br-bot | buildstream: merge request (chandan/pypi-prep-1.2->bst-1.2: setup.py: Add fields required for publishing on PyPi) #699 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/699 | 11:11 |
| gitlab-br-bot | buildstream: merge request (valentindavid/faster_startup-1.2->bst-1.2: Use ArtifactCache's get_cache_size when calculating the quota) #701 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/701 | 11:24 |
| gitlab-br-bot | buildstream: merge request (valentindavid/faster_startup-1.2->bst-1.2: Use ArtifactCache's get_cache_size when calculating the quota) #701 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/701 | 11:25 |
| tristan | qinusty, I don't really have time to review thoroughly the logging refactor, but I've added some more comments to here: https://gitlab.com/BuildStream/buildstream/merge_requests/670 | 11:26 |
| tristan | qinusty, in general, I worry that this branches direction favors removing LOC too much, risking legibility and coherency in favor of less LOC | 11:28 |
| *** claro has joined #buildstream | 11:33 | |
| gitlab-br-bot | buildstream: issue #578 ("--debug option leads to a traceback") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/578 | 11:33 |
| tristan | Oh crap | 11:35 |
| tristan | /o\ | 11:35 |
| valentind | juergbi, do you have an opinion on !669? How to do it differently? It is a potentially serious bug, and I think we need a fix in 1.2. | 11:35 |
| tristan | https://gitlab.com/BuildStream/buildstream/merge_requests/699 gets *merged*, while I have a pipeline that took hours to complete, which was ahead of 699 in the queue :'( | 11:35 |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location-1.2->bst-1.2: Don't strip pathname from debug symbol files) #698 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/698 | 11:36 |
| cs-shadow | tristan: i'm sorry, i didn't know you had other things in the queue already | 11:37 |
| tristan | https://gitlab.com/BuildStream/buildstream/merge_requests/698 ... that'll only take another one hundred and twenty three minutes and fourty one seconds | 11:37 |
| juergbi | valentind: I've taken a look but need to give it some more thoughs | 11:37 |
| tristan | cs-shadow, it's just frustrating, here I am trying to close things :-/ | 11:38 |
| tristan | cs-shadow, I really didn't expect your pipeline, which was started a good twenty minutes later, to beat me heh | 11:38 |
| juergbi | tristan: maybe we're too fast moving for semi-linear history | 11:38 |
| tristan | juergbi, I don't know; it's a balance; the problem is surely compute anyway | 11:39 |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location->master: Don't strip pathname from debug symbol files) #697 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/697 | 11:39 |
| cs-shadow | tristan: I can definitely understand. I lost couple of races when trying to land the source transform stuff | 11:39 |
| tristan | juergbi, sometimes a run takes < 30min, other times > 100min | 11:39 |
| juergbi | yes, if/when CI gets faster, it should be less of an issue | 11:39 |
| tristan | I can force this one through, in fact | 11:40 |
| tristan | but meh | 11:40 |
| tristan | disheartening | 11:40 |
| cs-shadow | Anything over 30mins is really bad | 11:40 |
| valentind | juergbi, OK. Thank you. | 11:43 |
| tristan | We could disable the test CI on master and protected branches at least | 11:48 |
| tristan | And only do the docs build + publish part when running on master | 11:48 |
| tristan | That way we have only pre-merge CI for the heavy duty stuff | 11:48 |
| tristan | well, not *that* heavy duty, but still will reduce a lot of the load | 11:49 |
| tristan | its already redundant that the tests run pre/post merge | 11:49 |
| qinusty | Cheers tristan, I'll have a look through the comments and leave some of my thoughts, we can come back to this next week or so if it needs more thought. It don't see it being backported so there no urgency on getting it done | 11:52 |
| *** tristan has quit IRC | 12:05 | |
| gitlab-br-bot | buildstream: merge request (BenjaminSchubert/fix-quota-tests->master: Mock storage space checks for tests.) #702 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/702 | 12:09 |
| gitlab-br-bot | buildstream: merge request (fix-quota-tests->master: Mock storage space checks for tests.) #635 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/635 | 12:10 |
| *** tristan has joined #buildstream | 12:18 | |
| *** ChanServ sets mode: +o tristan | 12:18 | |
| toscalix | valentind: who needs to approve https://gitlab.com/BuildStream/buildstream/merge_requests/701 ? | 12:26 |
| valentind | toscalix, not sure. | 12:43 |
| valentind | jonathanmaw, I am backporting one commit from you. Can you look at !701 to say if you are fine with that? | 12:43 |
| gitlab-br-bot | buildstream: merge request (valentindavid/faster_startup-1.2->bst-1.2: Use ArtifactCache's get_cache_size when calculating the quota) #701 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/701 | 12:45 |
| jonathanmaw | valentind: 👍 | 12:47 |
| *** noisecell has quit IRC | 12:49 | |
| gitlab-br-bot | buildstream: merge request (tristan/debug-symbols-location-1.2->bst-1.2: Don't strip pathname from debug symbol files) #698 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/698 | 13:04 |
| gitlab-br-bot | buildstream: merge request (Qinusty/597-non-alias-url-fix->master: Fix ValueError traceback for URL without alias or URI scheme) #700 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/700 | 13:07 |
| qinusty | https://gitlab.com/BuildStream/buildstream/merge_requests/700 is ready for review if anyone fancies a quick review | 13:08 |
| bochecha | when `bst build` finds everything in the remote cache and just pulls it all, why does it then pushes it all back? that seems like a big waste of time :-/ | 13:09 |
| qinusty | It doesnt push bochecha, *Dives into issues to find this* | 13:10 |
| qinusty | https://gitlab.com/BuildStream/buildstream/issues/551 | 13:11 |
| bochecha | qinusty: interesting | 13:12 |
| qinusty | Basically, BuildStream still goes into the Push job, but the cache server will already have the artifact. Therefore skip | 13:12 |
| bochecha | qinusty: I do see some of those SKIPPED messages | 13:12 |
| bochecha | but not always | 13:12 |
| qinusty | Oh really? buildstream version? | 13:12 |
| bochecha | should be on the 1.2 branch, commit hash is 970c6696f2eef0bde282f82eb95ae4708ccbf308 | 13:13 |
| qinusty | Has it pulled an artifact generated by an earlier version of buildstream? | 13:13 |
| bochecha | for example, I see: | 13:13 |
| bochecha | [--:--:--][b7f2a886][ pull:basesdk-debug.bst ] INFO Downloaded artifact b7f2a886 | 13:13 |
| bochecha | [00:13:21][b7f2a886][ pull:basesdk-debug.bst ] SUCCESS freedesktop-sdk/basesdk-debug/b7f2a886-pull.106.log | 13:13 |
| bochecha | followed further down by: | 13:13 |
| bochecha | [--:--:--][b7f2a886][ push:basesdk-debug.bst ] INFO Pushed artifact b7f2a886 | 13:13 |
| bochecha | [00:00:06][b7f2a886][ push:basesdk-debug.bst ] SUCCESS Pushing artifact b7f2a886 | 13:13 |
| gitlab-br-bot | buildstream: merge request (coldtom/autotools-libtool->master: Upstream freedesktop-sdk autotools config) #683 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/683 | 13:14 |
| bochecha | qinusty: no, we have definitely rebuilt (and thus pushed) stuff since the last time we updated bst | 13:14 |
| bochecha | qinusty: here's a job as an example: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/91231669 | 13:15 |
| qinusty | Interesting, I would have to assume it is something to do with cache keys etc. It is actually being timed as 6s which confirms that it's not skipping. | 13:20 |
| bochecha | all those few seconds on many elements add up :) | 13:21 |
| bochecha | (granted, that's certainly not the worst offender for why our CI takes so long :P) | 13:21 |
| qinusty | Agreed | 13:21 |
| qinusty | It is definitely an issue | 13:22 |
| qinusty | I will happily look into it, just trying to think of how to reproduce this without triggering your build of x hours :P | 13:22 |
| gitlab-br-bot | buildstream: merge request (valentindavid/faster_startup-1.2->bst-1.2: Use ArtifactCache's get_cache_size when calculating the quota) #701 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/701 | 13:22 |
| bochecha | should I reopen #551? | 13:22 |
| qinusty | Go ahead, it's clearly a separate issue. If you could link that CI job and reference an element which you see the issue on (b7f2a886) then it'll help | 13:23 |
| bochecha | by "a separate issue", you mean I should open a new one? | 13:24 |
| qinusty | No, sorry. Separate from what I thought initially | 13:24 |
| qinusty | Same issue, incorrect assumed solution :D | 13:25 |
| gitlab-br-bot | buildstream: issue #551 ("Artifacts are being pushed just after being pulled from the CAS cache") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/551 | 13:25 |
| bochecha | qinusty: done, thanks for the hints :) | 13:28 |
| jmac | jennis: I want to try and reproduce issue #601 - is it bootstrap.bst from freedesktop-sdk? Anything unusual about your environment like unix? | 13:29 |
| qinusty | Could I clone freedesktop-sdk and reproduce this build locally using the testcache to see the issues? | 13:29 |
| qinusty | bochecha ^ | 13:29 |
| bochecha | qinusty: sure… you won't be able to push, though, I guess :-/ | 13:29 |
| qinusty | Makes sense... But I could get to the point at which it TRIES to push | 13:30 |
| qinusty | One would assume | 13:30 |
| bochecha | probably | 13:30 |
| qinusty | Do we have any documentation surrounding the install of bst-external? | 13:34 |
| *** noisecell has joined #buildstream | 13:36 | |
| bochecha | qinusty: not that I know of, I just `pip install --user git+https://…` | 13:36 |
| coldtom | i clone and pip install --user . | 13:38 |
| coldtom | but i can't recall documentation on it | 13:38 |
| bochecha | there seems to be nothing at buildstream.gitlab.io at least | 13:39 |
| jmac | tristan: This VTE fix you've pushed breaks on systems which don't define VTE_VERSION | 13:39 |
| qinusty | I'll raise an issue :D | 13:39 |
| jennis | jmac, no in my case it's a tarball of an image | 13:55 |
| valentind | tristan, maybe you can review a last time !685. | 13:55 |
| jmac | jennis: Is it private data? | 13:57 |
| jennis | jmac, yes | 13:59 |
| jmac | Drat. Well, I'll have another look through the code and see if I can figure out what's hapenning anyway | 14:00 |
| jennis | qinusty, I'll add a installation note the bst-external repo | 14:01 |
| qinusty | cheers jennis | 14:04 |
| jennis | jmac, ok, I've given as much context as I could on the issue | 14:17 |
| valentind | Is the CI down? | 14:24 |
| tristan | jmac, it *breaks* ? | 14:32 |
| *** CTtpollard has quit IRC | 14:32 | |
| tristan | I thought accesses to os.environ['sdfsdfds'] were safe for non-existing env vars | 14:33 |
| jmac | It's a KeyError here | 14:33 |
| *** xjuan has joined #buildstream | 14:33 | |
| tristan | maybe the int(None) is not raising the expected ValueError ? | 14:33 |
| tristan | hmmm | 14:33 |
| tristan | jmac, looks like a one line fix... I'm just stopping by after gym and on my way to meet people for dinner; I can fix it tmw first thing | 14:34 |
| tristan | sorry for breaking the world :-S | 14:34 |
| jmac | tristan: No problems, I don't think many people have seen it | 14:35 |
| tristan | it's only when xterm TERM doesnt set VTE_VERSION | 14:35 |
| jmac | Yeah, which seems to only be Ubuntu 17.10 IME. 18.04 sets it. | 14:35 |
| *** tpollard has joined #buildstream | 14:35 | |
| jmac | It's easy to work around, obviously | 14:35 |
| tristan | but, seeing as KeyError is thrown, I'd better check for KeyError when checking the TERM var also | 14:35 |
| *** CTtpollard has joined #buildstream | 14:35 | |
| tristan | TERM is almost *certainly* set, but nevertheless | 14:36 |
| tristan | or os.environ.get() I suppose is the right thing | 14:36 |
| valentind | tristan, I cannot get any build on the CI. Is someone taking care of that? | 14:44 |
| qinusty | jjardon knows about CI? | 14:45 |
| bochecha | valentind: is that digital ocean machines? we're having trouble with the fdo-sdk CI as well, and it seems they are having a downtime: https://status.digitalocean.com/ | 14:45 |
| gitlab-br-bot | buildstream: issue #602 ("Cannot mount disk image in sandbox") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/602 | 15:07 |
| cs-shadow | tristan: when you get a chance, can you please create an account on https://pypi.org/ and https://test.pypi.org? Also, ping me your username when you're done. Not urgent | 15:09 |
| *** WSalmon has quit IRC | 15:11 | |
| *** tristan has quit IRC | 15:24 | |
| *** tristan has joined #buildstream | 15:24 | |
| qinusty | bochecha: Any chance you see a trend in the elements which are pulled and pushed? https://gitlab.com/BuildStream/buildstream/issues/551#note_96195990 | 15:45 |
| bochecha | qinusty: I'll have a look | 15:58 |
| qinusty | I would also be interested to see whether it was the same elements each time? Do you have more CI builds which show this happening? | 16:00 |
| toscalix | tpollard: tiago will be back next wednesday | 16:09 |
| qinusty | Has anyone noticed how gitlab logs when viewed as raw are a little.... Screwed? https://gitlab-artifacts.s3.amazonaws.com/5c/63/5c63681383a059a6dab32deb7a44d5cf535132c0c0c35105f4ce1334be7e8bef/2018_08_22/91186759/78635410/job.log?response-content-type=text/plain%3B%20charset%3Dutf-8&response-content-disposition=inline&X-Amz-Expires=600&X-Amz-Date=2 | 16:09 |
| qinusty | 0180822T160854Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJTOFWQ3GL4O3Q3FA/20180822/us-east-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=fccb58256eb9f300b9aa588357ea5e7b3bc77f25c9795f3f7decbd8cde0ac3df | 16:09 |
| qinusty | I assume due to the coloured text etc | 16:10 |
| qinusty | https://gitlab.com/gitlab-org/gitlab-runner/issues/2250 ? | 16:12 |
| bochecha | qinusty: I have this one: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/jobs/91298063 | 16:13 |
| qinusty | You only have the single remote cache right bochecha? | 16:28 |
| bochecha | qinusty: yes | 16:28 |
| bochecha | qinusty: basically, if you clone freedesktop-sdk, then you have the same config as I have :) | 16:29 |
| qinusty | I have :D, what're you building? | 16:29 |
| qinusty | bst build ... | 16:29 |
| coldtom | all.bst it looks like | 16:30 |
| *** solid_black has quit IRC | 16:33 | |
| qinusty | I'll run a build on a modified version of 1.1.6 to check the cache keys of the element during pull and push jobs. | 16:34 |
| *** noisecell has quit IRC | 16:48 | |
| *** leopi has quit IRC | 16:53 | |
| *** jonathanmaw has quit IRC | 17:03 | |
| *** toscalix has quit IRC | 17:08 | |
| bochecha | qinusty: sorry, missed your messages; yes, the CI builds all.bst | 17:15 |
| gitlab-br-bot | buildstream: merge request (valentindavid/faster_startup-1.2->bst-1.2: Use ArtifactCache's get_cache_size when calculating the quota) #701 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/701 | 17:28 |
| *** leopi has joined #buildstream | 17:29 | |
| *** rdale has quit IRC | 17:51 | |
| *** alatiera_ has quit IRC | 17:51 | |
| *** alatiera_ has joined #buildstream | 17:53 | |
| *** alatiera_ has quit IRC | 18:25 | |
| *** alatiera_ has joined #buildstream | 18:25 | |
| *** leopi has quit IRC | 18:30 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/inconsistant-workspace->master: Improve error message for deleted open workspaces) #703 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/703 | 18:34 |
| gitlab-br-bot | buildstream: merge request (chandan/fix-mainfest-dev-requirements->master: MANIFEST.in: Fix include for dev-requirements.txt) #704 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/704 | 18:37 |
| gitlab-br-bot | buildstream: merge request (chandan/fix-mainfest-dev-requirements->master: MANIFEST.in: Fix include for dev-requirements.txt) #704 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/704 | 18:37 |
| gitlab-br-bot | buildstream: merge request (chandan/fix-mainfest-dev-requirements->master: MANIFEST.in: Fix include for dev-requirements.txt) #704 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/704 | 18:38 |
| gitlab-br-bot | buildstream: merge request (chandan/fix-mainfest-dev-requirements->master: MANIFEST.in: Fix include for dev-requirements.txt) #704 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/704 | 18:38 |
| *** alatiera_ has quit IRC | 19:18 | |
| gitlab-br-bot | buildstream: merge request (chandan/fix-mainfest-dev-requirements-1.2->bst-1.2: MANIFEST.in: Fix include for dev-requirements.txt) #705 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/705 | 20:13 |
| *** TingPing has left #buildstream | 20:49 | |
| *** tristan has quit IRC | 21:18 | |
| gitlab-br-bot | buildstream: merge request (chandan/fix-mainfest-dev-requirements-1.2->bst-1.2: MANIFEST.in: Fix include for dev-requirements.txt) #705 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/705 | 23:44 |
| gitlab-br-bot | buildstream: merge request (chandan/fix-mainfest-dev-requirements->master: MANIFEST.in: Fix include for dev-requirements.txt) #704 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/704 | 23:44 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!