| *** bochecha has quit IRC | 00:31 | |
| *** bochecha has joined #buildstream | 00:31 | |
| *** bochecha has quit IRC | 00:36 | |
| *** bochecha has joined #buildstream | 00:36 | |
| *** ppt has joined #buildstream | 01:06 | |
| ppt | CALL 415-349-5666 L0DE RADIO HOUR | 01:06 |
|---|---|---|
| ppt | https://www.youtube.com/watch?v=DIZqYgaOchY | 01:07 |
| ppt | JOIN #LRH EFNET TO JOIN THE DISCUSSION | 01:07 |
| ppt | bochecha tristan saxa jude juergbi laurenceurhegyi paulsherwood nexus tlater hergertme gitlab-br-bot csoriano inigomartinez pro[m] benbrown jjardon[m] cgmcintyre[m] kailueke[m] mattiasb mrmcq2u[m] waltervargas[m] ptomato[m] persia brlogger ironfoot | 01:07 |
| *** ppt has left #buildstream | 01:07 | |
| *** bochecha has quit IRC | 01:39 | |
| *** tristan has quit IRC | 07:06 | |
| *** tristan has joined #buildstream | 07:31 | |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: setup.py: Pinning coverage at 4.4.0 instead of 4.2) https://gitlab.com/BuildStream/buildstream/commit/a6334521433a8a917b82d826f600d854604e9ed7 | 09:28 |
| *** bochecha has joined #buildstream | 10:34 | |
| *** tristan has quit IRC | 10:35 | |
| *** valentind has joined #buildstream | 11:35 | |
| valentind | There was a change recently that seems to untar symbolic links into directories. | 11:37 |
| valentind | alsa-lib has a symlink to . as include/alsa. | 11:37 |
| valentind | Then unpacked it is replaced as an empty directory | 11:38 |
| *** tristan has joined #buildstream | 11:50 | |
| valentind | tristan, found a bug with symlinks in tar | 11:55 |
| valentind | alsa-lib has include/also to be a symlink to '.' | 11:56 |
| valentind | And when unpacked, it becomes an empty directory. | 11:56 |
| gitlab-br-bot | buildstream: merge request (tar-symlinks->master: Fix untar of symlinks. Only hardlinks are relative to top of archive and should be normalized.) #163 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/163 | 12:22 |
| gitlab-br-bot | buildstream: merge request (fix-compose-delete-with-symlink-in-path->master: Remove non canonical path from manifest after integration commands in compose plugin.) #161 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/161 | 12:27 |
| tristan | valentind, we intentionally normalize symlinks to be relative paths; this was initially an insurance policy to ensure that while operating on files outside of a sandbox, we never accidentally touch files outside of the staging directory due to following absolute symlink paths | 12:51 |
| tristan | but, that should indeed not effect symlinks to already relative paths | 12:52 |
| tristan | that is done deep in the staging process, I think you are talking about a different normalization that I did recently in tar plugin, though | 12:53 |
| tristan | while fixing the unpacking of the adwaita icon theme thing | 12:53 |
| tristan | valentind, I will be flying to europe early tomorrow morning, so I expect I'll be looking at stuff again on monday morning EU time | 12:54 |
| tristan | by the way :) | 12:54 |
| valentind | tristan, I verified the tarball in the origin bug, and it was not symlinks. | 12:55 |
| valentind | I think symlinks are always relative. | 12:55 |
| tristan | yes, but for python tarfile I believe the paths are stored in the same attribute for hardlinks and symlinks | 12:55 |
| valentind | Yes, this is why in my PR I check the type. | 12:56 |
| tristan | the things is, it has to work with the normalization in both cases, because we offer the feature of unpacking a subdir of a tar (not only the root) | 12:56 |
| tristan | I'm not saying your PR is wrong btw :) | 12:56 |
| tristan | I'll have to look at it bit closer too; can you also include a regression test for your issue in tests/sources/tar.py please ? | 12:57 |
| valentind | I can add that yes. | 12:57 |
| tristan | thanks :) | 12:57 |
| tristan | its just from the title "only hardlinks are relative to toplevel" I wonder if its the correct fix (admittedly without really looking at it) | 12:58 |
| valentind | Maybe I am wrong about "hardlink". | 12:59 |
| valentind | Maybe it is just that tar has absolute and relative links. | 12:59 |
| tristan | valentind, from what I understand; a tarball can always include a symlink to an absolute path, or even a symlink to a path that is not inside the tarball | 12:59 |
| tristan | but indeed, the hardlinks are going to be treated differently | 12:59 |
| tristan | we may have another issue I have to fix there, I think we may need to be fixing the internal inode number; or doing the preprocessing of fixing the tarinfo objects in a stage before extraction | 13:00 |
| valentind | symlinks can be absolute, but then it is to the filessytem. Not in the tarball. | 13:00 |
| * tristan kind of swept that under the rug but should take the time to fix it properly | 13:01 | |
| tristan | because they dont get extracted as hardlinks, they become separate files | 13:01 |
| tristan | (it's kindof minor, because at least until we fix #38 we still lose the significance of intentional hardlinks, because ostree doesnt care about those and blindly dedups everything) | 13:02 |
| valentind | But this is less likely to be problematic. But sure a fix would be nice. But I do not think anybody would rely on it. | 13:02 |
| tristan | nah, nobody really cares for now, but for correctness especially when rolling out bootable images for production systems | 13:02 |
| tristan | it will be important to retain which files simply are the same, vs which files are intentionally hardlinked | 13:02 |
| valentind | tristan, one of my PR got timeout in integration_unix, and then after a rebuild after rebasing, it passed. Is integration_unix known to be unstable? | 13:35 |
| valentind | brb | 13:36 |
| *** valentind has quit IRC | 13:36 | |
| *** valentind has joined #buildstream | 13:39 | |
| bochecha | valentind: it's known to be very slow | 14:02 |
| bochecha | at least it was incredibly slow for all my PRs | 14:02 |
| bochecha | Gitlab has a timeout after which it considers jobs to be failed | 14:03 |
| valentind | It stopped at exactly 1 hour. | 14:03 |
| valentind | The other build was 45 minutes. So it is not that much difference. | 14:04 |
| bochecha | yeah, I think that's the gitlab timeout by default | 14:04 |
| bochecha | on another project I had to increase that timeout to 4 hours :P (it's not tests, it's an actual cross-arch build, it's expected to be long) | 14:04 |
| tristan | hmmm, I had cranked that up | 14:06 |
| tristan | ahhh | 14:06 |
| bochecha | it's in Settings -> CI / CD -> General Pipeline Settings | 14:06 |
| tristan | valentind, because it will be in your fork repo, you can crank up the time limit in the settings | 14:06 |
| tristan | exact | 14:06 |
| tristan | and yeah it's known and integration tests are a bit disgusting for now, needs some love | 14:06 |
| valentind | tristan, I cannot use check_output because it tries to decode it as UTF-8 | 14:43 |
| valentind | I tried to use "call" and git a temp file as stdout. But then there is an exception thrown because I gave stdout parameter. | 14:44 |
| valentind | I will lift checks on stdout and stderr and I will use a temporary file. | 14:47 |
| gitlab-br-bot | buildstream: merge request (lzip->master: Add support for lzip in tar source) #162 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/162 | 14:55 |
| *** semanticdesign has joined #buildstream | 15:11 | |
| tristan | valentind, that works too ! | 15:14 |
| tristan | valentind, also, there is an API for temp files... | 15:14 |
| tristan | http://buildstream.gitlab.io/buildstream/buildstream.source.html#buildstream.source.Source.tempdir | 15:14 |
| tristan | with self.tempdir() as tmpdir: | 15:15 |
| tristan | valentind, that makes sure we cleanup any temp files within the context manager, in case of process termination etc | 15:15 |
| valentind | TemporaryFile gives a deleted file. So it is already cleanup. | 17:57 |
| valentind | If I wanted a named file I would have used that self.tempdir() however. | 17:59 |
| *** tristan has quit IRC | 18:28 | |
| *** tristan has joined #buildstream | 18:29 | |
| * tristan got disconnected, repaste just in case | 18:29 | |
| tristan | <tristan> valentind, self.tempdir() gives you a directory which buildstream cleans up when it sends it's child tasks SIGTERM and forcefully _exit()'s | 18:29 |
| tristan | <tristan> that wont happen with python's tempdir | 18:29 |
| tristan | <tristan> or python's whatever thing | 18:29 |
| tristan | so anyway, named or not; temporary files created in child tasks *must* be managed for forceful termination; the way to do that is with Plugin.tempdir() | 18:32 |
| *** semanticdesign has quit IRC | 19:14 | |
| valentind | tristan, do you mean termination of the process? | 19:57 |
| valentind | With TemporaryFile, when the fd is closed, the inode is automatically removed from the disk since it has already been unlinked. | 19:59 |
| *** tristan has quit IRC | 20:31 | |
| gitlab-br-bot | buildstream: merge request (add-accept-header->master: Replace call to legacy urllib.request.urlretrieve by urllib.request.urlopen and…) #164 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/164 | 21:16 |
| *** valentind has quit IRC | 23:03 | |
| *** bochecha has quit IRC | 23:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!