| valentind | tristan, I have found a weird bug. localedef crash when using SafeHardLinks. Sometimes mmap returns a pointer that is not properly mapped. | 00:14 |
|---|---|---|
| *** tristan has quit IRC | 00:42 | |
| *** bochecha has joined #buildstream | 04:15 | |
| gitlab-br-bot | push on buildstream@zip (by Mathieu Bridon): 12 commits (last: Catch attempts to compose a list) https://gitlab.com/BuildStream/buildstream/commit/8c4d8ca2b7c886690fb933428c6768c745667db7 | 05:42 |
| gitlab-br-bot | buildstream: merge request (zip->master: WIP: Add a new zip source) #131 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/131 | 05:42 |
| gitlab-br-bot | push on buildstream@zip (by Mathieu Bridon): 3 commits (last: tar: Move most of the code to a new base class) https://gitlab.com/BuildStream/buildstream/commit/e45e566ec1829e76a61961ee5b18b7d5e97238a7 | 05:43 |
| gitlab-br-bot | buildstream: merge request (zip->master: WIP: Add a new zip source) #131 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/131 | 05:43 |
| *** bochecha has quit IRC | 06:14 | |
| *** tristan has joined #buildstream | 06:47 | |
| tristan | valentind, care to file it with the whole story ? | 06:49 |
| gitlab-br-bot | push on buildstream@zip (by Mathieu Bridon): 4 commits (last: tests: Reuse utils.sha256sum) https://gitlab.com/BuildStream/buildstream/commit/91df512270ea1d530df1dd76e48c622076f4db31 | 08:25 |
| gitlab-br-bot | buildstream: merge request (zip->master: WIP: Add a new zip source) #131 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/131 | 08:25 |
| *** persia has quit IRC | 09:12 | |
| *** persia has joined #buildstream | 09:12 | |
| *** tlater has joined #buildstream | 09:35 | |
| *** ssam2 has joined #buildstream | 09:35 | |
| valentind | tristan, Yes. I will. I will try to make a simple C program that reproduce it. | 09:45 |
| valentind | I think it is: call open read+write+create on a non existing file. Write on it, mmap that fd, hard link the file to another name and unlink the original name. | 09:47 |
| valentind | Then try to access the mmaped memory. | 09:47 |
| tristan | ouch, interesting | 09:48 |
| valentind | I run localedef with a prefix on /tmp for now and then move the result. | 09:50 |
| tristan | valentind, fwiw - I have an idea about the correct fix - I think some of it is discussed in this ostree report: https://github.com/ostreedev/ostree/issues/925 | 09:51 |
| tristan | valentind, at least on our side, we should not be re-duplicating files from hardlinks, when they dont link outside of the sandbox | 09:52 |
| valentind | Oh, I had weird bugs sometimes where tar would not work because files changed. | 09:52 |
| tristan | Also, when we stage hardlinks from the artifact into the sandbox - we should preserve the state of which files were linked together, in case they are intentionally hardlinks; but fixing that depends on fixing the upstream ostree bug as well, because ostree assumes that "every identical file should be deduplicated as a hardlink" | 09:53 |
| tristan | which loses the precious information of which files were initially linked together at ostree commit time | 09:53 |
| tristan | so that is impossible to restore at checkout time | 09:53 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 2 commits (last: plugins/sources/tar.py: Ignore possible leading '.' directory.) https://gitlab.com/BuildStream/buildstream/commit/9870257fd81299c282f909d04b88d821f90b3743 | 10:03 |
| gitlab-br-bot | push on buildstream@multiple_targets (by Tristan Maat): 19 commits (last: Adjust build command to multiple targets) https://gitlab.com/BuildStream/buildstream/commit/80dbc08ba07743f78e51bb8f452c03e92ab72303 | 10:15 |
| tristan | tlater, minor nitpick, lets make the name of the arguments consistent | 10:19 |
| tristan | tlater, before we had 'element' and 'target', I guess we started with 'target' and in some cases decided the element is not used in a 'target' context | 10:19 |
| tristan | tlater, lets call them 'elements' across the board then ? | 10:20 |
| tristan | Note; this will effect absolutely nothing except for man pages and `bst --help` output | 10:20 |
| tlater | tristan: Yup, that's a good note. | 10:20 |
| valentind | tristan, open+write+mmap is enough to reproduce the bug. | 10:24 |
| tristan | valentind, ok thanks, please file it at https://gitlab.com/BuildStream/buildstream/issues :) | 10:27 |
| tristan | I got loads going on, lets not forget, it's probably a critical bug | 10:27 |
| tristan | valentind, better to mention that it effects localedef, also | 10:27 |
| gitlab-br-bot | buildstream: issue #143 ("mmap does not work properly on SafeHardlinks") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/143 | 10:32 |
| tristan | valentind, thanks ! | 10:35 |
| valentind | I wonder how fuse deals with r+w mmaps. | 10:36 |
| valentind | Does it make write calls on flush? | 10:36 |
| tristan | Dont know, what we use is an internal fork of fusepy with a minor fix for bsd I think | 10:39 |
| tristan | or for something | 10:39 |
| ssam2 | it's a fix to add ppc64 support :-) | 10:39 |
| tristan | valentind, we use a higher level API of fuse which doesnt care about mmap - I assume this means do nothing and let the underlying fs handle it passthrough style | 10:40 |
| tristan | which seems appropriate, you would need to open for writing first to get an fd to mmap | 10:40 |
| valentind | I looked at the fuse C api, and there was no mmap. | 10:40 |
| valentind | Apart for block device. | 10:41 |
| valentind | Seems related https://sourceforge.net/p/fuse/mailman/message/32694975/ | 10:49 |
| *** givascu has joined #buildstream | 11:03 | |
| *** tristan has quit IRC | 11:03 | |
| *** tlater has quit IRC | 11:13 | |
| *** tlater has joined #buildstream | 11:14 | |
| gitlab-br-bot | buildstream: merge request (incremental-build->master: Add support for doing incremental builds) #126 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/126 | 11:14 |
| *** valentind has quit IRC | 11:23 | |
| *** adds68_ has joined #buildstream | 11:30 | |
| *** adds68 has quit IRC | 11:31 | |
| ssam2 | updating .expected files in the test suite when rebasing is very tedious! | 11:49 |
| ssam2 | is there a shortcut way? | 11:49 |
| gitlab-br-bot | push on buildstream@multiple_targets (by Tristan Maat): 3 commits (last: main.py: Make CLI wording for elements consistent) https://gitlab.com/BuildStream/buildstream/commit/d9bc4ae4e62a1b38bdb75ea78c14c74da2772b12 | 11:49 |
| ssam2 | i guess i can script it myself | 11:50 |
| tlater | ssam2: There's no shortcut way yet, no | 11:53 |
| ssam2 | for future reference: https://pastebin.com/v1c8aNu1 | 11:58 |
| tlater | \o/ | 12:08 |
| gitlab-br-bot | push on buildstream@multiple_targets (by Tristan Maat): 10 commits (last: _loader.py: Adjust the loader to support multiple targets) https://gitlab.com/BuildStream/buildstream/commit/afb03daad6700a3f06554059fc3cb6a71e98ef6c | 12:10 |
| gitlab-br-bot | push on buildstream@sam/1.0-pre (by Sam Thursfield): 10 commits (last: buildelement.py: Issue #121 - Remove traces of pre-/post- commands) https://gitlab.com/BuildStream/buildstream/commit/506196de5c9cbf2aca172b1e224d24794e533fbb | 12:12 |
| gitlab-br-bot | push on buildstream@multiple_targets (by Tristan Maat): 4 commits (last: Stop using pipeline.target for UI widgets) https://gitlab.com/BuildStream/buildstream/commit/dd5fdea5e295a74df5baaed0eb983ff7655a0881 | 12:13 |
| gitlab-br-bot | buildstream: merge request (multiple_targets->master: Multiple targets) #135 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/135 | 12:18 |
| gitlab-br-bot | push on buildstream@multiple_targets (by Tristan Maat): 3 commits (last: main.py: Make CLI wording for elements consistent) https://gitlab.com/BuildStream/buildstream/commit/e99f1ee71111540bf1e55285e40a2774336e41b2 | 12:19 |
| gitlab-br-bot | buildstream: merge request (multiple_targets->master: Multiple targets) #135 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/135 | 12:19 |
| * tlater ended up splitting this off the tracking changes branch, because it was actually a fairly large change | 12:21 | |
| tlater | It had a lot of collateral | 12:21 |
| gitlab-br-bot | push on buildstream@multiple_targets (by Tristan Maat): 3 commits (last: main.py: Make CLI wording for elements consistent) https://gitlab.com/BuildStream/buildstream/commit/194603ece50bb120748a5d0d4f6ee9aa857dd453 | 13:29 |
| gitlab-br-bot | buildstream: merge request (multiple_targets->master: Multiple targets) #135 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/135 | 13:29 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 12 commits (last: _platform/linux.py: Add preflight check to detect user namespaces) https://gitlab.com/BuildStream/buildstream/commit/a9a2bcece3b90092d9260f5c46675a0e17df381d | 13:39 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: WIP: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 13:39 |
| *** WSalmon has joined #buildstream | 14:50 | |
| tlater | ssam2: Do you have a test element for the unhandled compose-list-exception you recently fixed? | 15:36 |
| tlater | ssam2: Gah, sorry, nevermind, I found your test cases. | 15:37 |
| *** adds68__ has joined #buildstream | 16:00 | |
| *** adds68_ has quit IRC | 16:00 | |
| *** WSalmon has quit IRC | 16:19 | |
| ssam2 | the bst-external repo isn't working for me | 16:30 |
| ssam2 | i cloned it and did `pip3 install --user .` | 16:30 |
| ssam2 | but i still get: Error loading pipeline: No Element type registered for kind 'x86image' | 16:30 |
| ssam2 | ah, https://gitlab.com/BuildStream/buildstream/merge_requests/125 | 16:33 |
| ssam2 | except that that branch is empty | 16:33 |
| ssam2 | seems I need to use buildstream-external:x86-image for the `kind` | 16:34 |
| * ssam2 will send a patch to update the readme at https://gitlab.com/BuildStream/defs2bst/ | 16:35 | |
| tlater | ssam2: Do you know what tristan aims for in this comment: https://gitlab.com/BuildStream/buildstream/merge_requests/129#note_44913951 ? | 16:36 |
| tlater | I see no convenient way to keep a stack of provenances, is he suggesting a global variable into which every provenance is written as we parse? | 16:37 |
| ssam2 | i didn't think on it too deeply | 16:40 |
| ssam2 | it does seem like we lose the provenance inside the _yaml module | 16:40 |
| ssam2 | so it can't report useful errors itself, and has to depend on the code calling into it to attach provenance info | 16:40 |
| ssam2 | not sure if a global variable is the best way; perhaps a keyword argument to the functions would be better ? | 16:42 |
| tlater | Simplest way to do that is an optional kwarg named 'provenance', but it seems like tristan had a very specific solution in mind... | 16:42 |
| ssam2 | that may be the case; but we'll have to wait for him to explain it if so | 16:42 |
| tlater | I suppose I'll do that for now and pester tristan about all the things I need to pester him about tomorrow. | 16:43 |
| tlater | ta though, ssam2 :) | 16:43 |
| gitlab-br-bot | buildstream: merge request (benchmark->master: WIP: Benchmark) #136 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/136 | 16:48 |
| *** tristan has joined #buildstream | 16:59 | |
| tristan | Odd, we have https://gitlab.com/BuildStream/buildstream/-/jobs/38687858 failing and https://gitlab.com/BuildStream/buildstream/-/jobs/38698246 passing... same master tip | 17:03 |
| tristan | error is expected cached, but received downloadable state | 17:04 |
| ssam2 | tests/frontend/pull.py::test_push_pull_all FAILED | 17:05 |
| tristan | seems like we have a race condition when it comes to trusting ostree to report that something is cached directly after caching it ? | 17:05 |
| ssam2 | could be | 17:05 |
| gitlab-br-bot | buildstream: Tristan Maat created branch 142-potentially-printing-provenance-more-than-once-in-loaderrors | 17:07 |
| gitlab-br-bot | push on buildstream@142-potentially-printing-provenance-more-than-once-in-loaderrors (by Tristan Maat): 1 commit (last: Issue #142: Ensure we don't append provenances twice) https://gitlab.com/BuildStream/buildstream/commit/7d8b0a95cca0391d998f5853da79d766332c393f | 17:08 |
| *** givascu has quit IRC | 17:11 | |
| gitlab-br-bot | buildstream: issue #144 ("Possible race condition - inconsistent passing test") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/144 | 17:13 |
| adds68__ | In https://buildstream.gitlab.io/buildstream/elements/autotools.html#module-elements.autotools what format are the conf-args expected as? | 17:16 |
| adds68__ | It looks like a string in that example, but i'm not sure | 17:17 |
| tristan | adds68__, all "variables" are strings | 17:17 |
| adds68__ | tristan, i'm struggling to write them out using bstDumper | 17:18 |
| adds68__ | tristan, i first thought i could pass the conf-args as a list of strings, but those get prefixed with '-' when i dump them | 17:18 |
| tristan | adds68__, I cant really help much at 2am :-/ | 17:18 |
| * tristan is just doing a drive by really | 17:19 | |
| adds68__ | tristan, haha understood | 17:19 |
| ssam2 | adds68__, a list of strings is not the same as a string | 17:19 |
| tristan | adds68__, it is only ever *one string* | 17:19 |
| tristan | adds68__, every variable itself is one string | 17:19 |
| gitlab-br-bot | buildstream: issue #145 ("Tar Source should normalize leading `.`") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/145 | 17:19 |
| *** tiagogomes has quit IRC | 17:20 | |
| *** tiagogomes has joined #buildstream | 17:20 | |
| tristan | adds68__, one tip about bstDumper ... it's known that in some cases, serializing strings which have trailing whitespace results in ugly serialization (i.e. quoted with escape sequences looking all jumbled) | 17:22 |
| tristan | if you run into ugly quoted strings in the output, it may be work-aroundable by stripping trailing whitespace | 17:22 |
| adds68__ | ssam2, aye, guess i'll just having to convert my list into a string manually | 17:23 |
| adds68__ | tristan, cheers shall keep that in mind :) | 17:23 |
| tristan | " ".join(list_of_configure_flags) | 17:23 |
| adds68__ | tristan, i need to append the commands with new line delimiters in order to make them look as expected | 17:24 |
| adds68__ | should be trivial really | 17:24 |
| tristan | adds68__, also make sure that it really is conf-args you want - I suspect you probably want conf-extra instead | 17:27 |
| adds68__ | tristan, for some reason i'm getting a '-' appended after conf-args: | | 17:28 |
| tristan | adds68__, i.e. conf-args is more general, we probably want to set it up on a per-project basis, conditional on whether you're building a flatpak (different prefix ?) or not | 17:28 |
| adds68__ | tristan, https://pastebin.com/MT361PKZ | 17:28 |
| adds68__ | Not sure why | 17:28 |
| tristan | while conf-extra is more intended for individual modules tweaking the commands | 17:29 |
| tristan | or for sweeping statements, it depends, just saying give that some thought | 17:29 |
| tristan | it might be actually best to override configure-commands by appending to it, giving more flexibility to how you use the project | 17:30 |
| tristan | adds68__, take a look at how we're converting JHBuild modulesets as an example, appending module specific args to the configure-commands lines allows the project to define conf-extra on a project wide basis, without screwing with individual module configure args | 17:31 |
| adds68__ | tristan, i'm passing these commands to conf-args https://github.com/flatpak/freedesktop-sdk-images/blob/da3ab5e75220e2edbcd219a959519ad29a3a3af5/org.freedesktop.Sdk.json.in#L132 | 17:31 |
| adds68__ | tristan, will do, stuff like this which takes some though in order to make it work for multiple modules etc | 17:31 |
| tristan | that is build-commands | 17:32 |
| tristan | I think you rather mean like this: https://github.com/flatpak/freedesktop-sdk-images/blob/da3ab5e75220e2edbcd219a959519ad29a3a3af5/org.freedesktop.Sdk.json.in#L2349 | 17:32 |
| tristan | e.g. "config-opts", is the topic | 17:33 |
| adds68__ | tristan, oh sorry wrong line linked | 17:33 |
| adds68__ | tristan, https://github.com/flatpak/freedesktop-sdk-images/blob/da3ab5e75220e2edbcd219a959519ad29a3a3af5/org.freedesktop.Sdk.json.in#L1031 | 17:33 |
| tristan | right | 17:33 |
| tristan | adds68__, https://gitlab.com/BuildStream/jhbuild2bst/blob/master/jhbuild/modtypes/autotools.py#L79 | 17:34 |
| adds68__ | tristan, that looks useful! Shall take a look tomorrow, got to run, thanks for your help and get some sleep :P | 17:35 |
| tristan | adds68__, with jhbuild, we override configure-commands for each element, and append the module specific configure options, like: %{configure} %{conf-args} --enable-module-specific-option | 17:35 |
| tristan | that lets us do project-wide tricks later with %{conf-extra} | 17:35 |
| paulsherwood | tristan: when i finally get around to running buildstream, i hope i don't find it way too complicated :) | 17:36 |
| paulsherwood | it seems to be growing loads of functionality | 17:36 |
| paulsherwood | and 14kloc now... | 17:38 |
| paulsherwood | excluding tests | 17:38 |
| tristan | paulsherwood, its a bit biggish, but since you ask; count with radon is better | 17:52 |
| tristan | note that for bst, ~16K SLOC yields ~8K LLOC | 17:52 |
| tristan | ybd checkout is at 2.7K SLOC to 2.1K LLOC (i.e., almost no whitespace or comments there) | 17:53 |
| tristan | anyway, I dont expect you to find it too complicated | 17:54 |
| *** ssam2 has quit IRC | 18:09 | |
| *** valentind has joined #buildstream | 19:27 | |
| *** tristan has quit IRC | 21:22 | |
| *** tristan has joined #buildstream | 22:12 | |
| *** valentind has quit IRC | 23:02 | |
| *** tristan has quit IRC | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!