| *** jonathanmaw has joined #buildstream | 09:06 | |
| *** valentind has joined #buildstream | 09:20 | |
| valentind | tristan, I realize forgot to mention the issue in the commit message like you asked. I can fix that later today. | 09:25 |
|---|---|---|
| *** adds68 has joined #buildstream | 09:32 | |
| *** bethw has joined #buildstream | 10:13 | |
| *** tristan has quit IRC | 10:15 | |
| gitlab-br-bot | push on buildstream@juerg/recursive-pipelines (by Jürg Billeter): 15 commits (last: .gitlab-ci.yml: Use specific version of buildstream-fedora Docker image) https://gitlab.com/BuildStream/buildstream/commit/5c37208c5c5df1925a0bb280933772feb04cb674 | 10:30 |
| gitlab-br-bot | buildstream: merge request (juerg/recursive-pipelines->master: WIP: Junctions and subprojects) #160 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/160 | 10:30 |
| *** tristan has joined #buildstream | 10:49 | |
| *** valentind has quit IRC | 10:59 | |
| gitlab-br-bot | buildstream: merge request (fix-compose-delete-with-symlink-in-path->master: Remove non canonical path from manifest after integration commands in compose plugin.) #161 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/161 | 11:04 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Remove all deleted paths from manifest after integration commands in) https://gitlab.com/BuildStream/buildstream/commit/02cc8d03faf9201847e173d2d1474f8dd0fd3c30 | 11:04 |
| gitlab-br-bot | buildstream: merge request (sam/compose-log-splits->master: WIP: Log details of artifact splitting when building 'compose' elements) #140 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/140 | 11:07 |
| *** ssam2 has joined #buildstream | 11:19 | |
| juergbi | ta ssam2 | 11:21 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 11 commits (last: _artifactcache.py: Fixed stack trace regression in pushing of artifacts.) https://gitlab.com/BuildStream/buildstream/commit/628e9a23f8b9d14f0216f85240b5ed148a08b117 | 11:22 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 11:22 |
| tlater | tristan: Would you mind checking my new test case a bit more thoroughly once you get around to reviewing again? I'm not 100% sure it tests properly, since I don't think I entirely understand no-strict mode. | 11:30 |
| gitlab-br-bot | push on buildstream@juerg/recursive-pipelines (by Jürg Billeter): 15 commits (last: Remove all deleted paths from manifest after integration commands in) https://gitlab.com/BuildStream/buildstream/commit/02cc8d03faf9201847e173d2d1474f8dd0fd3c30 | 11:50 |
| gitlab-br-bot | buildstream: merge request (juerg/recursive-pipelines->master: WIP: Junctions and subprojects) #160 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/160 | 11:50 |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: Testing distchecks) https://gitlab.com/BuildStream/buildstream/commit/923f093a804ff7264484013dc497fd347182ffc4 | 11:52 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 2 commits (last: _pipeline.py: Merge the track queue into the scheduler run) https://gitlab.com/BuildStream/buildstream/commit/bf27bbabdeede96e118c1fa403898cd2cd284fb3 | 12:00 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 12:00 |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: fixup) https://gitlab.com/BuildStream/buildstream/commit/03069b83522c5fe1365a16ce00f45c8a3f360988 | 12:00 |
| tristan | tlater, non-strict mode still requires that "the output of a full build be based on the exact versions of every element" | 12:03 |
| gitlab-br-bot | push on buildstream@sam/multiple-caches (by Sam Thursfield): 3 commits (last: Filter out 0 length URLs) https://gitlab.com/BuildStream/buildstream/commit/dde84624cf309ce4c9cfa79bf7ee487757bcd263 | 12:03 |
| gitlab-br-bot | buildstream: merge request (sam/multiple-caches->master: WIP: multiple remote cache support) #166 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/166 | 12:03 |
| tristan | tlater, without requiring that elements be built against the exact version of their dependencies | 12:03 |
| tristan | so consider that for the full array of elements; you will always get all the correct versions, but you wont know that reverse dependencies have been rebuilt or not | 12:04 |
| tristan | So your question is, how do we treat this with build & track in the same session | 12:04 |
| tristan | tlater, I would argue that you *still* want to block reverse dependency builds of things which have not yet been tracked | 12:05 |
| tristan | I think that is the straight answer | 12:05 |
| tristan | tlater, in *any* case, you cannot construct the cache keys, even in non-strict mode, until you can get the cache key of the dependencies | 12:05 |
| tristan | and you *must not* be able to know the cache key of an element which is going to be tracked, but has not yet been tracked | 12:06 |
| tristan | tlater, does that answer it ? | 12:06 |
| tlater | Yup, ta :) | 12:06 |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: Fixup) https://gitlab.com/BuildStream/buildstream/commit/b783112359d37719b71654760ae9d6409d53f655 | 12:08 |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: Fixup) https://gitlab.com/BuildStream/buildstream/commit/26fe1d7431464d7291c02d413732ccc10af0ad1e | 12:10 |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: Fixup) https://gitlab.com/BuildStream/buildstream/commit/4f7292372b8d695416f42e349d6e8a20b0660182 | 12:30 |
| juergbi | tristan: for subproject-specific artifact servers, do we care about this only for push or also for pull? | 12:38 |
| juergbi | i.e., it looks like it would actually be pretty simple to just add pull urls from subprojects and then allow using all artifact servers for pulling elements from any project in the pipeline | 12:38 |
| juergbi | the only question would be how to handle priorities for urls coming from subprojects | 12:39 |
| juergbi | or shall i keep pull urls separate for each subproject (as i originally planned), same as for push? | 12:39 |
| jonathanmaw | hrm, I'm having trouble running builds for some reason https://pastebin.com/W11Y3xF9 | 12:45 |
| jonathanmaw | when staging dependencies, it complains about "Artifact missing" | 12:45 |
| juergbi | dangling items in the artifact cache? | 12:46 |
| jonathanmaw | juergbi: I'll try running with a blank cache and see if that works | 12:47 |
| juergbi | ok. if you haven't manually manipulated the cache it shouldn't happen, though | 12:47 |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: Fixup) https://gitlab.com/BuildStream/buildstream/commit/c219573296bf7907f92b0e6bc4c9b958f8914a2b | 12:50 |
| jonathanmaw | I'm seeing "Artifact missing for $foo" when using an empty cache | 12:52 |
| jonathanmaw | once this build fails I'll see what refs are in the ostree cache | 12:52 |
| tristan | juergbi, I think limiting the access on per project basis is the principal of least surprise | 12:53 |
| tristan | juergbi, this will probably collide a bit with sam's work on multiple cache configurations | 12:53 |
| ssam2 | what i've currently done is merge all the caches together, effectively | 12:54 |
| tristan | I guess there are a few approaches; but I think it should be explicit - if you want to build a subproject pushing into your depending project's artifact share; then we can add something for that | 12:54 |
| ssam2 | so if any cache has a ref, it gets pulled from that cache. (with priority ordering taken into account) | 12:54 |
| tristan | ssam2, right, but you've done that in a setting where only one project exists in a session | 12:54 |
| ssam2 | yeah | 12:55 |
| ssam2 | so adding per-project rules will make it a bit more complex | 12:55 |
| tristan | projects declare their artifact caches | 12:55 |
| tristan | so those should be the one(s) used for each project respectively in a session | 12:55 |
| tristan | we can however have the junction element which bridges with a depending project, have some configuration to force onto it's sub projects | 12:56 |
| tristan | I dont think that should block landing of the junction/multiple project work, but it's probably desirable to let the project override artifact servers of the projects it depends on | 12:56 |
| *** bochecha has joined #buildstream | 13:54 | |
| *** xjuan has joined #buildstream | 14:16 | |
| *** mcatanzaro has joined #buildstream | 14:44 | |
| nexus | What would be the best "simple project" to use for a "Your first project" tutorial/doc | 14:55 |
| *** jude has quit IRC | 14:58 | |
| nexus | also, what base platform do we want to use in them? | 14:59 |
| *** xjuan_ has joined #buildstream | 15:04 | |
| *** bochecha has quit IRC | 15:05 | |
| *** xjuan has quit IRC | 15:06 | |
| nexus | It seems like having "'" in the "name" field in "project.conf" causes an invalid refspec error | 15:09 |
| jonathanmaw | my build continued and failed, the only ref in the ostree was for baserock/gnu-toolchain-base | 15:11 |
| jonathanmaw | i.e. the element that was pulled straight from the cache | 15:12 |
| jonathanmaw | and it looks like my build failed because stage1-binutils was trying to fetch itself from cache while staging dependencies | 15:12 |
| jonathanmaw | I'll make sure I'm using the latest version of buildstream | 15:12 |
| jonathanmaw | well, it seems to be working now I'm using the latest master, so I guess problems solved _o_ | 15:35 |
| *** jude has joined #buildstream | 15:45 | |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: Testing distchecks) https://gitlab.com/BuildStream/buildstream/commit/3215d9ef2f10ece0b9b85e898cf581779ebae657 | 16:01 |
| juergbi | ssam2: i see the following issue with pytest on your branch: | 16:35 |
| juergbi | self.urls = list(utils.deduplicate( | 16:36 |
| juergbi | > project_extra_urls + project.artifact_urls + context.artifact_urls)) | 16:36 |
| juergbi | E TypeError: can only concatenate list (not "NoneType") to list | 16:36 |
| juergbi | context.artifact_urls is None for some reason. have you already looked into this? | 16:36 |
| ssam2 | perhaps not, let me run the full test suite | 16:37 |
| juergbi | looking at the code context.artifact_urls should be initialized as part of Context.load(). not sure how it can still be None | 16:38 |
| ssam2 | probably it's being overridden by a test somewhere | 16:39 |
| juergbi | ah, maybe a test is failing to call Context.load() | 16:41 |
| juergbi | we could just initialize artifact_urls to [] instead of None | 16:42 |
| ssam2 | true, not much point initializing it to a value that break subsequent code | 16:43 |
| juergbi | yes, all tests pass now, even with my branch that supports subproject-specific options | 16:44 |
| juergbi | that said, i haven't written any tests yet for push/pull with subprojects | 16:45 |
| gitlab-br-bot | push on buildstream@sam/multiple-caches (by Sam Thursfield): 1 commit (last: Initialize content.artifact_urls to []) https://gitlab.com/BuildStream/buildstream/commit/ab50eb38ffd67796118a474e2b5d741e35496ab1 | 16:45 |
| gitlab-br-bot | buildstream: merge request (sam/multiple-caches->master: WIP: multiple remote cache support) #166 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/166 | 16:46 |
| gitlab-br-bot | push on buildstream@juerg/recursive-pipelines (by Jürg Billeter): 20 commits (last: Filter out 0 length URLs) https://gitlab.com/BuildStream/buildstream/commit/37fa828949f9e1e162fdca0e82d9c47e98e12fc3 | 17:09 |
| gitlab-br-bot | buildstream: merge request (juerg/recursive-pipelines->master: WIP: Junctions and subprojects) #160 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/160 | 17:09 |
| gitlab-br-bot | push on buildstream@juerg/recursive-pipelines (by Jürg Billeter): 1 commit (last: _artifactcache: Use project-specific remotes for subprojects) https://gitlab.com/BuildStream/buildstream/commit/60d75b1aa18d3603f12d035237fe9e28198e6503 | 17:14 |
| gitlab-br-bot | buildstream: merge request (juerg/recursive-pipelines->master: WIP: Junctions and subprojects) #160 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/160 | 17:14 |
| gitlab-br-bot | push on buildstream@tracking-changes (by Tristan Maat): 2 commits (last: _pipeline.py: Merge the track queue into the scheduler run) https://gitlab.com/BuildStream/buildstream/commit/2b337c4ae408b74cdaf2fb0653c395fca0e025ed | 17:15 |
| gitlab-br-bot | buildstream: merge request (tracking-changes->master: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/119 | 17:15 |
| *** mcatanzaro has quit IRC | 17:18 | |
| tristan | ironfoot, ok lets disable those push notifications... where can I modify something ? | 17:19 |
| ironfoot | tristan: oh, no, I failed to do the promised update :( | 17:27 |
| ironfoot | tristan: here is the configuration: https://gitlab.com/baserock/infrastructure/blob/master/baserock_webserver/gitlab-bot/config.js | 17:29 |
| * tristan gets himself a clone | 17:30 | |
| ironfoot | clone myself too, and I'll fix the bot | 17:32 |
| tristan | ironfoot, how do you mean fix it ? | 17:34 |
| ironfoot | to enable push notifications only in master | 17:34 |
| tristan | I mean, I was about to just disable push notifications, but if you're going to allow us to have push notifications only for master (even if it's a piece of hard code for push notifications), then I'll just not :) | 17:34 |
| tristan | Ok ! | 17:34 |
| tristan | awesome | 17:34 |
| ironfoot | tbh, i don't know when that's going to happen, but when it does, I'll poke you, in case you want to enable them again. | 17:35 |
| ironfoot | (i've been a bit busy lately) | 17:35 |
| tristan | what what what ? | 17:36 |
| tristan | ironfoot, ok so... I was getting tired of the push notifications that are unrelated to master... and I was just going to disable them | 17:36 |
| tristan | but then you said your gonna clone yourself and fix the bot | 17:37 |
| tristan | now you say you dont know when ? | 17:37 |
| tristan | just... lemme know... what's the story :D | 17:37 |
| ironfoot | hehe, do you have a clonning machine? | 17:37 |
| ironfoot | no, I don't know when it's going to happen, sorry | 17:37 |
| ironfoot | please, remove the push notifications for now | 17:37 |
| *** mcatanzaro has joined #buildstream | 17:38 | |
| tristan | ironfoot, ok... I was going to add some of the other buildstream side repos while I'm at it too | 17:38 |
| tristan | ironfoot, do I have developer status on that repo, so I can push an MR branch for you ? | 17:38 |
| nexus | Hi, I'm getting an error during my build "Command 'ldconfig' failed with exitcode 1" "Sandbox directory: /root/.cache/buildstream/build/step7-q475f7j8" | 17:40 |
| nexus | anyone know what's causing this? | 17:40 |
| tlater | ^ Working in bst-here | 17:40 |
| nexus | ^ | 17:40 |
| tlater | You'll probably have to modify the script to use --privileged | 17:41 |
| nexus | which script? | 17:41 |
| tlater | I think we have an open issue for that; for some reason the permissions aren't enough currently | 17:41 |
| tlater | bst-here | 17:41 |
| nexus | hmm, 1 sec | 17:42 |
| nexus | that seemed to work | 17:43 |
| nexus | why is that a thing? | 17:43 |
| tlater | Yeah, context: https://gitlab.com/BuildStream/buildstream/issues/156 | 17:43 |
| gitlab-br-bot | push on buildstream@distcheck-testing (by Tristan Van Berkom): 1 commit (last: .gitlab-ci.yml: Use source distribution tarballs in all tests) https://gitlab.com/BuildStream/buildstream/commit/3cb272e5fbf47c8a269564c27ce798a874baac84 | 17:43 |
| tlater | That issue should probably be marked as a 'bug', considering this makes bst-here unusable | 17:47 |
| ironfoot | tristan: yes | 17:50 |
| juergbi | tristan: regarding test coverage, besides pushreceive, a big part we're missing may be the source bundle functionality | 17:56 |
| tristan | ironfoot, https://gitlab.com/baserock/infrastructure/merge_requests/21 | 17:59 |
| tristan | juergbi, indeed that part is also untested | 17:59 |
| tristan | we should be able to test that in the regular test suite, actually | 17:59 |
| tristan | (i.e. as opposed to requiring third party downloads and qualifying as an "integration test") | 18:00 |
| juergbi | may be possible as that's not sandboxed | 18:00 |
| tlater | tristan: Regarding https://gitlab.com/BuildStream/buildstream/issues/156, do you think it would be sensible to make `--privileged` the default for now, until someone has time to look into the actual correct permissions? It seems bad to just leave a broken script lying around. | 18:02 |
| tristan | tlater, yes | 18:02 |
| juergbi | ssam2: are you working (or planning to work) on pushreceive tests or shall i look into those? | 18:03 |
| ssam2 | i was planning on doing it, but happy for you to take it on :-) | 18:03 |
| ssam2 | i have to do tests for pulling from multiple caches, but seems better to rewrite frontend/pull.py for that | 18:03 |
| tristan | Hmmm | 18:04 |
| tristan | I wonder if the backdoor we have in place for local paths to artifact shares is relevant | 18:05 |
| tristan | or if it's just a local helper for us | 18:05 |
| juergbi | you mean whether we want to officially support this or not? | 18:05 |
| ssam2 | i don't see a use for it outside tests, really | 18:05 |
| *** jonathanmaw has quit IRC | 18:05 | |
| tristan | on the one hand, yes ssam2 has a point; it seems orthogonal activities | 18:05 |
| tristan | on the other hand, if we can properly test push/pull, then we have no need anymore for that hack, and could remove it from the codebase | 18:06 |
| tristan | if we keep it, then it should be supported | 18:06 |
| tristan | "officially" | 18:06 |
| tristan | if we have proper tests for push/pull that dont need that local trick, then we should remove it | 18:06 |
| ssam2 | it's not really at all intrusive to support file:// URLs, though | 18:06 |
| tristan | I agree | 18:06 |
| ssam2 | we could remove the hack that allows that /path/foo instead of file://path/foo | 18:07 |
| juergbi | yes, not sure we gain a lot in maintainability by removing it | 18:07 |
| juergbi | starting with / is a valid url, though | 18:07 |
| tristan | Well, if we have both, both paths should be tested | 18:07 |
| tristan | not that that is a horrible thing or anything :) | 18:07 |
| tristan | just bringing it up, I mean there is the burden of testing it, and if it's practically worthless, maybe it's then worth removing | 18:08 |
| ssam2 | the issue about only testing the ssh:// codepath is that we might only run that test on hosts with opensshd available | 18:08 |
| tristan | if it's useful for anything, I have no issues with keeping it, though | 18:08 |
| *** jude has quit IRC | 18:08 | |
| tristan | ssam2, that's not a huge concern, if something cannot be tested, we should skip it with a warning as we do for some host-tool specific tests | 18:09 |
| juergbi | i'd tend to keep it | 18:09 |
| tristan | yeah, I dont particularly mind keeping it | 18:09 |
| gitlab-br-bot | push on buildstream@sam/plugin-log (by Sam Thursfield): 1 commit (last: plugin.py: Add log() method) https://gitlab.com/BuildStream/buildstream/commit/3ec023aa86a593c0eb9dbe1e5f6634b1f5544e6a | 18:09 |
| tristan | it's a little bit of an eye sore in the artifact cache code, but not terribly so | 18:09 |
| ssam2 | having two test cases for the same thing isn't great, though | 18:09 |
| tristan | right, ideally what we should have is additional parameterization to the push/pull tests | 18:10 |
| *** gitlab-br-bot has quit IRC | 18:10 | |
| *** gitlab-br-bot has joined #buildstream | 18:10 | |
| tristan | so run the whole battery against the local path, and then again against the ssh:// path | 18:10 |
| ssam2 | yeah. if the logic goes into the create_artifact_share() helper then that should be fine | 18:10 |
| gitlab-br-bot | buildstream: merge request (sam/plugin-log->master: plugin.py: Add log() method) #168 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/168 | 18:11 |
| tristan | anyway; from my perspective, I wont block multiple artifact shares on the absence of the test scaffolding that we dont have yet but would be very nice to have | 18:11 |
| tristan | so I'm totally ambivalent about whether juergbi or ssam2 take it on :D | 18:11 |
| tristan | (which was the original topic I think) | 18:12 |
| ssam2 | yeah :-) | 18:12 |
| juergbi | given that i can actually test subproject-specific artifact servers also with local push, i probably first do that | 18:12 |
| juergbi | but i can look into pushreceive tests after that | 18:12 |
| ssam2 | i'll look at modifying tests/frontend/pull.py to properly test that multiple caches work | 18:12 |
| ssam2 | that should be done tomorrow and we can think about merging the multiple cache support branch then | 18:13 |
| ssam2 | after that. .. i would like to look more at the various slownesses i found last week | 18:13 |
| ssam2 | although we also should come up with a plan for GPG signing this week | 18:13 |
| ssam2 | to see if there's anything needed that would block 1.0 (in terms of changing the artifact format) | 18:13 |
| tristan | juergbi, it seems sensible to me that you stick with the local method... and then whoever tests the pushreceive code paths implements that unilaterally across the existing related push/pull tests | 18:13 |
| juergbi | for the junction-specific push/pull test, local server may suffice as it's orthogonal | 18:14 |
| tristan | right, probably we can even have the ssh related tests only present in the push/pull stuff, if the junction specific tests dont add edge cases to the underlying artifact exchange code anyway | 18:16 |
| juergbi | but my tests should probably build upon the multi cache tests as those aspects are tighter connected | 18:18 |
| tristan | aha... | 18:18 |
| tristan | ok so... juergbi do you think it's more sensible to land the junctions *after* ssam2's branch ? | 18:19 |
| juergbi | tristan: the subproject-specific artifact server support builds upon ssam2's branch | 18:20 |
| juergbi | so either we land that first or we land junction support without subproject-specific artifact server support first | 18:20 |
| juergbi | i'd go with the former unless we hit a blocker there | 18:20 |
| tristan | ahhh ok I wasnt aware of that part | 18:20 |
| tristan | Ok in any case your work is a lot more complex, I expect that ssam2's branch will be much easier to land too | 18:21 |
| juergbi | yes, sounds fine to me | 18:21 |
| tristan | there is a variance of like ~20 min between test runs of integration_tests | 18:21 |
| juergbi | btw: the first few commits of my branch are refactorings that could also land before the feature work | 18:22 |
| tristan | seems like sometimes we get a host that is closer to the storage device | 18:22 |
| tristan | juergbi, sure I am totally open to landing those in advance and lessening, however slightly, the effort of landing the bigger branch :) | 18:22 |
| * tristan did that as well with the format changes a month or two ago | 18:23 | |
| *** ssam2 has quit IRC | 18:23 | |
| tristan | pushed the refactorings down the history in rebase, and landed them in advance preparation | 18:23 |
| juergbi | yes, i already prepared the branch history for this. should probably still get a review of those, though, as sanity check | 18:25 |
| tristan | OK! | 18:34 |
| tristan | Now the CI tests all operate on a source distribution tarball | 18:34 |
| tristan | so we should in theory, catch any problems with tarballs at pre-merge time | 18:34 |
| juergbi | \o/ | 18:37 |
| *** tristan has quit IRC | 18:40 | |
| *** bethw has quit IRC | 18:51 | |
| jjardon[m] | Hi, what is source type I should use if its a file but its in a URL somewhere? Does bst supports that? | 19:05 |
| jjardon[m] | this is the use case: https://github.com/flatpak/freedesktop-sdk-images/blob/9995653f1bcc3c57351da0cd80af63fbc29ad57e/org.freedesktop.Sdk.json.in#L1847 we can download the file and store it locally so we can use local: but I wonder is something else exist | 19:06 |
| jjardon[m] | mmm, local: doesnt have ref: neither ; how do you know you are using exactly the same thing? (I can change the file and keep the name, for example) | 19:08 |
| *** bethw has joined #buildstream | 19:16 | |
| *** bochecha has joined #buildstream | 19:43 | |
| *** tristan has joined #buildstream | 19:57 | |
| *** valentind has joined #buildstream | 20:25 | |
| gitlab-br-bot | buildstream: merge request (lzip->master: Add support for lzip in tar source) #162 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/162 | 20:57 |
| juergbi | jjardon[m]: i don't think we support this right now. it is indeed a bit odd that source plugins are a mix of file type and protocol | 21:22 |
| juergbi | we should support downloading blobs and patches from http, and for consistency we should probably also support local tar and zip files (although i don't think the latter will be very useful in practice) | 21:24 |
| juergbi | in this particular case, you could use a git source where you only use a single file, although this may also not be ideal | 21:25 |
| juergbi | for local sources you shouldn't need a ref as they are typically version controlled in the same [git] repo as the .bst file | 21:26 |
| juergbi | (and the cache key does include the sha256 hash, so there is no risk with regards to rebuilds) | 21:27 |
| jjardon[m] | juergbi: ok, thanks. I understand there is not risk about rebuilds, but for the user can be a bit confussing that exactly the same bst cause a rebuild, only because it turns out the contents of that file have changed | 21:33 |
| juergbi | if we generalize local (and patch) to also support http, we could probably easily support ref as well there | 21:34 |
| juergbi | however, i don't think we want to require this for local sources | 21:35 |
| juergbi | this generalization is probably something that we should discuss as soon as possible with tristan, in case we want to change something before 1.0 | 21:36 |
| gitlab-br-bot | buildstream: issue #163 ("[Feature Request] Buildstream should support downloading blobs and patches from http") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/163 | 21:37 |
| juergbi | ta | 21:37 |
| *** bethw has quit IRC | 21:45 | |
| juergbi | the local source plugin also supports whole directories, so it may make sense to keep this as is | 21:46 |
| juergbi | however, we could add a file source plugin | 21:47 |
| *** bethw has joined #buildstream | 21:47 | |
| jjardon[m] | yup | 22:02 |
| *** bochecha has quit IRC | 22:24 | |
| *** bethw has quit IRC | 22:41 | |
| tristan | jjardon[m], now we have a private _downloadablefile.py abstract source, used for both tar & zip | 22:47 |
| tristan | making a raw file downloader I think makes sense | 22:47 |
| tristan | of course it will expect the checksum never changes, so should not be used with something that changes, or will trigger errors | 22:48 |
| *** xjuan_ has quit IRC | 23:25 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!