| *** cs-shadow has quit IRC | 01:58 | |
| *** tristan has quit IRC | 02:14 | |
| juergbi | jmac: possibly relevant to your work: https://gitlab.com/BuildStream/buildstream/merge_requests/858 | 05:20 |
|---|---|---|
| *** catonano has joined #buildstream | 06:00 | |
| *** abderrahim has quit IRC | 06:19 | |
| *** abderrahim has joined #buildstream | 06:19 | |
| *** catonano_ has joined #buildstream | 06:45 | |
| *** catonano has quit IRC | 06:46 | |
| *** catonano_ is now known as catonano | 06:46 | |
| *** catonano has quit IRC | 07:48 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/staging_order_and_link_files->master: WIP: Make link_files and copy_files and behave independently to staging order) #858 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/858 | 07:53 |
| gitlab-br-bot | buildstream: merge request (valentindavid/staging_order_and_link_files->master: WIP: Make link_files and copy_files and behave independently to staging order) #858 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/858 | 07:57 |
| gitlab-br-bot | buildstream: merge request (valentindavid/staging_order_and_link_files->master: WIP: Make link_files and copy_files and behave independently to staging order) #858 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/858 | 07:58 |
| gitlab-br-bot | buildstream: merge request (valentindavid/staging_order_and_link_files->master: Make link_files and copy_files and behave independently to staging order) #858 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/858 | 08:10 |
| *** catonano has joined #buildstream | 08:13 | |
| *** toscalix has joined #buildstream | 08:34 | |
| gitlab-br-bot | buildstream: merge request (danielsilverstone-ct/bwrap-check-runtime-only->master: WIP: Make bwrap check runtime only) #847 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/847 | 08:39 |
| *** raoul has joined #buildstream | 08:42 | |
| *** jonathanmaw has joined #buildstream | 08:48 | |
| jmac | juergbi: Interesting, thanks | 08:59 |
| gitlab-br-bot | buildstream: merge request (willsalmon/outOfSourecBuild->master: Out of source builds) #776 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/776 | 09:02 |
| gitlab-br-bot | buildstream: merge request (danielsilverstone-ct/bwrap-check-runtime-only->master: WIP: Make bwrap check runtime only) #847 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/847 | 09:04 |
| lchlan | tristan: Sorry to bother, have you had a chance to look at !836 at all? | 09:04 |
| gitlab-br-bot | buildstream: merge request (willsalmon/outOfSourecBuild->master: Out of source builds) #776 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/776 | 09:05 |
| gitlab-br-bot | buildstream: merge request (danielsilverstone-ct/bwrap-check-runtime-only->master: WIP: Make bwrap check runtime only) #847 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/847 | 09:13 |
| gitlab-br-bot | buildstream: merge request (valentindavid/staging_order_and_link_files->master: WIP: Make link_files and copy_files and behave independently to staging order) #858 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/858 | 09:17 |
| gitlab-br-bot | buildstream: merge request (danielsilverstone-ct/bwrap-check-runtime-only->master: WIP: Make bwrap check runtime only) #847 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/847 | 09:20 |
| *** bochecha has joined #buildstream | 09:23 | |
| gitlab-br-bot | buildstream: merge request (danielsilverstone-ct/bwrap-check-runtime-only->master: Make bwrap check runtime only) #847 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/847 | 09:23 |
| gitlab-br-bot | buildstream: merge request (danielsilverstone-ct/bwrap-check-runtime-only->master: Make bwrap check runtime only) #847 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/847 | 09:23 |
| gitlab-br-bot | buildstream: issue #621 ("Unhandled exception when terminating failed build") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/621 | 09:25 |
| *** tristan has joined #buildstream | 09:27 | |
| gitlab-br-bot | buildstream: issue #696 ("Missing tests for Linux-non-bwrap-non-FUSE situations") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/696 | 09:28 |
| *** lachlan has joined #buildstream | 09:29 | |
| Kinnison | juergbi: Thanks for the approval on 847, unfortunately I'm not a committer so I can't merge it. Could you set it to merge when the tests pass? | 09:29 |
| juergbi | sure | 09:30 |
| Kinnison | thakns | 09:30 |
| Kinnison | thanks too | 09:30 |
| * Kinnison needs to learn to type | 09:30 | |
| *** catonano has quit IRC | 09:32 | |
| *** catonano has joined #buildstream | 09:32 | |
| gitlab-br-bot | buildstream: issue #644 ("Allow BuildStream to be installed in a remote-only mode") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/644 | 09:46 |
| gitlab-br-bot | buildstream: merge request (danielsilverstone-ct/bwrap-check-runtime-only->master: Make bwrap check runtime only) #847 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/847 | 09:46 |
| * Kinnison wonders if anyone has seen tristan today | 09:47 | |
| *** ChanServ sets mode: +o tristan | 09:50 | |
| * tristan waves at Kinnison | 09:51 | |
| Kinnison | ohai :-D | 09:51 |
| Kinnison | tristan: I'm trying to help lachlan finish up on !836 and wondered if you might have a chance to look at his test changes? | 09:51 |
| * tristan looks | 09:52 | |
| Kinnison | thank you | 09:52 |
| *** lachlan has quit IRC | 10:14 | |
| tristan | Kinnison, Comments up | 10:19 |
| Kinnison | tristan: Thank you. | 10:20 |
| *** catonano has quit IRC | 10:24 | |
| *** lachlan has joined #buildstream | 10:32 | |
| *** tristan has quit IRC | 10:34 | |
| *** raoul has quit IRC | 10:38 | |
| *** raoul has joined #buildstream | 10:38 | |
| *** tristan has joined #buildstream | 10:48 | |
| gitlab-br-bot | buildstream: merge request (coldtom/strip-rules->master: WIP: Upstream freedesktop-sdk strip rules) #750 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/750 | 10:53 |
| gitlab-br-bot | buildstream: merge request (coldtom/strip-rules->master: WIP: Upstream freedesktop-sdk strip rules) #750 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/750 | 10:53 |
| *** catonano has joined #buildstream | 11:04 | |
| *** catonano has quit IRC | 11:04 | |
| *** lachlan has quit IRC | 11:04 | |
| *** lachlan has joined #buildstream | 11:18 | |
| *** raoul has quit IRC | 11:21 | |
| *** raoul has joined #buildstream | 11:30 | |
| *** ahmed89 has joined #buildstream | 11:32 | |
| *** ChanServ sets mode: +o tristan | 12:02 | |
| tristan | lachlan, I was going to discuss YamlCache here but jonathanmaw beat me to it on the MR, just updated that: https://gitlab.com/BuildStream/buildstream/merge_requests/836/diffs#note_106500470 | 12:02 |
| tristan | And one more edit on that comment, missed one of jonathanmaw's details | 12:06 |
| tristan | Do we have anymore minor fixes for bst-1.2 ? | 12:18 |
| tristan | valentind, anything besides !849 ? | 12:19 |
| tristan | We have very good material for a 1.2.3 now, so I'd like to push out a release (few good fixes from yesterday, and immense performance boost from batch files support) | 12:20 |
| valentind | tristan, there is something not merge is the fixes for cache server filling up. But Freedesktop SDK is now running the fix, and I will to wait for the cache to fill up to see how it holds. | 12:21 |
| *** dtf has joined #buildstream | 12:21 | |
| valentind | So maybe for 1.2.4. | 12:21 |
| laurence | I'm was just looking for the website url and couldn't quite find it - any objections to adding a hyperlink to the gitlab README? | 12:26 |
| laurence | Will create an MR, assuming none | 12:26 |
| tristan | laurence, Of course, makes sense :) | 12:32 |
| gitlab-br-bot | buildstream: merge request (patch-2->master: README: Update to add link to website) #859 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/859 | 12:32 |
| gitlab-br-bot | buildstream: merge request (patch-2->master: README: Update to add link to website) #859 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/859 | 12:41 |
| *** ahmed89 has quit IRC | 12:42 | |
| *** ahmed89 has joined #buildstream | 12:44 | |
| *** ahmed89 has quit IRC | 12:47 | |
| *** ahmed89 has joined #buildstream | 12:47 | |
| gitlab-br-bot | buildstream: merge request (juerg/remote->master: element.py: Prepare local sandbox for bst checkout and bst shell) #860 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/860 | 12:52 |
| *** finn has quit IRC | 12:57 | |
| *** finn__ has joined #buildstream | 12:59 | |
| *** finn_ has quit IRC | 13:00 | |
| valentind | This mix of symlinks and directories in staging is really annoying. So for example if I have an element that in directory /some/path has a link to ../something. Then I have another element that has /some/path to be a symlink to other/path and /some/other/path is a directory, then the link will go in /some/other/path and point to /some/other/something instead of /some/something. | 13:07 |
| valentind | Or is it? | 13:07 |
| *** finn has joined #buildstream | 13:07 | |
| *** finn has quit IRC | 13:11 | |
| *** finn__ has quit IRC | 13:11 | |
| *** finn has joined #buildstream | 13:23 | |
| *** finn__ has joined #buildstream | 13:23 | |
| gitlab-br-bot | buildstream: issue #667 ("Add regression test for #666") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/667 | 13:40 |
| gitlab-br-bot | buildstream: merge request (Qinusty/634-workspace-failed-builds->master: Do not save workspace on failed build) #812 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/812 | 13:43 |
| gitlab-br-bot | buildstream: merge request (Qinusty/634-workspace-failed-builds->master: Do not save workspace on failed build) #812 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/812 | 13:47 |
| gitlab-br-bot | buildstream: issue #634 ("bug in build after failed build with workspace's") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/634 | 14:11 |
| gitlab-br-bot | buildstream: merge request (Qinusty/634-workspace-failed-builds->master: Do not save workspace on failed build) #812 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/812 | 14:11 |
| finn | hi everyone, this is buildstream executing remotely | 14:26 |
| finn | https://asciinema.org/a/gJ1tdOoGWlF346ZeQ75XpPXoz | 14:26 |
| finn | This is part of a larger demonstration I will be presenting at BazelCon | 14:26 |
| tristan | very nice show finn :) | 14:26 |
| finn | (we've snuck in buildstream) | 14:26 |
| tristan | We will want to see slides, remember to blog after that ! | 14:27 |
| tiagogomes | nice :) | 14:28 |
| tristan | valentind, please let me know once you've landed https://gitlab.com/BuildStream/buildstream/merge_requests/849 on 1.2, I hope to release with that fix tomorrow | 14:42 |
| tristan | valentind, symlink questions are hard ones indeed :-/ | 14:43 |
| tristan | valentind, The root of the "symlink problem" here is that we do not run builds on a single persistent sysroot, and instead split up and stage artifacts | 14:44 |
| *** rdale has joined #buildstream | 14:45 | |
| tristan | valentind, So there is one very important intention here: If a low level component creates, for example, a `/usr/lib -> /store/lib` symlink, and then many elements install stuff into /usr/lib... then when they get *staged*, the symlink is there first; and the symlink *beat* the reverse dependencies which each have /usr/lib as a directory | 14:46 |
| tristan | I.e. the artifacts have /usr/lib/libfoo.so, but depend on an artifact with /usr/lib -> /store/lib, so at staging time, libfoo.so is staged at /store/lib/libfoo.so | 14:47 |
| *** finn has quit IRC | 14:49 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/rmtree_oserror->master: Catch correct exception from shutil.rmtree) #849 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/849 | 14:56 |
| gitlab-br-bot | buildstream: merge request (valentindavid/rmtree_oserror-1.2->bst-1.2: Catch correct exception from shutil.rmtree [bst-1.2]) #861 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/861 | 14:57 |
| *** tristan has quit IRC | 14:59 | |
| *** tristan has joined #buildstream | 15:06 | |
| *** ChanServ sets mode: +o tristan | 15:06 | |
| gitlab-br-bot | buildstream: merge request (valentindavid/rmtree_oserror-1.2->bst-1.2: Catch correct exception from shutil.rmtree [bst-1.2]) #861 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/861 | 15:16 |
| valentind | tristan, It is merged in 1.2 | 15:18 |
| tristan | valentind, Thanks ! | 15:21 |
| *** finn has joined #buildstream | 15:23 | |
| valentind | tristan, Could we enforce that we have dependencies when a directory happens to be a symlink? That would not solves all the bugs we have, but that would cut down the most complex ones. | 15:25 |
| tristan | valentind, I don't know what you mean | 15:27 |
| tristan | valentind, for instance, one of the popular use cases is to stage /lib -> usr/lib, /bin -> usr/bin (i.e. usr merge system) | 15:28 |
| tristan | Where that happens right down at the bottom | 15:28 |
| tristan | valentind, from my perspective; any artifact can have a symlink in the output, that symlink can be a symlink to a directory, a file, or be an intentionally broken symlink | 15:29 |
| tristan | At stage time, the first to be staged wins | 15:29 |
| valentind | tristan, the problem is now the order of element that are not dependent to each others should not be important, but it is. Sometimes it builds, sometimes fails. | 15:29 |
| tristan | And you express that in your project with runtime dependencies | 15:29 |
| valentind | And just changing some dependency changes the order. I think even the command line might change the order. | 15:30 |
| valentind | So it works for a while. | 15:30 |
| tristan | That should not be the case at all | 15:30 |
| valentind | And then one day, it fails. And you do not know why. | 15:30 |
| valentind | It should always fail, if it can fail. | 15:30 |
| tristan | valentind, At load time, we sort all the deps for every element in a stable way | 15:30 |
| valentind | tristan, I have seen this case. | 15:30 |
| valentind | tristan, if you rename an element for example, the order change. | 15:31 |
| tristan | staging order, whether we have explicitly set one with a dependency or not; will be the same for every build | 15:31 |
| tristan | valentind, Yes, that is *true* | 15:31 |
| tristan | valentind, that is not random at all | 15:31 |
| gitlab-br-bot | buildstream: issue #153 ("Unhandled exception in utils._force_rmtree()") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/153 | 15:31 |
| gitlab-br-bot | buildstream: merge request (valentindavid/rmtree_oserror->master: Catch correct exception from shutil.rmtree) #849 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/849 | 15:31 |
| valentind | tristan, yes, but changing one dependency for example can also change order. | 15:31 |
| valentind | And sometimes in a way that a human being cannot understand. | 15:31 |
| tristan | valentind, If you did not care to express a dependency because of a necessary symlink, and explicitly enforce your staging order, then you need to correct that with a dependency | 15:31 |
| tristan | valentind, The problem, if there is one, is informing the user of this in the right way | 15:32 |
| valentind | In freedesktop sdk we do not mix symlinks and directory, because this case comes back to haunt us if we do. | 15:32 |
| tristan | I believe freedesktop-sdk *is* a usr merge system | 15:32 |
| valentind | tristan, yes, so I want to verify the order is correct. | 15:32 |
| tristan | Only the user (project author), knows the correct outcome | 15:33 |
| valentind | tristan, no we make sure all element install in usr. | 15:33 |
| tristan | And that author needs to setup the staging order explicitly, using dependencies. | 15:33 |
| valentind | tristan, exactly, we need to do that. The problem is now we do not get error or warning if we did not set the dependencies. | 15:34 |
| valentind | It works for a while. And then it breaks one day. | 15:34 |
| tristan | What error would you want ? | 15:34 |
| tristan | That is the question you know | 15:34 |
| valentind | And that is a pain for contributors because they have no idea what is happening. | 15:35 |
| tristan | valentind, Sure, but what error is it that we want ? | 15:35 |
| tristan | It looks like we are closing in on actually understanding what needs to be done | 15:35 |
| valentind | I would want to detect that we put files in a path that is using symlinks, but that symlinks is not from a dependency (either direct or indirect). | 15:35 |
| tristan | I dont understand that sentence | 15:36 |
| valentind | During the staging of an element, I have to copy a file from the artifact to the stage area. If the path of the dirname of this file in the stage area has any symlink, that is it is not a canonical path, then I need to make sure that those symlinks were creating by staging elements that the current element depends on (and it can be indirect dependency as well). | 15:38 |
| tristan | valentind, You want it to be an error when I install into /usr/lib, but then later stage against a dependency which has set /usr/lib -> /store/lib to be an error ? | 15:38 |
| tristan | That would be wrong, because it's exactly intentional; otherwise every element would have to know about it | 15:38 |
| *** lachlan has quit IRC | 15:39 | |
| valentind | tristan, not a dependency. | 15:39 |
| valentind | When it is not a dependency. | 15:39 |
| tristan | Instead, if you want /opt/org to be a symlink to /store/org, one day, because you ran out of space in the /opt partition... then you just add a symlink once and rebuild the whole system | 15:39 |
| valentind | Let say a.bst creates /usr/lib -> /store/lib, and b.bst installs a files in /usr/lib. all.bst depends on both a.bst and b.bst. | 15:40 |
| tristan | valentind, Ok I *think* I see where you are going with your longer sentence above | 15:40 |
| valentind | I want to make sure that there is a dependency from b.bst to a.bst. | 15:41 |
| valentind | The problem now is that if a.bst is staged before b.bst, it works. If b.bst is staged before a.bst, it fails. | 15:41 |
| valentind | So when I am staging b.bst after to a.bst I want to detect that the symlink was created by a dependency of b.bst. | 15:42 |
| valentind | And if not, warn, possibly configurable as a fatal error. | 15:42 |
| tristan | valentind, Can you please take the time to write this up to the mailing list, problem statement, and suggested new warning ? | 15:42 |
| valentind | tristan, OK | 15:43 |
| tristan | It's worth more eyes, and maybe additional edge cases can be explored at the same time | 15:43 |
| valentind | The warning is because it would change the behavior of 1.2 and I think this is a fix we need in 1.2. | 15:43 |
| valentind | Anyway, then there is the bug I talked about earlier with relative symlink. But I will have dinner now. | 15:43 |
| tristan | I dont think it should be an error at all | 15:44 |
| valentind | I have test cases. | 15:44 |
| tristan | I mean; I think it should be configurable as an fatal-warning, to be consistent with other such things | 15:44 |
| tristan | Also it seems to be on the feature side of things, not really appropriate for backporting to 1.2 | 15:45 |
| tristan | But then, if it saves lives, maybe we can consider making an exception for this feature addition (it would also be an API addition, which is kindof against the general rule) | 15:45 |
| laurence | I'm about to look through the BuildStream 'newcomer' or 'low hanging fruit' bugs to see if there are any specifically related to Remote Execution client | 15:48 |
| laurence | since some project team members will be at BazelCon in NY next week, and there's a hackaton on the thursday, which would be a good time to try to address some | 15:48 |
| toscalix | sadly I haven't finished the roadmapping proposal on time for the event | 15:55 |
| gitlab-br-bot | buildstream: merge request (willsalmon/outOfSourecBuild->master: Out of source builds) #776 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/776 | 15:55 |
| *** lachlan has joined #buildstream | 16:06 | |
| tiagogomes | bst checkout --deps none $artifact doesn't do anything | 16:12 |
| jennis | tiagogomes: https://gitlab.com/BuildStream/buildstream/merge_requests/819 | 16:23 |
| jennis | Issue #670 describes this, cs-shadow has a fix for it within that MR, there is also the intention to backport this: https://gitlab.com/BuildStream/buildstream/merge_requests/828 | 16:24 |
| jennis | These seem to be sitting idly because of a change I suggested that I'm not sure tristan agree's with, awaiting his response: https://gitlab.com/BuildStream/buildstream/merge_requests/819#note_105454242 | 16:27 |
| tiagogomes | thanks | 16:32 |
| jennis | Is #477 blocked by `bst artifact` commands? https://gitlab.com/BuildStream/buildstream/issues/477 or are we able to discuss some sort of implementation for it soon? | 16:39 |
| jennis | Just too add weight, I've heard numerous requests for this | 16:41 |
| *** lachlan has quit IRC | 16:51 | |
| *** toscalix has quit IRC | 17:05 | |
| *** alatiera_ has quit IRC | 17:21 | |
| *** lachlan has joined #buildstream | 17:25 | |
| *** raoul has quit IRC | 17:29 | |
| *** lachlan has quit IRC | 17:42 | |
| *** rdale has quit IRC | 17:45 | |
| *** jonathanmaw has quit IRC | 18:03 | |
| *** xjuan has joined #buildstream | 18:18 | |
| tristan | jennis, I think the answer for that one ultimately is to (A) Allow the user to ignore remote cache servers and (B) Implement `bst artifact delete` | 18:18 |
| tristan | So yeah, blocked, we're pretty close to finishing on the proposal stage, I have to write up a summary after all the churn throughout that proposal | 18:19 |
| tristan | But implementation wise, should be fairly easy | 18:19 |
| jennis | ahh thanks tristan, and yes, a summary write up of that discussion would be hugely appreciated! | 18:33 |
| * jennis has found it hard to keep track of that one | 18:34 | |
| *** xjuan has quit IRC | 18:45 | |
| *** xjuan has joined #buildstream | 18:46 | |
| *** Prince781 has joined #buildstream | 19:46 | |
| *** xjuan has quit IRC | 21:55 | |
| *** xjuan has joined #buildstream | 22:21 | |
| *** tristan has quit IRC | 23:07 | |
| *** xjuan has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!