| *** mohan43u has quit IRC | 00:08 | |
| *** mohan43u has joined #buildstream | 00:12 | |
| *** Prince781 has quit IRC | 00:20 | |
| *** Prince781 has joined #buildstream | 00:22 | |
| *** Prince781 has quit IRC | 00:27 | |
| *** Prince781 has joined #buildstream | 00:28 | |
| *** Prince781 has quit IRC | 01:27 | |
| *** Prince781 has joined #buildstream | 02:18 | |
| *** Prince781 has quit IRC | 04:53 | |
| *** tristan has joined #buildstream | 04:55 | |
| laurence | juergbi, a few weeks back we were discussing FUSE and possible staging optimizations, i think i remember that you you mentioned doing some performance tests which proved FUSE was not really an option because of the overhead? | 07:08 |
|---|---|---|
| laurence | I may have some of the details slightly incorrect there ^ | 07:08 |
| laurence | Anyway I was looking at this and wondering if we could add those results to gitlab show them in public? https://gitlab.com/BuildStream/buildstream/issues/56#note_65236835 | 07:09 |
| tristan | laurence, we tried some tests, including testing with a C implementation, and the overhead was a factor of... more than twice the time it takes | 07:09 |
| juergbi | yes, I built gtk+ under different conditions as a simple benchmark | 07:12 |
| laurence | tristan, juergbi, right, ok. is it worth sharing the results to show we've 'done the homework'? or not needed? i dunno | 07:13 |
| juergbi | would probably make sense to put the results somewhere. however, it was done in context of #38 | 07:13 |
| laurence | ok | 07:13 |
| juergbi | I could add it to #38 and then link to it in #56 | 07:13 |
| tristan | I think some of the discussion can be found in https://irclogs.baserock.org/buildstream | 07:14 |
| juergbi | yes but that's not as easy to find | 07:14 |
| laurence | alright, great, can always link to the chat from the issue if we want | 07:15 |
| tristan | maybe if there was a related email that day, can find the date | 07:15 |
| tristan | laurence, preferably I like to keep stuff copied *in* the issues rather than linked, in general | 07:15 |
| tristan | links have a tendency of expiring over time | 07:15 |
| * tristan was just saying, it might be helpful to juergbi to find / remember | 07:16 | |
| tristan | but maybe he has an easier route | 07:16 |
| juergbi | I have it in my notes | 07:16 |
| tristan | :) | 07:16 |
| juergbi | tristan: btw: interesting idea in #321 | 07:17 |
| * juergbi likes reusing existing generic facilities | 07:18 | |
| tristan | ah yes | 07:18 |
| tristan | I thought of that late last night :) | 07:18 |
| *** toscalix has joined #buildstream | 07:35 | |
| *** noisecell has joined #buildstream | 08:03 | |
| *** valentind has joined #buildstream | 08:06 | |
| skullman | Was there a resolution to the cause of Nexus' weird test hanging problem in Unix mode? I'm getting it in my tests now. | 08:18 |
| skullman | it was something fuse related IIRC | 08:20 |
| gitlab-br-bot | buildstream: merge request (jennis/correct-installing-documentation->master: install.rst: Inform user to restart terminal once they have adjusted their path) #340 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/340 | 08:23 |
| jennis | Is anyone familiar with click and knows a quick fix to rectify why our documentation is showing options like this for various commands: --track <track_> (https://buildstream.gitlab.io/buildstream/invoking.html#cmdoption-bst-build-track) | 08:29 |
| jennis | Ideally we'd want it to say "--track <element>" | 08:30 |
| jennis | I know it's to do with the way we've invoked click.option( but I haven't managed to find how we can correct this | 08:31 |
| jennis | For this specific case: https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/_frontend/cli.py#L173 | 08:31 |
| juergbi | skullman: there were two issues. stack traces from that: https://pastebin.com/LaXXJPXK | 08:48 |
| juergbi | I fixed the directory issue | 08:48 |
| juergbi | however, the issue of hang on test failure might stillb e there (second traceback) | 08:49 |
| juergbi | I thought we filed an issue for that but I can't find it right now | 08:49 |
| Nexus | juergbi: I think i remember tlater saying he was working on it | 08:50 |
| juergbi | Mar 15 18:08:52 <tlater> Nexus already has an issue for 2 open | 08:51 |
| Nexus | https://gitlab.com/BuildStream/buildstream/issues/298 | 08:51 |
| Nexus | :) | 08:51 |
| Nexus | thats a different thing i think | 08:51 |
| Nexus | but thats the only one i remember making | 08:52 |
| juergbi | it might be that | 08:52 |
| Nexus | that's about getting spammed with unmounted stuff of you force cancel a test | 08:53 |
| juergbi | it's not related to the second stack trace from my pastebin? | 08:53 |
| Nexus | i don't believe so | 08:53 |
| juergbi | hm | 08:53 |
| juergbi | skullman: the second stack trace in the pastebin might be related to the hang you're seeing, though | 08:56 |
| juergbi | skullman: as a quick test you could add -l to the umount invocation | 08:58 |
| *** bethw has joined #buildstream | 09:04 | |
| tlater | juergbi: Yep, sorry, it's in my backlog - it's not the highest priority atm and it's not turned out to be a quick fix, unfortunately | 09:04 |
| juergbi | tlater: is this all part of #298 or is there / do we need a second issue? | 09:05 |
| tlater | I thought we had an issue for it, let me double check | 09:06 |
| tlater | juergbi: Ah, yes, it's exactly #298 - skullman might be seeing a different hang, of course | 09:06 |
| * tlater hasn't looked at the paste in detail yet | 09:06 | |
| Nexus | it seems to be the same thing | 09:06 |
| juergbi | if it's the same stack trace, we should probably copy it into the issue | 09:07 |
| juergbi | same/related | 09:07 |
| juergbi | otherwise open a new issue and add it there | 09:08 |
| tlater | skullman: Think you could produce a stacktrace so we can be sure? I'll dig up the problem in a second, in case you want to try and manually debug. | 09:08 |
| Nexus | how much of the stracktrace do you want? | 09:08 |
| skullman | https://pastebin.com/BqaK0sQb is what I've got so far | 09:08 |
| skullman | :/ I think I must have caused some unrelated issue, since other tests are now failing | 09:09 |
| tlater | Hm, I'm not sure that's the same issue | 09:10 |
| tlater | Though, it's usually removing some tempdir with a yet-to-be unmounted directory | 09:10 |
| tlater | skullman: I think the issue is this helper: https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/sandbox/_mounter.py#L93 | 09:11 |
| tlater | We use signals.terminator, but it doesn't catch any exception - if you get an IO exception it will fail to catch and clean up, causing the directory to stay mounted | 09:12 |
| tlater | That in turn causes some variant of rm to fail, and often leaves a child process hanging | 09:12 |
| tlater | This can only happen in the unix tests | 09:12 |
| tlater | I thought it would just be a case of adding another `try: catch:` block, but that doesn't seem to work :[ | 09:13 |
| *** toscalix has quit IRC | 09:37 | |
| *** toscalix has joined #buildstream | 09:38 | |
| skullman | aha! it's about error handling when /bin/sh is missing | 09:40 |
| tristan | Aha this ! | 09:40 |
| * skullman didn't care how the tests failed so wasn't bothered when under linux it was failing, but the error handling is more fragile without bubblewrap | 09:41 | |
| tristan | So I have been thinking for a looooong time, that could have something better for reporting errors for missing dependencies *in the runtime* | 09:41 |
| tristan | We handle this for host tools (primarily in Plugin.preflight()) elegantly, and not all elements will require a "shell", while many elements require "a shell and more" | 09:42 |
| tristan | Wouldnt it be nice to have a nicer failure from say, the autotools element, if automake is not found in the staged runtime ? | 09:43 |
| tristan | That said, I've been pretty unbothered by the standard errors one gets when we just try to run something in the runtime that doesnt exist, so it's not been a priority | 09:43 |
| tristan | juergbi, thoughts ? | 09:43 |
| juergbi | tristan: for cases where the standard errors are reasonably clear, I wouldn't bother | 09:52 |
| juergbi | people that see such errors are typically familiar with building and thus standard errors | 09:52 |
| juergbi | if missing /bin/sh causes some odd error message, that's different | 09:53 |
| juergbi | btw: in my `include` support test, I automatically depend on automake (and more) by including autotools.inc | 09:54 |
| *** ernestask has joined #buildstream | 09:58 | |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 09:58 |
| *** valentind has quit IRC | 10:04 | |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 10:05 |
| tristan | juergbi, fair; still I think that a solution to Paul's original cryptic error stemming from not having a shell, should be localized to BuildElement, which is what is requiring /bin/sh | 10:06 |
| skullman | adding `-l` to the umount command doesn't seem to fix it | 10:07 |
| tristan | jjardon[m], in light of my last comment https://gitlab.com/BuildStream/buildstream/issues/321#note_65232874, do you think it's sensible that we can close 321 ? | 10:12 |
| tristan | jjardon[m], I think that with existing features, it pretty much gives you *exactly* what you wanted, without any new feature :) | 10:12 |
| juergbi | tristan: Element.integrate also depends on 'sh' | 10:13 |
| tristan | juergbi, sigh... you are right :'( | 10:13 |
| jjardon[m] | tristan sure, but lets close it when It gets documented ; I think its an important use case that almost anyone using junctions would need to use at some point | 10:13 |
| juergbi | tristan: make this generic and provide a suitable error message when the sandbox command can't be found? | 10:13 |
| juergbi | (not sure what the error is right now) | 10:13 |
| skullman | :/ I can think of a way to reliably wait for a mount to finish under linux, by getting the mount ID from /proc or name_to_handle_at and adding a wait loop until the mount ID changes, but given the whole point is that we can't use Linuxisms in Unix mode this is of no help | 10:14 |
| tristan | juergbi, in the long run; I dont want to depend on any specific shell, but rather have ways in which the user can select an interpretor for the commands they run | 10:14 |
| tristan | juergbi, or allow direct invocation of binaries | 10:15 |
| tristan | juergbi, rpm spec files for instance, allow selecting interpretors for postinst instructions, which is interesting | 10:15 |
| skullman | I think bubblewrap gets away from this issue because the mount namespace lets it terminate everything in the background rather than explicitly unmounting and waiting for that to complete. | 10:15 |
| tristan | juergbi, also, it seems weird to require a shell when there are not integration commands, one may very well want to run `compose` on something that does not have a shell; so we have to be careful to not trigger an assertion "where a shell is not required" | 10:16 |
| juergbi | tristan: yes, I'd like that. I'm actually about to look into a related topic: remote execution of commands. so I'll keep that aspect in mind | 10:16 |
| persia | I like "DVD Mastering" as a good use case to cover this class of problem. Requires custom image format, custom bytecode interpreter to execute the payload, etc. | 10:16 |
| tristan | jjardon[m], Agreed; this is documented to the extent that it is reasonable within reference material, but it deserves a citation in user facing "Examples" / "How do I ... ?" style documentation, for which a standard is currently being developed | 10:17 |
| tristan | jjardon[m], I will repurpose 321 accordingly | 10:18 |
| jjardon[m] | tristan: great, thanks; I think there is a lot of valuable info in that issue and I think it is a waste that it gets lost | 10:19 |
| gitlab-br-bot | buildstream: issue #323 ("Formatting within the invoking documentation does not appear correct") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/323 | 10:20 |
| tristan | jjardon[m], updated :) | 10:20 |
| juergbi | skullman: I assume we can at least properly handle the case where /bin/sh is missing without complex mount handling, can't we? | 10:20 |
| juergbi | skullman: in the end, the unix backend can't ever be as robust as system-specific ones, afaict. POSIX is completely useless for such things | 10:21 |
| tristan | juergbi, skullman; one interesting property here, is that the `Sandbox` object is always given the full command line, and does not make any assumption about what shell | 10:22 |
| skullman | juergbi: so long as we do the check before mounting anything, it we should be able to make the check | 10:22 |
| juergbi | tristan: hence my suggestion to make a generic check | 10:22 |
| tristan | juergbi, yes that sounds good to me, at that location | 10:22 |
| juergbi | skullman: but we can't handle similar failures in a generic way? e.g., if we check for /bin/sh, it's still possible that ld-linux.so is missing or something like that | 10:23 |
| tristan | Currently we often pass it 'sh', which magically "works" by discovering things in PATH (in some way that I dont particularly understand) | 10:23 |
| juergbi | I hope it uses the configured PATH, not the host PATH | 10:23 |
| *** aday has joined #buildstream | 10:23 | |
| tristan | juergbi, well, we run it in a subprocess with an overridden environment, so I would presume it has to | 10:24 |
| juergbi | tristan: right, it's bwrap/chroot that does it, so that's fine | 10:24 |
| tristan | in any case, we can check if A.) it is an absolute path, where we just assert that it exists, or B.) it is not an absolute path, in which case we assert that it exists somewhere in PATH | 10:25 |
| juergbi | right | 10:25 |
| juergbi | I would definitely expect us to be able to avoid a mount issue even without special handling | 10:26 |
| tristan | I dont know about missing ld-linux.so or missing shared libs which a program we try to run cannot find | 10:26 |
| juergbi | special handling may be required for better error message | 10:26 |
| tristan | that seems overkill for now unless you have some magic idea :) | 10:26 |
| juergbi | so I would actually first want to fix the mount issue / hang | 10:26 |
| tristan | yeah that is a separate issue | 10:27 |
| tristan | I recall dealing with it one time I think with YBD, I was very confused | 10:27 |
| juergbi | if that's fixed, maybe even the standard error is sufficient | 10:27 |
| juergbi | but maybe not | 10:27 |
| juergbi | skullman: do you have a stack trace of the hang? | 10:27 |
| tristan | It happens in /proc, and it cannot be unmounted because of something else inside /proc being mounted too; which might be automatically mounted depending on what is being run in the sandbox | 10:28 |
| tristan | And in that case, it can be dangerous to your host to force unmount | 10:28 |
| tristan | (or require a reboot or doctoring) | 10:28 |
| skullman | http://paste.debian.net/1016924/ | 10:28 |
| juergbi | I'm missing the connection to the missing /bin/sh | 10:28 |
| juergbi | why can't we handle /bin/false but not missing file? | 10:29 |
| juergbi | ah, it's a python exception in the unix case | 10:29 |
| juergbi | we don't actually use the chroot binary | 10:29 |
| * tristan was mixing up umount issues and just contributing noise to the conversation, sorry for that | 10:30 | |
| skullman | sorry, I've run out of time to look at this issue until next week | 10:30 |
| tristan | in any case, that looks like if we handle the FileNotFound error there, we can do the correct umount and not jump directly to force removing things which hits -EBUSY | 10:31 |
| juergbi | hm, shouldn't popen be moved out of the 'with'? | 10:32 |
| juergbi | that probably doesn't solve the issue but looks wrong | 10:34 |
| * tristan has remained happily ignorant of SandboxChroot() for a while... | 10:35 | |
| tristan | We really need a proper unix platform to test on and bring that backend closer to a projection ready state | 10:35 |
| tristan | probably starting with BSD | 10:35 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 10:47 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 11:02 |
| tlater | jennis: Do we have a fix for the arpy import yet? | 11:02 |
| tlater | Or did you work around that by installing arpy locally? | 11:03 |
| jennis | I just installed it locally | 11:03 |
| jennis | So that was because I was unable to compile the docs | 11:03 |
| tlater | Right, I'll put up an issue about it then | 11:04 |
| * tlater just ran into it on his machine as well after rebasing | 11:04 | |
| *** mohan43u has quit IRC | 11:05 | |
| *** mohan43u has joined #buildstream | 11:08 | |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 11:10 |
| jennis | yes so just to reiterate: If you don't have arpy installed locally, you'll get import errors + you won't be able to `make -C doc` | 11:12 |
| jennis | Once you have it installed locally, it's fine. | 11:12 |
| jennis | tlater ^ | 11:13 |
| tlater | Hm, those are two separate issues I think | 11:13 |
| jennis | Well installing arpy locally seemed to "fix" both of them | 11:13 |
| tlater | Nicer to create a single issue with two points, I suppose | 11:13 |
| tlater | Actually, best to reopen tristan's issue and add these as comments | 11:14 |
| gitlab-br-bot | buildstream: issue #317 ("New deb source arpy dependency is too hard") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/317 | 11:17 |
| tlater | tristan: I reopened ^, because it wasn't fully fixed. I don't think the critical marker is appropriate anymore though, do you have an opinion on this? | 11:18 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 11:29 |
| * tlater just ran into a test that started failing because of the default %{install-root} change - the exception thrown talks about a 'read-only file system' | 11:32 | |
| *** bethw has quit IRC | 11:32 | |
| tlater | Should we maybe change that in case someone attempts to run a newer version of buildstream on a project that happens to write to '/buildstream/install' directly? | 11:33 |
| tlater | I can imagine it being quite confusing, I'm glad I remembered this change happened. | 11:33 |
| Nexus | "'/buildstream/install' directly" as in hard coded path? tlater | 11:35 |
| tlater | Nexus: Yep, '/buildstream/install' instead of '%{install-root}' - probably not the most common thing, but I've written a few tests like that because I frankly keep forgetting variables exist. | 11:36 |
| Nexus | IMO those people should be caught and shot, but i guess we could put in a ncier message... | 11:36 |
| Nexus | nicer* | 11:36 |
| tlater | Hum, looks like %{install-root} -> /buildstream/install: https://gitlab.com/BuildStream/buildstream/-/jobs/59691997 | 11:42 |
| tlater | Nexus: Any clue why that might happen? | 11:42 |
| Nexus | tlater: not sure why it's getting read-only, but /buildstream/install is correct... | 11:43 |
| tlater | I thought we'd changed the default installation directory? | 11:43 |
| tlater | Ah, no, just the default build dir | 11:44 |
| tlater | Hrmm | 11:44 |
| Nexus | we did, from buildstream-install | 11:44 |
| tlater | In either case, that's not related then | 11:44 |
| * tlater wonders what else might have changed | 11:44 | |
| Nexus | nope | 11:44 |
| Nexus | try running as root? | 11:44 |
| Nexus | as if its a permissions thing | 11:44 |
| tlater | You already automatically run as root, Nexus - it looks like the /buildstream/install directory is mounted RO. | 11:47 |
| * tlater isn't sure why that would happen, but suspects it's a recent commit or part of this branch | 11:47 | |
| Nexus | how odd | 11:47 |
| tlater | /buildstream/install might not be mounted at all... | 11:48 |
| * tlater checks the sandboxing commands | 11:48 | |
| tlater | Nexus: It looks like '/buildstream/install' isn't mounted, but 'buildstream-install' is. | 11:51 |
| Nexus | okayyyy, how does that work? | 11:52 |
| Nexus | buildstream-install shouldn't appear anywhere in code | 11:52 |
| * tlater is trying to figure that out, might paste a few things once I've explored more :) | 11:53 | |
| Nexus | kk | 11:53 |
| tlater | Looks like there are some leftover references to that dir in our tests | 11:55 |
| Nexus | there shouldn't be... where? | 11:55 |
| Nexus | yeah i see them | 11:56 |
| tlater | None related to the integration tests though | 11:57 |
| jennis | is there a way to find which MR a commit was part of? | 12:00 |
| Nexus | git blame? | 12:02 |
| tlater | jennis: gitlab shows somewhere on the page, but not all commits were part of MRs | 12:02 |
| Nexus | find the sha, find the commit, find the push | 12:02 |
| jennis | right ok, tlater I think in this case it was the latter. Thanks | 12:08 |
| tristan | jennis, tlater; fwiw; we have higher restrictions for being able to build docs, I would consider it acceptable to update HACKING.rst to mention that you need to install arpy, alongside the additional sphinx,sphinx-click,etc docs-related deps | 12:09 |
| tlater | Nice, that makes solving that issue pretty trivial | 12:10 |
| tristan | conditional docs builds is just nonsense anyway; we build it all or nothing | 12:11 |
| tristan | until we find a weird circumstance that we cannot anymore (win32 specific elements in a weird and unlikely future ?) | 12:12 |
| *** bethw has joined #buildstream | 12:12 | |
| tlater | Nexus: This is a bit very obvious: https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/data/projectconfig.yaml#L60 | 12:15 |
| * tlater went through the effort of finding the correct provenance, expecting it to be something in just the tests directory | 12:15 | |
| tlater | That makes me wonder why %{install-root} expands to /buildstream/install though | 12:16 |
| Nexus | wtf... it got changed... the latest diff on the merged MR shows it being changed from /buildstream/install to /buildstream-install | 12:17 |
| Nexus | that should never have happened | 12:17 |
| Nexus | change it back in your MR? | 12:18 |
| Nexus | i don't know what's going on... | 12:18 |
| tlater | Odd, that change is yours, but yeah, I'll change it back | 12:18 |
| tlater | Nexus: Any idea why %{install-root} isn't set to that though? | 12:19 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 12:20 |
| *** tristan has quit IRC | 12:21 | |
| Nexus | tlater: no idea, maybe for testing it is being reset? | 12:23 |
| Nexus | but i removed all instances of that, that i could find | 12:23 |
| tlater | I'm a bit concerned that the actual %{install-root} resolves to something different than what is set in projectconf.yaml... The tests run buildstream as normal, and there are no interfering variables in the specific test I had. | 12:25 |
| tlater | I think we broke the ability to change `install-root` | 12:25 |
| Nexus | :p | 12:26 |
| *** xjuan has joined #buildstream | 12:32 | |
| tlater | Now a bunch of tests in variables.py fail, because the install-root is different... | 12:35 |
| * tlater wonders how any of this ever worked | 12:35 | |
| Nexus | :p | 12:35 |
| Nexus | some merge must have messed up | 12:35 |
| tlater | This isn't great, variables look to be broken on master? | 12:36 |
| tlater | Or at least that one variable is | 12:36 |
| Nexus | So, i seem to have `./tests/cachekey/update.py` pointing at bst-external for some reason | 12:36 |
| Nexus | then why does master work? | 12:36 |
| gitlab-br-bot | buildstream: merge request (jmac/performance-ci->master: WIP: .gitlab-ci.yml: Add the performance test.) #341 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/341 | 12:37 |
| Nexus | only your branch seems to have this issue | 12:37 |
| Nexus | ohh | 12:38 |
| Nexus | i have an idea | 12:38 |
| Nexus | tlater: add `install-root: /buildstream/install` below `variables:` in buildstream/plugins/elements/script.yaml | 12:38 |
| tlater | Nexus: master still works because install-root is hardcoded to /buildstream-install | 12:39 |
| tlater | Once you change it everything breaks | 12:39 |
| tlater | Nexus: I don't see how that fixes buildstream resolving `install-root` incorrectly - it's resolving it "fine" now I've changed it in projectconfig.yaml, but it cannot be changed. | 12:40 |
| gitlab-br-bot | buildstream: issue #324 ("Preserve hardlinks in artifacts") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/324 | 12:46 |
| juergbi | laurence: ^^ #324 is the hardlink aspect of #38 | 12:53 |
| laurence | juergbi, tvm for that | 12:55 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 13:01 |
| *** mohan43u has quit IRC | 13:05 | |
| *** mohan43u has joined #buildstream | 13:09 | |
| Nexus | all is well in the world | 13:19 |
| tlater | \o/ | 13:19 |
| Nexus | apart from compose | 13:19 |
| tlater | Well, except in compose.py, perhaps | 13:19 |
| Nexus | someone wake up tristan :D | 13:19 |
| tlater | juergbi: Think you could review https://gitlab.com/BuildStream/buildstream/merge_requests/315 again for me? | 13:20 |
| tlater | You must be getting really tired of that MR... | 13:20 |
| * Nexus is please that his MR does in fact work | 13:21 | |
| juergbi | tlater: sure, will take a look | 13:22 |
| * laurence crosses fingers and hopes that #315 can be merged this time. | 13:22 | |
| * Nexus also crosses his fingers | 13:22 | |
| Nexus | so tlater, 13:36 < Nexus> So, i seem to have `./tests/cachekey/update.py` pointing at bst-external for some reason | 13:23 |
| tlater | Right, yeah | 13:23 |
| tlater | It's because the testutils module is the same in buildstream and bst-external | 13:23 |
| tlater | So python gets confused | 13:23 |
| tlater | We want to import the testutils from buildstream anyway though | 13:23 |
| tlater | Temporary workaround is uninstalling bst-external, so maybe we want to temporarily rename the module in bst-external? | 13:24 |
| tlater | ^ Might be a while until we get around to opening up the test API, so this might be worthwhile | 13:24 |
| * Nexus rm -rf 's bst-external | 13:25 | |
| tlater | Oh, that might actually not work, Nexus | 13:27 |
| tlater | You'd want to pip3 remove bst-external | 13:27 |
| Nexus | it worked :) | 13:27 |
| juergbi | tlater: just one squash missing and then I think we can land this | 13:31 |
| juergbi | let's hope we haven't missed anything in the core logic | 13:32 |
| tlater | juergbi: Ah right, I couldn't find where those imports were made unused | 13:32 |
| tlater | tyvm! | 13:32 |
| juergbi | I just used git log -p buildstream/source.py | 13:32 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 13:33 |
| * tlater tries this command | 13:33 | |
| * tlater is amused that #215 is being solved by !315 | 13:35 | |
| tlater | juergbi: Regarding the python3.4 CI change | 13:36 |
| Nexus | :) | 13:36 |
| tlater | I think I prefer setting images for all tests, even if it's the default image - we'll hopefully have a lot of platforms soon-ish, which will make one test *not* setting an image stick out like a sore thumb. | 13:37 |
| tlater | The default should probably still be debian though, didn't consider that `setup.py sdist` should work on python3.4 as well... | 13:37 |
| juergbi | tlater: the one downside I see here is that we have to update multiple places when updating the debian image | 13:38 |
| tlater | Hm, I suppose this is one of those spots where you just end up arguing forever, so I'll go with what the maintainer says | 13:39 |
| juergbi | lots of platforms: do you mean non-Linux? how does gitlab-ci actually work in that case? it's not like you can run FreeBSD in docker | 13:39 |
| tlater | juergbi: That's true, I suppose. Probably better to say lots of distributions, we'll need different runners for platforms | 13:40 |
| juergbi | I just suspect someone will forget to update it in one place | 13:40 |
| juergbi | if we can specify images indirectly via variables, I'd be fine referring to a particular image variable in each job | 13:41 |
| juergbi | don't know whether that's possible in the file format | 13:41 |
| gitlab-br-bot | buildstream: merge request (146-use-minimum-python-version-3-4-for-integration-tests->master: Resolve "Use minimum python version (3.4) for integration tests") #266 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/266 | 13:41 |
| tlater | juergbi: Don't think so, I could check | 13:41 |
| tlater | But I think that's more effort than it's worth | 13:41 |
| juergbi | are we planning to run CI on more distros? I'm not aware of this | 13:42 |
| tlater | tristan has been talking about this for a while | 13:42 |
| * tlater thinks it's a sensible idea to at least run it for a handful of distros | 13:43 | |
| gitlab-br-bot | buildstream: issue #215 ("Workspace builds might not rebuild correctly when dependecies are updated") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/215 | 13:43 |
| gitlab-br-bot | buildstream: merge request (215-workspace-builds-might-not-rebuild-correctly-when-dependecies-are-updated->master: Resolve "Workspace builds might not rebuild correctly when dependecies are updated") #315 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/315 | 13:43 |
| tlater | \o/ | 13:43 |
| Nexus | \o/ | 13:43 |
| Nexus | laurence: ^ | 13:43 |
| * tlater hopes using debian-8 as the default doesn't break docs, otherwise that branch should hopefully be ready today as well | 13:44 | |
| juergbi | sounds good | 13:45 |
| gitlab-br-bot | buildstream: merge request (bst-here-update->master: bst-here: Add '-u' flag to upgrade buildstream (Issue #291)) #311 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/311 | 13:49 |
| tlater | Ack, arpy isn't in the debian image yet :| | 13:49 |
| gitlab-br-bot | buildstream: merge request (bst-here-update->master: bst-here: Add '-u' flag to upgrade buildstream (Issue #291)) #311 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/311 | 13:50 |
| Nexus | Dear Buildstream: Please stop failing tests! | 13:53 |
| Nexus | Kind Regards, Nexus | 13:53 |
| tlater | Do we have a maintainer for buildstream-docker-images? | 13:56 |
| Nexus | tlater: well volunteered :) | 13:59 |
| * tlater thinks he lacks the reviewing superpowers you need for the job | 14:00 | |
| tlater | Plus, I'd rather not review my own patches, even if they're one-liners ;P | 14:00 |
| juergbi | approved | 14:03 |
| gitlab-br-bot | buildstream: merge request (bst-here-update->master: bst-here: Add '-u' flag to upgrade buildstream (Issue #291)) #311 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/311 | 14:10 |
| Nexus | when a bst command is called in the tests, is it called using all of the arguments? | 14:16 |
| tlater | Nexus: It's given a few arguments and then yours are added | 14:18 |
| Nexus | explain further pls? | 14:18 |
| jmac | Typically we invoke bst through the cliapp system rather than using a shell | 14:18 |
| jmac | In tests, I mean | 14:18 |
| tlater | Nexus: I think you'll have to be more specific with your question | 14:19 |
| Nexus | tlater: so, i was testing if my code worked with "--no-cache" active, however it seemed to run when i reverse the defaults, implying it was being called | 14:20 |
| tlater | Nexus: What was being called? | 14:22 |
| Nexus | workspace open --no-cache | 14:22 |
| tlater | Sorry for being slow, I have very little context on what you're working on right now | 14:22 |
| tlater | There's a --no-cache command there? | 14:22 |
| tlater | Huh | 14:22 |
| Nexus | there is now :) | 14:22 |
| tlater | Nexus: You can look at what exactly is being run by raising an exception in the cli class, just before the command is executed | 14:23 |
| gitlab-br-bot | buildstream: merge request (test-run-doc->master: HACKING.rst: Add integration and pytest notes) #319 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/319 | 14:46 |
| *** aday has quit IRC | 14:49 | |
| gitlab-br-bot | buildstream: merge request (test-run-doc->master: HACKING.rst: Add integration and pytest notes) #319 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/319 | 14:58 |
| gitlab-br-bot | buildstream: issue #325 ("bst is pushing artifact it just have pulled") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/325 | 14:59 |
| *** aday has joined #buildstream | 15:02 | |
| gitlab-br-bot | buildstream: merge request (soft-arpy-dependency->master: WIP: Soft arpy dependency) #342 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/342 | 15:04 |
| *** Prince781 has joined #buildstream | 15:06 | |
| *** Prince781 has quit IRC | 15:07 | |
| *** Prince781 has joined #buildstream | 15:08 | |
| gitlab-br-bot | buildstream: merge request (146-use-minimum-python-version-3-4-for-integration-tests->master: Resolve "Use minimum python version (3.4) for integration tests") #266 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/266 | 15:08 |
| * tlater crosses fingers | 15:09 | |
| *** toscalix has quit IRC | 15:22 | |
| gitlab-br-bot | buildstream: merge request (soft-arpy-dependency->master: Soft arpy dependency) #342 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/342 | 15:27 |
| tlater | Hm, looks like one of the recent tests tickles out an issue with _yaml.py in python3.4: https://gitlab.com/BuildStream/buildstream/-/jobs/59750556 | 15:27 |
| * tlater isn't sure what to do about this | 15:28 | |
| gitlab-br-bot | buildstream: issue #326 ("undocumented behaviour when Buildstream rebuilds all elements locally") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/326 | 15:32 |
| gitlab-br-bot | buildstream: merge request (146-use-minimum-python-version-3-4-for-integration-tests->master: Resolve "Use minimum python version (3.4) for integration tests") #266 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/266 | 15:35 |
| gitlab-br-bot | buildstream: issue #327 ("There is no link to Gitlab from the docs") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/327 | 16:00 |
| gitlab-br-bot | buildstream: merge request (jmac/performance-ci->master: WIP: Add benchmark to CI test and document CI.) #341 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/341 | 16:07 |
| gitlab-br-bot | buildstream: merge request (jmac/performance-ci->master: WIP: Add benchmark to CI test and document CI.) #341 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/341 | 16:08 |
| jmac | !341 is my attempt to add benchmark tests to BuildStream's CI; could anyone take a look at it and see what they think? | 16:09 |
| jmac | Not urgent | 16:09 |
| laurence | tlater, cs-shadow is the maintainer for the docker-images repo | 16:13 |
| laurence | he only pops up on here occasionally, though | 16:13 |
| tlater | Ah, that's news | 16:14 |
| * tlater will keep this in mind | 16:14 | |
| gitlab-br-bot | buildstream: merge request (jjardon/ref-to-code->master: docs: Add basic resources page in the fron page) #343 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/343 | 16:19 |
| *** valentind has joined #buildstream | 16:20 | |
| jmac | tlater: I'm not sure what you mean by 'target' in your comment on !341 | 16:22 |
| tlater | jmac: Err, I think the terminology is jobs in gitlab yaml | 16:22 |
| tlater | Sorry, they look like makefile targets to me | 16:22 |
| jmac | Heh :) | 16:22 |
| jmac | You mean stages? | 16:23 |
| tlater | Yes! | 16:23 |
| jmac | So the `before_script` bit is common code for unix-test and linux-test, but we don't want that bit for linux-perf-tests | 16:23 |
| jmac | So we could add that bit to both unix-test and linux-test but that means duplicate code | 16:23 |
| tlater | I think we want entirely separate environments for these tests | 16:24 |
| tlater | Perhaps we shouldn't call before_script before_script, but something else | 16:24 |
| tlater | And only run it before unix/linux | 16:24 |
| jmac | before_script is a GitLab thing | 16:24 |
| tlater | We can define a stage instead though :) | 16:24 |
| jmac | I don't think you can conditionally execute a stage | 16:25 |
| tlater | jmac: Isn't that what dependencies do? | 16:25 |
| tlater | Like we depend on source-dist? | 16:25 |
| jmac | hmm | 16:26 |
| * tlater generally dislikes if statements in .gitlab-ci.yaml, but if there's no other way it's not a huge issue | 16:26 | |
| jmac | Yes, that might work, I'll give it try | 16:27 |
| jmac | I wasn't keen on the if statement either | 16:27 |
| gitlab-br-bot | buildstream: merge request (jmac/performance-ci->master: WIP: Add benchmark to CI test and document CI.) #341 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/341 | 16:31 |
| gitlab-br-bot | buildstream: merge request (jjardon/ref-to-code->master: docs: Add basic resources page in the fron page) #343 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/343 | 16:32 |
| *** noisecell has quit IRC | 16:39 | |
| *** Prince781 has quit IRC | 16:44 | |
| *** tiago has quit IRC | 16:45 | |
| jmac | tlater: Unfortunately, it looks like each stage is a separate sandbox - so setting up a user in a previous stage won't make a user available in linux-tests | 16:59 |
| jmac | That's what appears to be going on in https://gitlab.com/BuildStream/buildstream/pipelines/19583998 anyway | 16:59 |
| tlater | jmac: Aww :[ | 17:00 |
| jmac | One alternative is to use YAML references, as in https://gitlab.com/gitlab-org/gitlab-ce/issues/15403#note_27079119 | 17:00 |
| jmac | I haven't seen YAML references being used much in baserock or buildstream - are they a problem? | 17:00 |
| tlater | jmac: I've never even seen the feature | 17:02 |
| tlater | I'd say try it out, quite possible none of us have come across it :) | 17:02 |
| gitlab-br-bot | buildstream: merge request (146-use-minimum-python-version-3-4-for-integration-tests->master: Resolve "Use minimum python version (3.4) for integration tests") #266 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/266 | 17:03 |
| tlater | Not sure adding commits named 'Temporary commit' is best practice for sharing code across computers... | 17:04 |
| * Nexus looks at tlaters worksapce changes and cries | 17:04 | |
| gitlab-br-bot | buildstream: merge request (jmac/performance-ci->master: WIP: Add benchmark to CI test and document CI.) #341 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/341 | 17:13 |
| gitlab-br-bot | buildstream: merge request (jmac/performance-ci->master: WIP: Add benchmark to CI test and document CI.) #341 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/341 | 17:16 |
| *** ernestask has quit IRC | 17:32 | |
| *** bethw has quit IRC | 17:34 | |
| *** valentind has quit IRC | 17:35 | |
| *** valentind has joined #buildstream | 17:35 | |
| *** tristan has joined #buildstream | 17:59 | |
| gitlab-br-bot | buildstream: merge request (jjardon/pip3->master: HACKING.rst: Be specific about the only pip packages required are the python3 ones) #344 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/344 | 18:00 |
| gitlab-br-bot | buildstream: merge request (jjardon/pip3->master: HACKING.rst: Be specific about the only pip packages required are the python3 ones) #344 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/344 | 18:00 |
| gitlab-br-bot | buildstream: merge request (jennis/correct-installing-documentation->master: install.rst: Inform user to restart terminal once they have adjusted their path) #340 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/340 | 18:05 |
| gitlab-br-bot | buildstream: merge request (jennis/correct-installing-documentation->master: install.rst: Inform user to restart terminal once they have adjusted their path) #340 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/340 | 18:17 |
| *** aiden has quit IRC | 18:25 | |
| *** adds68 has quit IRC | 18:25 | |
| *** paulsherwood has quit IRC | 18:25 | |
| *** laurence has quit IRC | 18:25 | |
| *** paulsherwood has joined #buildstream | 18:25 | |
| *** aiden has joined #buildstream | 18:26 | |
| *** laurence has joined #buildstream | 18:26 | |
| *** slaf- has joined #buildstream | 18:27 | |
| *** slaf- has joined #buildstream | 18:28 | |
| *** slaf has quit IRC | 18:29 | |
| *** slaf- is now known as slaf | 18:29 | |
| *** aiden has quit IRC | 18:35 | |
| *** tlater has quit IRC | 18:35 | |
| *** laurence has quit IRC | 18:35 | |
| *** milloni has quit IRC | 18:35 | |
| *** aiden has joined #buildstream | 18:41 | |
| *** benbrown has quit IRC | 18:44 | |
| *** paulsherwood has quit IRC | 18:44 | |
| *** jmac has quit IRC | 18:44 | |
| *** Nexus has quit IRC | 18:44 | |
| *** paulsherwood has joined #buildstream | 18:46 | |
| *** paulsherwood has joined #buildstream | 18:46 | |
| *** laurence has joined #buildstream | 18:47 | |
| *** tlater has joined #buildstream | 18:54 | |
| *** milloni has joined #buildstream | 18:55 | |
| gitlab-br-bot | buildstream: merge request (jjardon/index_problems->master: Use toctree instead normal list so correct index always appear in the left menu) #345 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/345 | 19:10 |
| gitlab-br-bot | buildstream: merge request (jjardon/index_problems->master: Use toctree instead normal list so correct index always appear in the left menu) #345 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/345 | 19:10 |
| gitlab-br-bot | buildstream: merge request (jjardon/index_problems->master: Use toctree instead normal list so correct index always appear in the left menu) #345 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/345 | 19:10 |
| gitlab-br-bot | buildstream: merge request (jjardon/index_problems->master: Use toctree instead normal list so correct index always appear in the left menu) #345 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/345 | 19:12 |
| *** aday has quit IRC | 20:02 | |
| *** adds68 has joined #buildstream | 20:52 | |
| *** mohan43u has quit IRC | 20:56 | |
| *** mohan43u has joined #buildstream | 21:00 | |
| *** mohan43u has quit IRC | 21:05 | |
| *** mohan43u has joined #buildstream | 21:08 | |
| *** aday has joined #buildstream | 21:34 | |
| *** xjuan has quit IRC | 21:42 | |
| *** valentind has quit IRC | 21:55 | |
| *** aday has quit IRC | 21:57 | |
| *** mohan43u has quit IRC | 22:06 | |
| *** mohan43u has joined #buildstream | 22:10 | |
| *** Prince781 has joined #buildstream | 22:40 | |
| *** Prince781 has quit IRC | 23:09 | |
| *** mohan43u has quit IRC | 23:27 | |
| *** mohan43u has joined #buildstream | 23:30 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!