| *** slaf has joined #buildstream | 00:14 | |
| *** Prince781 has joined #buildstream | 03:09 | |
| *** Prince781 has quit IRC | 04:35 | |
| *** tristan has joined #buildstream | 08:14 | |
| *** toscalix has joined #buildstream | 08:48 | |
| *** valentind has joined #buildstream | 08:59 | |
| *** jennis has quit IRC | 09:04 | |
| *** ssam2 has joined #buildstream | 09:05 | |
| *** jennis has joined #buildstream | 09:23 | |
| gitlab-br-bot | buildstream: merge request (sam/install-tweaks->master: doc/source/install.rst: Clarify install instructions) #252 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/252 | 09:30 |
|---|---|---|
| gitlab-br-bot | buildstream: merge request (sam/plugin-docs->master: doc: Promote documention on how to use external plugins up one level) #209 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/209 | 09:32 |
| gitlab-br-bot | buildstream: merge request (sam/symlinks-optimization->master: Improve tests for symlink handling) #246 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/246 | 09:34 |
| ssam2 | https://gitlab.com/BuildStream/bst-external/merge_requests/9 should be ready for merge now, can anyone take a look ? | 09:36 |
| *** tiago has joined #buildstream | 09:58 | |
| tristan | ssam2, looks alright to me, just went through chasing what is happening with the exceptions... also looks like it chose to use `requests` library | 09:58 |
| tristan | which jumps out at me a bit | 09:58 |
| juergbi | i have one comment but it looks like gitlab is failing... | 09:59 |
| tristan | I'm not sure why we chose to use urllib for tar & zip sources and now requests for docker sources | 09:59 |
| tristan | Also looks like we need to work out a better story on sharing our test scaffolding with third party plugin repos (but we knew that...) | 10:00 |
| juergbi | ok, added comment about relative paths | 10:01 |
| ssam2 | requests is generally better | 10:01 |
| ssam2 | i can switch to to use urllib if preferred | 10:01 |
| juergbi | damn it, lost a comment on another MR because of broken gitlab | 10:01 |
| tristan | ssam2, I dont want to block on that really; and I dont have a preference for urllib | 10:02 |
| juergbi | ah no, it got through | 10:02 |
| *** jonathanmaw has joined #buildstream | 10:02 | |
| tristan | ssam2, but if requests is generally better (google shows me various SO posts which seem to recommend requests for convenience)... it might be worth filing a bug about using requests uniformly throughout the plugins we maintain | 10:03 |
| *** tiago has quit IRC | 10:03 | |
| *** tiago has joined #buildstream | 10:04 | |
| ssam2 | the in-tree ones as well ? ok | 10:04 |
| tristan | ah nice catch juergbi | 10:04 |
| gitlab-br-bot | buildstream: merge request (sam/symlinks-optimization->master: Improve tests for symlink handling) #246 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/246 | 10:14 |
| tristan | errr | 10:16 |
| tristan | juergbi, ssam2 ... the above merge 246 adds a test to the integration subdirectory | 10:16 |
| tristan | And does not mark it with @pytest.mark.integration | 10:16 |
| tristan | What do we think about keeping all the integration style tests in the same directory ? | 10:17 |
| *** aday has joined #buildstream | 10:17 | |
| juergbi | oh, i missed that. it looks like it ran pretty fast | 10:18 |
| tristan | I dont personally care about keeping them all in the same directory | 10:18 |
| juergbi | tristan: it has pytestmark = pytest.mark.integration | 10:19 |
| juergbi | so it is properly skipped outside integration tests | 10:19 |
| tristan | oh ? | 10:19 |
| tristan | aha strange | 10:20 |
| juergbi | it does it for the whole file instead of individual tests | 10:20 |
| tristan | it *is* an integration test I think anyway | 10:20 |
| *** dominic has joined #buildstream | 10:20 | |
| juergbi | yes, it is | 10:20 |
| tristan | because it refers to this `base.bst` as a dependency but doesnt provide it | 10:20 |
| juergbi | (pulls in base.bst) | 10:20 |
| tristan | so I presume that may be a magick string | 10:20 |
| *** dominic has quit IRC | 10:21 | |
| juergbi | not magic, just reusing base from other integration tests | 10:21 |
| *** dominic has joined #buildstream | 10:21 | |
| juergbi | (extending shared project) | 10:21 |
| tristan | yeah, it must be doing some magick though; like copying in something | 10:21 |
| juergbi | no, base.bst already existed in that directory | 10:21 |
| tristan | ahaaaaa | 10:22 |
| tristan | I see what he did there | 10:22 |
| gitlab-br-bot | buildstream: merge request (sam/symlink-speedup->master: utils.py: Wrap calls to os.path.realpath() in an LRU cache) #247 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/247 | 10:26 |
| *** valentind has quit IRC | 10:33 | |
| *** ernestask has joined #buildstream | 10:40 | |
| gitlab-br-bot | buildstream: issue #174 ("utils._ensure_real_directory() very slow in some cases") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/174 | 10:54 |
| gitlab-br-bot | buildstream: merge request (sam/symlink-speedup->master: utils.py: Wrap calls to os.path.realpath() in an LRU cache) #247 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/247 | 10:54 |
| tristan | ssam2, so do we have your ostree-push fork available in a docker image usable by gitlab runners yet ? | 11:20 |
| tristan | I'm trying to see if I can put out one of these fires, basically I'd like to tie up the loop of debootstrap-ostree | 11:20 |
| tristan | instead of having that on a dedicated machine, I'd like the debootstrap process to run when changes get merged to the repo | 11:21 |
| tristan | I *can* work around this ala flatpak-build-scripts, and record the latest sha/arch to avoid too many debootstrap refreshes | 11:22 |
| tristan | but it looks like the gitlab route is the elegant one | 11:22 |
| adds68 | tristan, sorry to bother you with non-tech questions, but is there any reason buildstream isn't using http://docs.readthedocs.io/en/latest/getting_started.html ? | 11:24 |
| adds68 | tristan, i think it's easy to use as you already generate the docs with sphinx | 11:24 |
| adds68 | tristan, much nicer UI and formatting than the current docs page also, i could raise an issue if this is something that could be explored further | 11:25 |
| gitlab-br-bot | buildstream: issue #245 ("Logging issues") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/245 | 11:28 |
| paulsherwood | adds68: +1 for the readthedocs approach | 11:31 |
| tristan | adds68, We're using gitlab pages simply because it's less moving parts and it lets us auto-update the docs when things get landed to master | 11:36 |
| tristan | adds68, there is a bug in there that is important to fix, though: https://gitlab.com/BuildStream/buildstream/issues/178 | 11:36 |
| paulsherwood | tristan: can't we still acheive that with readthedocs approach? | 11:37 |
| tristan | Maybe ? | 11:37 |
| tristan | Why do we want our docs at "readthedocs.com" anyway ? | 11:37 |
| tristan | seems rather arbitrary | 11:37 |
| adds68 | tristan, it's just a well used place to store docs for things built using Python | 11:38 |
| paulsherwood | tristan: it's the layout i'm interested in | 11:38 |
| tristan | adds68, we are not a part of the python ecosystem and wont really be | 11:38 |
| adds68 | paulsherwood, tristan another approach is to change the current HTML style to use the Mesa style https://www.mesa3d.org/intro.html | 11:38 |
| tristan | And the layout can be controlled somewhat with themes right ? | 11:39 |
| * paulsherwood currently feels like it's far from obvious how to get started, is all | 11:39 | |
| tristan | Also wont we lose horizontal screen realestate with something like readthedocs ? | 11:39 |
| adds68 | tristan, understood, it was just a quick fix suggestion is all | 11:39 |
| ssam2 | tristan, no, it's not in any of the images yet | 11:42 |
| ssam2 | can add it fairly easily... or since it's tiny and in python, it would be fairly quick just to git clone in the .gitlab-ci.yml each time | 11:43 |
| tristan | ssam2, nice ok... will try that | 11:43 |
| ssam2 | on the subject of docs, we still have the issue of https://gitlab.com/BuildStream/buildstream/issues/178 | 11:43 |
| ssam2 | i'd be in favour of any solution which can fix that | 11:44 |
| juergbi | hm, seeing unexpected gitlab test failures on master | 11:44 |
| juergbi | PermissionError: [Errno 13] Permission denied: '/builds/BuildStream/cache/integration-cache/artifacts' | 11:44 |
| tristan | ssam2, Agreed, that is a pain point | 11:45 |
| juergbi | do we need to clear the runner cache? and do we need to fix something in the ci instructions? | 11:45 |
| tristan | If moving to readthedocs solves that; then I dont immensely mind, although I do have reservations about getting tangled into a readthedocs dependency | 11:46 |
| tristan | there is something clean and nice about "When we run CI, the static HTML is generated, and can simply be placed <here>" | 11:46 |
| adds68 | tristan, i just think it's a shame how when i point people to the project they get put off with the current docs situation | 11:46 |
| tristan | Okay. | 11:47 |
| aday | tristan: i'm no expert, but i'd surprised if you couldn't do that with sphinx and get readthedocs at the same time | 11:47 |
| ssam2 | the complex part is we do custom doc auto-generation from the element and source plugins | 11:48 |
| ssam2 | readthedocs seems to want to control the whole doc generation process, so i don't know if we could extend it to understand our plugins | 11:48 |
| adds68 | ssam2, if that is done using Sphinx, read the docs will just import what ever is generated | 11:48 |
| ssam2 | ok, that doesn't sound too hard then | 11:48 |
| ssam2 | if it's that easy, anyone fancy doing a prototype ? :-) | 11:49 |
| aday | yeah, we have generated api reference as part of the flatpak readthedocs site | 11:50 |
| aday | it can handle plain html pages | 11:52 |
| ssam2 | however, this is orthogonal to the complaint that "it's far from obvious how to get started" | 11:53 |
| ssam2 | which i agree is an issue; but it raises the immediate question of "get started with *what*" | 11:53 |
| ssam2 | e.g. if you want to get started building GNOME, you start by reading https://wiki.gnome.org/Newcomers/BuildSystemComponent | 11:53 |
| ssam2 | other projects using BuildStream should have similar such guides | 11:54 |
| adds68 | ssam2, yea i feel that is the main issue, i feel GNOME is too big of an example to start with | 11:54 |
| ssam2 | most people aren't going to think "I want to use buildstream" | 11:54 |
| jmac | I disagree | 11:54 |
| ssam2 | they will think "I want to use buildstream *to do X*" | 11:54 |
| ssam2 | developers being told "your new project is to work on buildstream's codebase" are an exception here, yet are the majority of people currently looking at docs | 11:54 |
| jmac | I really don't think they're an exception | 11:55 |
| adds68 | ssam2, that is what my friend asked me when i sent him to the docs | 11:55 |
| ssam2 | sure, his case is the sort of thing we should be looking at | 11:55 |
| ssam2 | what is his use case ? | 11:55 |
| adds68 | ssam2, well basically they build lost of compilers cross platform and he said Buildstream sounds like it could remove the need for their custom/buggy tool they use | 11:56 |
| adds68 | ssam2, but i don't fully understand his use case as he can't explain a lot due to work contract etc | 11:56 |
| ssam2 | so broadly, the use case there is writing elements to build stuff | 11:57 |
| adds68 | ssam2, yea i tried to explain to him about how you can write custom elements, but he then i had to explain buildstream-external etc | 11:57 |
| adds68 | then i* | 11:58 |
| ssam2 | that's an advanced topic | 11:58 |
| ssam2 | https://gitlab.com/BuildStream/buildstream-examples should help a lot here | 11:59 |
| ssam2 | if we could just link to a few simple, well-documented examples in that repo from the top of the main documentation ... | 12:00 |
| ssam2 | of course, we don't yet have any examples (although i think tlater has one on image building nearly complete) | 12:00 |
| adds68 | Ok i've pinged him and he said he is going to write down what he is trying to solve and some tips as to why the docs didn't answer his questions | 12:01 |
| ssam2 | thanks! | 12:02 |
| adds68 | Hopefully have some actual facts people can view rather than my poor explanations | 12:02 |
| jmac | Did I understand correctly that we don't treat build-commands and install-commands as different in any way? | 12:47 |
| ssam2 | as far as I know, that's correct | 12:48 |
| jmac | Good | 12:49 |
| ssam2 | certainly, in the context of running the build we don't: https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/buildelement.py#L200 | 12:49 |
| *** mcatanzaro has joined #buildstream | 13:22 | |
| tristan | aha ! | 13:24 |
| tristan | mcatanzaro, I'm back from one week off (looks like I chose the wrong week ?), end of day now and have been trying to put out these fires | 13:24 |
| mcatanzaro | tristan: Welcome back! | 13:25 |
| tristan | yeah, what a mess when I come back haha :) | 13:25 |
| mcatanzaro | I'm going to be busy with meetings for the next 2 hours, so looks like we both picked bad timezones, but I think the only problem that could block this week's release is that I can't add new system dependencies | 13:26 |
| mcatanzaro | Basically I have no chance without a solution for that | 13:26 |
| tristan | mcatanzaro, so there are a couple things... one is [re]automating debootstrap-ostree; I wanted to put the process on gnome infra via gitlab but thats a little bit challenging | 13:26 |
| tristan | right, I added the things you wanted, that's up for now; but unfortunately it doesnt close the problem | 13:26 |
| tristan | One thing is cantarell-fonts now wants fontmake (after changing to meson it seems) | 13:27 |
| mcatanzaro | Might be that you just have to add the deps this week as I request them | 13:27 |
| tristan | and that is only available in sid | 13:27 |
| mcatanzaro | Oh, so, do we have to update the base system to sid, then? | 13:27 |
| tristan | That is not a problem for me tbh, but I'm currently building a sid debootstrap and see if that works well | 13:27 |
| tristan | the debootstrap process is passing, have to try to run a build | 13:27 |
| mcatanzaro | It's also odd that the debootstrap is shared with non-GNOME projects | 13:28 |
| tristan | pango has a harder problem to solve, too | 13:28 |
| mcatanzaro | So really would be nicer to have on our infrastructure... what's wrong with pango? | 13:28 |
| mcatanzaro | I mean, why is its dependency hard to solve? | 13:28 |
| tristan | which is... I think changing to meson made that dep you added non-optional | 13:28 |
| tristan | and with it available from stretch, it has a bug | 13:29 |
| * tristan had the bug report open, lemme find it | 13:29 | |
| tristan | mcatanzaro, I'm getting this error: https://bugs.freedesktop.org/show_bug.cgi?id=82548 :-/ | 13:30 |
| tristan | or pretty much similar error | 13:30 |
| mcatanzaro | Is pango using -Werror? | 13:30 |
| mcatanzaro | pango is owned by mclasen, should be no problem to get it fixed | 13:31 |
| tristan | not explicitly, it's using meson now | 13:31 |
| mcatanzaro | Surely meson does not add -Werror automatically? | 13:31 |
| mcatanzaro | BTW that dependency is new in git master, so it won't block this release once we convert the buildstream projects to tarballs... (but we should fix it anyway, because you can't build anything that depends on GTK+ currently ;) | 13:32 |
| tristan | I dont think it adds -Werror, and the bug report fwiw speaks of -Wundef | 13:33 |
| mcatanzaro | So what is the error then? | 13:33 |
| * tristan pushed convo to #release-team :) | 13:34 | |
| *** tristan has quit IRC | 13:41 | |
| gitlab-br-bot | buildstream: merge request (juerg/scheduler->master: scheduler.py: Do not prematurely terminate loop after skipping jobs) #270 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/270 | 14:00 |
| juergbi | ssam2: i believe we've agreed to merge https://gitlab.com/BuildStream/bst-external/merge_requests/9 as is, right? | 14:12 |
| ssam2 | i think so yeah | 14:12 |
| gitlab-br-bot | buildstream: merge request (versioned_yaml->master: WIP: Version workspaces.yml) #271 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/271 | 14:13 |
| juergbi | ok, i pressed the button ;) | 14:13 |
| ssam2 | thanks! | 14:14 |
| ssam2 | so now buildstream supports devops ;-) | 14:14 |
| *** tristan has joined #buildstream | 14:21 | |
| *** csoriano has quit IRC | 14:46 | |
| *** csoriano has joined #buildstream | 14:46 | |
| jmac | Since "sam/use-recursive-pipelines" was merged into freedesktop-sdk, bst complains about the 'junction' keyword | 14:56 |
| ssam2 | update your version of buildstream | 14:56 |
| ssam2 | support for junction was merged into master last Friday | 14:56 |
| ssam2 | freedesktop SDK 1.8 is still "unstable" at present, so can play fast and loose with build tool requirements :-) | 14:57 |
| jmac | Update worked, thanks | 14:58 |
| nexus | Looking into !245 again, i seem to be unable to reproduce the error now | 15:08 |
| juergbi | nexus: do you mean the original issue or the error you saw with the first patch? | 15:09 |
| nexus | the original issue, if i remove all of my "fixes" and run the code, it works fine | 15:09 |
| juergbi | hm, was there a ruamel upstream change? | 15:13 |
| nexus | have you reproduced it on your end? | 15:13 |
| nexus | i'll have a look | 15:13 |
| juergbi | i can still reproduce it at least with the raw python3 (no buildstream) test case | 15:14 |
| juergbi | let me try the bst test case | 15:14 |
| juergbi | that's still reproducible here | 15:15 |
| nexus | hmm | 15:16 |
| juergbi | nexus: what happens on your system if you track the mini project from the issue description? | 15:22 |
| juergbi | maybe also put the output on pastebin | 15:23 |
| nexus | it works exactelyit works perfectly as far as i can tell | 15:24 |
| nexus | https://paste.codethink.co.uk/?3993 | 15:24 |
| gitlab-br-bot | buildstream: merge request (212-git-source-needs-a-way-to-disable-checking-out-submodules->master: Resolve "Git source needs a way to disable checking out submodules") #259 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/259 | 15:26 |
| *** noisecell has quit IRC | 15:27 | |
| juergbi | nexus: yes, that looks fine. commit 3573f99 mentioned in the log doesn't exist in the buildstream repo, though. are you sure you're testing the version without any fixes? | 15:34 |
| nexus | juergbi: yep, i just used bst master and it worked fine | 15:43 |
| *** noisecell has joined #buildstream | 15:44 | |
| * nexus is at a loss | 15:54 | |
| juergbi | nexus: did you also try the buildstream-independent python test case in the comment? | 16:00 |
| nexus | yes, that still has the bug | 16:01 |
| *** dominic has quit IRC | 16:01 | |
| juergbi | can you double check that you're really testing bst master and not a version you installed before or similar? | 16:01 |
| juergbi | the commit id in the pastebin was not master, afaict | 16:02 |
| *** lantw44 has quit IRC | 16:05 | |
| *** lantw44 has joined #buildstream | 16:13 | |
| *** noisecell has quit IRC | 16:18 | |
| nexus | juergbi: i'm running this on 0f430622e4539b5cc9209384e32980dc086c29df | 16:25 |
| *** jonathanmaw has quit IRC | 16:26 | |
| juergbi | nexus: do you now also see that commit sha in bst --version? | 16:27 |
| nexus | 0f43062 | 16:30 |
| *** noisecell has joined #buildstream | 16:35 | |
| *** bethw has joined #buildstream | 16:39 | |
| juergbi | i updated to the latest ruamel.yaml-0.15.35 and can still reproduce it | 16:40 |
| *** bethw has quit IRC | 16:40 | |
| *** jonathanmaw has joined #buildstream | 16:40 | |
| jmac | I've got the latest master of freedesktop-sdk and buildstream now. I can now build base/glib.bst and base-stack.bst, but I can't checkout base/glib.bst, which is a dependency of base-stack.bst | 16:41 |
| jmac | I don't understand how I can use something as a dependency for a build but not check it out | 16:41 |
| ssam2 | what happens when you try to check it out ? | 16:42 |
| jmac | This: https://paste.gnome.org/pyyox95zk | 16:43 |
| jmac | It can't find `glib-compile-schemas`, which is one of the binaries produced in the build | 16:43 |
| ssam2 | hmmm.... | 16:43 |
| ssam2 | no, it can't find a shell | 16:43 |
| jmac | Oh, I see | 16:43 |
| jmac | So glib.bst has a custom integration-command | 16:44 |
| jmac | Which will need a shell, of course | 16:44 |
| ssam2 | yeah | 16:44 |
| persia | missing runtime dependency? | 16:44 |
| ssam2 | you could also `bst checkout --no-integrate` | 16:44 |
| jmac | persia: This element only has build dependencies. | 16:45 |
| ssam2 | although it does suggest something is wrong in the dependency list | 16:45 |
| jmac | Do integration commands explicitly use runtime dependencies? | 16:45 |
| ssam2 | i'm not sure how that works in detail actually | 16:46 |
| ssam2 | i know that `bst checkout` should get you the runtime dependenciues | 16:46 |
| ssam2 | so in this case, the integration command would work if you had a runtime dependency on the 'base' binaries | 16:46 |
| persia | jmac: That's a gap, but likely not an uncommon one. In Debian, for example, everything depends on a base environment, which is allowed to assume things like a working shell, so nobody bothers to document the dependencies all the way down. | 16:46 |
| ssam2 | in general avoid thinking about this by always runtime-depending on the 'base' | 16:47 |
| ssam2 | *I avoid thinking about it | 16:47 |
| persia | I believe integration commands *may* require runtime dependencies, and the code should allow it to work that way (but I don't know if the code does allow it to work that way) | 16:47 |
| jmac | In this particular project almost everything depends on "bootstrap-import.bst", but it's marked as a build dependency only | 16:47 |
| jmac | I wouldn't have thought of checking out as "running", so it seems odd to need that as a runtime dependency for this. | 16:47 |
| jmac | Now that I know that, it's easily fixed. | 16:47 |
| ssam2 | checking out is considered as "I want to do something with this artifact" | 16:47 |
| jmac | "do something" includes building with... and we have build-time dependencies | 16:48 |
| gitlab-br-bot | buildstream: merge request (jonathan/bzr-source-method->master: Jonathan/bzr source method) #242 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/242 | 16:48 |
| gitlab-br-bot | buildstream: merge request (jonathan/bzr-source-method->master: Jonathan/bzr source method) #242 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/242 | 16:49 |
| ssam2 | i think that in buildstream, we consider build-time as something that only happens *inside* BuildStream | 16:49 |
| *** dominic has joined #buildstream | 16:49 | |
| jmac | That's fine, I think it might be implicit knowledge at the moment though | 16:49 |
| ssam2 | I imagine so | 16:50 |
| persia | Doesn't buildstream also permit one to execute an arbitrary command with checkout or shell, such that if one doesn't have a shell as a runtime dependency, one can run something else? | 16:50 |
| ssam2 | conceptually, we don't want to allow building e.g. GLib against a sysroot, then exporting just the binaries and linking them against your host | 16:50 |
| persia | (but not having shell as runtime dependencies for things that require shell for integration commands is bad: if buildstream is directly executing the integration commands (outside shell), it is maybe less bad) | 16:50 |
| ssam2 | that way lies madness, if for example you had a different libc inside the sandbox than on your host | 16:50 |
| ssam2 | not sure this is clearly documented or even clearly thought through though; we probably need to get tristan to clearly explain the "vision" somewhere | 16:51 |
| * ssam2 wins 10 points for saying "thought through though" | 16:52 | |
| persia | And, likely, a metal-coloured star of some description | 16:52 |
| juergbi | ssam2: when you have a minute, i'd appreciate a review / sanity check of https://gitlab.com/BuildStream/buildstream/merge_requests/270 | 16:56 |
| *** toscalix has quit IRC | 16:56 | |
| *** valentind has joined #buildstream | 17:11 | |
| gitlab-br-bot | buildstream: merge request (jonathan/bzr-source-method->master: Jonathan/bzr source method) #242 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/242 | 17:18 |
| nexus | in regards to issue 166 https://gitlab.com/BuildStream/buildstream/issues/166 Did we consider the option of simply having the tracking number in quotes? as this would seem to fix the issue | 17:24 |
| jmac | Someone would probably forget at some point, and it could take a while to figure out what the problem was | 17:26 |
| juergbi | nexus: this would likely cause people to trip over the same issue again when someone forgets the quotes | 17:26 |
| juergbi | also, enforcing quotes would break format compatibility | 17:26 |
| jmac | If I ever write my syntax checker for the yaml we might be able to catch that problem and give a more helpful error, but I don't have time for that right now | 17:27 |
| ssam2 | i had a quick play with readthedocs.org and got this: https://buildstream.readthedocs.io/en/latest/ | 17:37 |
| ssam2 | it seems fine apart from https://buildstream.readthedocs.io/en/latest/main_authoring.html#main-authoring doesn't link to any of the plugin docs | 17:37 |
| gitlab-br-bot | buildstream: merge request (issue-166_yaml_removing_underscores->master: WIP: Issue 166 yaml removing underscores) #245 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/245 | 17:37 |
| ssam2 | which is because we have custom logic to autogenerate that which is in a Makefile, and I don't see a way to get readthedocs.org to call it ... | 17:37 |
| ssam2 | ah, also the sphinx-click documentation isn't generated: https://buildstream.readthedocs.io/en/latest/invoking.html#invoking | 17:39 |
| tristan | ssam2, you would want to populate readthedocs with the output of a build of the docs | 17:40 |
| ssam2 | I can't find anything in the UI that would allow me to do that | 17:40 |
| tristan | there should probably be a way, and it would be something we'd want to trigger from a gitlab job | 17:41 |
| ssam2 | https://stackoverflow.com/questions/27228115/is-it-possible-to-upload-sphinx-pre-generated-html-documents-into-readthedocs-si suggests it isn't possible | 17:41 |
| tristan | (i.e. the upload to readthedocs.io we'd want to keep that automated) | 17:41 |
| ssam2 | the upstream issue is https://github.com/rtfd/readthedocs.org/issues/1083 | 17:42 |
| tristan | ah too bad, I was rather hoping that we might be able to get around our multi-branch docs problem | 17:42 |
| ssam2 | yeah, that would have been great | 17:42 |
| ssam2 | i'll note this in issue #178 for when the next round of "just move to readthedocs.org!" comes in :-) | 17:42 |
| tristan | yeah; I have a hard time believing that it's the packaging rather than the content which "drives people away" when reading our docs | 17:44 |
| tristan | I actually think the alabaster theme is quite nice, too | 17:44 |
| ssam2 | i do wish the table of contents on the left hand side had more stuff in | 17:44 |
| tristan | for that you have to wrestle sphinx | 17:44 |
| jmac | +1 | 17:45 |
| tristan | I've wasted an afternoon or two on that | 17:45 |
| ssam2 | did it ask you a riddle ? | 17:45 |
| jmac | I'd like the TOC to be a static list of all the top-level and 2nd-level headers | 17:45 |
| ssam2 | if we all want that, i'll open another issue | 17:46 |
| tristan | mmm that's more or less what it is | 17:46 |
| ssam2 | currently it changes on each page, though | 17:46 |
| tristan | jmac, except they are page contextual; i.e. you only get the first/second level titles of the page currently shown | 17:46 |
| persia | I remember a couple "hack around toctree" items raised in the past, so I wonder if we're using toctree as effectively as we might | 17:46 |
| tristan | so it's not very helpful when considering the docs as a book | 17:46 |
| jmac | tristan: Indeed. I don't really like them being contextual. That's just me. | 17:47 |
| tristan | this is also a reason why I struggle to keep section names nicely below around 15 chars (so they dont wrap in the sidebar toc) | 17:47 |
| tristan | jmac, oh I dont really much either; just saying if you want to improve it, get ready for sumo wrestling with the beast that is sphinx :) | 17:48 |
| jmac | E.g.: https://buildstream.gitlab.io/buildstream/docker.html#docker - no link back to the table of contents or any other documentation | 17:49 |
| jmac | tristan: Indeed | 17:49 |
| tristan | could always get lucky though and find the pro tip that I was unable to figure out :) | 17:49 |
| tristan | jmac, indeed very annoying, and this silly "search" box keeps appearing, it really thinks its so useful | 17:50 |
| gitlab-br-bot | buildstream: issue #246 ("Documentation sidebar changes from page to page") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/246 | 17:55 |
| *** ssam2 has quit IRC | 17:58 | |
| gitlab-br-bot | buildstream: merge request (juerg/scheduler->master: scheduler.py: Do not prematurely terminate loop after skipping jobs) #270 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/270 | 18:04 |
| *** slaf- has joined #buildstream | 18:15 | |
| *** slaf has quit IRC | 18:16 | |
| *** slaf- is now known as slaf | 18:16 | |
| tristan | any idea what's causing these stack traces https://gitlab.com/BuildStream/buildstream/-/jobs/52234336 ? | 18:33 |
| juergbi | i suspect it's fallout from the integration test refactoring but not sure what exactly is happening. i thought it was fixed | 18:35 |
| tristan | I was thinking that yeah | 18:35 |
| tristan | that job is from your scheduler fix | 18:36 |
| tristan | so nothing there which would cause EPERM like that | 18:36 |
| juergbi | yes, it's definitely a CI issue, not an issue with that particular change | 18:36 |
| juergbi | also saw it recently on master after the identical commit was fine in the branch CI | 18:37 |
| juergbi | there was an issue where mkdir was running as the unprivileged user instead of root but tlater added a mkdir to the privileged/root part which should have fixed this | 18:37 |
| tristan | hhhhmmmmmm | 18:38 |
| tristan | juergbi, smells like assumptions around permissions and shared gitlab caches ? | 18:38 |
| juergbi | there is a chmod -R in before_script | 18:39 |
| tristan | well, I dont know what it is, but it happened again | 18:39 |
| juergbi | :/ | 18:39 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/jobs/52234336 | 18:39 |
| tristan | retry again, but there is something obviously broken | 18:39 |
| *** dominic has quit IRC | 18:40 | |
| juergbi | we could clear the runner cache in case there was a one-time issue somewhere at gitlab | 18:40 |
| juergbi | but we shouldn't have to do this over and over | 18:41 |
| tristan | juergbi, not sure; it's up to you | 18:42 |
| tristan | juergbi, on one hand, we definitely have a problem; and now we are luckily reproducing it | 18:42 |
| tristan | so it may be the best opportunity for tlater to jump in and save the day | 18:43 |
| tristan | on the other hand, we may want to land commits more than we want to fix this issue | 18:43 |
| juergbi | tlater is off this week, unfortunately | 18:43 |
| tristan | third time in a row https://gitlab.com/BuildStream/buildstream/-/jobs/52238104 | 18:43 |
| juergbi | will take a quick look, maybe i'll spot something | 18:44 |
| tristan | well, before_script does some chowns and stuff in `.` (wherever that happens to be) | 18:46 |
| tristan | while it sets the env var INTEGRATION_CACHE to a much more specific location | 18:46 |
| * tristan is unsure how the user is expected to work with INTEGRATION_CACHE, and if `./setup.py test` will at least ensure that it's integration tests dont reuse the users caches by default | 18:47 | |
| juergbi | default is temp directory but that should never be used in CI where INTEGRATION_CACHE is set | 18:49 |
| tristan | good good :) | 18:50 |
| juergbi | tristan: is there somewhere i can log into to fetch the cache? (IP 10.131.43.226) | 18:51 |
| tristan | Maaaan that is gitlab territory, my default is to refer to Sam or jjardon[m] | 18:52 |
| tristan | heh | 18:52 |
| tristan | from what I hear though, you cannot login to any runner | 18:52 |
| tristan | and the shared cache... I dont know how to download | 18:52 |
| tristan | juergbi, you might be able to spoof it by creating a branch with a modified .gitlab-ci.yml | 18:52 |
| juergbi | i thought this was CT infrastructure | 18:53 |
| tristan | which makes the shared cache an artifact of a job | 18:53 |
| juergbi | you might be right about this being a path issue | 18:53 |
| tristan | Ummm, I dont think so really, parts of it are | 18:53 |
| tristan | juergbi, I think CT has contributed runners by digitalocean, all on cloud | 18:54 |
| juergbi | based on the log it seems the PWD at the beginning is /builds/BuildStream/buildstream/ | 18:54 |
| tristan | but I may be wrong | 18:54 |
| juergbi | so creating a subdirectory cache in there doesn't make sense if INTEGRATION_CACHE is set to /builds/BuildStream/cache | 18:54 |
| juergbi | but i might be missing something, it's a bit confusing | 18:55 |
| tristan | yup, and we error on /builds/BuildStream/cache/integration-cache/artifacts | 18:55 |
| tristan | so it's plausible that gitlab up and changed the PWD | 18:56 |
| tristan | or, maybe we should be declaring more, and telling gitlab where things should go (like the PWD) | 18:57 |
| jjardon[m] | juergbi: give me a public SSH key and I will give you access to that machine with the gitlab cache | 18:57 |
| tristan | a quick fix seems like changing INTEGRATION_CACHE to point to /builds/BuildStream/buildstream/cache | 18:58 |
| juergbi | jjardon[m]: ta, sent by mail | 18:59 |
| jjardon[m] | note gitlab cache is a "best efford" thing; Its not garantee to be there | 19:00 |
| jjardon[m] | also "Note that since cache is shared between jobs, if you're using different paths for different jobs, you should also set a different cache:key otherwise cache content can be overwritten." | 19:00 |
| jjardon[m] | from https://docs.gitlab.com/ce/ci/yaml/README.html#cache | 19:00 |
| juergbi | ok, we do set a cache key since the integration test merge, which should avoid conflicts | 19:01 |
| tristan | juergbi, that separation should effectively obsolete the chown hacks in before_script it seems | 19:02 |
| tristan | well, at least the ones regarding the cache | 19:02 |
| juergbi | we still need chown because we run the tests as buildstream user while the cache zip file is extracted as root | 19:02 |
| tristan | ewww, it wont restore those ? | 19:03 |
| tristan | wasnt this discussion the other way around when we were blaming tar for restoring uids all by itself when run as root ? | 19:04 |
| tristan | my the tables have turned... | 19:04 |
| juergbi | well, gitlab uses zip... | 19:04 |
| tristan | yeah, I was answering myself, who was complaining that zip was not smart enough to do it when awarded proper perms :) | 19:05 |
| * juergbi no longer likes the UID approach of security in general for some time now | 19:06 | |
| persia | UID was useful when people shared large machines. It isn't useful in distributed environments. | 19:08 |
| juergbi | yes, that's one issue but there are others such as making it difficult to have more fine grained granularity than user | 19:14 |
| jjardon[m] | juergbi: 46.101.48.48 | 19:15 |
| persia | juergbi: How do you mean? When do you want that? | 19:15 |
| jjardon[m] | that's the public IP for 10.131.43.226 | 19:15 |
| jjardon[m] | juergbi: you should have access to root@46.101.48.48 | 19:16 |
| juergbi | ta | 19:17 |
| juergbi | persia: the usual sandboxing use cases. this is typically solved differently because UID is painful to use for this | 19:18 |
| jjardon[m] | juergbi: tristan Im doing some cleanup of some runners that seems to be stall for a long time, sorry if some build fails | 19:19 |
| juergbi | however, a single approach to split a system into users and a user into sandboxes would make more sense, imo | 19:19 |
| juergbi | jjardon[m]: i can log in but i have no idea where on that machine i would find the runner cache | 19:19 |
| juergbi | any hints? | 19:19 |
| persia | Yeah, I don't see any reason that a UID has to be associated with a person. At one of my unis, we each had several UIDs in NIS, with different sorts of access permissions, etc. Of course, it means that people have to use `su` more, and think about "UID" in terms that don't map one-to-one to human operator :) | 19:20 |
| persia | But I do see a problem with using more than one machine without overarching NIS, or passing data between machines, which is part of why so many of us all use UID 1000 to do everything. | 19:21 |
| persia | (but global federated identity is well outside the scope of buildstream) | 19:21 |
| jjardon[m] | juergbi: Its the minio server, I think in /export/runner/ | 19:22 |
| juergbi | ta | 19:24 |
| juergbi | linux-tests-_3 there is timestamped Feb 7 16:09 and is an empty cache directory | 19:24 |
| juergbi | which could indeed be the case given the configuration, however, if the cache is always essentially empty, i don't see why it sometimes works and sometimes doesn't | 19:25 |
| juergbi | maybe it actually only worked with a non-empty cache | 19:26 |
| juergbi | i'll push a branch for debugging | 19:27 |
| jjardon[m] | juergbi: tristan cleanup finish (just in time for your build juergbi) | 19:31 |
| juergbi | ta | 19:32 |
| juergbi | all jobs are currently paused, though | 19:38 |
| jjardon[m] | juergbi: I have retry and It seems to work now? | 19:47 |
| jjardon[m] | It can be slow as the runners have to be created from scratch instead be already there and reused | 19:48 |
| juergbi | ok | 19:49 |
| juergbi | yes, seems to work fine now | 19:50 |
| *** ernestask has quit IRC | 20:01 | |
| *** valentind has quit IRC | 20:05 | |
| *** valentind has joined #buildstream | 20:05 | |
| *** Prince781 has joined #buildstream | 20:07 | |
| *** Prince781 has quit IRC | 20:25 | |
| juergbi | paulsherwood: video is up now: https://video.fosdem.org/2018/K.3.201/introducing_buildstream.webm | 20:32 |
| gitlab-br-bot | buildstream: merge request (juerg/ci->master: Fix CI cache) #272 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/272 | 20:37 |
| gitlab-br-bot | buildstream: merge request (juerg/ci->master: Fix CI cache) #272 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/272 | 20:43 |
| gitlab-br-bot | buildstream: merge request (juerg/scheduler->master: scheduler.py: Do not prematurely terminate loop after skipping jobs) #270 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/270 | 20:44 |
| *** jonathanmaw has quit IRC | 21:02 | |
| *** jonathanmaw has joined #buildstream | 21:04 | |
| gitlab-br-bot | buildstream: issue #236 ("Elements not building when using "bst build --track-all --track-save"") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/236 | 21:08 |
| gitlab-br-bot | buildstream: merge request (juerg/scheduler->master: scheduler.py: Do not prematurely terminate loop after skipping jobs) #270 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/270 | 21:08 |
| *** valentind has quit IRC | 21:49 | |
| *** juergbi has quit IRC | 21:50 | |
| *** juergbi has joined #buildstream | 21:53 | |
| *** Prince781 has joined #buildstream | 21:55 | |
| *** aday has quit IRC | 22:21 | |
| gitlab-br-bot | buildstream: issue #247 ("Print summary of failed modules when there is a build failure") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/247 | 22:55 |
| gitlab-br-bot | buildstream: issue #248 ("Do not automatically update git checkout with refs") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/248 | 23:35 |
| *** tristan has quit IRC | 23:40 | |
| *** tristan has joined #buildstream | 23:42 | |
| gitlab-br-bot | buildstream: issue #249 ("Impossible to remove workspace for module that no longer exists") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/249 | 23:49 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!