| *** tristan has joined #buildstream | 03:01 | |
| *** semanticdesign has joined #buildstream | 06:20 | |
| *** semanticdesign_ has joined #buildstream | 06:20 | |
| ltu | tristan, i'm thinking of adding a page here - https://wiki.gnome.org/Projects/BuildStream | 06:40 |
|---|---|---|
| ltu | to link to a page about the release schedule | 06:41 |
| ltu | basically summarising the discussion we had last week | 06:41 |
| ltu | which can be updated over time to be a general page for any release info | 06:41 |
| ltu | since, until we get a dedicated webiste, i guess that's BuildStream's 'homepage' | 06:42 |
| ltu | let me know if there's a better place that could go :) | 06:42 |
| tristan | ltu, I think that anyway, if we want a "webpage", that wont be the place to talk about release schedule stuff :) | 06:44 |
| tristan | ltu, Ummm, so yeah I guess we could do that - I feel like this should probably be the beginnings to a guide / checklist for maintainers | 06:45 |
| tristan | but anyway, add it how you envision it | 06:45 |
| tristan | I think eventually we'll want to document steps to making a release, what mailing lists it should be announced on, how to bump the versions and branch/tag accordingly, etc | 06:46 |
| tristan | And release schedule fits into that | 06:46 |
| ltu | ok | 06:46 |
| ltu | yes those kind of additions were what i was thinking of, eventually | 06:47 |
| ltu | for now i just thought to get a page about all things release | 06:47 |
| ltu | a dedicated page rather than it being just captured in those meeting minutes | 06:47 |
| tristan | yep | 06:49 |
| tristan | hrmmm, would be nice to relax the integration tests a bit | 06:59 |
| tristan | testing for exactness of everything is not always the point; for instance compose tests care about which files were split in which way, not about exactness of binaries | 07:00 |
| tristan | build tests care about building software that works, etc | 07:00 |
| tristan | a simple change like adding --compress-debug-sections to the strip commands doesnt necessarily break these constraints | 07:01 |
| * tristan now runs the integration tests along side a build of GNOME on his overheating laptop to update the tests he broke :-S | 07:02 | |
| tristan | ltu, fwiw, looking at https://wiki.gnome.org/Projects/BuildStream/Monthly-Meeting/20171017 ... you did pretty awesome there | 07:28 |
| tristan | ltu, However I think it's too much work - what would be nice is to have a short summery of actions and resolutions from you (much less detailed); and a raw IRC log... for the IRC log we'll be getting some automation too so you can just link to it | 07:29 |
| ltu | tristan, cool, that'd be good. | 07:30 |
| tristan | ltu, it would be nice also, to re-post to the buildstream-list ML the summary and link (but again; I hope this is much less effort on your part - I dont think we need to eat that much of your time) | 07:30 |
| ltu | yeah it didn't take too long because i was able to do it straight after the meeting, but i may not always have that luxury. | 07:30 |
| tristan | yeah | 07:30 |
| tristan | I'm also used to this kind of thing being mostly volunteer driven, in all aspects I'm trying to keep maintenance burden to a minimum | 07:31 |
| ltu | i did mail it out to the list with also a link to the wiki page though | 07:31 |
| ltu | tristan, yeah, definitely, less is more i think, will keep it as tight as possible :) | 07:32 |
| *** valentind has joined #buildstream | 07:43 | |
| *** anahuelamo has quit IRC | 08:21 | |
| *** anahuelamo has joined #buildstream | 08:22 | |
| *** tiagogomes has joined #buildstream | 08:23 | |
| *** tiagogomes has quit IRC | 08:26 | |
| *** tiagogomes has joined #buildstream | 08:29 | |
| *** jonathanmaw has joined #buildstream | 08:40 | |
| jonathanmaw | tristan: I'm having a look at integrating documentation building with setup.py (iirc that was a request at one point, can't remember if it's got an issue) | 09:03 |
| jonathanmaw | AIUI I can add custom commands, but I haven't been able to dig up whether there's a default "documentation" command. googling for how to build documentation is difficult :/ | 09:04 |
| jonathanmaw | what with google just showing me the setuptools documentation in whole | 09:04 |
| *** tlater has joined #buildstream | 09:09 | |
| *** ssam2 has joined #buildstream | 09:12 | |
| tlater | jonathanmaw: I'm fairly sure there's no default documentation command for setuptools | 09:13 |
| tristan | Nah | 09:21 |
| tristan | jonathanmaw, We are using makefiles for docs | 09:21 |
| tristan | jonathanmaw, also, note that we are generating custom templates for plugins specifically, even more custom | 09:21 |
| tristan | https://gitlab.com/BuildStream/buildstream/blob/master/doc/Makefile#L33 | 09:22 |
| jonathanmaw | tristan: yep. I was wondering whether we wanted documentation building to be accessible via setup.py | 09:23 |
| tristan | jonathanmaw, I wouldnt waste any breath on that | 09:26 |
| jonathanmaw | ok, that makes things simpler for me :) | 09:28 |
| tristan | jonathanmaw, generally, I'm hoping we can one day migrate away from setuptools entirely and maybe use meson | 09:30 |
| tristan | which is another reason to not waste breath | 09:30 |
| *** valentind has quit IRC | 09:30 | |
| tristan | It turns out, it's just incredibly hard to fit the kind of thing we want to do (i.e. pretty standard stuff for pretty much any build system) into the particularly shaped and opinionated hole that is setuptools | 09:31 |
| tristan | Nice, I seem to have reproduced a nasty bug | 09:36 |
| * tristan has several 'Track' tasks which have been hanging for almost 50 minutes now | 09:37 | |
| tristan | Without failing, without retrying, without ever timing out, just hang indefinitely | 09:37 |
| tristan | Great thing for autobuilders too | 09:37 |
| tristan | Makes sure that one task holds the build lock indefinitely until you log in and kill all the waiting cron jobs locking on the lock file | 09:37 |
| tristan | If of course, you still have enough free pid's to spawn 'kill' | 09:38 |
| ssam2 | tristan, I guess we just https://github.com/mesonbuild/meson/issues/2377 to be fixed :-) | 09:50 |
| ssam2 | *just need | 09:50 |
| tristan | ssam2, ummm, yeah and then a bunch of work needs to be done on our side after :) | 09:58 |
| ssam2 | fun work though. it's always nice removing crappy build systems :-) | 09:58 |
| tristan | ssam2, hey by the way; where *is* the cross-bootstrap stuff, is there a buildstream project I can point to somewhere ? | 09:58 |
| ssam2 | best thing to link to is probably http://wiki.baserock.org/guides/how-to-cross-bootstrap/ | 09:59 |
| ssam2 | it uses baserock definitions repo... but depends on some changes that are outstanding in GitLab still :-( | 09:59 |
| tristan | So basically, there's no way somebody can just "look at it" | 09:59 |
| tristan | ? | 09:59 |
| tristan | That really sucks yeah, to me it's like just ... seems lost | 09:59 |
| ssam2 | i mean, if we ever get Baserock CI working again, the branches can be merged | 10:00 |
| tristan | Myeah | 10:00 |
| ssam2 | and then you look at the baserock definitions repo, it's like a first class citizen there | 10:00 |
| tristan | Still | 10:00 |
| ssam2 | were you expecting me to start and maintain a whole separate project for that ? | 10:00 |
| tristan | I was expecting that a sample might exist *somewhere* | 10:00 |
| tristan | Not necessarily maintained | 10:01 |
| tristan | ssam2, adds68 is doing the real work of superseding that, and I wonder if he really knows how to extract that work from baserock in some contorted way | 10:01 |
| tlater | tristan: On the ^C regression, I don't think we call subprocess.poll() on any of the processes we call os.waitpid() on | 10:01 |
| ssam2 | tristan, he'll need some pointers i'm sure, but i don't see that as an issue | 10:02 |
| tristan | ssam2, although I think still there is no hurry has he has to do the flatpak json bits for org.freedesktop.Sdk first... | 10:02 |
| tristan | ssam2, but then, there is also valentind who is trying to figure out bootstrapping a runtime on his own, and popping up and asking me questions on IRC over the weekend | 10:02 |
| ssam2 | ah ok | 10:03 |
| tristan | he stepped out a while ago but I told him to come around on monday so I could point to the place where the real up-to-date bootstrap of a runtime exists | 10:03 |
| ssam2 | ah right. well it is a shame that baserock work all got cancelled before we finished it all, it would have been good to just point to master | 10:04 |
| tristan | jinja2 is not bailing out on undefined variables, just evaluating to false | 10:04 |
| * tristan is trying to fix jinja2 handling right now | 10:04 | |
| tristan | fwiw, the original test case with all that undefined variable exception handling, also does not properly fail for 'plover == pony' | 10:05 |
| ssam2 | oh :-( | 10:05 |
| ssam2 | i just ripped that code from Ansible, I figured it'd have been tested well enough there | 10:05 |
| tristan | ssam2, it might work only for the 'is defined' stuff | 10:06 |
| ssam2 | yeah, i thought that all it did was catch undefined variable errors, if it found 'is defined' in the expression | 10:06 |
| ssam2 | which suggests that there normally *should* be undefined variable errors | 10:06 |
| tristan | tlater, I cant say exactly why; run `bst build --track apps.bst` a bunch of times for https://gnome7.codethink.co.uk/gnome-modulesets.git | 10:07 |
| tristan | tlater, you will see either A.) Status area bits going up in the rolling log (cause of the offset of the subprocess messages) | 10:07 |
| tristan | Or B.) Enough of those subprocess messages to also show up passed the buggy status area | 10:08 |
| ssam2 | but.. on testing, seems that it does just evaluate to false ! | 10:08 |
| * tristan thinks there should be a way to detect this and is searching for it | 10:08 | |
| tristan | in jinja2 docs currently | 10:08 |
| tristan | But - the *real* problem is rust :) | 10:09 |
| tristan | After all the noise I have to sort though, that's the bottom line | 10:09 |
| tristan | beat up rust until it behaves properly | 10:09 |
| ssam2 | building components written in rust, you mean ? | 10:10 |
| tristan | yeah | 10:10 |
| tristan | https://gitlab.com/BuildStream/buildstream/issues/123 | 10:10 |
| ssam2 | that's gonna be a pain, yeah... | 10:10 |
| ssam2 | at least we're not the first | 10:10 |
| tristan | I have a temporary workaround in mind | 10:11 |
| tristan | I've started out by installing cargo-vendor on gnome7 | 10:11 |
| tlater | tristan: That issue may be related to what I raised a week or so ago - I think it's the tracking that's causing that issue. | 10:11 |
| tristan | And will cook up a little task which "vendors" the dependencies into a directory, committing it to ostree | 10:12 |
| tristan | ssam2, and then have rust-depending elements in GNOME depend on an element that imports *that*, setting something like CARGO_HOME to ensure it doesnt go all whacky and looking up things on the internet | 10:12 |
| tristan | But the real solution is gonna be much more painful, yes | 10:13 |
| tristan | tlater, soooo.... can you please take care of it ? | 10:13 |
| tlater | tristan: I'll try, though I might need a bit of help with it. I couldn't figure it out when I raised it originally. | 10:14 |
| tristan | this certainly was not happening before, it looks exactly like os.waitpid() being called on a reaped process | 10:14 |
| tristan | tlater, if it "happened" then you filed a bug and have some log about it | 10:14 |
| tlater | tristan: This one https://gitlab.com/BuildStream/buildstream/issues/107 | 10:15 |
| * tristan closes duplicate | 10:18 | |
| ssam2 | tristan, seems like `jinja2.Environment(undefined=jinja2.StrictUndefined)` to construct the environment is what we want | 10:19 |
| tristan | ssam2, AWESOME | 10:21 |
| tristan | Thanks a lot | 10:21 |
| tristan | ssam2, I have a test case already, and now with that; it passes \o/ | 10:21 |
| ssam2 | :-) | 10:21 |
| tristan | whoa, freaky | 10:23 |
| tristan | tlater, weird: https://gitlab.com/BuildStream/buildstream/issues/107#note_44249263 | 10:28 |
| tristan | So, those tracking tasks which took an hour, did not block the scheduler from returning | 10:29 |
| tristan | they were just artificially there in the status area, ticking away and waiting for the child tasks to return (when they probably did a long time ago, probably around the same time those messages reach the console) | 10:29 |
| tlater | tristan: That's interesting, they did cause the scheduler to block for me iirc. | 10:31 |
| tristan | Note, those tracks were unsuccessful | 10:37 |
| tlater | Yes, this seems to only happen on unsuccessful tracks | 10:39 |
| tristan | tlater, I *very highly* doubt that statement | 10:39 |
| tristan | I have been building a ton this weekend, usually after a fresh conversion and new track | 10:40 |
| tristan | A.) It *always* prints these messages | 10:40 |
| tristan | B.) This seems to be the first time that anything failed to be tracked, maybe my network dropped off for a moment or smth I dont know | 10:40 |
| tlater | tristan: Somehow the job_done function in queue.py seems to be called twice | 10:58 |
| tlater | tristan: That causes it to raise an exception (because the job has already been removed from the active_jobs list) and not to remove the job from the frontend | 11:00 |
| tlater | I'm still not sure *why* all the job exit tasks seem to be run twice though | 11:00 |
| tristan | sounds like you're hot on the trail | 11:00 |
| *** ChanServ sets mode: +o ironfoot | 11:14 | |
| tlater | tristan: If you set PYTHONASYNCIODEBUG=1 you can get more detailed messages :) | 11:22 |
| tlater | tristan: Can you confirm that this change is what's causing it? https://gitlab.com/BuildStream/buildstream/commit/84c871b8a60f7826392c35a36735ac52df29ef74 | 11:42 |
| * tlater has no idea why it would, perhaps attaching event loops doesn't work quite as you'd expect it to? | 11:43 | |
| jonathanmaw | tristan: Do you think the bst-external documentation should be on a separate site (and linked to), or should I try to merge the documentation into buildstream's documentation when generating it? | 12:12 |
| tristan | tlater, YES | 12:22 |
| tristan | tlater, I believe so, only tried once, reverted it, stopped happening, unrevert... happens again | 12:23 |
| tlater | \o/ Now just to figure out the *why* :) | 12:26 |
| *** tristan has quit IRC | 12:29 | |
| *** cs_shadow_ has joined #buildstream | 12:32 | |
| *** tlater has quit IRC | 13:27 | |
| *** tlater has joined #buildstream | 13:30 | |
| tlater | Hm, I'm frankly not sure what happens here: https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/_scheduler/job.py#L46 | 13:48 |
| tlater | The commit I linked earlier is definitely causing the process issues, but I can't really debug it because I don't understand how the jobs are supposed to interact with it. | 13:51 |
| tlater | It slightly weirds me out that the watcher isn't called as a contextmanager, since the documentation says it's supposed to, but then we don't use the normal Process class. | 13:53 |
| * tlater is a little lost here | 13:53 | |
| *** tristan has joined #buildstream | 15:32 | |
| tristan | ssam2, nice comment on pip issue, I was thinking the same thing :) | 15:37 |
| tristan | in fact, once you generate .bst files from a package, it then becomes easy to `bst track` until upstream python module grows new deps | 15:38 |
| ssam2 | yeah | 15:38 |
| ssam2 | finding the upstream repo can be a challenge, the metadata doesn't always specify that | 15:39 |
| * tristan was thinking of raising something on the ML today as this is going to be a bit of a "hot topic", for pip, npm, cargo, other similar "distro-ish" indexers | 15:39 | |
| ssam2 | yeah | 15:39 |
| ssam2 | all of them create headaches | 15:39 |
| tristan | yes | 15:39 |
| ssam2 | some rubygem developers consider rubygems.org to be the official source code location for their project, and do weird things in their git repo (or hide it completely) | 15:40 |
| ssam2 | although buildstream might be able to just consume the gem source tarballs, so that might not be as bad as it was in Baserock where we were intent on getting actual git repos | 15:40 |
| ssam2 | and npm... wat | 15:40 |
| tristan | And I feel its really not ideal to be using them in our context - alot of the time (I think this is untrue for rust), you only publish to an indexer like PyPI when you release | 15:40 |
| tristan | So, what happens when you have a full custom linux firmware build, and you want to patch one of these modules you are fetching from some third party indexer/distro-ish thing ? | 15:41 |
| ssam2 | yeah | 15:41 |
| tristan | another good argument to at least not recommend this practice | 15:41 |
| ssam2 | to be fair, finding the repo is often less work than doing the patch, so it's not such a bad case | 15:41 |
| tristan | mmmyeah | 15:42 |
| tristan | but think about rpms for an example | 15:42 |
| ssam2 | source rpms or binary rpms ? | 15:42 |
| tristan | people use patches against release tarballs, they could find the upstream, but they dont | 15:43 |
| ssam2 | right, yeah | 15:43 |
| tristan | cause it's just not obvious, it's an extra step in their workflow | 15:43 |
| ssam2 | i agree with you in principle, but having tackled the issue of finding upstreams for a long time as part of the baserock-import project, i got the impression that it's a tedious and not very fruitful battle trying to connect every python or ruby projects with a GIt repo | 15:44 |
| ssam2 | npm is actually great here because the project *is* the repo | 15:44 |
| tristan | cargo also I *think* | 15:44 |
| ssam2 | right, I think that didn't even exist back then :-) | 15:44 |
| tristan | Seems it will be easier to write such a script for cargo/npm | 15:44 |
| tristan | yeah, rust is eating my brain :-/ | 15:45 |
| tristan | but I think a sane solution will be possible | 15:45 |
| ssam2 | i think i've asked this before but -- does the "event plugins" feature need discussing on the ML? or should I just propose it in a bug report ? | 15:49 |
| *** adds68 has quit IRC | 16:02 | |
| *** adds68 has joined #buildstream | 16:08 | |
| *** jude has quit IRC | 16:14 | |
| tristan | ssam2, we can talk about it on the ML I think it would be good | 16:32 |
| tristan | ssam2, to be honest, it's because I have no idea what it's going to look like | 16:32 |
| tristan | I feel like some kind of 'hooks' API might be nice | 16:32 |
| tristan | i.e. run a script, doesnt even have to be python | 16:32 |
| tristan | exec this thing when such and such happens ? | 16:33 |
| ssam2 | right; that might be nicer than adding a new class of plugins | 16:33 |
| tristan | I dont know... lets see what you design | 16:33 |
| ssam2 | or it could expose a TCP socket or something and use web hooks | 16:33 |
| tristan | :) | 16:33 |
| ssam2 | i say "use web hooks" without really knowing what that means :-) | 16:33 |
| tristan | mmyyeaaah but buildstream is not a web app, it's a shell tool | 16:33 |
| ssam2 | yeah, just something in the project.conf to say 'exec this when that happens' might be better | 16:34 |
| tristan | it can easily invoke something that posts something | 16:34 |
| ssam2 | i was just thinking about ... exposing a message bus that people can monitor | 16:34 |
| tristan | not sure if it's project related | 16:34 |
| tristan | rather invocation context related no ? | 16:34 |
| ssam2 | vs. hooking up commands to internal hooks | 16:34 |
| ssam2 | oh yeah, project isn't really the place | 16:35 |
| tristan | I mean, just because running builds on this server triggers things, does it mean that users running builds also do that ? | 16:35 |
| ssam2 | you're right, user config makes more sense | 16:35 |
| tristan | anyway... take your time :) | 16:35 |
| tristan | dont rush it:) | 16:35 |
| tlater | Is the new pre- and post- command support documented yet? | 16:50 |
| tlater | Ah, nvm, it's in the buildelement documentation. | 16:51 |
| tlater | It looks like it isn't included in the sphinx documentation though. | 16:53 |
| tlater | Is that intentional? | 16:53 |
| tristan | tlater, pre-/post- is not yet removed, unfortunately I did not pre-remove it from the docs, I should have | 16:54 |
| tristan | they are *all* going away though, any pre-/post- command is not 1.0 material and will be removed | 16:54 |
| * tristan not sure, did I remove it ? I think my branch which removes the feature still contains some docs removals | 16:55 | |
| tlater | tristan: Did the (<) notation not makeit in the end? | 16:56 |
| tristan | tlater, that is there and documented | 16:56 |
| tristan | it replaces pre-/post- commands | 16:56 |
| tlater | tristan: If that's supposed to be on the documentation website under buildelement, it's not showing up atm | 16:57 |
| tristan | pre-/post- commands are not yet removed, considering that some things might need to be migrated to use the proper method first | 16:57 |
| tlater | But I guess the branch has not been merged yet? | 16:57 |
| tristan | tlater, look at my MR | 16:57 |
| tristan | I'm not merging it, cause waiting on baserock | 16:57 |
| tristan | jhbuild migrations have changed to no longer use pre-/post- | 16:58 |
| tlater | Ah, sorry, I didn't see the link on the issue | 16:58 |
| *** ssam2 has quit IRC | 16:58 | |
| *** xjuan has joined #buildstream | 17:10 | |
| *** tiagogomes has quit IRC | 17:12 | |
| *** jonathanmaw has quit IRC | 17:26 | |
| *** tlater has quit IRC | 17:28 | |
| *** valentind has joined #buildstream | 18:57 | |
| *** jude has joined #buildstream | 19:23 | |
| *** jude has quit IRC | 20:08 | |
| *** tristan has quit IRC | 21:16 | |
| *** xjuan has quit IRC | 21:58 | |
| *** valentind has quit IRC | 22:08 | |
| *** julien has joined #buildstream | 23:10 | |
| *** julien has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!