| *** tristan has joined #buildstream | 06:14 | |
| *** ChanServ sets mode: +o tristan | 06:14 | |
| *** bochecha has joined #buildstream | 08:05 | |
| gitlab-br-bot | buildstream: merge request (cross_platform->master: Cross platform) #81 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/81 | 08:30 |
|---|---|---|
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 12 commits (last: widget.py: Replace tail with a python implementation) https://gitlab.com/BuildStream/buildstream/commit/be20f4113dd2c0b67e6223f83c86b9490224584a | 08:30 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch cross_platform | 08:30 |
| *** jonathanmaw has joined #buildstream | 08:52 | |
| *** ssam2 has joined #buildstream | 08:57 | |
| *** tlater has joined #buildstream | 09:03 | |
| tlater | Oh, the cross platform branch is finally merged \o/ | 09:11 |
| tristan | Yay ! | 09:12 |
| tristan | Another one bites the dust | 09:12 |
| bochecha | oh, that should help with installing bst-receive-artifacts without having bwrap installed, then, shouldn't it? | 09:19 |
| bochecha | (that was 91) | 09:20 |
| tristan | Oh that... | 09:23 |
| tristan | I dont think no... but we should be able to do something about that | 09:24 |
| tristan | Sheesh /me hates this setuptools | 09:24 |
| * tristan thought he was doing the right thing setting up this python project, to follow all the standard practices | 09:25 | |
| tristan | Sometimes standard practices are just so ... insufficient, though :-/ | 09:25 |
| tristan | bochecha, fwiw; it looks like in some near-ish future; all of our python hacks will explode in our face :) | 09:27 |
| tristan | https://www.python.org/dev/peps/pep-0552/ | 09:27 |
| tristan | of course, building older pythons will still remain a thing, so python hacks will have to stay around but eventually; it will explode and we'll have to make them conditional | 09:28 |
| bochecha | yeah, the whole python packaging space is (finally) evolving (one could say "getting fixed") | 09:29 |
| bochecha | that will definitely break things, but hopefully it should also bring a lot of sanity :) | 09:29 |
| paulsherwood | is there an understanding of the need for provenance in the 'getting fixed' approach? | 09:30 |
| paulsherwood | (ie some confidence that we know whose source we are building) | 09:30 |
| tristan | paulsherwood, I guess that would be in the domain of distutils and pip, I am not aware of signing mechanisms and such in their deployment infra (but there may be) | 09:33 |
| tristan | bochecha, btw, I'm not sure how your suggestion for setup.py works; I added a print("{}".format(sys.argv)) to setup.py and ran this: https://bpaste.net/show/494818725dba | 09:33 |
| bochecha | tristan: iirc the `install` command class has a `user_options` attribute which will contain the options passed by the user to the cli | 09:35 |
| bochecha | I guess you need a custom install command class which handles those | 09:35 |
| bochecha | that's from memory, though | 09:35 |
| tlater | bochecha: You're right on that one. It's moreso for commands than flags though | 09:35 |
| tlater | Hmmm, actually, you'd just have to override the default install command | 09:36 |
| bochecha | yes, override it with a custom class which only adds support for the desired options, and defers everything else to the defaul command | 09:43 |
| * tristan had tried something like that before | 09:44 | |
| tristan | I think I was fiddling with trying to automate the generation of man pages | 09:44 |
| bochecha | that's another painful thing with Python | 09:44 |
| bochecha | there's a patch posted on the Python bug tracker to have argparse generate man pages | 09:45 |
| bochecha | which is a pretty nice idea (at least for simple cases which can be handled automatically from the automatic argparse help) | 09:45 |
| tristan | Except, like with everything else python; the thing is the standard library is the shitty one nobody wants to use, except for the most basic of purposes | 09:45 |
| tristan | Click generates man pages from the help output, and has just a few formatting issues | 09:46 |
| tristan | s/the thing is/the thing in/ | 09:46 |
| bochecha | ah, I didn't know click did that | 09:47 |
| bochecha | I'm not a big fan of click, though... I guess I'm just used to argparse :) | 09:47 |
| tristan | its an extension yeah https://github.com/click-contrib/click-man | 09:47 |
| tristan | well click solves more problems (and *cough* would solve them better if the maintainer looked at my patches) than just command line parsing | 09:48 |
| tristan | whats really nice is the decorators create a whole data model which can be applied to generating help output, man pages, automation of bash completion, etc | 09:48 |
| tristan | and then also gives you the things you will want for cli apps (prompting the user, echoing with colors, automatically avoid ANSI escape codes when not connected to terminal) | 09:49 |
| bochecha | I think click didn't do all that last time I looked at it | 09:50 |
| bochecha | you're making me want to have another look :) | 09:50 |
| tristan | heh maybe | 09:50 |
| tristan | it has some history and was forked out of another pocoo project I think | 09:50 |
| bochecha | yeah, it came out of flask | 09:50 |
| bochecha | now that I think about it, the last time I used click was probably in June 2014 | 09:51 |
| bochecha | I probably should have a new look, lets could have happened since then :x | 09:51 |
| bochecha | s/lets/lots/ | 09:51 |
| tristan | bochecha, maybe you have a trick for this; I have this command: https://bpaste.net/show/d709b0036ea5 and that successfully picks up the arg | 10:32 |
| tristan | I can probable put all of the system assertions directly and conditionally in the install.run() as well | 10:33 |
| tristan | Only thing is; it's impossible to conditionally set the entry_points | 10:33 |
| tristan | afaics | 10:33 |
| bochecha | well, first, I'd use the other form of writing entrypoints | 10:33 |
| tristan | What is that ? | 10:33 |
| bochecha | because then it's a bit nicer and easier to append to a list rather than having strings | 10:33 |
| bochecha | this: https://framagit.org/bochecha/org.ideascube.Example/blob/master/setup.py#L46-53 | 10:34 |
| bochecha | but for your actual question... yeah, I don't know how to conditionally pass entrypoints | 10:34 |
| tristan | Ah | 10:34 |
| bochecha | it might be too late in fact, since they are supposed to be passed to the setup function :-/ | 10:35 |
| tristan | bochecha, well in that case, it's a dict, so it's mutable, maybe we'll get lucky :) | 10:35 |
| bochecha | true | 10:35 |
| bochecha | but it's possible setuptools already did what it had to do with that dict, and so modifying it won't change the outcome | 10:35 |
| bochecha | maybe in case the option was passed to only install the receiver, you can just remove the `bst` script :P | 10:36 |
| tristan | it worked \o/ | 10:40 |
| tristan | Yeah I did the opposite, append the `bst` entry point if that's not specified | 10:40 |
| bochecha | that feels like such an awful hack :) | 10:40 |
| bochecha | but hey, if it works... | 10:40 |
| tristan | I dont think we'll see better :-/ | 10:41 |
| bochecha | I hope the pyproject.toml stuff, when it takes over, will have something for deps which aren't Python modules, and conditional builds | 10:41 |
| ssam2 | fwiw i've been building a python app with Meson just fine | 10:41 |
| bochecha | yeah | 10:42 |
| ssam2 | i guess it lacks support for python dep management though | 10:42 |
| bochecha | pip already has support for different build systems (iirc it supports distutils, setuptools and flit) | 10:42 |
| bochecha | adding support for meson in pip would be pretty great, because then everything is possible :) | 10:42 |
| bochecha | you'd `pip install foo` | 10:43 |
| bochecha | pip would download and install the deps | 10:43 |
| bochecha | and then use meson to build/install foo | 10:43 |
| tristan | Eh I have little faith | 10:44 |
| bochecha | well, Python packaging has always been awful, so it can only get better ;) | 10:44 |
| tristan | heh | 10:44 |
| bochecha | tristan: to get back to a more constructive conversation, did you have any time to think about what to do with those pyc files? | 10:45 |
| bochecha | i.e should I spend time trying to handle those corner-cases Flatpak handles, or do we say "just don't do that, it's awful"? | 10:45 |
| tristan | bochecha, yeah I just didnt get to merging it, I think lets go with the dd trick you did without any additional crazy hacks | 10:45 |
| bochecha | works for me | 10:46 |
| bochecha | I'll push the fix for the integration tests then :) | 10:46 |
| bochecha | and open an MR for it | 10:46 |
| tristan | I dont want to add those at all, and if we must; we should have documentation about which specific cases those are (they do ugly things) | 10:46 |
| tristan | I mean, installing the same .py file is already nonsense; should only happen in the case of circular deps I suppose | 10:47 |
| bochecha | yeah, and py/pyc files should really come together IMHO | 11:02 |
| bochecha | installing just the py (you don't trust the bytecode optimisations?) or just the pyc (proprietary software?), I can understand | 11:03 |
| bochecha | if installing both, then they really should be coming together | 11:03 |
| *** semanticdesign has joined #buildstream | 11:04 | |
| gitlab-br-bot | push on buildstream@91-allow-only-receiver-install (by Tristan Van Berkom): 2 commits (last: setup.py: Added --artifact-receiver-only install option) https://gitlab.com/BuildStream/buildstream/commit/559f9e38cf27269a9d421c18a71d39db91d57d73 | 11:15 |
| gitlab-br-bot | buildstream: merge request (91-allow-only-receiver-install->master: Resolve "bst-artifact-receive does not require bubblewrap") #99 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/99 | 11:15 |
| tristan | bochecha, yeah I was going to do a fix for the tests and push that, an MR will be helpful | 11:24 |
| tristan | bochecha, of course CI will fail heh... but no worry I'll just merge them both and things will land rightside up | 11:24 |
| tristan | crap, racing against 1hour deadline CI now :-/ | 11:26 |
| tlater | It seems to be about 50/50 whether the unix tests manage to succeed in < 1 hour | 11:26 |
| ssam2 | i'm sure that can be altered | 11:27 |
| bochecha | tristan: https://gitlab.com/BuildStream/buildstream-tests/merge_requests/17 | 11:27 |
| tristan | yeah I think it's a limit imposed by the free gitlab runners (in which case, totally reasonable) | 11:27 |
| gitlab-br-bot | buildstream: merge request (fix-pyc-timestamps->master: Fix timestamps in Python bytecode) #98 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/98 | 11:29 |
| gitlab-br-bot | buildstream: merge request (91-allow-only-receiver-install->master: Resolve "bst-artifact-receive does not require bubblewrap") #99 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/99 | 11:30 |
| gitlab-br-bot | push on buildstream@91-allow-only-receiver-install (by Tristan Van Berkom): 2 commits (last: setup.py: Added --artifact-receiver-only install option) https://gitlab.com/BuildStream/buildstream/commit/b8b85e6639d1df06dd3480878f2c7177d5c92e42 | 11:30 |
| gitlab-br-bot | push on buildstream@fix-pyc-timestamps (by Mathieu Bridon): 16 commits (last: Fork and embed fusepy) https://gitlab.com/BuildStream/buildstream/commit/bf0f2863e8cea73e41d5d9efca3126e4075fc3ae | 11:30 |
| gitlab-br-bot | buildstream: merge request (fix-pyc-timestamps->master: Fix timestamps in Python bytecode) #98 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/98 | 11:30 |
| bochecha | tristan: ok, both MRs are in, I removed the WIP status from the first one, tests are running (and will fail obviously), should be all ready to be merged :) | 11:31 |
| tristan | yep, I'll do on command line... | 11:37 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: Fix timestamps in Python bytecode) https://gitlab.com/BuildStream/buildstream/commit/4a29f7eb2c53cd3d06038d3cbb44d1ce835de647 | 11:39 |
| gitlab-br-bot | buildstream: issue #94 ("BuildStream could be more helpful when build Python code") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/94 | 11:39 |
| gitlab-br-bot | buildstream: merge request (fix-pyc-timestamps->master: Fix timestamps in Python bytecode) #98 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/98 | 11:39 |
| bochecha | ** Couldn't install with extension module, trying without it... | 11:42 |
| bochecha | ** BuildFailed: command 'gcc' failed with exit status 1 | 11:42 |
| bochecha | I guess the tests might run a bit faster if the C extension was compiled? | 11:43 |
| bochecha | (that's from https://gitlab.com/BuildStream/buildstream/-/jobs/34488586) | 11:43 |
| tristan | Hmmm, I thought we fixed that by removing the --upgrade options when installing with pip | 11:44 |
| *** semanticdesign has quit IRC | 11:44 | |
| *** semanticdesign has joined #buildstream | 11:45 | |
| gitlab-br-bot | push on buildstream@91-allow-only-receiver-install (by Tristan Van Berkom): 3 commits (last: Fix timestamps in Python bytecode) https://gitlab.com/BuildStream/buildstream/commit/4a29f7eb2c53cd3d06038d3cbb44d1ce835de647 | 11:45 |
| gitlab-br-bot | buildstream: merge request (91-allow-only-receiver-install->master: Resolve "bst-artifact-receive does not require bubblewrap") #99 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/99 | 11:45 |
| bochecha | wow, I never realized the tests were so awfully slow in the CI :o | 11:46 |
| tristan | bochecha, the I/O is terrible | 11:47 |
| bochecha | is that because they run in Docker, where OSTree isn't as fast as it could be? (I remember Flatpak does some magic with ostree's fuse-rofiles to speed everything up, which doesn't work inside Docker) | 11:47 |
| tristan | processing is not great, but I/O is horrible | 11:47 |
| tristan | I implemented a job which clones a bunch of gits and runs a conversion of one builder format to another... maybe 6 months ago | 11:47 |
| tristan | I implemented a change which doubled the I/O | 11:48 |
| tristan | On my laptop, this turned 4 minutes into 5 minutes | 11:48 |
| tristan | On the gitlab runner, it turned 45minutes into an hour and a half :-/ | 11:48 |
| bochecha | ouch | 11:48 |
| tristan | was a bit hilarious too... people were telling me that my last commit made the task take and hour and a half | 11:51 |
| tristan | I was debugging it for a day, and then later learned that it was originally taking 45min | 11:51 |
| tristan | heh | 11:51 |
| bochecha | and I find my CI is long, at ~10 minutes :x | 11:52 |
| tristan | I think we can dedicate our own runners though | 11:52 |
| tristan | and get better performance | 11:52 |
| bochecha | probably if you stop using docker as well | 11:52 |
| bochecha | gitlab ci allows running the ci jobs either in docker images, or as simple shell scripts directly on the runner host | 11:53 |
| *** tlater has quit IRC | 11:53 | |
| tristan | yeah not sure... I feel like it might be interesting to have the .gitlab-ci.yml declaring the exact image I'm going to test on | 11:54 |
| tristan | And then even iterate through various images, ensuring today's buildstream works with last years fedora or such | 11:54 |
| *** tlater has joined #buildstream | 11:58 | |
| *** semanticdesign has quit IRC | 12:07 | |
| tristan | oh yeah | 12:07 |
| *** semanticdesign has joined #buildstream | 12:08 | |
| tristan | Once we churn out some tiny base runtimes... we should refactor the integration tests to use those | 12:08 |
| tristan | A huge bottleneck is also that some of the tests use the freedesktop flatpak base runtime | 12:08 |
| *** adds68 has quit IRC | 12:12 | |
| ssam2 | tiny base runtimes ... got to about 300MB in Baserock | 12:15 |
| ssam2 | it's hard to get a G++ toolchain smaller than that I think | 12:15 |
| tristan | yeah, which is much smaller :) | 12:16 |
| tristan | ssam2, once you have the issue of hosting sorted out, we should change our integration tests to use that instead | 12:17 |
| ssam2 | sounds good | 12:18 |
| tristan | I would be interesting if we could even share code between the "Example projects" and the integration tests... but meh, I wont hold my breath | 12:18 |
| *** adds68 has joined #buildstream | 12:18 | |
| gitlab-br-bot | buildstream: issue #91 ("bst-artifact-receive does not require bubblewrap") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/91 | 12:34 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 2 commits (last: setup.py: Added --artifact-receiver-only install option) https://gitlab.com/BuildStream/buildstream/commit/14fa10f490b59a55307378b565cb517ad96d5aab | 12:34 |
| gitlab-br-bot | buildstream: merge request (91-allow-only-receiver-install->master: Resolve "bst-artifact-receive does not require bubblewrap") #99 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/99 | 12:34 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch 91-allow-only-receiver-install | 12:34 |
| tristan | Aright it passed ! | 12:35 |
| tristan | let friday night begin ! | 12:35 |
| *** tristan has quit IRC | 12:39 | |
| gitlab-br-bot | push on buildstream@81-non-empty-read-only-directories-not-handled-during-bst-build-and-others (by Tristan Maat): 1 commit (last: Ensure that artifact file permissions are set in the right order) https://gitlab.com/BuildStream/buildstream/commit/264aa2dab0886d22655814dad2e25258c70e24a7 | 13:03 |
| *** tlater has quit IRC | 13:13 | |
| gitlab-br-bot | buildstream: Mathieu Bridon deleted branch fix-pyc-timestamps | 13:17 |
| *** tlater has joined #buildstream | 13:19 | |
| *** xjuan has joined #buildstream | 13:30 | |
| *** reto has joined #buildstream | 14:02 | |
| *** ssam2 has quit IRC | 15:55 | |
| gitlab-br-bot | push on buildstream@81-non-empty-read-only-directories-not-handled-during-bst-build-and-others (by Tristan Maat): 1 commit (last: Ensure that artifact file permissions are set in the right order) https://gitlab.com/BuildStream/buildstream/commit/97cf18248edc049c35fb03523cdcaf532617b1af | 17:18 |
| gitlab-br-bot | push on buildstream@81-non-empty-read-only-directories-not-handled-during-bst-build-and-others (by Tristan Maat): 1 commit (last: Ensure that artifact file permissions are set in the right order) https://gitlab.com/BuildStream/buildstream/commit/ac3150c1fc01ab7474827616abec66582cd6b25a | 17:20 |
| *** jonathanmaw has quit IRC | 17:20 | |
| gitlab-br-bot | push on buildstream@81-non-empty-read-only-directories-not-handled-during-bst-build-and-others (by Tristan Maat): 1 commit (last: Ensure that artifact file permissions are set in the right order) https://gitlab.com/BuildStream/buildstream/commit/b8b7b22e46ab64124a9ae744510aff646cf88e31 | 17:22 |
| gitlab-br-bot | buildstream: merge request (81-non-empty-read-only-directories-not-handled-during-bst-build-and-others->master: Ensure that artifact file permissions are set in the right order) #100 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/100 | 17:24 |
| tlater | I made !100 \o/ | 17:25 |
| gitlab-br-bot | push on buildstream@81-non-empty-read-only-directories-not-handled-during-bst-build-and-others (by Tristan Maat): 16 commits (last: widget.py: Replace tail with a python implementation) https://gitlab.com/BuildStream/buildstream/commit/be20f4113dd2c0b67e6223f83c86b9490224584a | 17:28 |
| gitlab-br-bot | buildstream: merge request (81-non-empty-read-only-directories-not-handled-during-bst-build-and-others->master: Ensure that artifact file permissions are set in the right order) #100 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/100 | 17:28 |
| *** tlater has quit IRC | 17:28 | |
| *** tristan has joined #buildstream | 20:17 | |
| *** tristan has quit IRC | 21:31 | |
| *** semanticdesign has quit IRC | 23:20 | |
| *** xjuan has quit IRC | 23:40 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!