*** jswagner has quit IRC | 03:46 | |
*** jswagner has joined #buildstream | 03:47 | |
*** jswagner has joined #buildstream | 03:50 | |
*** jswagner has joined #buildstream | 03:53 | |
*** jswagner has joined #buildstream | 03:57 | |
*** jswagner has joined #buildstream | 03:59 | |
*** jswagner has joined #buildstream | 04:00 | |
*** jswagner has joined #buildstream | 04:02 | |
*** jswagner has joined #buildstream | 04:04 | |
*** tristan has quit IRC | 05:42 | |
*** tristan has joined #buildstream | 05:52 | |
*** bochecha has joined #buildstream | 07:50 | |
juergbi | benschubert: do you know what's wrong in my setup if I get this: | 08:27 |
---|---|---|
juergbi | TypeError: C function buildstream._yaml.node_keys has wrong signature (expected PyObject *(PyObject *, int __pyx_skip_dispatch), got PyObject *(struct __pyx_obj_11buildstream_5_yaml_Node *, int __pyx_skip_dispatch)) | 08:27 |
juergbi | I have rerun pip install but I haven't recreated the venv (yet) | 08:28 |
juergbi | is there a bug in automatic rebuilding? | 08:28 |
benschubert | juergbi: argh, apparently a file depending on _yaml.pxd didn't notice _yaml.pxd got changed. Can you delete src/buildstream/_*.c* ? | 08:28 |
benschubert | yep | 08:28 |
benschubert | I can't find the cause of this | 08:28 |
juergbi | that helped, thanks | 08:29 |
juergbi | might there be an easy way to regenerate all .c files whenever any .pxd changes. wouldn't be optimal but better than rebuilding too little | 08:31 |
benschubert | agreed, I've been looking into it, the dependency is apaprently there, but the system doesn't realize it changed | 08:34 |
*** raoul has joined #buildstream | 08:46 | |
jennis | juergbi, benschubert, we found that `git clean -xfd` before the pip install also worked | 08:47 |
benschubert | ^ don;t do this if you have an IDE configuration in the directory | 08:49 |
jennis | yeah, I should have said this can be dangerous | 08:49 |
aevri | This worked for me: git clean *.c -fx | 08:50 |
*** tristan has quit IRC | 08:50 | |
juergbi | git clean -dfX is something I sometimes use in C projects | 08:51 |
juergbi | only deletes gitignored files | 08:51 |
juergbi | as long as gitignore only includes generated files, this works really well | 08:51 |
benschubert | but people with vscode/pycharm will have a local configuration for the IDE in the directory, so it will wipe it too :) | 08:52 |
juergbi | benschubert: is your vscode config covered by .gitignore? | 08:52 |
aevri | `.tox` dir is also precious :) | 08:52 |
juergbi | true, .tox is not so cheap to rebuild | 08:52 |
benschubert | juergbi: I have a global gitignore for all those kind of folders | 08:53 |
juergbi | ah, git clean probably uses that as well | 08:53 |
benschubert | sadly yep :) | 08:53 |
gitlab-br-bot | aevri opened (was WIP) MR !1391 (aevri/supersuper->master: Refactor 'super(cls, self)' -> 'super()') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1391 | 09:00 |
*** tristan has joined #buildstream | 09:03 | |
*** phildawson_ has joined #buildstream | 09:03 | |
gitlab-br-bot | BenjaminSchubert approved MR !1391 (aevri/supersuper->master: Refactor 'super(cls, self)' -> 'super()') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1391 | 09:05 |
gitlab-br-bot | marge-bot123 merged MR !1391 (aevri/supersuper->master: Refactor 'super(cls, self)' -> 'super()') on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1391 | 09:09 |
aevri | whoosh :) | 09:10 |
benschubert | Question: I have a git repository containing multiple directories with each having a project I want to build. Is there currently any way to only re-build something if the subdirectory has changed and not everytime the main git source changes? | 09:14 |
*** jonathanmaw has joined #buildstream | 09:21 | |
tpollard | jennis: just looking at the artifact subcommand MR you pointed out, I think a lot of it can probably be simplified now with the mapping of the element to it's artifact class member | 09:34 |
jennis | oh that sounds like good news | 09:36 |
jennis | We'd still need to load ArtifactElement's for specified refs though too, right? | 09:37 |
tpollard | I'd have to dive into it more, and think about AaaP in this scenario too | 09:41 |
*** lachlan has joined #buildstream | 09:41 | |
tpollard | storing metadata in the proto, and expanding on what is serialised into it at creation gives some nice scope for fleshing out Artifact referencing | 09:46 |
gitlab-br-bot | aevri opened (was WIP) MR !1382 (aevri/win32_temptext->master: _artifact: use _TempTextBuffer instead of tmp file) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1382 | 09:49 |
*** samkirkham has joined #buildstream | 09:52 | |
*** ChanServ sets mode: +o tristan | 09:54 | |
tristan | benschubert, I think I understood the question | 09:54 |
tristan | benschubert, What you mean is this is a git repository of source code, and you are referring to that git repository in an element, but only use a subset of that element... something like this correct ? | 09:55 |
tristan | benschubert, The answer is unfortunately no - but there was some discussion on build avoidance on the source and artifact level which would be a way forward to reduce this | 09:55 |
tristan | and avoid a rebuild | 09:55 |
tristan | benschubert, However that approach I'm thinking of might not work if for instance, you need to actually stage the whole tree | 09:57 |
tristan | benschubert, i.e. the idea is essentially... if everything that was staged has not changed (or matches a key), then we can reuse the output - even if the refs have technically changed - this would also avoid rebuilds on say, superficial git merge commits and the like | 09:58 |
tristan | benschubert, for the case of "building a subdir" (like we do for example when we build spidermonkey only, but need to build it from the mozilla source tree), then we'd have the source stage only the subdir we need to build - some caching magic would allow us to identify that we've already built something with exactly the same input (even if surrounding directories have changed) | 10:00 |
benschubert | tristan: yep you got it :) I only need the subtree :) | 10:01 |
tristan | Yup, that would be a great feature and would help at multiple levels :) | 10:03 |
tristan | From what Sander and I had discussed before... it would also address another point... for instance let's say that the code *has* changed, but it produces the same output | 10:04 |
tristan | Then we could alias that output to the new key, and at least avoid builds of reverse dependencies | 10:04 |
tristan | I mean basically, the same thing that we could do with SourceCache, we could also do with ArtifactCache | 10:04 |
paulsherwood | benschubert: why are all of thes projects in the same repo? | 10:20 |
* paulsherwood is concerned that the projects might overlap, so that a change in one subdir might actually require rebuild of something in another | 10:20 | |
*** tpollard has quit IRC | 10:23 | |
*** tpollard has joined #buildstream | 10:23 | |
*** lachlan has quit IRC | 10:27 | |
samkirkham | Is there a way to set the uid/gid in the integration sandbox (not the build sandbox) of an element? | 10:32 |
benschubert | paulsherwood: no idea, and I don't have control over that | 10:32 |
benschubert | but I've been guaranteed they don't overlap :D | 10:33 |
tristan | samkirkham, nice observation | 10:36 |
tristan | samkirkham, currently the uid/gid configuration is bound to the element, not specific sandbox invocations | 10:36 |
tristan | samkirkham, so I suppose it would be a bug that currently we consider those to be the same (i.e. if we wanted to run build commands as a regular user with uid/gid 1000, it would still be wrong that we are running integration commands not as root) | 10:37 |
samkirkham | Hmm, ok. I'm having problems with some integration commands I want to run. I think it's related to this. atm I can't see a reliable way to test these commands either because the same issue crops up (wrong uid in a shell for testing) | 10:39 |
samkirkham | It may be better for me to just run these commands on first boot, but I'm non sure what the canonical way to do that is. I'd rather not add *too* much complexity. | 10:40 |
tristan | samkirkham, an option would be to `bst checkout` the tree and try running bwrap on it directly ? | 10:40 |
tristan | for testing / debugging | 10:40 |
tristan | samkirkham, What kind of commands are these btw ? | 10:41 |
samkirkham | ssh-keygen so I can have an openssh server service running via systemd | 10:41 |
tristan | Know that BuildStream currently does not preserve uid/gid bits or any fs attributes beyond 0755/0644, no setuid, nothing | 10:41 |
samkirkham | Yeah, I'm expecting to have to set some attribute bits when I start encountering problems. | 10:42 |
samkirkham | Not ideal but I've been told this already. | 10:42 |
tristan | sec... | 10:43 |
tristan | https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/tristan/gnome-boot/elements/vm/desktop-vm-image-x86_64.bst#L31 | 10:43 |
tristan | Attributes set in the sandbox directly before wrapping up an image with mkfs works | 10:43 |
tristan | samkirkham, but iirc, you know that because we discussed this on #freedesktop-sdk ? | 10:44 |
tristan | samkirkham, So another thing... I mean... you technically *can*, but it's really not intended that you should be able to run ssh-keygen in the build sandbox at all | 10:44 |
tristan | samkirkham, i.e... like we first tried to build systems without having any /dev/urandom at build time, but that just doesnt work | 10:45 |
samkirkham | I do. My plan was to use a different tool for creating the FS because of some weirdness on my particular embedded platform | 10:45 |
tristan | but generating a key that is intentionally random means that your builds are intentionally non-reproducible | 10:45 |
samkirkham | Yeah maybe ssh-keygen is better run on first boot anyway. | 10:45 |
samkirkham | Although again I'm not sure what the recommended way to do that is here. | 10:47 |
tristan | Hmm, systemd may have a facility for this ? | 10:47 |
*** lachlan has joined #buildstream | 10:47 | |
tristan | one-off initialization things ? | 10:48 |
tristan | it does other stuff, like generating users - probably one can set something up to run once | 10:48 |
samkirkham | I suspect it does. Just wasn't sure if there was a bst-canon way to do it. Seems like not so I can just see what systemd has to offer =) Thanks. | 10:49 |
tristan | samkirkham, just seems logical to me... also using a .service file would allow you to set other service files to depend on that thing having already been done once | 10:49 |
tpollard | you'll probably want systemd-firstboot.service | 11:07 |
*** bochecha_ has joined #buildstream | 11:15 | |
*** bochecha has quit IRC | 11:18 | |
*** bochecha_ is now known as bochecha | 11:18 | |
*** bochecha has quit IRC | 11:42 | |
*** bochecha has joined #buildstream | 11:43 | |
*** lachlan has quit IRC | 12:07 | |
*** tristan has quit IRC | 12:12 | |
gitlab-br-bot | aevri opened (was WIP) MR !1393 (aevri/psutil_volume->master: utils._get_volume_size: use shutil.disk_usage) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1393 | 12:28 |
*** phil has joined #buildstream | 12:42 | |
*** CTtpollard has joined #buildstream | 12:43 | |
*** CTtpollard has quit IRC | 12:43 | |
*** CTtpollard has joined #buildstream | 12:43 | |
*** phildawson_ has quit IRC | 12:44 | |
*** tpollard has quit IRC | 12:44 | |
gitlab-br-bot | aevri opened (was WIP) MR !1390 (aevri/pylint_argsdiff->master: enable PyLint "arguments-differ" error, and fixup complaints) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1390 | 13:06 |
* tlater[m] can't seem to run the integration tests because of an import failure for some of the magic cython functions | 13:08 | |
tlater[m] | Did I miss a PSA about refreshing a cache or somesuch? | 13:09 |
aevri | Is it this? TypeError: C function buildstream._yaml.node_keys has wrong signature (expected PyObject *(PyObject *, int __pyx_skip_dispatch), got PyObject *(struct __pyx_obj_11buildstream_5_yaml_Node *, int __pyx_skip_dispatch)) | 13:09 |
aevri | If so `git clean *.c -fx` and re-doing the pip install should do the trick. | 13:10 |
tlater[m] | aevri: Not quite. It's `extract_depends_from_node` in `loader.py:31` | 13:10 |
tlater[m] | Trying `git clean` in case it's some pycache screwup anyway... | 13:10 |
aevri | Can you paste the error line? Maybe it's some new fun :) | 13:11 |
tlater[m] | aevri: Is pip install required now? | 13:11 |
aevri | Oh, if you haven't done that since the cython changes then sort of yes | 13:11 |
tlater[m] | I thought tox was smarter than that | 13:11 |
aevri | Oh tox should be fine afaik, you will need to `tox -r` at least once I think though, to get cython in the first place | 13:12 |
tlater[m] | Gotcha, thanks :) | 13:13 |
* tlater[m] has indeed not run the test suite in a while | 13:13 | |
*** lachlan has joined #buildstream | 13:15 | |
tlater[m] | aevri: And `tox -r` did indeed resolve the problem. I need to sit down and grog tox at some point. | 13:15 |
aevri | Ah great, yeah me too :) | 13:16 |
*** phil has quit IRC | 14:07 | |
*** phil has joined #buildstream | 14:07 | |
*** rdale has quit IRC | 14:43 | |
*** rdale has joined #buildstream | 14:44 | |
tlater[m] | So I'm trying to get freedesktop-sdk to build as part of the BuildStream examples docs - this works, for the most part, but it requires bst-external, which - because of some recent change - doesn't work with master at all anymore. | 14:54 |
tlater[m] | Instead I should probably be using bst-plugins-experimental anyway | 14:54 |
tlater[m] | The only problem with this is that freedesktop-sdk depends on specifically bst-external, and the two packages don't share a package name. | 14:54 |
tlater[m] | Why did we change the package name? Is there some way to reconcile this for when any projects requiring those dependencies will inevitably need to convert? | 14:55 |
tlater[m] | Should I start a freedesktop-sdk-buildstream-master branch? | 14:55 |
tlater[m] | Would there be any problem with simply renaming the bst-plugins-experimental package so that pip dependencies still resolve correctly? | 14:57 |
*** CTtpollard is now known as tpollard | 14:59 | |
* tlater[m] also wonders if git-tag should perhaps be merged into git as part of all these plugin changes | 15:01 | |
tlater[m] | That would resolve a lot of freedesktop-sdk's burden | 15:01 |
*** tristan has joined #buildstream | 15:04 | |
tpollard | tlater[m]: does it depend of bst-external specifically, I'm sure I built some stuff with plugins-experimental recently? | 15:07 |
tlater[m] | tpollard: https://gitlab.com/freedesktop-sdk/freedesktop-sdk/blob/master/project.conf#L284 | 15:09 |
tlater[m] | And alas, experimental won't work with that, since this makes it a non-drop-in replacement: https://gitlab.com/BuildStream/bst-plugins-experimental/blob/master/setup.py#L32 | 15:10 |
tlater[m] | If it worked for you (which it did for me), this is likely because there were no breaking changes for the particular plugins you were using | 15:10 |
tlater[m] | But now I get ```[synthetic node]: Value of 'submodules' is not of the expected type 'Mapping'``` | 15:11 |
tlater[m] | Which is resolved in a recent commit on experimental | 15:11 |
tlater[m] | But breaks every external plugin | 15:11 |
tpollard | Yep :/ | 15:14 |
gitlab-br-bot | marge-bot123 merged MR !1393 (aevri/psutil_volume->master: utils._get_volume_size: use shutil.disk_usage) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1393 | 15:14 |
benschubert | tlater[m]: we do build freedesktop as part of the CI every night, can you use the same branch? | 15:24 |
tlater[m] | benschubert: Which branch is that? | 15:26 |
benschubert | tlater[m]: freedesktop-sdk-18.08.31.1-2-g5c639b8d553016ea6c28ea160a572da2e189745e | 15:28 |
benschubert | you can see that in .overnight-tests in .gitlab-ci.yml | 15:28 |
* tlater[m] can't see freedesktop-sdk in .gitab-ci.yml | 15:28 | |
benschubert | https://gitlab.com/BuildStream/buildstream/blob/master/.gitlab-ci.yml#L226 | 15:29 |
tlater[m] | benschubert: of buildstream-plugins-experimental? | 15:29 |
benschubert | tlater[m]: buildstream | 15:29 |
benschubert | the full link is ^ :) | 15:29 |
tlater[m] | Or some other repository? | 15:30 |
tlater[m] | Ah, gotcha | 15:30 |
tlater[m] | Hrm, I suppose with any luck that will be merged once 2.0 lands, so this workaround will drop when the docs are actually in use. | 15:31 |
* tlater[m] is still not sure if the migration story is set straight, though | 15:33 | |
benschubert | probably not, but buildstream 1.4 might help a bit :) | 15:35 |
gitlab-br-bot | juergbi opened MR !1395 (juerg/cas-dead-code->master: cascache.py: Remove unused methods commit() and _commit_directory()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1395 | 15:48 |
gitlab-br-bot | BenjaminSchubert approved MR !1395 (juerg/cas-dead-code->master: cascache.py: Remove unused methods commit() and _commit_directory()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1395 | 15:50 |
*** lachlan has quit IRC | 15:56 | |
*** bochecha has quit IRC | 16:05 | |
*** lachlan has joined #buildstream | 16:07 | |
*** phil has quit IRC | 16:32 | |
*** phil has joined #buildstream | 16:32 | |
gitlab-br-bot | marge-bot123 merged MR !1395 (juerg/cas-dead-code->master: cascache.py: Remove unused methods commit() and _commit_directory()) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1395 | 16:46 |
*** tpollard has quit IRC | 16:50 | |
*** samkirkham has quit IRC | 17:00 | |
*** adds68 has quit IRC | 17:00 | |
*** ikerperez has quit IRC | 17:01 | |
*** adds68 has joined #buildstream | 17:10 | |
*** paulsherwood has quit IRC | 17:14 | |
*** benbrown has quit IRC | 17:15 | |
*** adds68 has quit IRC | 17:16 | |
*** paulsherwood has joined #buildstream | 17:17 | |
*** rdale has quit IRC | 17:18 | |
*** jennis has quit IRC | 17:20 | |
*** WSalmon has quit IRC | 17:20 | |
*** bethw has quit IRC | 17:21 | |
*** johnward has quit IRC | 17:21 | |
*** valentind has quit IRC | 17:21 | |
*** laurence has quit IRC | 17:21 | |
*** valentind has joined #buildstream | 17:23 | |
*** laurence has joined #buildstream | 17:23 | |
*** valentind has quit IRC | 17:30 | |
*** laurence has quit IRC | 17:30 | |
*** paulsherwood has quit IRC | 17:30 | |
*** phil has quit IRC | 17:32 | |
*** bochecha has joined #buildstream | 17:42 | |
*** bochecha has quit IRC | 17:43 | |
*** lachlan has quit IRC | 17:45 | |
*** lachlan has joined #buildstream | 17:53 | |
gitlab-br-bot | BenjaminSchubert opened MR !1397 (bschubert/create-node-on-set-for-mapping->master: _yaml: Create Node when setting a dictionary) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1397 | 17:55 |
*** lachlan has quit IRC | 18:00 | |
*** jonathanmaw has quit IRC | 18:04 | |
*** lachlan has joined #buildstream | 18:17 | |
*** lachlan has quit IRC | 18:33 | |
*** bochecha has joined #buildstream | 19:21 | |
*** toscalix has joined #buildstream | 19:57 | |
gitlab-br-bot | BenjaminSchubert closed MR !1397 (bschubert/create-node-on-set-for-mapping->master: _yaml: Create Node when setting a dictionary) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/1397 | 21:23 |
*** paulsherwood has joined #buildstream | 21:33 | |
*** bethw has joined #buildstream | 21:45 | |
*** jennis has joined #buildstream | 21:46 | |
*** valentind has joined #buildstream | 21:46 | |
*** WSalmon has joined #buildstream | 21:47 | |
*** laurence has joined #buildstream | 21:47 | |
*** johnward has joined #buildstream | 21:47 | |
*** toscalix has quit IRC | 22:00 | |
*** bochecha has quit IRC | 22:06 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!