| juergbi | ironfoot: thanks for the MR, good catch | 07:26 |
|---|---|---|
| *** tlater has joined #buildstream | 07:47 | |
| *** tiagogomes has joined #buildstream | 08:25 | |
| *** tiagogomes has quit IRC | 08:29 | |
| *** jonathanmaw has joined #buildstream | 08:39 | |
| ironfoot | juergbi: i wondered if storing "project_variant" makes sense, instead of only "True" | 08:45 |
| ironfoot | i don't know if it's possible that an element appears twice in the pipeline with different variants | 08:46 |
| *** ssam2 has joined #buildstream | 08:48 | |
| *** tiagogomes has joined #buildstream | 08:54 | |
| juergbi | ironfoot: it will stop at the first element with a conflicting project variant, so i don't think it's necessary to store the project variant in the dict | 08:58 |
| juergbi | a single element must only be used in one variant in a pipeline. the earlier element variant resolution should cover this | 08:59 |
| juergbi | i'd say it's fine to merge as is but as tristan has already commented, i'll let him handle this, probably tomorrow | 09:01 |
| ironfoot | that's fine : ) | 09:07 |
| jonathanmaw | trying to figure out why the cache key changes before vs. after build. It looks like it wasn't just the contents of get_unique_key(). | 09:55 |
| jonathanmaw | I think it might be self.__public differing, but it's hard to read with all the provenance data | 09:55 |
| jonathanmaw | I'll strip out provenance and see what's different then. | 09:55 |
| juergbi | jonathanmaw: isn't it just self.commands as we discussed last week? | 09:56 |
| juergbi | or are you looking into a different issue? | 09:56 |
| jonathanmaw | juergbi: apparently not. I've changed it so that there's a self.__original_commands used in get_unique_key instead, which doesn't change during build | 09:57 |
| jonathanmaw | before and after build, they have the same contents now | 09:57 |
| jonathanmaw | cache key is still differing, at a guess I'm suspecting self.__public, so I'm having a look at that | 09:57 |
| juergbi | hm, i thought i diff'd the dict used to generate the key and only saw a difference in commands | 09:58 |
| jonathanmaw | juergbi: the only difference I see now is public data, possibly provenance. over 2000 lines of stuff before the build, and about 60 lines after the build. | 10:10 |
| jonathanmaw | aha, here's the actual differences: https://pastebin.com/HSpk6qRD | 10:14 |
| jonathanmaw | looks like self.__public is being changed, even when we're trying to write to self.__dynamic_public for that part. | 10:14 |
| *** locallycompact has joined #buildstream | 10:15 | |
| jonathanmaw | aha! the problem was https://gitlab.com/BuildStream/buildstream/blob/master/buildstream/element.py#L1016 | 10:27 |
| jonathanmaw | that doesn't copy the dict, so writing one is writing the other. | 10:27 |
| jonathanmaw | "self.__dynamic_public = dict(self.__public)" does the job, but I don't know if provenance also needs changing | 10:28 |
| jonathanmaw | I can look in _yaml to see if there's an appropriate way to copy dicts | 10:28 |
| *** jonathanmaw has quit IRC | 10:30 | |
| *** jonathanmaw has joined #buildstream | 10:33 | |
| *** jonathanmaw has quit IRC | 10:44 | |
| *** jonathanmaw has joined #buildstream | 10:45 | |
| *** ssam2 has quit IRC | 10:48 | |
| juergbi | ah, the base dict is not copied, even though domain data dicts are copied | 10:56 |
| gitlab-br-bot | buildstream: merge request (jonathan/dpkg-build-key-fix->master: Fix cache key changes in dpkg-build elements) #64 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/64 | 12:41 |
| *** tlater has quit IRC | 13:09 | |
| *** tlater has joined #buildstream | 14:25 | |
| tlater | Hm. Using a tarfile as the ostree repo makes extraction incredibly slow | 14:31 |
| juergbi | tlater: right, the extract directories are currently kept around, though, so it might not be too bad | 14:38 |
| juergbi | (with OSTree, these are just hardlinks, not occupying additional blocks) | 14:38 |
| tlater | Ah, yeah, that won't work with a tarfile | 14:38 |
| juergbi | it will take additional disk space but it would still work, right? | 14:39 |
| juergbi | assuming the tar files are compressed, it should at least not require double the space | 14:39 |
| tlater | A hardlink into a tarfile? | 14:39 |
| juergbi | no, caching extract directories | 14:39 |
| tlater | Oh, yeah, that works fine | 14:39 |
| tlater | It just a very long time, which is especially annoying for base images. | 14:40 |
| tlater | I've been waiting for ~30 mins now... | 14:40 |
| juergbi | oh :/ | 14:42 |
| tlater | It's a lot quicker when I use native tar | 14:43 |
| tlater | Hmmm | 14:43 |
| jonathanmaw | hrm, https://gitlab.com/jonathanmaw/buildstream-tests/-/jobs/24102881 is failing | 14:48 |
| jonathanmaw | the actual error seems to be "Error opening file /builds/jonathanmaw/buildstream-tests/buildstream-tests/dpkg-build-test/keys/gnome-sdk.gpg: No such file or directory" | 14:48 |
| jonathanmaw | hrm, it's not visible in the commit, either... | 14:49 |
| jonathanmaw | mystery solved. | 14:50 |
| tlater | Whoo, finally found an implementation that does it in 2 minutes rather than 2 hours :) | 15:46 |
| tlater | Oh, nevermind, this one doesn't extract anything | 15:47 |
| jonathanmaw | lol | 15:48 |
| tlater | I think I'll have to use a native shell instead of python methods :/ Let's hope AIX and Solaris have reasonable implementations. | 15:49 |
| *** ssam2 has joined #buildstream | 16:06 | |
| *** ssam2 has quit IRC | 16:18 | |
| *** ssam2 has joined #buildstream | 16:35 | |
| *** jonathanmaw has quit IRC | 16:47 | |
| *** tlater has quit IRC | 16:51 | |
| *** locallycompact has quit IRC | 17:07 | |
| *** locallycompact has joined #buildstream | 17:07 | |
| *** locallycompact has quit IRC | 17:12 | |
| *** jjardon[m] has quit IRC | 18:03 | |
| *** waltervargas[m] has quit IRC | 18:03 | |
| *** mattiasb has quit IRC | 18:03 | |
| *** waltervargas[m] has joined #buildstream | 19:31 | |
| *** ssam2 has quit IRC | 20:04 | |
| *** mattiasb has joined #buildstream | 20:22 | |
| *** jjardon[m] has joined #buildstream | 20:58 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!