IRC logs for #buildstream for Wednesday, 2020-01-15

« Tuesday, 2020-01-14 Index Thursday, 2020-01-16 »
gitlab-br-botcs-shadow opened MR !1796 (chandan/py38-cov->master: Bump coverage version, fix coverage on Python 3.8) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179600:12
gitlab-br-botcs-shadow closed MR !1796 (chandan/py38-cov->master: Bump coverage version, fix coverage on Python 3.8) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179600:19
*** phildawson has quit IRC00:39
*** xjuan has quit IRC01:55
*** brlogger` has joined #buildstream08:27
*** cs-shadow_ has joined #buildstream08:27
*** robjh has quit IRC08:27
*** laurence has quit IRC08:27
*** milloni has quit IRC08:27
*** cs-shadow has quit IRC08:27
*** ironfoot has quit IRC08:27
*** pro[m] has quit IRC08:27
*** connorshea[m] has quit IRC08:27
*** jjardon[m] has quit IRC08:27
*** ssssam[m] has quit IRC08:27
*** asingh_[m] has quit IRC08:27
*** tlater[m] has quit IRC08:27
*** persia has quit IRC08:27
*** benbrown has quit IRC08:27
*** jjardon has quit IRC08:27
*** lchlan has quit IRC08:27
*** brlogger has quit IRC08:27
*** cs-shadow_ is now known as cs-shadow08:27
*** lchlan has joined #buildstream08:28
*** jjardon has joined #buildstream08:29
*** ChanServ sets mode: +o jjardon08:29
*** benbrown has joined #buildstream08:29
*** ironfoot has joined #buildstream08:29
*** ChanServ sets mode: +o ironfoot08:29
*** persia has joined #buildstream08:30
*** milloni has joined #buildstream08:30
*** robjh has joined #buildstream08:33
*** laurence has joined #buildstream08:34
*** laurence has quit IRC09:04
*** persia has quit IRC09:04
*** ironfoot has quit IRC09:04
*** lchlan has quit IRC09:04
*** cs-shadow has quit IRC09:04
*** slaf has quit IRC09:04
*** tristan has quit IRC09:04
*** delli3_ has quit IRC09:04
*** tpollard has quit IRC09:04
*** douglaswinship has quit IRC09:04
*** mohan43u has quit IRC09:04
*** juergbi has quit IRC09:04
*** benschubert has quit IRC09:04
*** jswagner has quit IRC09:04
*** coldtom has quit IRC09:04
*** swick has quit IRC09:04
*** skullone[m] has quit IRC09:04
*** reuben640[m] has quit IRC09:04
*** rafaelff[m] has quit IRC09:04
*** m_22[m] has quit IRC09:04
*** krichter[m] has quit IRC09:04
*** dineshdb[m] has quit IRC09:04
*** Demos[m] has quit IRC09:04
*** cgmcintyre[m] has quit IRC09:04
*** awacheux[m] has quit IRC09:04
*** waltervargas[m] has quit IRC09:04
*** theawless[m] has quit IRC09:04
*** tchaik[m] has quit IRC09:04
*** mrmcq2u[m] has quit IRC09:04
*** albfan[m] has quit IRC09:04
*** gimpnet-irc[m] has quit IRC09:04
*** segfault3[m] has quit IRC09:04
*** mattiasb has quit IRC09:04
*** abderrahim[m] has quit IRC09:04
*** verdre[m] has quit IRC09:04
*** milloni has quit IRC09:04
*** paulsherwood has quit IRC09:04
*** ikerperez has quit IRC09:04
*** bethw has quit IRC09:04
*** jward has quit IRC09:04
*** Trevinho has quit IRC09:04
*** lantw44 has quit IRC09:04
*** SotK has quit IRC09:04
*** hergertme has quit IRC09:04
*** milloni has joined #buildstream09:04
*** paulsherwood has joined #buildstream09:04
*** ikerperez has joined #buildstream09:04
*** bethw has joined #buildstream09:04
*** jward has joined #buildstream09:04
*** Trevinho has joined #buildstream09:04
*** lantw44 has joined #buildstream09:04
*** SotK has joined #buildstream09:04
*** hergertme has joined #buildstream09:04
*** laurence has joined #buildstream09:04
*** persia has joined #buildstream09:04
*** ironfoot has joined #buildstream09:04
*** lchlan has joined #buildstream09:04
*** cs-shadow has joined #buildstream09:04
*** slaf has joined #buildstream09:04
*** tristan has joined #buildstream09:04
*** delli3_ has joined #buildstream09:04
*** tpollard has joined #buildstream09:04
*** douglaswinship has joined #buildstream09:04
*** mohan43u has joined #buildstream09:04
*** juergbi has joined #buildstream09:04
*** benschubert has joined #buildstream09:04
*** jswagner has joined #buildstream09:04
*** coldtom has joined #buildstream09:04
*** swick has joined #buildstream09:04
*** skullone[m] has joined #buildstream09:04
*** waltervargas[m] has joined #buildstream09:04
*** verdre[m] has joined #buildstream09:04
*** theawless[m] has joined #buildstream09:04
*** tchaik[m] has joined #buildstream09:04
*** segfault3[m] has joined #buildstream09:04
*** reuben640[m] has joined #buildstream09:04
*** rafaelff[m] has joined #buildstream09:04
*** mrmcq2u[m] has joined #buildstream09:04
*** mattiasb has joined #buildstream09:04
*** m_22[m] has joined #buildstream09:04
*** krichter[m] has joined #buildstream09:04
*** gimpnet-irc[m] has joined #buildstream09:04
*** dineshdb[m] has joined #buildstream09:04
*** Demos[m] has joined #buildstream09:04
*** cgmcintyre[m] has joined #buildstream09:04
*** awacheux[m] has joined #buildstream09:04
*** albfan[m] has joined #buildstream09:04
*** abderrahim[m] has joined #buildstream09:04
*** irc.acc.umu.se sets mode: +o ironfoot09:04
*** traveltissues has joined #buildstream09:13
*** bochecha has joined #buildstream09:16
*** ssssam[m] has joined #buildstream09:24
*** connorshea[m] has joined #buildstream09:26
benschubertjuergbi: it doesn't hide an underlying bug :) the reason why this happens now is that the validation and recomputation is done in the fetch subprocess to give more breathing time to the scheduler :) Hence, the parent doesn't know that it has changed nor that it needs to invalidate it. We can however assume that the child returning success means we are now cached09:36
juergbibenschubert: right, I understand that. however, with the fixed is_cached setting, I would expect the asserts to pass09:37
juergbi(it's fine to remove them, though, as they are pretty much pointless now)09:37
juergbiassuming fetch_done is still called before the asserts09:37
benschubertjuergbi: the assert would actually not recompute the value09:38
benschubertah right09:38
benschubertthe fetch done at the breaking commit doesn't recompute, that's the reason for the breakage :)09:38
benschubertwhich is fine, because we know it09:38
benschubertso I'll go and merge both commits and ensure the rest passes09:39
*** tme5 has joined #buildstream09:47
*** tiagogomes has joined #buildstream10:01
*** jjardon[m] has joined #buildstream10:12
*** pro[m] has joined #buildstream10:24
*** lachlan has joined #buildstream10:34
benschubertjuergbi: around the breaking change there. What do you think is best:10:37
benschubert- Change the tested branch on BuildStream CI, merge in BuildStream, update bst-plugins-experimental, revert to a tagged release in BuildStream10:37
benschubert- Mark master on bst-plugins-experimental as incompatible with current release. Merge, tag, then update BuildStream tag, and then revert the tests on the bst-plugins-experimetnal repo?10:37
juergbibenschubert: would an alternative be to first make it compatible with both? I think you already did this for the core plugins, so I would expect it to work for the external plugins as well10:38
juergbi(implement both methods)10:38
juergbimight even be simpler overall given the CI issue10:38
benschubertAnd do a non-breaking change on bst-plugins-experimental?10:38
benschubertyep I can do that10:39
juergbiyes, first make experimental compatible and then change buildstream over10:39
benschubertSure I'll do that10:39
*** lachlan has quit IRC10:40
benschubertAlso, Also, I can't get 'easily' all the commits to build. An alternative would be to move more of them together (like the last 5 ones of the branch) but that might make a too big change in one go. What would you prefer?10:43
benschubertOtherwise I'd need temporary fixes all around10:43
benschubertI'm happy to jusst all merge them together10:43
juergbibenschubert: depends a bit on how much / why it is broken10:44
benschuberthttps://gitlab.com/BuildStream/buildstream/-/jobs/403622745 for https://gitlab.com/BuildStream/buildstream/commit/7fa1b4cf59ca20269027b38c134565ee1484dc7c10:44
benschubert9 tests10:44
benschubertthat have ramifications that get fixed later on10:45
benschubertbut I would need to add an intermediary step10:45
juergbimostly just the git validation stuff, so doesn't look too bad10:47
tpollardcs-shadow: got excited when I saw the coverage MR for 3.8 heh10:47
juergbibenschubert: I would expect the submodule check to be easily fixable by moving the `not` change to that commit but it's not super important, I suppose10:47
benschubertfair enough, I'll try to fix at least those :)10:48
juergbibenschubert: fix up what can easily be done and then leave it. let's not waste too much time on it10:49
juergbiI'd be more worried about core functionality that is temporarily broken10:49
benschubertfair point10:53
*** lachlan has joined #buildstream10:53
juergbitme5, coldtom: wondering whether it would make sense to merge git cleanups and (1) as a first step10:55
tme5yes, that seems sensible10:56
juergbiand to me (2) actually seems less problematic than (3)10:56
tme5also agreed10:56
coldtom(3) was always a not-ideal solution to the problem at hand10:56
tme5i'm writing to the mailing list rn10:56
juergbitme5: also, the reason why we created _GitSourceBase was such that it would be relatively easy to create a git plugin with custom tracking10:57
juergbimaybe just migrate git_tag on top of _GitSourceBase for now?10:57
coldtomjuergbi: it'd have to be public API first :P10:57
benschubertYeah, we should have made this public long ago :(10:57
juergbiwell, the story about that is...10:57
benschubertI think we can do that10:57
juergbithe goal was/is to move it out to a separate repo10:58
benschubertand it was meant to be public from the start10:58
juergbiand then make it public there10:58
juergbithe separate non-public class in core was a compromise as we weren't ready moving git out10:58
juergbibst-plugins-experimental git_tag could still use it given our plan and the whole thing being 'experimental10:59
tme5the GitSourceBase class isn't particularly amenable to overriding10:59
juergbimaybe we could improve that?10:59
coldtomwith the refactor things might be easier10:59
juergbiI think that would be worthwhile anyway because even if all git_tag features were merged in, it would still not work in all scenarios11:00
juergbi(e.g., it can't use the github release API, of course)11:00
benschuberttme5: I would definitely push improvements then :)11:00
juergbii.e., I think we definitely want a public GitSourceBase in the future11:01
juergbiwhich should make it easy to customize tracking as that's the main point11:01
juergbi(iirc)11:01
coldtomi believe there was also some talk about allowing something equivalent to subclassing plugins in general at some point11:02
juergbiI think there was a discussion a long time ago, possibly when we discussed the git issue11:03
cs-shadowtpollard: me too :) There were some recent bugfixes on coverage that got me excited and bst testsuite appeared to be happy. Turns out I was using the py37 env all along :facepalming:11:08
cs-shadowjuergbi, benschubert: the other thing that's makes GitSourceBase less than ideal for overriding currently is its relationship with GitMirror class. They kind of depend on each other and don't have well defined interface boundaries11:10
cs-shadowthat will probably need some thought as well when we think abut making it public11:10
tme5they don't depend on each other too greatly, and my MR separates them further. In fact, I was of the opinion that GitMirror is a far more useful public API class than GitSourceBase11:13
cs-shadowGitMirror is definitely essential for writing any git-based plugins currently. But it's not very intuitive for plugin authors. No other plugin forces you to have a corresponding mirror class for example11:20
*** lantw44 has quit IRC11:20
*** lachlan has quit IRC11:20
*** lantw44 has joined #buildstream11:20
tme5no other plugin or transport mechanism requires it11:21
tme5if we were to decide ostree behaviour needed overriding i'd imagine an OSTreeMirror would be sensible11:22
tme5actually maybe not. the main purpose of GitMirror to me seems to be for independent caching of submodules11:22
coldtomthe SourceFetcher API is public though, i think it's reasonable to say "only git uses it right now but some other plugins might need to eventually"11:23
*** jonathanmaw has joined #buildstream11:25
tme5the SourceFetcher API is confusing to me, I don't quite understand its purpose11:25
coldtomi always thought the purpose was "let us handle git submodules nicely"11:26
tme5anyway, this is clearly an issue with many facets. The discussion above seems sensible to me. We should formalise it though, on the ML and issue/MR comments11:27
*** tlater[m] has joined #buildstream11:38
*** santi has joined #buildstream11:39
*** asingh_[m] has joined #buildstream11:44
*** lachlan has joined #buildstream11:44
*** lachlan has quit IRC11:51
*** xjuan has joined #buildstream12:01
*** lachlan has joined #buildstream12:21
*** lachlan has quit IRC12:25
*** lachlan has joined #buildstream12:37
*** lachlan has quit IRC12:41
gitlab-br-bottpollard merged MR !1745 (tpollard/messagerate->master: Make message & status rendering be tick driven by default) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/174512:43
*** lachlan has joined #buildstream12:46
gitlab-br-bottpollard opened issue #1250 (Add test for 'throttle-ui-updates' config) on buildstream https://gitlab.com/BuildStream/buildstream/issues/125012:48
*** lachlan has quit IRC12:52
gitlab-br-botjjardon opened (was WIP) MR !1729 (jjardon/move_deb_source->master: Remove deb surce plugin, it has beem moved to bst-plugins-experimental) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/172913:01
coldtomhi, i'm currently experiencing a weird cache key issue: i'm trying to execute buildstream through bazel, specifically building a zip file of freedesktop-sdk. I have everything from this build cached locally, whether from a git checkout or the zip file, however the cache keys change when bazel has extracted the zip file. anyone got any suggestions13:08
coldtomas to why this might be?13:08
coldtomoh, and to make things more complicated, i still hit the cache for a fair few elements13:08
juergbicoldtom: is the zip file extracted into a directory that is referenced by a `local` source?13:12
coldtomjuergbi: the zip file is of freedesktop-sdk, so that bazel can invoke bst to build me something13:13
coldtomi'm wondering if there's something in the cache keys affected by permissions or whatnot13:13
juergbicoldtom: and on every run you get a different cache key?13:13
juergbipermissions shouldn't affect cache key, except for the executable bit13:14
coldtomi _think_ they're consistently different13:14
juergbinot sure what you mean with that. you always get the same cache key when running buildstream via bazel, but it's different from running buildstream without bazel?13:14
coldtomi think i get the same cache keys when running on a project extracted by bazel, not just when executing bst through bazel, but this is inconsistent to the cache keys from either git or manually unzipping13:16
coldtomhowever, i think bazel is setting the executable bit on everything when it extracts, which might be why13:17
*** lachlan has joined #buildstream13:17
juergbiah ok, yes that would explain it...13:17
gitlab-br-botvalentindavid approved MR !1729 (jjardon/move_deb_source->master: Remove deb surce plugin, it has beem moved to bst-plugins-experimental) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/172913:23
tme5coldtom, would using a tarball instead work?13:25
tme5since they store permissions i'd guess bazel would be more likely to not do weird things13:25
*** Trevinho has quit IRC13:27
*** Trevinho has joined #buildstream13:27
*** lachlan has quit IRC13:30
gitlab-br-botBenjaminSchubert approved MR !1729 (jjardon/move_deb_source->master: Remove deb surce plugin, it has beem moved to bst-plugins-experimental) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/172913:57
coldtoma tarball worked, ta tme514:01
gitlab-br-bottpollard opened MR !1797 (tpollard/multishell->master: element.py: Only assert __stage_sources_directory on initial shell) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179714:37
gitlab-br-botcoldtom approved MR !1797 (tpollard/multishell->master: element.py: Only assert __stage_sources_directory on initial shell) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179714:39
*** lachlan has joined #buildstream14:41
*** lachlan has quit IRC14:46
benschubertjuergbi: !1739 is ready and all tests pass for all commits. I'll rebase on master and prepare the bst-plugins-experimental branch14:50
gitlab-br-botMR !1739: WIP: Optimize consistency and state handling https://gitlab.com/BuildStream/buildstream/merge_requests/173914:50
juergbinice, will take another quick look14:55
*** bochecha has quit IRC14:57
tme5i was planning to make some little fixes and changes to the documentation. how would you prefer I submitted these? would you rather lots of smaller MRs or one larger one?15:00
benschuberttme5: I personnally prefer coherent block (So if it's the same change or a related change in many place, grouped, but if it can be split in logical units, potentially smaller?)15:01
juergbitme5: I think a single MR with multiple commits would be reasonable15:01
benschubert^ good point15:02
juergbibenschubert: commented with a minor nit. looks good overall, thanks15:06
benschubertawesome thanks!15:06
tme5alright, thanks both15:06
benschubertjjardon: if you have time https://gitlab.com/BuildStream/bst-plugins-experimental/merge_requests/68 :) (Or anybody else)15:29
tpollarddoes anyone have any ideas for naming the added variable here? https://gitlab.com/BuildStream/buildstream/merge_requests/179715:53
*** traveltissues has quit IRC15:58
juergbitpollard: commented with an alternative16:06
tpollardI think you're right too16:07
gitlab-br-bottpollard closed MR !1797 (tpollard/multishell->master: element.py: Only assert __stage_sources_directory is None on initial shell) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179716:09
* tpollard facepalms16:09
*** lachlan has joined #buildstream16:29
*** lachlan has quit IRC16:32
gitlab-br-bottpollard opened MR !1798 (tpollard/removeassert->master: element.py: Remove unused __staged_sources_directory & assert) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179816:35
tpollardthanks for catching the juergbi, I was trying to work around a condition that didn't actually need to exist anymore16:36
tpollard*that16:36
tpollardI need to catchup with the staging parts of the codebase16:39
tpollardi.e the shift to virtual dirs16:43
WSalmondo shells have $XDG_CACHE_HOME set? i need to make sure something can find something so it dosnt try and download it from in a build shell. i have done something similar for it in docker but now need to translate to bst16:57
WSalmonbuild shells in particula16:58
benschubertyou can set it in the 'environment' for your plugin16:58
benschubertbut I don't think we do set it16:58
coldtomi don't think any of the XDG dirs get set do they?16:58
WSalmonyep but i wonted to check that we dont already do something related that i might be upsetting16:59
benschuberta quick check would be running a build element and run 'printenv && exit 1'16:59
WSalmonthanks benschubert17:02
*** benschubert has quit IRC17:16
*** benschubert has joined #buildstream17:16
*** santi has quit IRC17:20
*** santi has joined #buildstream17:27
*** tme5 has quit IRC17:27
*** lachlan has joined #buildstream17:34
gitlab-br-botjuergbi approved MR !1798 (tpollard/removeassert->master: element.py: Remove unused __staged_sources_directory & assert) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179817:34
gitlab-br-botBenjaminSchubert approved MR !1798 (tpollard/removeassert->master: element.py: Remove unused __staged_sources_directory & assert) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179817:35
benschuberttpollard: thanks a lot for fixing this :)17:35
tpollardnp, hopefully we can catch these things sooner with expanding the use of pexpect17:37
gitlab-br-bottpollard closed issue #1249 (On build error, can't enter twice in the shell) on buildstream https://gitlab.com/BuildStream/buildstream/issues/124917:37
gitlab-br-bottpollard merged MR !1798 (tpollard/removeassert->master: element.py: Remove unused __staged_sources_directory & assert) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179817:37
*** lachlan has quit IRC17:39
*** phildawson has joined #buildstream17:43
*** tiagogomes has quit IRC17:56
*** ikerperez has quit IRC18:10
*** paulsherwood has quit IRC18:10
*** paulsherwood has joined #buildstream18:10
*** ikerperez has joined #buildstream18:11
*** robjh has quit IRC18:15
*** laurence has quit IRC18:15
*** bethw has quit IRC18:15
*** WSalmon has quit IRC18:15
*** jward has quit IRC18:15
*** bethw has joined #buildstream18:15
*** valentind has quit IRC18:15
*** jward has joined #buildstream18:16
*** robjh has joined #buildstream18:18
*** valentind has joined #buildstream18:20
*** laurence has joined #buildstream18:20
*** WSalmon has joined #buildstream18:20
*** phildawson has quit IRC18:27
*** lachlan has joined #buildstream18:27
*** benschubert has quit IRC18:29
*** Trevinho has quit IRC18:30
*** lchlan has quit IRC18:30
*** jjardon has quit IRC18:31
*** lachlan has quit IRC18:32
*** benschubert has joined #buildstream18:33
*** benschubert has quit IRC18:36
*** WSalmon has quit IRC19:00
*** jonathanmaw has quit IRC19:00
*** bethw has quit IRC19:00
*** robjh has quit IRC19:00
*** laurence has quit IRC19:01
*** douglaswinship has quit IRC19:01
*** paulsherwood has quit IRC19:01
*** jward has quit IRC19:01
*** ikerperez has quit IRC19:01
*** valentind has quit IRC19:01
*** lachlan has joined #buildstream19:02
*** phildawson has joined #buildstream19:05
*** santi has quit IRC19:13
*** santi has joined #buildstream19:13
*** Trevinho has joined #buildstream19:19
*** Trevinho has quit IRC19:26
*** lachlan has quit IRC19:28
*** lachlan has joined #buildstream19:47
*** santi has quit IRC19:51
*** lachlan has quit IRC19:56
*** ikerperez has joined #buildstream20:04
*** robjh has joined #buildstream20:04
*** jward has joined #buildstream20:04
*** bethw has joined #buildstream20:05
*** valentind has joined #buildstream20:05
*** WSalmon has joined #buildstream20:05
*** laurence has joined #buildstream20:05
*** paulsherwood has joined #buildstream20:07
*** douglaswinship has joined #buildstream20:07
*** lachlan has joined #buildstream20:22
*** lachlan has quit IRC20:25
*** cs-shadow has quit IRC20:35
*** phildawson has quit IRC20:48
*** jjardon has joined #buildstream21:08
*** ChanServ sets mode: +o jjardon21:08
*** phildawson has joined #buildstream21:11
gitlab-br-botcs-shadow opened MR !1799 (chandan/remove-bst-experimental-unused->master: tests/integration/project.conf: Remove unused external plugins) on buildstream https://gitlab.com/BuildStream/buildstream/merge_requests/179922:59
*** benschubert has joined #buildstream23:14
*** phildawson has quit IRC23:31
*** phildawson has joined #buildstream23:40
« Tuesday, 2020-01-14 Index Thursday, 2020-01-16 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!