IRC logs for #buildstream for Tuesday, 2017-10-31

*** tristan has quit IRC05:32
*** tristan has joined #buildstream05:59
gitlab-br-botpush on buildstream@sam/compose-list-exception (by Tristan Van Berkom): 11 commits (last: _pipeline.py: Make fetching of remote refs optional) https://gitlab.com/BuildStream/buildstream/commit/5c2ef6d076921bc0121e61efaa7a719c34ea191206:59
gitlab-br-botbuildstream: merge request (sam/compose-list-exception->master: Catch attempts to compose a list) #129 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/12906:59
gitlab-br-botbuildstream: issue #142 ("Potentially printing provenance more than once in LoadErrors") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/14207:07
*** semanticdesign has joined #buildstream08:12
*** valentind has joined #buildstream08:36
gitlab-br-botbuildstream: merge request (sam/compose-list-exception->master: Catch attempts to compose a list) #129 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/12908:42
gitlab-br-botpush on buildstream@master (by Tristan Van Berkom): 1 commit (last: Catch attempts to compose a list) https://gitlab.com/BuildStream/buildstream/commit/8c4d8ca2b7c886690fb933428c6768c745667db708:42
gitlab-br-botbuildstream: Tristan Van Berkom deleted branch sam/compose-list-exception08:42
gitlab-br-botpush on buildstream@options (by Tristan Van Berkom): 9 commits (last: tests/frontend/track.py: Added tests for recursive tracking and --except) https://gitlab.com/BuildStream/buildstream/commit/e472f5abd7f9aeff994a14e581c7a89199bd255f08:44
gitlab-br-botbuildstream: merge request (options->master: doc: Explain how to configure project options) #130 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13008:44
*** givascu has joined #buildstream08:54
*** tlater has joined #buildstream09:24
gitlab-br-botpush on buildstream@user-namespace-fallback (by Tristan Van Berkom): 4 commits (last: _frontend/main.py: Hold on to unconditional startup messages) https://gitlab.com/BuildStream/buildstream/commit/fc5213d03c012a5ec2213214d36f0097ede3f79109:58
gitlab-br-botbuildstream: merge request (user-namespace-fallback->master: User namespace fallback) #132 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13209:59
tristanjjardon[m], ping09:59
tristanjjardon[m], can you please try building GNOME with user-namespace-fallback branch ? i.e. the above MR: https://gitlab.com/BuildStream/buildstream/merge_requests/13210:00
tristanjjardon[m], if that passes integration tests on not arch linux, and it "worksforyou" on Arch linux, I'll merge10:01
*** ssam2 has joined #buildstream10:05
gitlab-br-botpush on buildstream@master (by Tristan Van Berkom): 2 commits (last: _frontend/main.py: Hold on to unconditional startup messages) https://gitlab.com/BuildStream/buildstream/commit/e8309af42d6e9ce554961503638b4925c1a8a53d10:06
gitlab-br-botbuildstream: Tristan Van Berkom deleted branch options10:06
gitlab-br-botbuildstream: merge request (options->master: doc: Explain how to configure project options) #130 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/13010:06
gitlab-br-botpush on buildstream@user-namespace-fallback (by Tristan Van Berkom): 6 commits (last: Catch attempts to compose a list) https://gitlab.com/BuildStream/buildstream/commit/8c4d8ca2b7c886690fb933428c6768c745667db710:07
gitlab-br-botbuildstream: merge request (user-namespace-fallback->master: User namespace fallback) #132 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13210:08
jjardon[m]tristan: https://paste.gnome.org/pxnkzmsah how can I force a rebuild in bst?10:20
ssam2delete your artifact cache ?10:24
*** valentind has quit IRC10:25
gitlab-br-botpush on buildstream@tracking-changes (by Tristan Maat): 5 commits (last: WIP: Add support for multiple arguments (*very* WIP)) https://gitlab.com/BuildStream/buildstream/commit/65dbf24543c4eafdb066558e0560f9b433fba91f10:28
gitlab-br-botbuildstream: merge request (tracking-changes->master: WIP: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/11910:28
tristanUmmm good question10:38
tristanjjardon[m], maybe build something a bit higher up ?10:38
tristanlike core-deps/glib.bst ?10:38
tristanBecause... downloading and constructing the base system is... ugh, long time10:38
tristanjjardon[m], another approach might be to first run `bst track base/ninja.bst` and rerun that with a newer ninja10:39
tristanthen you only rebuild ninja10:39
jjardon[m]tristan: mmm, not sure this is what we expected: https://paste.gnome.org/p5u3xdc6a10:45
tristanjjardon[m], tracking didnt pull a new version of ninja10:48
tristan:-/10:48
tristanIt looks like stderr is not caught by my simple minded call to subprocess.check_output(), and instead it gets rudely sent to the terminal10:48
tristanBut, that's not a huge deal I think10:48
tristanjjardon[m], just try `bst track core-deps/glib.bst` and try building that, certainly that will have a new commit to build10:49
tristanor maybe gettext10:49
tristanthose are pretty low level in the stack10:49
jjardon[m]tristan: same happened for glib, let me try gettext10:50
jjardon[m]same for gettext10:50
tristanjjardon[m], no new versions ?10:51
tristanjjardon[m], I wonder, how did you get all of those built in the first place ?10:51
tristanhere, I can produce some mumbo jumbo ostree command for you...10:52
tristanone sec10:52
jjardon[m]tristan: I followed the steps on the wiki10:52
tristanjjardon[m], yeah, but you have Arch, which means... you couldnt have built it :D10:53
tristanjjardon[m], you did it with commented lines in _sandboxbwrap.py I guess ?10:54
tristanjjardon[m], here: ostree refs --repo  ~/.cache/buildstream/artifacts/ostree --delete `ostree refs --repo ~/.cache/buildstream/artifacts/ostree | grep gnome/core-deps-gettext`10:54
tristanjjardon[m], that'll delete your local cache of gettext10:54
jjardon[m]tristan: yes, I commented the s.py line10:59
tristan:)10:59
tristanjjardon[m], so this is most probably going to work, at least we've seen in your output that we're detecting the incapability to do user namespaces properly there11:00
jjardon[m]tristan: same thing after trying that command :/11:00
tristanwhat ?!11:00
tristanjjardon[m], the ostree thing didnt delete your artifact ?11:00
tristanwhat happened, is it on an artifact share ?11:00
tristandid it download the artifact instead of building it ?11:00
tristandid you run `bst build core-deps/gettext.bst` after that ?11:01
tristanjjardon[m], gettext.bst is not a part of base.bst, so; you'd have to try to build it explicitly after deleting it11:01
jjardon[m]I did bst build base.bst, let me try that11:01
jjardon[m]ah!11:01
tristanunfortunately, pipelines are damn slow on gitlab today11:02
jjardon[m]tristan: something interesting: https://paste.gnome.org/pkejdhda711:02
tristanhrmmmm11:03
tristanjjardon[m], mkayyyyy...11:03
* tristan looks into that11:04
tristanohhhhh11:05
tristantlater, looks like your Platform.get_platform() is actually instantiating a new platform in the child tasks, is that intentional for any reason ?11:06
tlatertristan: Where does it do that? It's possible that that's used only to access a specific function, probably a bit heavy handed if so.11:07
tristantlater, in jjardon[m]'s log: https://paste.gnome.org/pkejdhda7#line-6211:10
tristanline 6211:10
tristanindicates that in the child task, we're instantiating a platform... via Element.__sandbox()11:10
* tristan also found another bug thanks to that stack trace11:10
tristanlooks like at times we can be logging integration commands to the wrong log file11:11
* tristan quickly fixes this, which should fix jjardon[m]'s issue11:11
tlaterHmm... Yeah, it's definitely not supposed to do that.11:11
tlaterI think for a while I had the platform as a singleton11:11
tristantlater, if you could keep your eye on that and fix it, I'd really like that11:12
tristanI bet it's an easy fix11:12
tristanmaybe you just need a toplevel variable around in platform.py, and refer to it as `global` from within that function11:12
* tlater will, do you prefer making it a singleton or trying to fiddle the pipeline into places it probably doesn't belong?11:12
tristanget_platform()11:12
tristanI dont know what you mean by fiddling the pipeline, ensuring it's singleton is better I think11:13
tristanI guess you might mean, passing it through a whole lot of internal apis11:13
tlaterYeah, I think I prefer that too. Technically the pipeline currently owns the platform11:13
tristanyeah dont do that11:13
tristanjust singleton, maybe you are just missing global11:13
tristantlater, note that a variable assignment in python triggers a new variable to be declared11:13
tristanwhich will not have any warning, but shadow existing variables11:14
tristanwhich is, totally horrible11:14
tristanand is a probable cause of a bug like this :D11:14
* tlater is aware, my linter warns me :)11:14
tristanHmmm, we should add another linter to our test suite too11:14
tristananyway11:14
gitlab-br-botpush on buildstream@master (by Tristan Van Berkom): 1 commit (last: _scheduler/job.py: Ensure messages are logged to the correct log file) https://gitlab.com/BuildStream/buildstream/commit/40f5fe7789d4085183717cfc158297959d8b380b11:16
gitlab-br-botpush on buildstream@user-namespace-fallback (by Tristan Van Berkom): 4 commits (last: _scheduler/job.py: Ensure messages are logged to the correct log file) https://gitlab.com/BuildStream/buildstream/commit/40f5fe7789d4085183717cfc158297959d8b380b11:16
tristanjjardon[m], wanna try it one more time with a fresh `user-namespace-fallback` branch ?11:17
jjardon[m]sure, give me a second11:17
jjardon[m]tristan: mmm, seems its downloading the artifact: https://paste.gnome.org/ptnfbvsx9 Is there any way to disable any network connection? (like -n in jhbuild)11:20
gitlab-br-botbuildstream: merge request (user-namespace-fallback->master: User namespace fallback) #132 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13211:21
tristanjjardon[m], ummmm, modify project.conf :)11:22
tristanjjardon[m], comment out the artifact share11:22
tristanjjardon[m], and of course, rerun that ostree mumbo jumbo11:23
jjardon[m]tristan: Its alive! https://paste.gnome.org/pzpj2xtzp11:28
tristanw00t \o/11:29
tristanjjardon[m], great !11:29
tristanjjardon[m], that status line (after start, before the warning) is because of the thing tlater needs to fix11:30
* tlater will be done in a second, is stumped by failing tests11:30
* jjardon[m] proceed to build the whole GNOME now11:30
tristanjjardon[m], I'll set this one to merge, and that messy thing will go away by itself after Platform becomes a proper singleton11:30
* tristan is heading out "early" today11:31
tristanhave dinner plans11:31
jjardon[m]tristan: I gues is normal that "bst build base.bst" doesnt do anything as the artifact cache of gettext is the same?11:31
tristanjjardon[m], gettext.bst is not a part of base.bst (it's a part of core-deps)11:32
tristanjjardon[m], base.bst is only the debootstrap runtime11:32
*** adds68_ has quit IRC11:32
jjardon[m]tristan: rigth, you said that before :) sorry11:32
*** adds68 has joined #buildstream11:32
tlatertristan: Would you have a bit of time to go over what I wrote for Planner.plan? It's a lot larger of a change than I'd like it to be :/11:33
tlatertristan: It's here atm: https://gitlab.com/BuildStream/buildstream/merge_requests/119/diffs#029d46ba7e1b2197fd4337e31e3d8422a390dad5_84_9411:33
tristanI have to go now though :-/11:33
tlaterAh, that's alright11:34
tlaterSorry, I should have asked sooner11:34
tristantlater, I was busy trying to nail the Arch thing anyway11:34
tristantlater, I think anyway, my hair brained ideas of reducing that code with Element.dependencies() are wrong11:35
tristanbecause of runtime dependencies anyway needing to be sorted differently11:35
tlatertristan: Well, sure, but there's a lot that changes about the actual algorithm because the original doesn't create a proper parallel sort for multiple root nodes.11:36
tristantlater, not looking too deeply at this, maybe some alternative which takes the toplevel elements you want to build and uses some virtual toplevel to depend on them might help reduce that11:36
* tlater is quite happy to wait until you can have a proper look :)11:37
tristantlater, also, maybe it doesnt really have to be an "Element" per se, just some simple spoofy thing which "quacks like an element"11:37
tristanjust an idea11:37
* tristan out11:37
tlatero/11:37
tristantlater, if you happen to be needing something smaller to take care of, you might look at this: https://gitlab.com/BuildStream/buildstream/issues/14211:38
tristanbut that is really "just in case" :D11:38
* tlater has enough to do :)11:38
*** tristan has quit IRC11:41
*** tristan has joined #buildstream11:48
tristanok friend is a bit late... gives me some minutes...11:52
tristantlater, ok so, it looks like your algo grew mostly in terms of comments11:52
tristantlater, are you able to create a link to that which is not a diff, but a revision ?11:52
* tristan tries11:53
tlatertristan: https://gitlab.com/BuildStream/buildstream/blob/486df70383c51a455f9d50963e1043ad471fee65/buildstream/_pipeline.py#L9411:53
tristanhttps://gitlab.com/BuildStream/buildstream/blob/tracking-changes/buildstream/_pipeline.py#L6111:53
tristanyep, approximately the same11:54
tristanso, to be honest, I dont want to review this part11:54
tristanjuergbi, are you able to take some time to ensure that https://gitlab.com/BuildStream/buildstream/blob/tracking-changes/buildstream/_pipeline.py#L61 is correct ?11:54
tristanplease ?11:54
juergbiwill take a look11:55
tristanNot necessarily ensure I guess11:55
* tlater will add tests to help with 'correctness' anyway11:55
tristantlater, I was going to say that :)11:55
tlaterI'd mostly like to know if this is a sensible change :)11:55
tristantlater, but you are on the ball anyway :D11:55
tristanI dont mind about the code size fwiw, not obsessive about LOC counts, I'm more obsessive about general complexity and not having duplication of logic11:56
ssam2I can see having one (?) per toplevel dict entry getting a bit restrictive12:14
ssam2e.g., if I have %{gcc-arch-flags} and %{gcc-libdir} variables in my element, and i have complex but separate logic for configuring each one ... i have to put both sets of conditionals under one (?)12:15
ssam2which is fine for now12:15
ssam2but maybe for 2.0 we should allow conditional entries like (?.1), (?.2), (?.foo) so that different concerns can be logically separated12:16
ssam2or maybe it's good that this discourages people from going too crazy with conditionals. i could certainly rewrite this whole bootstrap more clearly, if I had months spare to retest everything12:17
tristanssam2, I'm not sure this is needed; each toplevel is a list anyway right ?12:32
tristanAnd it's an if, if, if, if list12:32
tristannot an if, elif, elif, list12:32
tristanssam2, anyway I'm on my way out the door literally, maybe if you showed me an example I could maybe point out how to do it with the current thing12:33
tristanor, understand what it is that is impossible or unwieldy12:33
ssam2no rush12:34
ssam2once you see the baserock gnu-toolchain you'll be able to see what i mean, i expect12:34
ssam2once i'm done converting to conditionals, i mean12:34
*** adds68 has quit IRC12:39
*** adds68 has joined #buildstream12:39
gitlab-br-botpush on buildstream@platform_singleton (by Tristan Maat): 1 commit (last: Make the platform object a singleton) https://gitlab.com/BuildStream/buildstream/commit/dda93aa347d34204e1f22f65d41c0b778e8bf88913:26
gitlab-br-botbuildstream: merge request (platform_singleton->master: Make the platform object a singleton) #133 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13313:26
gitlab-br-botbuildstream: merge request (platform_singleton->master: Make the platform object a singleton) #133 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13313:27
gitlab-br-botpush on buildstream@platform_singleton (by Tristan Maat): 1 commit (last: Make the platform object a singleton) https://gitlab.com/BuildStream/buildstream/commit/5e28b4a3de1f42a1d4240d429c1c85641da5f40c13:27
gitlab-br-botpush on buildstream@platform_singleton (by Tristan Maat): 1 commit (last: Clean old Platform.get_platform references) https://gitlab.com/BuildStream/buildstream/commit/e268c297788f8175ae82e8978e44222da8acad6413:31
gitlab-br-botbuildstream: merge request (platform_singleton->master: Make the platform object a singleton) #133 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13313:31
gitlab-br-botpush on buildstream@tracking-changes (by Tristan Maat): 14 commits (last: Catch attempts to compose a list) https://gitlab.com/BuildStream/buildstream/commit/8c4d8ca2b7c886690fb933428c6768c745667db713:39
gitlab-br-botbuildstream: merge request (tracking-changes->master: WIP: Tracking changes) #119 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/11913:39
gitlab-br-botbuildstream: issue #92 ("Handle kernels which lack CONFIG_USER_NS") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/9213:40
gitlab-br-botbuildstream: merge request (user-namespace-fallback->master: User namespace fallback) #132 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/13213:40
gitlab-br-botbuildstream: Tristan Van Berkom deleted branch user-namespace-fallback13:40
gitlab-br-botpush on buildstream@master (by Tristan Van Berkom): 3 commits (last: _platform/linux.py: Add preflight check to detect user namespaces) https://gitlab.com/BuildStream/buildstream/commit/a9a2bcece3b90092d9260f5c46675a0e17df381d13:40
*** tiagogomes has quit IRC14:33
tlaterHm, I'm getting a lot of test failures due to the changes in the internal API15:18
tlaterLoader.load() and various other commands now expect lists15:18
tlaterShould I make those functions implicitly change single arguments to lists?15:18
* tlater feels like that might be a bad idea, but it would fix ~450 failing test cases15:19
tlaterHmm, nah, it makes no sense to change API to accomodate tests...15:20
* tlater stops being lazy15:20
*** semanticdesign has quit IRC15:31
gitlab-br-botbuildstream: merge request (provenance_fmt->master: WIP: De-dup brackets around provenance in error messages) #134 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/13416:08
*** jude has quit IRC16:46
gitlab-br-botbuildstream: merge request (incremental-build->master: WIP: Incremental build) #126 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/12617:06
gitlab-br-botbuildstream: merge request (incremental-build->master: WIP: Incremental build) #126 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/12617:06
gitlab-br-botpush on buildstream@multiple_targets (by Tristan Maat): 13 commits (last: _pipeline.py: Implement Planner.plan using Kahn's algorithm) https://gitlab.com/BuildStream/buildstream/commit/5ce8d1904e0e19434e584283433065452f4d1fee17:20
*** valentind has joined #buildstream17:28
gitlab-br-botbuildstream: merge request (incremental-build->master: WIP: Incremental build) #126 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/12617:30
gitlab-br-botpush on buildstream@multiple_targets (by Tristan Maat): 3 commits (last: Adjust `bst workspace` to multiple targets) https://gitlab.com/BuildStream/buildstream/commit/3e525e2506eabb2c947195b4b8e23585c7c86b5b17:34
gitlab-br-botpush on buildstream@multiple_targets (by Tristan Maat): 4 commits (last: main.py: Adjust `bst track` to multiple targets) https://gitlab.com/BuildStream/buildstream/commit/bec2330c50c7d52db9dba516f7cf59569ef4e38118:03
*** tlater has quit IRC18:04
gitlab-br-botbuildstream: merge request (incremental-build->master: WIP: Incremental build) #126 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/12618:16
*** ssam2 has quit IRC18:40
*** semanticdesign has joined #buildstream19:03
*** givascu has quit IRC20:27
*** adds68 has quit IRC21:34
*** valentind has quit IRC22:32

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!