*** Prince781 has joined #buildstream | 01:18 | |
*** Prince781 has quit IRC | 02:17 | |
*** mhm527 has joined #buildstream | 03:30 | |
*** mhm527 has quit IRC | 03:31 | |
*** Prince781 has joined #buildstream | 03:40 | |
*** Prince781 has quit IRC | 04:44 | |
*** Prince781 has joined #buildstream | 05:06 | |
*** Prince781 has quit IRC | 05:11 | |
gitlab-br-bot | buildstream: merge request (valentindavid/385_vte_notification->master: buildstream/_frontend/app.py: Notify VTE on failure in interactive mode.) #447 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/447 | 05:16 |
---|---|---|
*** Prince781 has joined #buildstream | 05:17 | |
*** tristan has joined #buildstream | 05:24 | |
gitlab-br-bot | buildstream: issue #383 ("AssertionError when caching artifact") changed state ("closed") https://gitlab.com/BuildStream/buildstream/issues/383 | 06:28 |
gitlab-br-bot | buildstream: merge request (juerg/non-strict-buildable->master: element.py: Fix buildable check in non-strict mode) #448 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/448 | 06:28 |
*** Prince781 has quit IRC | 06:32 | |
*** toscalix has joined #buildstream | 06:58 | |
gitlab-br-bot | buildstream: merge request (juerg/no-remote-summaries->master: Drop use of OSTree summary files and dynamically plan build dependencies) #446 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/446 | 08:31 |
*** aday has joined #buildstream | 08:35 | |
toscalix | valentind: based on tristan mail, I assigned https://gitlab.com/BuildStream/buildstream/issues/359 to you | 08:40 |
valentind | toscalix, ok | 08:40 |
laurence | cs_shadow, hello :) | 08:47 |
laurence | are you planning to go to GUADEC? | 08:47 |
*** jonathanmaw has joined #buildstream | 08:48 | |
*** dominic has joined #buildstream | 08:51 | |
tristan | juergbi, this rebase is going to be a real pain :-S | 08:53 |
jmac | thanks juergbi | 08:53 |
tristan | juergbi, now we actually hand the pipeline to the elements and let the elements themselves do things... I dont like how loopy this is | 08:54 |
juergbi | tristan: that's why I mentioned it a few days ago | 08:54 |
tristan | yeah | 08:54 |
*** aday has quit IRC | 08:54 | |
juergbi | yes, I was going forth and back on this one | 08:54 |
tristan | I wonder how this should ultimately look | 08:54 |
tristan | Is there an alternative without that ? | 08:54 |
juergbi | not extremely happy about it either, but each element can only be in a single pipeline, it doesn't seem unreasonable | 08:54 |
juergbi | one potential alternative would be something like a callback to keep it decoupled | 08:55 |
tristan | "a pipeline" | 08:55 |
tristan | Is there now more than one pipeline too ? | 08:55 |
juergbi | no, there isn't. it's just hypothetical | 08:55 |
tristan | So, I was thinking that one way forward was to have something additional, like the QueueMaster or something, sitting in between... that might mesh well with what tlater is doing | 08:56 |
tristan | But | 08:56 |
tristan | It looks like this needs knowledge of the planner | 08:56 |
*** bethw has joined #buildstream | 08:56 | |
tristan | If the segmented lists of things which "we only might build" were to be sorted and planned first, and only optionally built, then at least we wouldnt need that part | 08:57 |
tristan | I'm trying very hard here to think of an approach which doesnt involve mangling... where Elements dont know anything about their owners | 08:58 |
tristan | and we have a nice top-down ownership model | 08:58 |
*** aday has joined #buildstream | 08:59 | |
juergbi | hm, not sure how that would work with planning everything but still keeping some parts optional | 08:59 |
cs_shadow | laurence: hey! Not sure yet. I’ll have to check with others at office about the plan | 09:00 |
juergbi | each on-demand build dependency can have further dependencies, so in one way or another you have to do dynamic planning | 09:00 |
cs_shadow | Let me get back to you on that | 09:00 |
tristan | I thought that if we had scheduled the build-of-build dependencies *after* the reverse dependencies which pull them in, and then have them in WAIT state until their reverse dependencies pass through the PullQueue | 09:00 |
laurence | cs_shadow, cool. would be good to see you all there :) | 09:01 |
tristan | juergbi, We could then decide whether to put those elements in READY or SKIP state | 09:01 |
tristan | juergbi, but I recall you said something about this approach being overly complex | 09:01 |
juergbi | tristan: how to organize this with indirect dependencies, e.g., runtime dependencies of these build dependencies? | 09:01 |
juergbi | which could have multiple indirect reverse dependencies | 09:01 |
juergbi | I don't see a straight forward way right now but there might be something simple that still makes this possible | 09:02 |
tristan | Well, the knee-jerk response to that question is to filter those build-of-build branches properly, in fact; the --except logic does pretty much *exactly* this algorithm | 09:03 |
tristan | juergbi, i.e. excepting build-of-build from a specific element, does not filter out what is orthogonally depended on | 09:03 |
tristan | the result of that is the portion of the branch which needs to be queued after the dependency | 09:04 |
tristan | but... | 09:04 |
tristan | this is only a knee-jerk response to the question | 09:04 |
tristan | not completely thought out I suppose | 09:04 |
juergbi | tristan: not sure I understand. my question was more how to dynamically queue up such indirect dependencies, not how to filter them out at the beginning | 09:05 |
juergbi | or maybe top-down that would actually be pretty easy | 09:06 |
tristan | They would be queued as segments *after* the depending element instead of before them | 09:06 |
juergbi | so we would add everything to the pipeline/queues but keep some elements in an inactive/waiting state until we actually need them | 09:07 |
tristan | at the *basic* level, we would decide to skip them if we never needed to pull/build them, instead of adding them to the lists dynamically on demand; that is the basic concept | 09:07 |
tristan | exactly | 09:07 |
*** bochecha_ has joined #buildstream | 09:08 | |
tristan | it would introduce more iterations in the loop which searches for the next element that is ready to process | 09:08 |
tristan | but that seems like a negligible overhead | 09:08 |
juergbi | and as the 'active' state would be an Element property, an Element could activate its dependency without going through a Pipeline object | 09:09 |
tristan | I suppose it could be orchestrated with an active state | 09:09 |
tristan | juergbi, or with a _schedule_pull() and/or _schedule_assemble() sort of approach | 09:10 |
juergbi | 'active' was just a placeholder, point was that Elements would directly influence other Elements, not going through the Pipeline | 09:10 |
tristan | Yeah, basically I guess this comes down to... when an element fails in the PullQueue, it must propagate some state to it's dependencies | 09:11 |
tristan | telling it's build-of-build dependencies that they are needed for this run | 09:11 |
juergbi | would the number of session elements always include those potentially queued elements? | 09:15 |
juergbi | just a detail I guess | 09:15 |
*** bochecha_ has quit IRC | 09:16 | |
tristan | juergbi, I think that queueing build-of-build *after* the depending elements is in fact a meaningless operation, the more I think of it | 09:17 |
tristan | i.e. it's not important to put them after, the build plan is still sorted in an optimal way for "what we need first comes first" | 09:17 |
tristan | it's just a lot of elements get skipped at the beginning of the loop | 09:17 |
tristan | (or temporarily ignored, to avoid confusion with the word "skipped") | 09:18 |
tristan | juergbi, the more I have been thinking about session / total... the more I think that it is a weird prediction we make; it's somehow useful but does not match perfectly with the underlying machine | 09:19 |
tristan | if we have sessions where some elements are tracked, some elements are tracked and built, and other elements are only tracked (not extremely useful but it's an example): then session / total doesnt really capture something very meaningful | 09:20 |
*** tristan has quit IRC | 09:35 | |
gitlab-br-bot | buildstream: merge request (jennis/explanation-of-commands->master: commands.rst: Split up commands to easily allow for further description) #450 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/450 | 09:42 |
*** aday has quit IRC | 09:44 | |
*** aday has joined #buildstream | 09:54 | |
*** tristan has joined #buildstream | 09:58 | |
jennis | tristan, if you have a moment !450 (^^) is a small docs fix re the patch allowing for adding further description to commands | 10:00 |
tristan | jennis, let's merge | 10:03 |
*** aday has quit IRC | 10:04 | |
jennis | ok great | 10:06 |
gitlab-br-bot | buildstream: merge request (jennis/explanation-of-commands->master: commands.rst: Split up commands to easily allow for further description) #450 changed state ("merged"): https://gitlab.com/BuildStream/buildstream/merge_requests/450 | 10:07 |
gitlab-br-bot | buildstream: merge request (valentindavid/etag->master: Store etag along with cache.) #441 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/441 | 10:09 |
finn | Is there a variable I can use which contains the absolute project folder path? | 11:05 |
finn | e.g I have a file in files/whatever.tar | 11:05 |
finn | I'd like to source it, what should the URL look like? | 11:05 |
finn | At the moment, I have an alias: project_dir: file:///home/finn/alpine | 11:06 |
finn | Then in my .bst I have: url: project_dir:/files/whatever.tar | 11:06 |
Nexus | directory = self._main_options['directory'] | 11:07 |
Nexus | directory = os.path.abspath(directory) | 11:07 |
Nexus | is what i currently use | 11:07 |
Nexus | not sure if we store a hard path? | 11:07 |
tlater | Nexus: you can use python code in .bst files? ;p | 11:07 |
*** bethw has quit IRC | 11:08 | |
Nexus | sorry, i saw variable and assumed | 11:08 |
Nexus | don't our bst files just check relative to themselves? | 11:09 |
Nexus | e.g. gnome-modulesets/elements/apps.bst points at apps/foo.bst | 11:10 |
Nexus | where apps is a dir inside elements | 11:10 |
finn | Directory structure locally is: | 11:10 |
finn | -elements | 11:10 |
finn | -files | 11:10 |
gitlab-br-bot | buildstream: merge request (valentindavid/253_better_missing_variable_errors->master: Fix provenance in error message for undefined variables.) #449 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/449 | 11:10 |
tlater | Nexus: Yes, dependencies work that way | 11:10 |
tlater | But tar *sources* need to specify an absolute directory in their `path` | 11:11 |
tlater | Or `url` | 11:11 |
Nexus | well that sucks | 11:11 |
tlater | We work around it in the test suite using code, but I can't remember if I ever checked for a variable... | 11:11 |
finn | Do you format the project.conf? | 11:14 |
finn | project_dir: file://{project_dir} | 11:14 |
tlater | finn: yep, that's exactly what we do | 11:15 |
tlater | We might want to create such a variable - this has been annoying me since I started working on buildstream | 11:16 |
tlater | (Assuming it doesn't exist and I simply overlooked it) | 11:16 |
finn | This is basically for a docker workaround currently so it isn't vital. I'll keep absolute file path name until you make a decision :) | 11:17 |
jennis | juergbi, looks like !446 is ready to land? As far as I can see, there is no obvious blocker to my remote expiry, however I'd like to test against your changes to make sure | 11:34 |
gitlab-br-bot | buildstream: merge request (valentindavid/253_better_missing_variable_errors->master: Fix provenance in error message for undefined variables.) #449 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/449 | 11:44 |
*** csoriano has quit IRC | 11:48 | |
*** csoriano has joined #buildstream | 11:48 | |
gitlab-br-bot | buildstream: merge request (372-allow-queues-to-run-auxilliary-jobs-after-an-element-s-job-finishes->master: WIP: Resolve "Allow queues to run auxilliary jobs after an element's job finishes") #433 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/433 | 12:07 |
gitlab-br-bot | buildstream: merge request (372-allow-queues-to-run-auxilliary-jobs-after-an-element-s-job-finishes->master: WIP: Resolve "Allow queues to run auxilliary jobs after an element's job finishes") #433 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/433 | 12:09 |
gitlab-br-bot | buildstream: merge request (372-allow-queues-to-run-auxilliary-jobs-after-an-element-s-job-finishes->master: WIP: Resolve "Allow queues to run auxilliary jobs after an element's job finishes") #433 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/433 | 12:10 |
gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 12:12 |
gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 12:20 |
Nexus | I can't seem to come up with a way of omitting .bzr from the source.stage() function. The wya i understand it, i'd need to prevent .bzr from being checked out, without modifying the upstream branch (which i doubt we want to do?), i don't see how this is possible | 12:20 |
gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 12:22 |
valentind | tristan, Should we forbid relative path of elements to contain colon character? Or should we check for name with colon before we split with for junctions? | 12:25 |
*** bethw has joined #buildstream | 12:32 | |
*** aday has joined #buildstream | 12:40 | |
gitlab-br-bot | buildstream: merge request (372-allow-queues-to-run-auxilliary-jobs-after-an-element-s-job-finishes->master: WIP: Resolve "Allow queues to run auxilliary jobs after an element's job finishes") #433 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/433 | 12:45 |
gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 12:46 |
gitlab-br-bot | buildstream: merge request (jmac/virtual_directories->master: WIP: Abstract directory class and filesystem-backed implementation) #445 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/445 | 12:48 |
*** jonathanmaw has quit IRC | 13:02 | |
*** xjuan has joined #buildstream | 13:21 | |
gitlab-br-bot | buildstream: merge request (issue-21_Caching_build_trees->master: WIP: Issue 21 caching build trees) #372 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/372 | 14:09 |
jennis | when writing docs, is it ok to refer to "name:" "kind:" "sources:" etc.. as nodes? | 14:21 |
*** toscalix has quit IRC | 14:44 | |
*** mohan43u has quit IRC | 14:44 | |
finn | Are they points of intersection? | 14:46 |
finn | To me they look like keys | 14:47 |
finn | or types? | 14:48 |
tlater | They're certainly dictionary keys, but this seems like an odd term to use in user-facing documentation | 14:49 |
tlater | "Options", or "settings" perhaps? | 14:49 |
finn | as is a node, a node is generally something which has branches coming out of it | 14:49 |
finn | There is also an option called options | 14:49 |
tlater | I'm not opposed to nodes/keys, but I feel like it's not quite as obvious to users. | 14:49 |
finn | settings is probably better than options to avoid a name clash? | 14:50 |
tlater | Yeah, definitely | 14:50 |
finn | settings it is | 14:50 |
tlater | Hehe | 14:50 |
tlater | jennis: ^ | 14:50 |
tlater | (Obviously we'd need to make sure that this term is used accross the documentation, which I don't think it currently is) | 14:51 |
finn | So I've tarred up a docker image, sourced it in a .bst | 15:01 |
finn | run the build command successfully | 15:01 |
finn | when I try to run the shell, I get this: | 15:01 |
finn | bwrap: execvp sh: No such file or directory | 15:01 |
jmac | That means there's no /bin/sh in your artifact | 15:02 |
tlater | finn: have you checked that the platform ends up in the sandbox properly? You can assert that using `bst export` | 15:02 |
jmac | Usually it happens when you miss off a dependency | 15:02 |
tlater | If you export the element with the tarfile you should get a nice sysroot :) | 15:02 |
finn | Error: No such command "export". | 15:02 |
tlater | err checkout | 15:03 |
tlater | Sorry | 15:03 |
*** mohan43u has joined #buildstream | 15:03 | |
finn | checkout works | 15:04 |
tlater | Does the resulting directory contain bin/sh? | 15:04 |
finn | bst checkout base.bst directory/ | 15:04 |
finn | directory/image-tools/bin/sh exists | 15:05 |
tlater | Ah | 15:05 |
tlater | See, this won't work | 15:05 |
tlater | Because bin/sh isn't in the root directory :) | 15:05 |
finn | riiight | 15:05 |
tlater | This is a little tricky, I think you'll need to make sure buildstream doesn't extract it into a directory with the same name as the element | 15:06 |
tlater | There was a note about that in the documentation for tar sources, I think | 15:06 |
*** tiago has quit IRC | 15:06 | |
finn | Thanks | 15:13 |
finn | that worked | 15:13 |
finn | base-dir: '*' | 15:15 |
gitlab-br-bot | buildstream: merge request (jennis/initialise-elements-sub-dir->master: app.py: Ensure an elements sub-dir is created upon project initialisation) #451 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/451 | 15:15 |
jennis | Ok, I agree with settings | 15:17 |
jennis | Could someone give !451 a quick review, it's a pretty trivial change | 15:17 |
jennis | pls :) | 15:17 |
finn | what are all these triangles? | 15:18 |
finn | ▶ | 15:18 |
tlater | Eum, where? | 15:18 |
jennis | That's my terminal settings :p | 15:18 |
tlater | Ah | 15:19 |
tlater | x) | 15:19 |
finn | very pretty | 15:19 |
finn | though I'd remove them and have a simple shell result | 15:19 |
tlater | So much vertical space though! | 15:19 |
finn | Ah so you're creating an elements dir with the init command now | 15:20 |
gitlab-br-bot | buildstream: merge request (jennis/initialise-elements-sub-dir->master: app.py: Ensure an elements sub-dir is created upon project initialisation) #451 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/451 | 15:20 |
jennis | ^^ no more triangles | 15:21 |
finn | haha thanks | 15:21 |
jennis | yes, just because our `bst init` command will by defauly create a project.conf which has the SETTING "element-path: elements" | 15:21 |
*** aday has quit IRC | 15:23 | |
jennis | tlater, thoughts on scrapping the (broken) x86_image in examples and replace it with what finn has just come up with | 15:23 |
jennis | as the maintainer of said repo :p | 15:23 |
tlater | jennis: Different purposes, we should instead add finn's | 15:24 |
*** aday has joined #buildstream | 15:24 | |
jennis | Thought we didn't use that x86 image? | 15:24 |
tlater | Both will be broken until the docker plugin is fixed | 15:24 |
jennis | Finn's source is a tarball in files/ | 15:24 |
tlater | jennis: The x86image example is an example that shows how you create an x86image using the x86image plugin | 15:24 |
jennis | aha, makes sense... | 15:25 |
tlater | finn's example is an example of how you create a simple, small buildstream project | 15:25 |
jennis | ok, I agree with add | 15:25 |
tlater | In either case, I'm not sure how much I like the tarball in files/ | 15:25 |
tlater | We could also do that for the x86image plugin | 15:25 |
tlater | But it makes the example hard to run because you'd need to modify the url first | 15:26 |
finn | you still want to add things to this -examples repo then? | 15:26 |
jennis | How about hosting it somewhere? | 15:26 |
finn | I can create a workaround with a script | 15:26 |
finn | or host it somewhere | 15:26 |
finn | could host it on git... | 15:26 |
tlater | Script workaround isn't particularly user friendly, which is important for someone just starting off, I think | 15:27 |
tlater | I'm not sure what licensing we need to worry about with hosting it somewhere, or where we'd host it | 15:27 |
tlater | Preferably someone just fixes the docker plugin and we don't need to worry about this | 15:27 |
finn | How hard would this docker patch be> | 15:27 |
finn | This work isn't _essential_ | 15:27 |
*** aday has quit IRC | 15:27 | |
tlater | Not very, it's correcting the permissions of a few files | 15:28 |
finn | Wouldn't this make more sense then | 15:28 |
jennis | Why are you against storing it in files? | 15:28 |
jennis | We do this for our integration tests | 15:28 |
*** aday has joined #buildstream | 15:28 | |
tlater | jennis: Because you need an additional script to be able to run it then | 15:28 |
finn | I'd have to format my string | 15:28 |
tlater | I worry that this would be confusing to a beginning user. | 15:29 |
finn | agreed | 15:29 |
finn | it isn't the "normal" way | 15:29 |
tlater | "Do we need to run this weird script on a new buildstream project?" | 15:29 |
jennis | ok fair enough | 15:29 |
finn | ok, what about I make this docker patch my task on Tuesday | 15:29 |
tlater | That would be awesome :) | 15:29 |
finn | and jennis - write the docs as if this example existed | 15:29 |
jennis | + 1 | 15:29 |
finn | I can send you it if you really want it now | 15:29 |
jennis | No, it's ok :) | 15:30 |
finn | I can tar it up | 15:30 |
finn | ok | 15:30 |
*** tiago has joined #buildstream | 15:36 | |
*** tiago has quit IRC | 15:39 | |
tlater | ^ As an alternative to fixing the docker script, we should probably also discuss adding a project-directory variable to core buildstream | 15:57 |
tlater | Will probably have to wait until we're at a sensible time to talk to tristan, though ;) | 15:58 |
* tlater sets up an issue for this in the mean time | 15:58 | |
gitlab-br-bot | buildstream: merge request (jmac/virtual_directories->master: WIP: Abstract directory class and filesystem-backed implementation) #445 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/445 | 16:02 |
gitlab-br-bot | buildstream: merge request (jmac/virtual_directories->master: WIP: Abstract directory class and filesystem-backed implementation) #445 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/445 | 16:02 |
*** dominic has quit IRC | 16:05 | |
gitlab-br-bot | buildstream: issue #388 ("Allow including sources besides `local` from the project directory") changed state ("opened") https://gitlab.com/BuildStream/buildstream/issues/388 | 16:20 |
tlater | finn: I think I managed to make a convincing case there :) | 16:20 |
tlater | (This is all assuming it doesn't currently exist, but I can't find anything suggesting so, and nobody has told me it does yet) | 16:22 |
*** finn has quit IRC | 16:23 | |
gitlab-br-bot | buildstream: merge request (jmac/virtual_directories->master: WIP: Abstract directory class and filesystem-backed implementation) #445 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/445 | 16:25 |
*** jennis has quit IRC | 16:47 | |
*** aday has quit IRC | 16:50 | |
*** aday has joined #buildstream | 16:50 | |
*** bethw has quit IRC | 17:10 | |
gitlab-br-bot | buildstream: merge request (135-expire-artifacts-in-local-cache->master: WIP: Resolve "Expire artifacts in local cache") #347 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/347 | 17:20 |
*** xjuan has quit IRC | 20:25 | |
*** aday has quit IRC | 21:22 | |
*** aday has joined #buildstream | 21:23 | |
*** cs_shadow has quit IRC | 21:55 | |
*** aday has quit IRC | 22:17 | |
*** tristan has quit IRC | 23:38 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!