| *** tristan has joined #buildstream | 05:06 | |
| *** ChanServ sets mode: +o tristan | 05:11 | |
| *** tlater has joined #buildstream | 07:49 | |
| *** ssam2 has joined #buildstream | 08:30 | |
| *** ssam2 has quit IRC | 08:34 | |
| *** ssam2 has joined #buildstream | 08:35 | |
| *** jonathanmaw has joined #buildstream | 08:50 | |
| *** tiagogomes has joined #buildstream | 09:11 | |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 1 commit (last: _sandboxbwrap.py: Fix race condition when removing bwrap devices) https://gitlab.com/BuildStream/buildstream/commit/e72fb745e2c4d2934988403ea3de48d697672a3e | 09:20 |
|---|---|---|
| tristan | tlater, you need to fix your MR branch | 09:27 |
| tristan | tlater, it looks like you have some weird merges on your branch | 09:28 |
| tlater | I'll have a look in a bit, I had to fix a few commits because they included work on the source-bundle command | 09:29 |
| tristan | tlater, what you are going to need to do is: git remote add upstream git@gitlab.com:BuildStream/buildstream.git | 09:29 |
| tristan | git fetch upstream | 09:29 |
| tristan | git checkout master | 09:29 |
| tristan | git rebase upstream/master | 09:29 |
| tristan | git checkout except | 09:30 |
| tristan | git rebase master | 09:30 |
| tristan | and fix the conflicts | 09:30 |
| tristan | tlater, in other words, your proposed MR branches always have to be composed of only a clean series of commits ahead of upstream/master | 09:30 |
| tlater | I thought I did. Might have been the wrong branch | 09:31 |
| jonathanmaw | tristan: part of dpkg build elements is generating the split rules from what's been built. | 09:36 |
| jonathanmaw | looking at how the splits are stored (Element.__splits), it seems there won't be a simple way of altering it | 09:37 |
| jonathanmaw | my first idea was to give element.py a set_public_data method | 09:37 |
| jonathanmaw | but it seems I'd also have to call __init_splits() afterwards, anyway. | 09:38 |
| jonathanmaw | which if I understand python class idioms correctly, I should only use members with __ prefixed in the classes that own them | 09:38 |
| tristan | jonathanmaw, that's the part I wanted you to set aside for now right ? | 09:40 |
| tristan | jonathanmaw, dont bother with that part... it needs artifact refactoring first | 09:40 |
| tristan | jonathanmaw, what we're going to do, is allow the assemble method to set_public_data() before returning, and the final public data will be recorded in the artifact | 09:41 |
| jonathanmaw | tristan: ok, should I go straight into mangling __splits for my own purposes? | 09:41 |
| tristan | jonathanmaw, when the next element builds, it will have to reload some state taking artifact encoded metadata into account | 09:41 |
| tristan | jonathanmaw, I dont think you should do that at all no. | 09:42 |
| tristan | jonathanmaw, instead, you should go ahead and try to create a packaging element, with hardcoded data encoded into the split-rules | 09:42 |
| tristan | i.e. assume that all that is all taken care of | 09:42 |
| tristan | so you edit foo-build.bst and add stuff to the public split data | 09:42 |
| tristan | then you run foo-package.bst to package that element | 09:43 |
| jonathanmaw | ok | 09:43 |
| tristan | I had better implement globbing on the split rules before we start using them too much | 09:43 |
| tristan | juergbi, ssam2 what do you guys think about that too ? | 09:44 |
| tristan | My thoughts are that regexes are write-only read-never crazy crap that normal humans dont understand, including seasoned developers | 09:44 |
| tristan | and if we're wildcarding filename paths, glob is a much more human friendly syntax for that | 09:44 |
| ssam2 | hmm | 09:45 |
| tristan | Also, I have code which does proper globbing on lists of filenames in python | 09:45 |
| ssam2 | globs for spit rules would be fine I think | 09:45 |
| juergbi | yes, globs make more sense for custom split rules | 09:45 |
| juergbi | power of regexps shouldn't be needed for that | 09:45 |
| tristan | (the default python glob module only works with a filesys in context, but I have something already in tarball source plugin) | 09:45 |
| tristan | Ok good, we agree on that :) | 09:46 |
| ssam2 | I've not seen any rules in Baserock that actually did more than /usr/lib64/.* | 09:46 |
| ssam2 | another concern is how they override each other | 09:46 |
| tristan | ssam2, look at glibc | 09:46 |
| tristan | ssam2, that is actually not a concern, intentionally | 09:46 |
| tristan | ssam2, in otherwords, there is no rule that split rules may not overlap with eachother | 09:47 |
| ssam2 | ok, so in the case of overlaps how do you decide where to put a file ? | 09:47 |
| tristan | well, first, split rules is public data, so any element can do whatever they want with it | 09:48 |
| tristan | but I think your question is mostly regarding the compose element | 09:48 |
| tristan | which is the only element in buildstream which uses it :) | 09:48 |
| ssam2 | right | 09:48 |
| tristan | https://buildstream.gitlab.io/buildstream/elements/compose.html#module-elements.compose | 09:48 |
| tristan | ssam2, I think it's pretty straightforward docs, is there something still unclear after looking at that ? | 09:49 |
| ssam2 | hmm | 09:49 |
| ssam2 | so if I want to produce foo-bin and foo-devel, I'd have *two* compose elements, one for each ? | 09:49 |
| ssam2 | and foo-bin would include whatever domains existed in the elements.. yeah seems fine | 09:49 |
| tristan | ssam2, correct | 09:50 |
| tristan | ssam2, however I suspect that the debian packaging element for instance, will also consume split rules on it's own terms | 09:50 |
| ssam2 | OK cool. This sounds much more easy to reason about than the Baserock system, anyway | 09:50 |
| ssam2 | which was the main issue with it ... i could never actually figure it out :-) | 09:50 |
| tristan | so you wont need to compose things before feeding composed things to a dpkg creating element | 09:50 |
| tristan | ssam2, we have a thread about it's brokenness | 09:51 |
| ssam2 | right, I guess compose would be for things like the Freedesktop 'Base' and 'SDK' sysroots ? | 09:51 |
| tristan | I concluded that thread with a statement that nobody has actually been able to demonstrate that it actually ever worked, ever | 09:51 |
| tristan | And Paul made a smart alec retort that I was not able to prove that it never did :) | 09:52 |
| tristan | However it's irrelevant | 09:52 |
| tristan | ssam2, yeah compose will be useful to GNOME for composing flatpak runtimes indeed | 09:53 |
| tristan | ssam2, but we also use it for composing an initramfs system (integrated with some fat cut out of it) | 09:53 |
| tristan | and for composing a full system to feed to an x86image element (we want a staged, *integrated* system in a sub directory where we can wrap it all up into a bootable image) | 09:54 |
| ssam2 | ah yes | 09:54 |
| ssam2 | I even copied that initramfs.bst definition but obviously didn't read it enough :-) | 09:55 |
| tristan | now you know what it does :) | 09:55 |
| gitlab-br-bot | buildstream: merge request (except->master: WIP: Add a --except flag to show, fetch and track) #26 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/26 | 10:10 |
| tristan | nice, the storaged-project devs ate up my patches quickly | 10:35 |
| gitlab-br-bot | buildstream: merge request (except->master: Add a --except flag to show, fetch and track) #26 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/26 | 10:36 |
| * tristan now has a very strange bug | 10:36 | |
| tristan | It seems phodav builds fine in jhbuild but craps out in buildstream sandbox | 10:37 |
| tristan | and the really weird thing... is that it craps out on autogen.sh, before doing anything | 10:37 |
| tlater | tristan: Is the MR ok like that? | 10:37 |
| tristan | I am between things :) | 10:54 |
| tristan | lemme see | 10:55 |
| *** tristan has quit IRC | 11:54 | |
| *** tristan has joined #buildstream | 12:54 | |
| *** ChanServ sets mode: +o tristan | 12:56 | |
| *** tlater has quit IRC | 13:18 | |
| *** tlater has joined #buildstream | 13:19 | |
| tristan | oh holy crap | 13:34 |
| tristan | what did I do | 13:34 |
| tristan | nah, I dont think I screwed it up okay | 13:35 |
| tristan | whew | 13:35 |
| tristan | tlater, ok I dont know what your doing there, but you dont have a proper branch | 13:38 |
| tristan | it's all screwed with merges | 13:38 |
| tlater | :( | 13:38 |
| tristan | tlater, just never, ever, run "git merge" | 13:38 |
| tristan | always rebase | 13:38 |
| tristan | I just fetched again | 13:38 |
| tristan | and when I checkout the except branch and try rebase against master, it conflicts right away on: Applying: WIP: add `bst source-bundle` command | 13:39 |
| tristan | that commit should not even be on the branch | 13:39 |
| tristan | tlater, you can see there is something wrong right away looking at https://gitlab.com/BuildStream/buildstream/merge_requests/26 ... the merge request has 68 (!!) commits | 13:40 |
| tristan | tlater, at this point, I think you screwed your local workflow somehow, so I suggest you do something a bit different | 13:41 |
| tristan | oh, crap, ok so... you have commits on your fork's master, even | 13:41 |
| tristan | ouch | 13:41 |
| tlater | I've always merged, discovered rebase the other day... This should still be fixable, I think. | 13:42 |
| tristan | tlater, not sure exactly how you are going to untangle it but I suggest you start over... i.e.: locally keep your git repo, delete your gitlab fork and create a new one | 13:42 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 3 commits (last: sandbox.py: Added INTERACTIVE sandbox flag) https://gitlab.com/BuildStream/buildstream/commit/640d014d404198bd9d074cc88914989e636c710a | 13:42 |
| tristan | tlater, then, in your local checkout of your new gitlab fork, first *create a branch* (never commit to your fork's master, that'll break your workflow) | 13:43 |
| tristan | and then what you can do is restage new commits from your old repo that you keep on hand | 13:43 |
| tristan | tlater, I have a feeling that will be easier than trying to deal with the franken-git you have created | 13:44 |
| tlater | And then rebase the result? | 13:44 |
| tristan | It will already be rebased since it's a new fork | 13:44 |
| tlater | Well, I'll do that. Should hopefully not be too bad. | 13:45 |
| tristan | tlater, the point is that you always work on branches that you regularly rebase against upstream master | 13:45 |
| tlater | I did that - unfortunately I regularly merged with upstream master, instead of rebasing. | 13:45 |
| tristan | so, you will need that 'git remote add upstream <upstream-buildstream-url>' stuff I showed you above | 13:45 |
| tristan | and all of your work goes into local *branches* | 13:46 |
| tristan | you can regularly get *your* master in sync with upstream master, by doing `git fetch upstream && git checkout master && git rebase upstream/master && git push origin master` | 13:47 |
| tristan | that'll update your remote gitlab repo with the upstream changes in master | 13:47 |
| tlater | On the upside, realizing rebase exists is going to fix a lot of issues I've had in the past. | 13:47 |
| tristan | in fact, in that case, you can do `git merge upstream/master` and it will be equivalent, as long as you never merge *your* branches into *your* master | 13:48 |
| tristan | tlater, ok... so since you learned rebase... note that in your own wip branches, before submitting MRs, you should probably normally do `git rebase --interactive master` | 13:48 |
| tristan | tlater, that is one of the coolest things that git does for you | 13:48 |
| tristan | lets you prepare a sane history and remove/squash commits that are not sane for master's history, before submitting | 13:49 |
| tristan | I.e. we want a few, comprehensive commits in a merge request, not 20 commits that include reverts of things you were trying out while developing | 13:49 |
| tristan | tlater, I think what might help you stage new commits from your old git would be to, from your old git, do the `git remote add upstream && git fetch upstream`... and then, from your except branch, run `git diff upstream/master` | 13:54 |
| tristan | that will give you both changes which occurred in master and in your branch | 13:55 |
| tristan | but might be helpful | 13:55 |
| tristan | or maybe you have intimate knowledge enough of which patches/commits should be there and which should not | 13:55 |
| ssam2 | tristan, it seems that utils._relative_symlink_target() is causing an issue | 13:56 |
| ssam2 | what problem is that function there to solve? | 13:56 |
| tristan | ssam2, could be, that thing is a bit wild, what's the issue ? | 13:56 |
| ssam2 | urgh, the issue is also wild | 13:57 |
| tristan | ssam2, it ensures that we never stage any absolute path symlinks | 13:57 |
| ssam2 | basically my /lib in the armv8l64 rootfs contains this: | 13:57 |
| ssam2 | ld-linux-aarch64.so.1 -> ../tools/lib/ld-linux-aarch64.so.1 | 13:57 |
| ssam2 | now, that would be fine except /lib is actually a symlink to /usr/lib | 13:57 |
| tristan | Right | 13:57 |
| ssam2 | and from /usr/lib the relative symlink doesn't work any more | 13:57 |
| tristan | Ah interesting | 13:58 |
| tristan | ssam2, Ok so, I have a similar issue | 13:58 |
| ssam2 | this doesn't hit other platforms I guess cus they use /lib64, which isn't a symlink | 13:58 |
| ssam2 | i guess x86_32 must also suffer this, though | 13:58 |
| tristan | which is that /usr/sbin/init is a symlink to /usr/lib/systemd or similar | 13:58 |
| tristan | and then the link doesnt make sense at boot time | 13:58 |
| tristan | ssam2, do you think that using a realpath expansion on the target might work ? | 13:59 |
| ssam2 | i can try it | 13:59 |
| tristan | like, before relativ-izing it ? | 13:59 |
| ssam2 | hmm, no because we're not in a sandbox here | 14:00 |
| ssam2 | so os.realpath(target) gets /mnt/build/src/definitions/usr/lib | 14:00 |
| tristan | ssam2, fwiw, that code comes originally from YBD, without ensuring relative symlinks, YBD (running as root), silently overwrites things on your host while staging things | 14:00 |
| ssam2 | but I guess we could snip off the root afterwards | 14:00 |
| ssam2 | ha, i guess that's also bad | 14:01 |
| tristan | ssam2, yes, that is why I said *before relativizing it* | 14:01 |
| tristan | but it is a bit hairy in there I admit :) | 14:01 |
| gitlab-br-bot | buildstream: merge request (except->master: Add a --except flag to show, fetch and track) #26 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/26 | 14:12 |
| tlater | There we go. I just used cherry pick to apply the commits I wanted. Not the first time I've done this - I just always stupidly thought things got messy if you lag behind the main branch. | 14:13 |
| tristan | tlater, thats not stupid at all, just keep your main branch clean and dont put any commits on it | 14:19 |
| tristan | anyway, its good to constantly rebase against master (and keep your master up to date with upstream/master) | 14:21 |
| tlater | Yeah, this is going to change my workflow quite significantly. Either way, the MR is up (without merges this time), have a peek when you have time :) | 14:23 |
| tristan | ok, 10 commits is better | 14:26 |
| tristan | tlater, now you can do `git rebase --interactive master` in your except branch | 14:26 |
| tristan | tlater, there is no need for "Fix styling issues" commits in master | 14:27 |
| tlater | What do I do to those? | 14:27 |
| tristan | tlater, i.e. you should be able to compose a history that just does what you intend to submit, not all this history of fixups and stuff | 14:27 |
| tristan | eh, well in that case, since you have commits which go and modify many files at once | 14:28 |
| tristan | I would do a soft reset and reconstruct the branch from scratch | 14:28 |
| tlater | This could just be one commit, in this case | 14:28 |
| tristan | tlater, to make sure you dont get stuck in a corner... create a temp branch first | 14:29 |
| tristan | like | 14:29 |
| tristan | git checkout except && git checkout -b temp | 14:29 |
| tristan | then: git reset master | 14:29 |
| tristan | now you have all your changes against master unstaged | 14:29 |
| tristan | git diff --stat | 14:29 |
| tlater | I can also just squash them all | 14:29 |
| tristan | ok, git commit buildstream/_pipeline.py -m "Adding some API to pipeline ..." | 14:29 |
| tlater | And give them a better name | 14:30 |
| tristan | squash all into one commit ? | 14:30 |
| tristan | a bit yuck | 14:30 |
| tristan | it would be one huge commit with https://gitlab.com/BuildStream/buildstream/merge_requests/27/diffs | 14:30 |
| tristan | tlater, I think this MR is 3 commits logically | 14:31 |
| tristan | 1.) Add some API to the pipeline | 14:31 |
| tristan | 2.) Update frontend commands, removing the old unused function | 14:31 |
| tristan | 3.) Added test case | 14:31 |
| tlater | Ok, that's probably better | 14:31 |
| tristan | the code looks pretty fine fwiw, but lets get used to creating a nice history :) | 14:32 |
| tristan | it pays off in the long run to have a nice comprehensive history | 14:33 |
| tlater | Messy histories have been the bane of every project I've done by myself. This git command is a revelation to me. | 14:34 |
| tlater | Alright, so this should hopefully not be a problem anymore. | 14:51 |
| tlater | MR is up :) | 14:51 |
| ssam2 | yay! armv8l64 sysroot now has a working native compiler | 14:57 |
| ironfoot | ssam2: :D | 15:07 |
| jonathanmaw | Is there a good way of testing whether my split-rules have been applied? | 15:08 |
| tristan | jonathanmaw, I have to run but... compose element ? | 15:09 |
| jonathanmaw | tristan: aha | 15:10 |
| *** tristan has quit IRC | 15:13 | |
| gitlab-br-bot | buildstream: merge request (except->master: Add a --except flag to fetch, track and show) #27 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/27 | 15:23 |
| ssam2 | bah, scripts/config from linux.git requires bash | 15:55 |
| ssam2 | my plan for a stage2-linux.bst is perhaps foiled | 15:56 |
| ssam2 | ah wait, it's my host tools that need Bash | 15:57 |
| ssam2 | so that's easier | 15:57 |
| gitlab-br-bot | buildstream: merge request (except->master: Add a --except flag to fetch, track and show) #27 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/27 | 16:31 |
| gitlab-br-bot | buildstream: merge request (except->master: Add --except to fetch, track and show) #28 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/28 | 16:37 |
| *** tlater has quit IRC | 16:41 | |
| *** jonathanmaw has quit IRC | 17:01 | |
| *** ssam2 has quit IRC | 17:08 | |
| *** tristan has joined #buildstream | 18:02 | |
| *** ChanServ sets mode: +o tristan | 18:06 | |
| gitlab-br-bot | buildstream: merge request (sam/symlinks-fix->master: utils.py: Fix corner case in relative_symlink_path() that breaks builds) #29 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/29 | 18:25 |
| gitlab-br-bot | buildstream: merge request (jonathan/fix-bzr-source->master: bzr source: Make sure the stage directory exists before bzr fetching) #30 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/30 | 19:38 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!