| *** tristan has joined #buildstream | 08:03 | |
| *** tlater has joined #buildstream | 08:28 | |
| tlater | Morning o/ | 08:36 |
|---|---|---|
| tlater | My merge request is up, tristan: https://gitlab.com/BuildStream/buildstream-tests/merge_requests/8 | 08:38 |
| *** jonathanmaw has joined #buildstream | 08:50 | |
| *** ssam2 has joined #buildstream | 09:21 | |
| tristan | tlater, why the extra leading space in 'image: samthursfield/buildstream:0.1-20170621.1' ? | 09:24 |
| tlater | Oh, whoops, I used that to trigger the CI when it was stuck at some point | 09:26 |
| tlater | The space is gone now :) | 09:53 |
| jonathanmaw | tristan: looks like I need a moderator to approve my subscription to buildstream-list | 09:57 |
| jonathanmaw | AIUI only you or jjardon can approve me | 09:57 |
| ssam2 | that doesn't seem right | 09:59 |
| ssam2 | it should be a public mailing list | 09:59 |
| ssam2 | what's the exact issue you're getting? | 09:59 |
| tristan | jonathanmaw, didnt I reply to your email ? | 10:03 |
| tristan | jonathanmaw, anyway, I tried to login, jjardon[m] created the ML and I dont know how to login to admin | 10:04 |
| tristan | just please subscribe to the list first, then you wont have any problems | 10:04 |
| jonathanmaw | tristan: I tried to subscribe and it seems I'm still waiting approval. I've twice tried to send an E-mail to be notified that it's subscriber-only. | 10:04 |
| tristan | really ?? | 10:05 |
| tristan | that is weird | 10:05 |
| tristan | no no... it is *certainly* subscriber only | 10:05 |
| tristan | but approval is automatic | 10:05 |
| ssam2 | emails that you sent before you subscribed won't be automatically approved just because you subscribed | 10:07 |
| ssam2 | probably easiest to resend them once you have subscribed to the list | 10:07 |
| tristan | ssam2++ | 10:09 |
| jonathanmaw | ssam2: afaict I am in some kind of limbo between being subscribed and not | 10:09 |
| jonathanmaw | since I got an E-mail that someone (me) was trying to subscribe me to the mailing list I was already on. | 10:09 |
| ssam2 | weird | 10:10 |
| ssam2 | tristan, you could possibly ask for the list admin password in #sysadmin | 10:10 |
| ssam2 | since you are an admin | 10:10 |
| ssam2 | jonathanmaw you should be able to check your subscription status at https://mail.gnome.org/mailman/listinfo/buildstream-list ... | 10:11 |
| ssam2 | or try unsubscribing and resubscribing or something | 10:11 |
| * jonathanmaw tries unsubscribing | 10:12 | |
| tristan | juergbi, so I thought on that more and as you see, wrote too many emails, sorry :) | 10:16 |
| juergbi | processing now ;) | 10:17 |
| tristan | juergbi, the one I sent today contains thoughts I was having while running on the treadmill at ~1:30am... I have a feeling we're missing an ingredient, which is the related project that is being built | 10:17 |
| tristan | So first, yeah; I think we're over paranoid with the tunnel vision that "This can be built against any version of these dependencies", because it omits the part that, in any case the project will dictate exact revisions of everything | 10:18 |
| tristan | However | 10:18 |
| tristan | It is also true, that you may have multiple projects, which build the same thing against entirely different revisions of things | 10:19 |
| tristan | that is a bit mind boggling to think about | 10:19 |
| tristan | It renders it literally impossible to even have a concept of "Which weak key artifact is /closest/ or /most recent/ for my artifact" | 10:20 |
| tristan | although I'm not sure it matters in practice | 10:20 |
| tristan | Ah | 10:20 |
| tristan | scratch that; in *any* case, at least artifacts are stored within a "project name" namespace | 10:20 |
| tristan | that helps things a bit | 10:20 |
| tristan | I think | 10:21 |
| juergbi | yes, it's possible that i'm a bit paranoid about this. i just want to make sure we're not skipping a more reliable solution | 10:22 |
| tristan | I admit I dislike the idea of adding more than one 'ref' to sources... especially considering that a "ref" is an abstract thing which a source only names "ref" by convention | 10:23 |
| tristan | but also, I'm not sure if I've thoroughly understood (or if you have either) exactly what you are proposing | 10:24 |
| juergbi | btw: i wouldn't add a second tracking reference. you would typically keep both refs on the branch you're tracking | 10:25 |
| jonathanmaw | I unsubscribed, and re-subscribed. I've got the Welcome to the "Buildstream-list" mailing list E-mail and everything | 10:31 |
| jonathanmaw | and my third try also needs moderator approval | 10:31 |
| tristan | :-/ | 10:33 |
| juergbi | the mailman admin interface should show why | 10:33 |
| tristan | juergbi, yeah, exactly :-/ | 10:33 |
| tristan | sigh, so off to #sysadmin to ask for a password then | 10:34 |
| tristan | jjardon[m], do you have that password, though ? | 10:34 |
| tlater | tristan: Is the CI finished now? | 10:34 |
| tristan | tlater, \o/ | 10:35 |
| tristan | lets move on !! | 10:35 |
| tristan | I havent seen it run on buildstream repo yet (with latest changes) | 10:35 |
| tristan | but should be good | 10:35 |
| tlater | I just did | 10:35 |
| tlater | It passed \o/ | 10:35 |
| tristan | nice: https://gitlab.com/BuildStream/buildstream/pipelines/9600389 | 10:36 |
| tristan | https://gitlab.com/BuildStream/buildstream/-/jobs/21310273 | 10:36 |
| tristan | cute | 10:36 |
| tlater | So, for developer workspaces, should this be another command that creates a local copy of an element's source and allows the user to edit and build that as a a replacement for the original? | 10:38 |
| tristan | tlater, https://wiki.gnome.org/Projects/BuildStream/Roadmap/Workspaces | 10:38 |
| tristan | basically yes | 10:38 |
| tristan | tlater, so long as the workspace is "open", the workspace should override the element source(s) | 10:39 |
| tristan | The part that is mostly unclear here to me... is what do we do for elements with more than once source ? | 10:39 |
| tlater | We'd just need to check out all those sources, no? | 10:40 |
| tristan | Should the workspace be Source based instead of Element based, and have a simple syntax for elements with only one source (and otherwise require a full specification if the element has more than one source) ? | 10:40 |
| tristan | there are a lot of facets to this | 10:41 |
| tlater | There's also the question what happens when multiple elements depend on the same source... So yeah, it can't be element based. | 10:42 |
| tristan | tlater, first of all, for a vcs based source; ideally I want the checked out source to be routed to the real origin (I think we discussed this) | 10:42 |
| tristan | tlater, on that I think you are wrong | 10:42 |
| tristan | i.e. multiple elements -> same source | 10:42 |
| tlater | Why can't that happen? | 10:42 |
| tristan | in the data model, even if they happen to be the same upstream source, they are separate sources | 10:42 |
| tristan | elements own their sources, they do not refer to a shared source | 10:43 |
| tristan | it's a rather corner case | 10:43 |
| tlater | Would it make sense to share a source for the developer workspace? | 10:43 |
| tristan | but there is another side effect of that: https://gitlab.com/BuildStream/buildstream/issues/5 | 10:43 |
| tristan | I personally feel like that is going to add complexity to the thing, for relatively small added benefit | 10:44 |
| tlater | Well, if it ever turns out to be useful that can be added later, I suppose. | 10:45 |
| tristan | tlater, however; one way you can look at it is... for instance if I say `bst workspace open <element.bst> <directory>` | 10:46 |
| tristan | then say, by default it will try to put the source in that directory | 10:46 |
| tristan | but it could also have `bst workspace open --no-checkout <element.bst> <directory>` | 10:46 |
| tristan | tlater, i.e. you could hardwire yourself, two separate elements to have a workspace opened on the same linux directory ? | 10:47 |
| tlater | Yeah, that works | 10:47 |
| tlater | I guess technically you could also have a --force flag which just uses that directory for the source even if it contains something | 10:48 |
| tristan | In that case, we need to be careful to copy files into the staging area (and not hardlink them) | 10:48 |
| tristan | in case files are being modified by parallel builds, we wouldnt want that happening | 10:48 |
| tlater | That's almost a bit unfortunate, but probably easier than figuring out if the source is shared. | 10:49 |
| tlater | You also mentioned you wanted bst to keep track of changes on top of whatever VCS may be part of the source? | 10:50 |
| tristan | yeah anyway we already avoid hardlinks at source staging time | 10:50 |
| tristan | the git source has an explicit --no-hardlinks thing | 10:50 |
| tristan | staging sources is not really a bottleneck | 10:50 |
| tristan | tlater, nah... we discussed that but, we want instead to just generate the Source's unique key based on a recursive directory checksum | 10:51 |
| tristan | that will 'just work' for any kind of source | 10:51 |
| tristan | So asides from all of this | 10:52 |
| tristan | tlater, I think that we will need to start storing data in `<project-directory>/.bst/` | 10:52 |
| tristan | I'm not sure what else we could possibly want this for at the moment, but we'll want to store data about which workspaces are open (and where) in say: <project-dir>/.bst/workspaces | 10:53 |
| tristan | it cant really go into ~/.cache/buildstream | 10:54 |
| jjardon[m] | tristan: sorry, what password? | 10:56 |
| tlater | Password for the buildstream mailing list admin interface | 10:59 |
| tristan | jjardon[m], yeah that one... having trouble with jonathanmaw's subscription here | 11:01 |
| * tristan looked in his emails for 'buildstream-list' but did not find any "Welcome, you are an administrator, here is your password" kind of email | 11:01 | |
| tristan | the list says I'm an admin, though heh | 11:01 |
| tlater | tristan: Should the `bst workspace` command check out all sources of an element or allow picking individual ones? | 11:05 |
| tristan | tlater, so yeah I was thinking on that... others please feel free to chime in... | 11:05 |
| tristan | my feeling right now is that it should be individual sources | 11:05 |
| tristan | So... I think that for instance, you could optionally specify `--source 0` to specify the first source of an element | 11:06 |
| tristan | If an element has only one source (99% of the time that is the case anyway)... then you dont need to specify it | 11:06 |
| tristan | If an element has more than once source, and you fail to specify that, then bst errors out and says "Tell me which source you want, you ambiguous dimwit !" | 11:07 |
| tlater | There's no way to give the sources names, is there? | 11:07 |
| tristan | Nah sources are unnamed | 11:07 |
| tristan | There could, possibly, be a way to do it with the url, but even that is not reliable | 11:08 |
| tlater | Might share the url with a different ref, I suppose. | 11:09 |
| tristan | because that is again, only named 'url' by convention | 11:09 |
| tristan | So also... now I've said this a few times... but I'm starting to be skeptical about it... | 11:10 |
| tristan | I *want* a checked out source to be routed to the real origin, in the case of a VCS | 11:10 |
| tristan | I mean, the user would want that by default | 11:10 |
| tristan | but | 11:10 |
| tristan | is it that bad if we dont do it ? | 11:10 |
| tlater | It's a bit of a pain to route it manually, if needed. | 11:11 |
| tristan | Because as it stands, we can easily just stage the source into a directory, and we dont have to modify the Source API or anything | 11:11 |
| tristan | a bit of a pain yes | 11:11 |
| tristan | git remote set-url origin <url> | 11:11 |
| tristan | tlater, I'm also considering how painful it would be to support for every possible source type, though | 11:12 |
| tristan | so we mirror things from their origin, and we do so if possible with bare repos | 11:12 |
| tristan | and we want to move checkout from that, and reroute it's origin | 11:13 |
| tristan | is that going to be easy to do for bzr.... ? .... or an eventual svn or cvs ? | 11:13 |
| tristan | tlater, ok so here's what I think we do... we leave that part out | 11:14 |
| tristan | tlater, because implementing that is quite orthogonal to the rest of the work | 11:14 |
| tristan | So, if it turns out we can do it and its very desirable, we can do it as an enhancement, without effecting the core design of workspaces really | 11:14 |
| tristan | Makes sense ? | 11:14 |
| tlater | Yeah | 11:14 |
| tristan | jonathanmaw, I got into the admin.... | 11:15 |
| tristan | jonathanmaw, I approved your last post, and discarded the first two | 11:17 |
| tristan | jonathanmaw, but looking at the membership, nothing looks odd about your subscription | 11:17 |
| tristan | the only thing I can see different from the others, is you dont have a "name" associated to your email address | 11:19 |
| * tristan puts your name | 11:19 | |
| tlater | Do multiple workspaces of the same source make sense? If only one can be "open" at a time that won't cause issues, I don't think. | 11:22 |
| persia | If a user is expected to be able to switch between workspaces (e.g. having different workspaces for different integration projects), then I would assert the answer is "yes". There are a number of situations when working on two different systems that a user would want to have two different locations for a source with the same identifier (and perhaps local modifications, although, as noted above, that may affect things). | 11:32 |
| tristan | tlater, I think the same approach we discussed before addresses this: I can have two workspaces for 'linux' and for 'linux-api-headers' open to the same directory, or I can have them open to separate directories, it's my choice | 11:34 |
| tristan | tlater, also, certainly I want to be able to open multiple workspaces | 11:35 |
| tristan | And also, there is no issue if I have separate checkouts of buildstream projects, which open workspaces to the same source directory | 11:35 |
| tristan | of course, I may hurt myself in the process | 11:35 |
| tristan | tlater, one thing which certainly should be done, is in the beginning of a build, the summary should mention what workspaces are open and point to what local directories (if I have any open workspaces) | 11:36 |
| tlater | I specifically meant two checkouts of the same source in the same element - it gets ambiguous which one should be used for the build. | 11:36 |
| tristan | Ah | 11:36 |
| tristan | No, only one | 11:36 |
| tristan | tlater, I think this can be handled with sane command line options to the workspace commands | 11:37 |
| tristan | for instance... when closing the workspace, perhaps you want an explicit --delete option if you expect it to actually remove the directory | 11:37 |
| tristan | but by default, leave the user's work in tact | 11:38 |
| tristan | and then when calling `bst workspace open` and the element/source in question is already opened, bail with a warning, unless --force is specified | 11:38 |
| tristan | something like that | 11:39 |
| tlater | Alright | 11:39 |
| tlater | Why only one though? | 11:39 |
| tlater | If we can have multiple closed workspaces, but only one that's open, we get around the ambiguity | 11:39 |
| tristan | it's less information to manage, and does not deter from the UX afaics | 11:39 |
| tlater | Okay :) | 11:40 |
| tristan | tlater, my view of a closed workspace, is a workspace that buildstream knows nothing about anymore | 11:40 |
| tristan | it may be a directory the user has on disk | 11:40 |
| tristan | but nothing more | 11:40 |
| tlater | No re-opening? | 11:40 |
| tristan | tlater, that would be `bst workspace open --no-checkout <element> <directory>` | 11:40 |
| tristan | or such | 11:40 |
| tristan | would be the same, basically just opening a workspace on a directory you already have | 11:40 |
| tlater | Perhaps add a --reopen that does the same thing, but only if the workspace has been closed before. | 11:41 |
| tristan | it may just be a matter of wording, but --reopen and --open-on-this-directory-i-prepared-myself... is the same thing | 11:41 |
| tristan | makes things simpler | 11:41 |
| tristan | tlater, avoid tracking too much state | 11:42 |
| tlater | I'd think --reopen makes more sense from the user's perspective. | 11:42 |
| tristan | but implies we have to remember about a closed workspace | 11:42 |
| tlater | I guess adding a note 'can be used to "re-open" a closed workspace' to the help menu might achieve the same thing. | 11:42 |
| tristan | sure | 11:43 |
| tristan | tlater, and of course, load/save everything in yaml :) | 11:44 |
| tristan | yaml all the way | 11:44 |
| tlater | Heh, of course :) | 11:47 |
| gitlab-br-bot | push on buildstream@test-host-target (by Tristan Van Berkom): 2 commits (last: Add --host-arch and --target-arch, and 'host-arches' conditional) https://gitlab.com/BuildStream/buildstream/commit/039062218957bdae6473b482adc8800fd9be7ed3 | 12:10 |
| tristan | ssam2, I'm running it through the CI one time and want to merge it... | 12:14 |
| tristan | ssam2, can you also please prepare a separate patch to update: https://buildstream.gitlab.io/buildstream/format.html#architecture-conditionals ? | 12:15 |
| tristan | ssam2, perhaps a subsection in there that briefly outlines 'host-arches' prefixed with a sort of 'normally you dont have to worry about this' | 12:16 |
| ssam2 | doh, i thought i did that | 12:17 |
| ssam2 | ah, I did but very briefly | 12:17 |
| ssam2 | There is also a ``host-arches`` attribute, which operates in the same manner but | 12:17 |
| ssam2 | follows the *host* architecture rather than the *target* architecture. | 12:17 |
| ssam2 | ...i could expand that I guess | 12:17 |
| tristan | oh did I miss that in the patch ? sorry | 12:18 |
| * tristan was using git show to look through it and didnt see it there | 12:18 | |
| tristan | ssam2, ehh... it's not amazing but it's there :) | 12:20 |
| persia | Just to check, does BuildStream distinguish between all of "host architecture", "build architecture", and "target architecture"? | 12:20 |
| tristan | feels a bit terse and misguiding | 12:20 |
| tristan | persia, currently no, only host and target really matter I think from buildstream's perspective | 12:20 |
| ssam2 | yeah.. i didn't know how to explain further without going much deeper into detail | 12:21 |
| tristan | persia, afaics, it provides all the context which build instructions need to provide the three to the toolchain for a cross build | 12:21 |
| persia | In my past experiences, I've had many times I needed to do Canadian Cross. It isn't that different from basic cross support, but not adding it early makes it hard to untangle later, if something needs support. | 12:21 |
| tristan | and host-vs-build-vs-target is only really needed at the toolchain level | 12:21 |
| tristan | persia, ssam2 is technically doing a canadian cross (well it runs the same way if not technically called that), to cross build a native compiler | 12:22 |
| persia | Ah, so to do Canadian with BuildStream, one would have host as host, target as build, and then the system being built would be a cross compiler? | 12:22 |
| tristan | as I recall, there are 2 phases (before compiling the compiler again natively on the target) | 12:23 |
| tristan | you cross compile using an 'unknown' in your target triple | 12:23 |
| tristan | and use the 'unknown' triple to cross build the native | 12:24 |
| tristan | I guess buildstream would need another arch option, if it were required to build a real canadian cross | 12:26 |
| tristan | i.e. build in an x86_64 runtime; a cross compiler to run on i386 and produce arm | 12:27 |
| tristan | you'd need a way to specify that | 12:27 |
| tristan | but, I'm not sure we'll ever see the day that we require that, and if we do, we can add support (but, I hope not, I think cross building a native compiler for target should be enough) | 12:28 |
| gitlab-br-bot | buildstream: Tristan Van Berkom deleted branch test-host-target | 12:30 |
| persia | Sample use case is to use some linux server to build a system for delivery that runs on corporate desktops, and is used to build target systems. | 12:30 |
| gitlab-br-bot | push on buildstream@master (by Tristan Van Berkom): 2 commits (last: Add --host-arch and --target-arch, and 'host-arches' conditional) https://gitlab.com/BuildStream/buildstream/commit/039062218957bdae6473b482adc8800fd9be7ed3 | 12:30 |
| persia | Whether that's "real canadian" or not is trickier :) | 12:30 |
| gitlab-br-bot | buildstream: merge request (sam/host-and-target-arch->master: Add --host-arch and --target-arch, and 'host-arches' conditional) #34 changed state ("closed"): https://gitlab.com/BuildStream/buildstream/merge_requests/34 | 12:30 |
| ssam2 | the only use of --build at GCC level is to work around bad autodetection of the build machine | 12:30 |
| ssam2 | you obviously can't just pass --build=powerpc-... and magically turn your x86 machine into a POWER one | 12:31 |
| ssam2 | but we can work around that in the gcc.bst file, it wouldn't require the end user to pass in a flag to buildstream that we then pass on to GCC | 12:31 |
| ssam2 | or rather to GCC's configure script | 12:32 |
| persia | Well, you *can* magically change the arch of your machine with linux (qemu-static+binfmtmisc), but I was thinking more of the case where the BuildStream user needed to build a system that was itself a cross-compiler, perhaps on a different platform than that used by the cross-compiler users. | 12:34 |
| ssam2 | persia, sure but not by passing flags into GCC | 12:34 |
| ssam2 | right now I think it would the same way that it did with other Baserock build tools, i.e. separate .bst files for the cross toolchain | 12:35 |
| tristan | ssam2, you could achieve the same with variants | 12:36 |
| persia | Right. That makes sense. BuildStream only has "host+target": if the user wants something with a longer chain, the user would e.g. cross-compile a cross-compiler. | 12:36 |
| tristan | ssam2, and one set of bst files, have the variants define variables conditional to what you want to build | 12:36 |
| ssam2 | can you include 2 variants of the same .bst file in a single pipeline ? | 12:37 |
| ssam2 | if so, that should work indeed | 12:37 |
| tristan | ssam2, nope | 12:37 |
| tristan | ssam2, but you would anyway need two separate stages right ? | 12:37 |
| tristan | ssam2, first one is your regular cross compiler and second is the cross compiled cross compiler | 12:38 |
| ssam2 | yeah indeed | 12:38 |
| tristan | and those dont really go well into one reused element anyway | 12:38 |
| tristan | mkay, so finally figured out the sshd_config and am now getting a failure to load the OSTree typelib, which does not make sense because; buildstream loads it fine in the same environment | 12:54 |
| ssam2 | py2 vs py3 ? | 12:55 |
| tristan | nah, ostree-push is already python3 | 12:59 |
| tristan | ok | 13:19 |
| tristan | that works well | 13:19 |
| tristan | turns out, on my client, I was getting a traceback from the server | 13:19 |
| tristan | and the server needs ostree girs installed in /usr | 13:19 |
| tristan | or, needs to have GI_TYPELIB_PATH in the environment executing ostree-receive | 13:19 |
| tristan | but I dont know how to do that | 13:19 |
| tristan | Anyway | 13:20 |
| tristan | pushing the linker-priority.bst artifact, which was taking like 10 to 15 minutes to eventually fail with sshfd | 13:20 |
| tristan | completes in seconds with real ostree-push | 13:20 |
| tristan | good news | 13:20 |
| tristan | but I'll have to rework it | 13:21 |
| tristan | So, july 6th, hmmm | 13:21 |
| tristan | grind time :-/ | 13:21 |
| tristan | juergbi, alright, I'm going to try to nail that from now until monday | 13:22 |
| tristan | juergbi, I'm going to make the buildstream.conf specify 2 urls for artifact shares, one for pull and one for push; pull will be http[s] only, push will be ostree-push | 13:23 |
| tristan | cause, its just unusable for me unfortunately :-/ | 13:23 |
| juergbi | tristan: using temporary archive-z2 repository or does it work with bare-user as well? | 13:26 |
| tristan | temporary archive-z2 | 13:26 |
| tristan | so compress -> push -> delete | 13:27 |
| juergbi | ok. i guess there isn't really any other option | 13:27 |
| * tristan is trying on the bare-user repo right now but doesnt expect it to work | 13:28 | |
| tristan | nah | 13:29 |
| tristan | it gives me a remote stack trace | 13:29 |
| tristan | and the ref doesnt appear | 13:29 |
| tristan | juergbi, the compression though is really quite fast for a single artifact | 13:30 |
| tristan | probably reduces overall transmission time anyway | 13:30 |
| juergbi | SSH typically uses transparent compression | 13:30 |
| juergbi | but the sshfs approach is definitely much slower overall | 13:31 |
| gitlab-br-bot | buildstream: merge request (jonathan/dpkg-build->master: WIP: Jonathan/dpkg build) #37 changed state ("opened"): https://gitlab.com/BuildStream/buildstream/merge_requests/37 | 13:31 |
| tristan | ok so for now, I have changed the ssh config so the sshfs approach wont work for now with same url | 13:32 |
| tristan | juergbi, if you want to use sshfs in the meantime, use artifacts@gnome7.codethink.co.uk:/srv/artifacts/repo | 13:32 |
| tristan | that would work | 13:32 |
| juergbi | ok. me alone using it might not be too useful, though ;) | 13:33 |
| tristan | yeah, well lemme get through my hack... and then lets collect ssh keys from the team first | 13:34 |
| tristan | we should also think about some x86 autobuilders | 13:34 |
| tristan | to populate that cache with GNOME builds, might bust the disk space though | 13:34 |
| tristan | anyway we're testing in this phase | 13:34 |
| ssam2 | cache expiry is something to think about fairly soon | 13:35 |
| tristan | ssam2, indeed | 13:35 |
| juergbi | yes | 13:35 |
| tristan | its not the easiest of problems | 13:35 |
| juergbi | should get some Ryzen 7 ;) | 13:35 |
| tristan | on the other hand, the more we get reproducible builds, the less that cache grows | 13:35 |
| ssam2 | but to be fair, for cache.baserock.org we spent years just doing some shell one liner that deleted artifacts over a certain age | 13:35 |
| ssam2 | the same could be done on this cache -- it's a cache so deleting too much isn't a major issue | 13:35 |
| juergbi | yes, but fully reproducible builds might still be a bit out | 13:36 |
| tristan | juergbi, apparently flatpak is faaairly close... but yeah... only for "some stuff" :) | 13:36 |
| tristan | I think there is a hack specifically for python | 13:36 |
| tristan | but I dont expect most of the C stuff to be reproducible | 13:36 |
| ssam2 | most C stuff generally is, I think it's only things like embedding timestamps that causes problems | 13:37 |
| ssam2 | as long as you use the same compiler, that is | 13:37 |
| juergbi | yes, and compiler versions | 13:37 |
| juergbi | timestamps are also in things like generated man/infopages, though | 13:37 |
| ssam2 | but we do use the same compiler as we have fixed host tools | 13:37 |
| tristan | I thought there was an issue with uninitialized static buffers and debris in your ram | 13:38 |
| juergbi | i have specs somewhere where i have a mostly reproducible base system | 13:38 |
| tristan | depending on what scope it's declared in | 13:38 |
| ssam2 | we investigated this for baserock definitions a while ago and found that maybe 50% of things were already reproducible, with no effort at all | 13:38 |
| ssam2 | and debian-reproducible have been hard at work in the intervening 2 years | 13:38 |
| tristan | well, sounds quite promising then, actually | 13:38 |
| juergbi | yes, debian is pretty far, afaik | 13:38 |
| juergbi | i don't expect uninitialized buffers influencing the build result | 13:39 |
| * tristan seems to recall reading something about that on reproducible-builds website | 13:39 | |
| ssam2 | i think the main issue is the last 10% is another 90% of the time | 13:39 |
| tristan | I did give that a read some months back | 13:39 |
| tristan | i.e. regarding buffers | 13:39 |
| tristan | anyway, maybe it had to do with some specific contexts | 13:40 |
| tristan | afair, the way the C code is written can influence it's reproducibility, but maybe compilers are fixing that | 13:40 |
| jonathanmaw | argh! I forgot my mailing list etiquette and replied directly to you, instead of to the list | 13:52 |
| *** tristan has quit IRC | 13:52 | |
| jonathanmaw | and it looks like I'm still in some kind of weird mailing list limbo. my proper reply (to the list instead of tristan) is awaiting moderator approval | 13:57 |
| *** tristan has joined #buildstream | 14:19 | |
| tlater | Am I overlooking a handy checksum computation function anywhere in bst or do I make one? | 14:39 |
| ssam2 | python hashlib ? | 14:40 |
| tlater | I don't think it has anything that computes a recursive hash, that's what I'm looking for. | 14:41 |
| tlater | Ah, local sources have an implementation for this... It ignores file permissions though, as far as I understand, so changes to only permissions would collide. | 14:51 |
| *** jude has joined #buildstream | 15:37 | |
| tristan | tlater, look at local source | 16:36 |
| tristan | it does exactly what you need, we could make it a utility if that makes sense | 16:37 |
| tristan | jonathanmaw, I know whats wrong with your buildstream-list posts now | 16:40 |
| tristan | jonathanmaw, you are registered on the list as jonathan.maw@codethink.co.uk, as one might expect | 16:40 |
| tristan | jonathanmaw, but for some reason you are sending with From headers as jonathan.maw@codethink.com | 16:41 |
| jonathanmaw | aha | 16:41 |
| jonathanmaw | looks like webmail is playing a trick on me | 16:41 |
| * jonathanmaw subscribes in his secret new moniker | 16:47 | |
| *** jude has quit IRC | 16:53 | |
| gitlab-br-bot | push on buildstream@workspaces (by Tristan Maat): 2 commits (last: main.py: Add workspace commands) https://gitlab.com/BuildStream/buildstream/commit/5d40d89f79a9e5cd2a6dad1c0d63e0595e5c60d6 | 16:58 |
| *** jonathanmaw has quit IRC | 16:59 | |
| *** tlater has quit IRC | 17:01 | |
| *** ssam2 has quit IRC | 18:40 | |
| *** tristan has quit IRC | 19:38 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!