| *** gtristan has joined #baserock | 06:15 | |
| rjek | aren: Most people in here are in Europe I think, and are probably not close to their computers yet after a night of slumber; hopefully they'll be around in an hour or three. | 06:34 |
|---|---|---|
| paulsher1ood | aren: can you identify an example, please? | 06:54 |
| paulsher1ood | there's currently an issue with ybd and recursive submodules - https://gitlab.com/baserock/ybd/issues/243 | 06:55 |
| paulsher1ood | i'm trying to figure out how to address that, and would be happy to know of examples with nested submodules | 06:56 |
| *** paulw has joined #baserock | 07:21 | |
| *** ctbruce has joined #baserock | 07:29 | |
| *** toscalix has joined #baserock | 07:39 | |
| *** rdale has joined #baserock | 07:40 | |
| jjardon | Hi! Any idea why the install-files extension if failing with "env: /root/ybd/tmp/tmp2beYgl: No such file or directory" here ? https://gitlab.com/baserock/definitions/builds/5576416 | 08:10 |
| *** franred has joined #baserock | 08:24 | |
| *** jude_ has joined #baserock | 08:26 | |
| *** CTtpollard has quit IRC | 08:29 | |
| *** CTtpollard has joined #baserock | 08:32 | |
| *** jude_ has quit IRC | 08:32 | |
| *** jude_ has joined #baserock | 08:36 | |
| jjardon | what we need for this to happen? update lorry controller in our server? https://gerrit.baserock.org/#/c/2314/ | 09:16 |
| SotK | yes | 09:16 |
| SotK | but it has some missing dependencies in master of definitions nowadays | 09:17 |
| SotK | that I haven't had time to integrate | 09:17 |
| jjardon | SotK: rigth, can you set a -2 to the patch so it doesn't get accidentally merged until that happen | 09:18 |
| jjardon | ? | 09:18 |
| SotK | done | 09:18 |
| jjardon | thanks! | 09:19 |
| *** locallycompact has joined #baserock | 09:31 | |
| paulsher1ood | gtristan: wrt your email just now... we can't resolve ref:=>commit for non-sha case without access to the repo | 09:54 |
| paulsher1ood | so no, i don't think you're resolving my concern yet :) | 09:55 |
| gtristan | paulsher1ood, what is ref:=>commit | 09:56 |
| gtristan | paulsher1ood, I think your definition of ref, in that case, is a tracking branch, not a concrete ref | 09:56 |
| paulsher1ood | for case where ref: devel, say, or master | 09:56 |
| paulsher1ood | or even v5.3 | 09:56 |
| gtristan | right, thats the distinction we make | 09:56 |
| gtristan | tag/branch etc is not the ref | 09:57 |
| paulsher1ood | ref can be a sha or not in git-speak, iiuc | 09:57 |
| gtristan | in git speak, sure | 09:57 |
| paulsher1ood | and it can be a sha or not in definitions | 09:57 |
| gtristan | Ok, so we have these two concepts: A BuildSource *must* be able to express an exact version | 09:58 |
| gtristan | A BuildSource *may* express a something that can be resolved, a symbolic tracking branch or tag or such | 09:58 |
| gtristan | They are separate things | 09:58 |
| paulsher1ood | i'd rather understand this in our current vocabulary, before you go inventing new terms, if possible? | 09:59 |
| gtristan | I cannot | 09:59 |
| gtristan | Because our current vocabulary revolves entirely around git, and only has one word to express the ref | 09:59 |
| paulsher1ood | plain english should be possible :) | 09:59 |
| gtristan | Sure :) | 09:59 |
| gtristan | So I thought that was plain english, ok so with any VCS one should be able to provide a way to express an "exact set of sources" | 10:00 |
| gtristan | in gitspeak that would be the commit sha or tree sha | 10:00 |
| gtristan | represents something exactly specific | 10:00 |
| gtristan | In most, if not all VCSs, one can also represent a tag or branch | 10:01 |
| gtristan | We treat these two things separately | 10:01 |
| gtristan | Not one "ref:" property | 10:01 |
| paulsher1ood | gtristan: so i was answering your original email from the perspective of what i know in ybd. making the modification in ybd would not lead to simplification, and i've tried to express why | 10:02 |
| paulsher1ood | i'm happy to consider a new solution, once you've published it... but some of the use-cases are subtle | 10:03 |
| gtristan | paulsher1ood, right, my reply was not in context to what we can do with ybd in it's present form; for BuildStream we have to keep the door open to users who might not want to buy everything baserock all at once; which means they might not want to jump on the only-git bandwagon right away | 10:06 |
| gtristan | For this we have a pretty simple abstraction I think which allows BuildSources to be many things, all of which can conform to this rule, that an explicit exact version can be expressed, or an abstract tracking branch can also be expressed | 10:06 |
| paulsher1ood | ack | 10:08 |
| gtristan | paulsher1ood, in any case; I guess the conversation has two sides to it; one predominant question left unanswered is the state of affairs with YBD whilst we develop BuildStream | 10:08 |
| gtristan | I did not answer that question, sorry | 10:09 |
| gtristan | heh | 10:09 |
| * SotK wonders what BuildStream is | 10:09 | |
| paulsher1ood | locallycompact: am i correct in thinking that the submodules: approach doesn't work for nested/recursive submodules? | 10:09 |
| gtristan | I would suggest though, that ybd remain as is for Baserock at V8 | 10:09 |
| locallycompact | there's no reason it can't | 10:09 |
| tiagogomes_ | will BuildStream be another build tool? | 10:09 |
| gtristan | my submodules proposal is garbage, solved in a different way with BuildStream, in any case people obviously hate submodules | 10:10 |
| gtristan | tiagogomes_, yes. | 10:10 |
| paulsher1ood | locallycompact: well, one reason, based on lots of previous experience, may be if it's never been tested :) | 10:10 |
| locallycompact | do you mean does the implementation work right now | 10:11 |
| paulsher1ood | no. i'm asking if it was ever considered/tested? | 10:11 |
| locallycompact | I don't know | 10:12 |
| paulsher1ood | are there any actual examples of this anywhere? | 10:12 |
| * paulsher1ood thought locallycompact implemented it | 10:12 | |
| SotK | more build tools? o.O | 10:12 |
| * SotK gets back to work | 10:12 | |
| pedroalvarez | SotK: :) | 10:12 |
| paulsher1ood | SotK: not necessarily. maybe less. | 10:12 |
| pedroalvarez | indeed, they will be less | 10:12 |
| tiagogomes_ | yabt: yet another build tool | 10:12 |
| pedroalvarez | if it has tests, I'm all for it | 10:13 |
| paulsher1ood | tiagogomes_: that's only valid because build is overloaded. there are not so many things that do what morph, ybd etc do | 10:13 |
| tiagogomes_ | No. But there are at least two. I would think that would be enough. | 10:16 |
| gtristan | tiagogomes_, from what I understand, both of those tools are only useful to Baserock, only allow building from git, and only allow building an entire OS from the bootstrap up | 10:17 |
| paulsher1ood | bitbake, aboriginal, portage are three more that i'm aware of | 10:17 |
| * paulsher1ood had hoped that ybd would be useful outside baserock systems, but agrees with gtristan's summary | 10:18 | |
| locallycompact | the second is true, I don't know what the first one means and the last is plainly false | 10:18 |
| SotK | gtristan: I thought morph and ybd supported building of just parts of systems, and also that the "only building from git" was a deliberate design choice to aid in reproducibility? | 10:19 |
| gtristan | locallycompact, I guess the sysroot approach we took proves that last one false indeed | 10:19 |
| gtristan | although feels like a hack, it's true we did that | 10:19 |
| gtristan | SotK, a design choice which is good for the concept of Baserock, sure - not necessarily a good design choice for an abstract build tool usable in other scenarios | 10:20 |
| locallycompact | mmno, you can just declare an assemblage to build with the host tools | 10:20 |
| paulsher1ood | locallycompact: in which build tool? | 10:21 |
| locallycompact | V10 ybd | 10:21 |
| locallycompact | if they're bootstrap components it uses the host tools | 10:21 |
| locallycompact | the semantics are a little dry | 10:22 |
| locallycompact | but you can mix assemblages whatever | 10:22 |
| * tiagogomes_ will wait for more details about the new build tool and expanded scope of Baserock to further comment on. | 10:22 | |
| tiagogomes_ | I am with pedroalvarez, it has tests, go for it. | 10:23 |
| paulsher1ood | locallycompact: ybd master doesn't support V10 yet... and i'm struggling to understand your implementation, as i think you are aware | 10:23 |
| locallycompact | which implementation | 10:23 |
| paulsher1ood | V10 | 10:23 |
| locallycompact | to clarify: | 10:23 |
| locallycompact | you think what's in staging/010 is *more confusing* than what's in ybd master | 10:24 |
| paulsher1ood | to clarify: your brevity and choice of vocabulary makes much of what you say difficult to understand | 10:27 |
| paulsher1ood | case in point - were you asking, or stating, above? i should not have to guess. | 10:27 |
| locallycompact | brevity, yes, if you don't want brevity I will give up. I want building to be "enrich cache keys, flatten assemblage, get remotes, build" | 10:29 |
| locallycompact | not five files of side effecting loops | 10:29 |
| paulsher1ood | i'm fine with brevity of code. you just don't seem to care much whether your comments/explanations are understood or not | 10:29 |
| locallycompact | I do care, but you are far too sensitive to panic at any terminology you haven't heard | 10:32 |
| locallycompact | Especially since it's all comp sci terminology and I gave up speaking maths months ago | 10:32 |
| paulsher1ood | give up speaking comp sci too, and speak english. | 10:33 |
| locallycompact | so I'll just use words only from Sesame Street | 10:34 |
| pedroalvarez | please guys, this isn't being constructive at all | 10:35 |
| paulsher1ood | pedroalvarez: noted, thanks | 10:38 |
| jjardon | paulsher1ood: seems https://gitlab.com/baserock/definitions/builds/5592737 is not trying to upload the artifacts to the kbas server, any idea why? | 10:54 |
| paulsher1ood | jjardon: no, actually. are you sure the password is correct? | 11:06 |
| jjardon | let me recheck | 11:06 |
| gtristan | pedroalvarez, so I see that you moved all the genivi stuff under a separate directory | 11:15 |
| pedroalvarez | gtristan: yep | 11:15 |
| gtristan | pedroalvarez, for the immediate future, in a world without unmaintained systems, should we follow that model ? | 11:16 |
| gtristan | pedroalvarez, and what systems are important enough to keep around ? | 11:16 |
| gtristan | I see we have awkwardly many systems defined for each arch it supports, with the lack of variants; so there are still at least much less than what appears to be there | 11:17 |
| pedroalvarez | this was done to test our proposal for downstream consumers, to keep in a different folder their definitions | 11:17 |
| pedroalvarez | yes, I agree | 11:18 |
| gtristan | so should we do this for the gnome system ? | 11:18 |
| gtristan | better to have one homogeneous way of doing things at a time | 11:18 |
| pedroalvarez | put gnome things in gnome/ ? | 11:18 |
| pedroalvarez | yeah, I think it makes sense | 11:18 |
| gtristan | yeah, and only shared stuff in strata ? maybe we kee systems/ and clusters/ only for the minimal/devel basic systems based on generic bsps and coreutils ? | 11:19 |
| pedroalvarez | yeah | 11:21 |
| gtristan | pedroalvarez, what is build-system-* compared to devel-system-* ? | 11:21 |
| pedroalvarez | not sure if we will end up with loads of folders at the top level dir.. though.. | 11:22 |
| pedroalvarez | devel has more baserock development tools, like things you need to run the baserock-import-tool, and the tool itself | 11:22 |
| paulsher1ood | maybe go for a baserock directory, with common strata etc under that? | 11:22 |
| gtristan | is there a reason for maintaining both devel- and build- ? | 11:23 |
| pedroalvarez | i wonder if there is a reason to maintain any of them | 11:23 |
| pedroalvarez | the main reason was to trim the system used to just build, to be used in distbuild network | 11:24 |
| pedroalvarez | s | 11:24 |
| pedroalvarez | or to make lighter downloads for users | 11:24 |
| pedroalvarez | but nowadays, nobody uses baserock systems to build, right? | 11:24 |
| pedroalvarez | (well, I do, but I'm an old school baserocker) | 11:24 |
| paulsher1ood | pedroalvarez: it's a valid usecase, and i would do it myself, but for some reason i had unresolvable kernel panics running with vbox | 11:25 |
| leeming | baserock to build baserock you mean? | 11:25 |
| paulsher1ood | yup | 11:25 |
| gtristan | Well, I was thinking at the base of it, we have some minimal system all created with shared strata, and some development system that "has development tools" also stemming from those same basic strata | 11:26 |
| gtristan | seems to make sense to keep those simplistic things, capability to build a very basic system that has dev stuff on it | 11:26 |
| gtristan | compared to just a shell + coreutils | 11:26 |
| paulsher1ood | gtristan: it makes sense to keep others, too, for testing and 'historical reasons' | 11:27 |
| paulsher1ood | but i know you'd like to see some de-cluttering | 11:27 |
| gtristan | paulsher1ood, the goal of this exercise is to remove all of that; historical systems cannot work when based on an ever evolving trunk/master anyway | 11:28 |
| * paulsher1ood does occasionally see bugs only show up building qt, openstack, gnome etc | 11:28 | |
| gtristan | but in the future with a more modular approach, systems which become historical artifacts still build against a consistent trunk and should actually be repeatable | 11:28 |
| * gtristan finds himself doing svn-speak suddenly for some reason | 11:29 | |
| gtristan | pedroalvarez, so anyway, the goal of starting this conversation was to clearly define what stays off the chopping block; for build-* and devel-* I think we should keep only devel-* (because build looks like a subset of devel-* anyway, as diff devel-system build-system | grep "name:" tells me | 11:36 |
| gtristan | cross-bootstrap we should keep around I think for educational value, although I'm not sure anyone has tried that in a long time | 11:37 |
| pedroalvarez | I believe is broken, because of some bugs in GMP, but haven't tested it, no | 11:37 |
| pedroalvarez | build- is almost a subsystem of devel- | 11:38 |
| pedroalvarez | but the reasons that make it different (mason CI, and distbuild nodes) might not be interesting for anyone here anymore | 11:38 |
| gtristan | devel includes nodejs and other things | 11:39 |
| gtristan | which build does not | 11:39 |
| gtristan | pedroalvarez, ok so you are saying that maybe the added sugar in "devel-" systems are no longer valuable ? | 11:43 |
| gtristan | pedroalvarez, I would contend that it's unimportant what they contain specifically; keeping the devel- systems with "more stuff" lets us keep more stuff around in a sensible way | 11:44 |
| gtristan | pedroalvarez, i.e. in other words... if you say that you dont want to have "devel-" systems in the common basic systems/ directory, and nothing else uses nodejs, I will certainly argue for removing nodejs completely from strata/ | 11:46 |
| gtristan | some rampant lying around chunk that is not used is simply deadcode to me | 11:46 |
| paulsher1ood | gtristan: the downside of this is that although it's out of date, it has been a working integration example. if someone turns up in future and wants to untegrate nodejs, they'd have an easier time starting from that, than from scratch? | 11:48 |
| gtristan | paulsher1ood, that's why I'm arguing that we keep the devel- system around which includes nodejs | 11:49 |
| gtristan | i.e. basically this: At the toplevel directory we have strata, systems and clusters | 11:49 |
| gtristan | in the systems/ directory we have: A.) Minimal systems B.) Base systems C.) Devel Systems | 11:50 |
| gtristan | The devel systems consume everything in the base strata/ directory if that's possible and makes sense | 11:50 |
| gtristan | Stuff in genivi/ and gnome/ subdirectories contain only strata that is exclusively for those purposes | 11:51 |
| gtristan | Nothing that is unused lies around | 11:51 |
| gtristan | paulsher1ood, I have an interesting analogy: One day I spent hours and hours fixing a bug and wondering why the compiled result did not fix the bug | 11:52 |
| pedroalvarez | gtristan: I meant that build- has some extra sugar no longer valuable | 11:52 |
| gtristan | paulsher1ood, until someone let me in on the secret that the #if directive I was playing under, which contained thousands of lines of deadcode, was never reachable | 11:52 |
| gtristan | and was maintained only for historical purposes | 11:52 |
| pedroalvarez | so yes, keeping devel- and killing build- is good for me | 11:52 |
| gtristan | paulsher1ood, even though we had a perfectly working VCS, people wanted to keep deadcode floating around, wasting peoples time looking at files that did not contribute to the build in any way | 11:53 |
| gtristan | pedroalvarez, I did not notice that when diffing the x86 ones, I thought only devel- had the extra sugar :-/ | 11:54 |
| gtristan | pedroalvarez, in that case: make sure devel includes both added sugarings and lose less ? | 11:54 |
| pedroalvarez | as I said, I don't think we care about that anymore | 11:55 |
| gtristan | Ok. | 11:55 |
| gtristan | pedroalvarez, what is ivi-system, we have genivi so drop it, right ? | 11:55 |
| pedroalvarez | that question is for jjardon | 11:55 |
| paulsher1ood | jjardon: ^^ | 11:55 |
| pedroalvarez | i'd say that they are different things, but almost the same | 11:56 |
| pedroalvarez | maybe in the future genivi could be ivi+extra things | 11:57 |
| gtristan | I thought there was a lorry system, i.e. a system to run a trove no ? | 11:57 |
| gtristan | did that get blasted away ? | 11:57 |
| pedroalvarez | trove-system- | 11:57 |
| * gtristan was thinking that was one of the things we'd want to keep | 11:58 | |
| gtristan | oh | 11:58 |
| gtristan | pedroalvarez, so even though there's not much, I think that merits a separate directory no ? | 11:58 |
| gtristan | i.e. it targets something real | 11:58 |
| pedroalvarez | i'm not sure about that one | 11:58 |
| pedroalvarez | yes, that's true | 11:59 |
| gtristan | genivi, gnome, lorry-or-trove | 11:59 |
| pedroalvarez | openstack | 11:59 |
| pedroalvarez | I don't like lorry-or-trove | 11:59 |
| gtristan | Is that something that is maintained and we care about ? | 11:59 |
| gtristan | pedroalvarez, no, but either lorry or trove, maybe trove is better ? | 11:59 |
| gtristan | genivi, gnome, trove, openstack ? | 12:00 |
| pedroalvarez | looks to me that the name will imply only one system, I'd go for a more generic name | 12:00 |
| gtristan | which, for trove ? | 12:00 |
| pedroalvarez | but I don't know which one | 12:00 |
| pedroalvarez | yes | 12:00 |
| gtristan | isnt there only one trove system ? | 12:00 |
| gtristan | I dont think thats a bad thing, honestly | 12:01 |
| gtristan | there could be more for each arch that trove eventually supports | 12:01 |
| gtristan | gnome is also only one system | 12:01 |
| gtristan | it just has a "lotta stuff" | 12:01 |
| pedroalvarez | <pedroalvarez> not sure if we will end up with loads of folders at the top level dir.. though.. | 12:01 |
| gtristan | right, you hate folders ? | 12:02 |
| gtristan | hehe | 12:02 |
| pedroalvarez | hehe | 12:02 |
| gtristan | pedroalvarez, I think no because... first of all, I think we've already discussed all the systems we're planning to keep right ? | 12:02 |
| pedroalvarez | you are right, others are only one system too | 12:02 |
| pedroalvarez | one or two systems | 12:02 |
| gtristan | I mean, instead of running down the list, what other systems are maintained ? | 12:02 |
| gtristan | jjardon likes weston-system I think, I'm not sure why though, maybe it should be a kind of gnome system ? If I understand weston is a system to test the weston wayland compositor ? | 12:03 |
| pedroalvarez | what about creating a folder for "everything else" ? | 12:03 |
| paulsher1ood | i would just note that we need to continue with multi-arch examples, including be-arm | 12:03 |
| paulsher1ood | +1 to the everything else idea | 12:04 |
| pedroalvarez | to put things that we don't maintain, but are useful to keep? | 12:04 |
| pedroalvarez | or things that are done by contributors, that we don't garantee that are 100% working? | 12:04 |
| pedroalvarez | contrib/ | 12:04 |
| paulsher1ood | yup, but i thin gtristan wants a cleaner house than that :) | 12:04 |
| pedroalvarez | yes, I know | 12:05 |
| gtristan | Sigh, I dont like it, but I can bend for that one if you guys like that | 12:05 |
| gtristan | The end result will be that modules will be split up anyway | 12:05 |
| pedroalvarez | i don't really mind, tbh | 12:05 |
| pedroalvarez | it just makes it easier to discover done things than digging in git log | 12:05 |
| gtristan | Ok let me put it this way: If there is stuff in "everything else" (which we will call something like "unmaintained"); then it also means that ostree work such like Ana is doing, does not have to touch those directories at all in order to pass review. | 12:06 |
| gtristan | See what I mean ? | 12:06 |
| pedroalvarez | yep | 12:06 |
| pedroalvarez | no need to make things from that folder work for reviewing | 12:07 |
| paulsher1ood | right... but conversely if we do accept a contrib there... it crops up on other people's diffs? | 12:07 |
| * paulsher1ood is not clear on how ostree works | 12:07 | |
| gtristan | paulsher1ood, only until we split up definitions modules | 12:07 |
| pedroalvarez | i did't understand that, paulsher1ood | 12:08 |
| gtristan | iiuc, what paulsher1ood is saying is that if people make contributions to unmaintained/ it makes an otherwise sensible git log dirty with unrelated garbage we dont care about | 12:08 |
| paulsher1ood | not quite as harsh as that, but ok :) | 12:09 |
| gtristan | hehe | 12:09 |
| pedroalvarez | so, we are going to close the project to contributions if they aren't going to be maintained? | 12:10 |
| paulsher1ood | iiuc you're trying to get to a situation where downstream users can consume maintained stuff as tidily as possible | 12:10 |
| paulsher1ood | pedroalvarez: i think gtristan is trying to separate things out, so downstreams can consume only what they care about | 12:10 |
| pedroalvarez | (not against, any of these ideas, just wanted to clarify given that this has happened a lot in previous years) | 12:10 |
| pedroalvarez | right | 12:11 |
| pedroalvarez | I take we will have places to hold not maintained contributions | 12:11 |
| * gtristan returns from restroom | 12:11 | |
| * paulsher1ood had thought that separate directories was enough, but now it seems that there are effects in the git log, and possible ostree implications, | 12:11 | |
| gtristan | Ok, to clarify; and it's all in the email regarding the module splitting thing (which will not be done with submodules anyway)... | 12:12 |
| pedroalvarez | ( \o/ ) | 12:12 |
| gtristan | pedroalvarez, there are a few issues, and one of the most importance is that flow of changes is backwards, making it impossible for anyone to maintain a stable system in a collaborative way | 12:13 |
| gtristan | stable systems can only be maintained by forking/duplicating everything | 12:13 |
| locallycompact | in V8 | 12:13 |
| gtristan | pedroalvarez, when someone makes a commit to core strata, it should not be forced upon every maintainer of every system | 12:13 |
| pedroalvarez | I agree there, gtristan | 12:14 |
| pedroalvarez | and other systems update whenever maintainers can | 12:14 |
| gtristan | instead, the maintainer of the gnome system should at times upgrade core, they do this by taking the latest stable branch of core and pointing to a new ref of core | 12:14 |
| gtristan | then, their shit breaks, and they report the bug, and a patch *if they have time* to the core maintainers | 12:15 |
| gtristan | their release timeline is not blocked by the whims of the maintainers of core.morph | 12:15 |
| paulsher1ood | isn't a likely/logical implication of this that every 'stratum' or 'assemblage' will end up in its own repo? | 12:15 |
| gtristan | paulsher1ood, yes, although they will not be joined by submodules or git subrepos | 12:16 |
| gtristan | paulsher1ood, actually no | 12:16 |
| gtristan | sorry | 12:16 |
| gtristan | I misread your question | 12:16 |
| gtristan | that would be one way to organize it, but I would not organize it as such | 12:16 |
| gtristan | I would instead have one module for bootstrap, and another module for everything GNU/Linux including bsps, foundation strata and core strata | 12:17 |
| gtristan | and then have modules like gnome, which consume strata from those modules | 12:17 |
| paulsher1ood | ack | 12:17 |
| gtristan | otherwise there would be a huge amount of modules, I dont think it's necessary to split *everything* like that | 12:18 |
| paulsher1ood | acck - that was my worry | 12:18 |
| gtristan | not even sure it makes sense to split out the bootstrap, but it could | 12:18 |
| paulsher1ood | worth thinking about for sure | 12:20 |
| locallycompact | gnome strata in and of themselves shouldn't need or have knowledge of core strata | 12:20 |
| locallycompact | containing assemblages do | 12:20 |
| locallycompact | bc if I want to build gnome on top of jibberish-root, I don't want gnome strata module to reference every conceivable core/root | 12:22 |
| jjardon | gtristan: ivi and weston systems are important and maintained, please do not remove them | 12:33 |
| jjardon | gtristan: build system should be keep; its the system that can be used to build any other baserock system; also Its the system we generate the baserock chroots from | 12:38 |
| tiagogomes_ | Is there a case for the build systems anymore, ybd is capable of building every system as long as the dependencies are installed | 12:42 |
| jjardon | paulsher1ood: ok to merge https://gitlab.com/baserock/ybd/merge_requests/258 ? | 13:28 |
| jjardon | paulsher1ood: maybe you prefer to build a single stratum (or a chunk), instead a whole system? | 13:29 |
| jjardon | actually, I think build build-essential should be enough, I will update the patch | 13:33 |
| gtristan | jjardon, cant we build another baserock system with the -devel system ? | 13:42 |
| gtristan | jjardon, if not, why do we have 2 ? | 13:42 |
| jjardon | gtristan: I think -devel is like "include everything" system, and Im not sure anyone is using it, but I can be wrong | 13:43 |
| gtristan | jjardon, right, so whats wrong with using that to build "another baserock system" ? | 13:43 |
| pedroalvarez | size | 13:44 |
| gtristan | Ummm, okaaaay | 13:44 |
| pedroalvarez | (just anticipating his answer) | 13:44 |
| gtristan | So anyway... "Any system with bash, posix compliant coreutils and a compiler can be used to build a baserock system" | 13:44 |
| pedroalvarez | (still waiting for it, though) | 13:44 |
| gtristan | That has to be true at all times | 13:45 |
| gtristan | Or well, +python | 13:45 |
| pedroalvarez | and build tool and dependencies | 13:45 |
| gtristan | Right, yaml and git | 13:46 |
| jjardon | gtristan: AFAIK thats what the build system is | 13:46 |
| gtristan | jjardon, yeah, is there an excuse we can come up with to cut some fat, please ? | 13:47 |
| tiagogomes_ | Why would someone use the build-system if they can built with the distro tools? | 13:47 |
| pedroalvarez | tiagogomes_: because of some distros, people sometimes use chroots | 13:48 |
| pedroalvarez | deployments are very tied to the host | 13:49 |
| gtristan | Ok ok host tooling is a bit of a problem we need to solve, the bootstrap itself could be improved as well with inspiration from the aboriginal stuff | 13:49 |
| jjardon | tiagogomes_: I can not use ybd or moprh without being root: I need a chroot, and the build system give me a minimal chroot system to build wherever I want | 13:49 |
| jjardon | also reproducibility | 13:49 |
| pedroalvarez | I'd like to have a very minimal build system | 13:50 |
| pedroalvarez | less than 200M | 13:50 |
| jjardon | me too | 13:50 |
| pedroalvarez | but i thought the goal was to make everything run outside any kind of chroot | 13:51 |
| jjardon | linux-user-root and bwrap should allow that | 13:52 |
| gtristan | those both have issues presently, some of which could be circumvented by placing constraints on the definitions | 13:54 |
| gtristan | (i.e. no creating device nodes or such) | 13:55 |
| jjardon | ok, lets do that then? | 13:55 |
| jjardon | I think we need to do that to support ostree anyway | 13:55 |
| gtristan | place restrictions/constraints ? | 13:55 |
| jjardon | no creating device nodes | 13:55 |
| gtristan | I am hopeful that we can get away with something else | 13:56 |
| jjardon | mmm, problems again cloning repos: https://gitlab.com/baserock/definitions/builds/5598981 | 13:56 |
| gtristan | maybe a fakeroot implementation with ptrace() could do it | 13:56 |
| gtristan | on the other hand, yocto also does not support that | 13:57 |
| tiagogomes_ | I have been looking at that :P | 13:57 |
| *** mdunford has joined #baserock | 13:57 | |
| gtristan | tiagogomes_, at what ? yocto ? ptrace() ? | 13:57 |
| gtristan | or the gitlab hehe | 13:57 |
| tiagogomes_ | attempting to do a build without being a root. There is already a fakeroot implementation that uses ptrace, Unfortunately, it doesn't support all the arches we care about. | 13:59 |
| gtristan | There is ? | 13:59 |
| gtristan | cool ! | 13:59 |
| tiagogomes_ | fakeroot-ng | 13:59 |
| gtristan | Hmmm, can it be made to ? | 13:59 |
| gtristan | Or are there restrictions / roadblocks to that ? | 14:00 |
| tiagogomes_ | yes, some one that knows kernel stuff should be able to do it in a day, couple of days | 14:00 |
| gtristan | Oh cool | 14:00 |
| pedroalvarez | that sounds good | 14:00 |
| gtristan | So that's an alternative | 14:00 |
| tiagogomes_ | The whole project is 5000 lines of code. The platform specific code could be 200 | 14:00 |
| gtristan | I may have to be a part of bubblewrap proper though | 14:00 |
| gtristan | tiagogomes_, would you be able to tell me if that would work recursively ? I.e. try running fakeroot-ng, and inside run bwrap to create a child namespace, and in that process... syscalls are still trapped by fakeroot-ng ? | 14:02 |
| tiagogomes_ | No, you have to do it the other way around I think. bbwrap -> fakeroot-ng | 14:03 |
| gtristan | hmmm, a sandbox could still be fashioned to do that | 14:03 |
| gtristan | well | 14:03 |
| gtristan | tiagogomes_, are you sure ? | 14:04 |
| gtristan | tiagogomes_, if you cannot, then... I worry that it will still not work, i.e. when running a build, there are many child processes going on | 14:04 |
| gtristan | fakeroot-ng would have to act recursively on *those* at least for the mknod/chown etc commands launched from recursive build scripts to get caught | 14:05 |
| tiagogomes_ | tiago@debian:~$ fakeroot-ng bwrap --bind / / sh | 14:06 |
| tiagogomes_ | No permissions to creating new namespace, likely because the kernel does not allow non-privileged user namespaces. On e.g. debian this can be enabled with 'sysctl kernel.unprivileged_userns_clone=1'. | 14:06 |
| tiagogomes_ | tiago@debian:~$ bwrap --bind / / fakeroot-ng echo hello world | 14:06 |
| tiagogomes_ | hello world | 14:06 |
| gtristan | the fakeroot-ng inside bubblewrap sandbox we could potentially handle (but seems more obvious the other way around) | 14:06 |
| tiagogomes_ | I was still researching how this could work, but I don't have time to complete my research at the moment | 14:07 |
| tiagogomes_ | There is also pseudo to consider. | 14:08 |
| gtristan | tiagogomes_, does fakeroot-ng sport the -s option ? And if you try to make it mknod, can you see what it looks like after ? | 14:08 |
| tiagogomes_ | gtristan it does. But on my approach I chose to get rid of device nodes entirely. | 14:09 |
| * gtristan starts to think that's the only sensible option, but reluctantly | 14:09 | |
| gtristan | in *any* case, current fhs-dirs cannot work | 14:09 |
| tiagogomes_ | I can't see much advantages in keeping device nodes tbh | 14:10 |
| gtristan | because you want/need /dev to be full of real devices | 14:10 |
| gtristan | at build time | 14:10 |
| gtristan | tiagogomes_, the ability to create tarballs with device nodes, and deploy systems which already have device nodes, without having to specify any additional metadata about them, or expecting that your system is something so specific as say, systemd based | 14:11 |
| tiagogomes_ | gtristan, solved with `bwrap --bind / / --dev /dev` . And because you are doing `--dev /dev` instead of `--dev-bind /dev /dev` , you get a sanitized list of device nodes at build time. | 14:11 |
| gtristan | right, I know that as a regular user, you cannot use fhs-dirs the way they currently are | 14:12 |
| gtristan | you cannot stage them and expect to redirect output to /dev/null and have a real /dev/null, or use any of that at build time | 14:12 |
| gtristan | but I dont like that you cannot distribute a system with device nodes | 14:13 |
| tiagogomes_ | what metadata about device nodes? | 14:15 |
| gtristan | I mean, in say an rpm based system, one would normally specify stuff in the package that says "when you install with root privileges on a system, install this device" | 14:16 |
| gtristan | thats how they solve the problem | 14:16 |
| gtristan | it's yuck, nicer to have something virtual you just tar up the results, even if it's virtual in the sandbox; in the tarball it can be a real device node (the tarball belongs to the user, not the device node represented in the tarball after all) | 14:17 |
| tiagogomes_ | that is what fakeroot would be useful. But like I said, I don't think we need to build or deliver device nodes | 14:19 |
| radiofree | there's also the yocto fakeroot replacement https://www.yoctoproject.org/tools-resources/projects/pseudo | 14:23 |
| jjardon | tiagogomes_: you mean remove the "devices" section from fhs-dirs ? | 14:24 |
| radiofree | some things will fail to build if you do that | 14:26 |
| radiofree | without replacing it with something else | 14:26 |
| radiofree | e.g webkit | 14:26 |
| pedroalvarez | Of course the idea would be to replace it with something else? | 14:27 |
| tiagogomes_ | I mentioned pseudo above | 14:30 |
| tiagogomes_ | The idea is to bind-mount them from the host | 14:31 |
| *** brlogger has joined #baserock | 14:56 | |
| *** brlogger` has quit IRC | 14:57 | |
| locallycompact | I rebased on ybd master and now I get https://gitlab.com/baserock/ybd/builds/5603128 | 14:58 |
| locallycompact | but not locally | 15:00 |
| locallycompact | no yes locally | 15:01 |
| locallycompact | are we sure master bootstraps? | 15:03 |
| leeming | well... a patch I added for bwrap brakes on a chroot bootstrap.. so i guess it does : | 15:07 |
| leeming | :) | 15:07 |
| jjardon | locallycompact: to be sure in the future, that's why I submitted https://gitlab.com/baserock/ybd/merge_requests/258 | 15:11 |
| jjardon | locallycompact: (I built a minimal system earlier today with current master with no problem) | 15:11 |
| locallycompact | certainly without that bubblewrap commit it's working, with it's broke | 15:12 |
| leeming | yes, this was raised before. Non bubblewrap backends fail with that patch | 15:13 |
| leeming | the PR was merged without testing | 15:13 |
| locallycompact | werp | 15:18 |
| leeming | no one confirmed if it broke with linux-user-chroot though | 15:23 |
| leeming | so not sure if it is a broken implementation of chroot in the sandbox, or something else | 15:23 |
| leeming | locallycompact, re S.I commands. have you tried with bwrap backend? | 15:29 |
| leeming | that mounts things as rw | 15:29 |
| leeming | instead of just ro | 15:29 |
| locallycompact | is that so | 15:29 |
| leeming | well | 15:29 |
| leeming | given some spec from ybd yes | 15:29 |
| locallycompact | guide me | 15:30 |
| leeming | if ybd tells it a mount is rw then it makes it so | 15:30 |
| leeming | or for a hack, you can set writable_paths:'all' | 15:30 |
| leeming | https://gitlab.com/baserock/ybd/blob/staging/010/ybd/sandbox.py#L141 | 15:32 |
| leeming | that is the config that specifies if mounts are ro/rw | 15:32 |
| locallycompact | ah that's what I missed | 15:32 |
| * locallycompact slaps | 15:32 | |
| leeming | not sure how linux-user-chroot/chroot behave | 15:33 |
| locallycompact | 'system' keyword | 15:33 |
| leeming | oh right :) | 15:33 |
| locallycompact | tiagogomes_, I seem to recall you had a branch that tidied up splitting logic, did that not make it in? | 15:47 |
| *** gtristan has quit IRC | 15:48 | |
| tiagogomes_ | I wouldn't call it a clean up. It was more about fixing a particular bug. I never created the PR though | 15:50 |
| tiagogomes_ | https://gitlab.com/baserock/ybd/commit/2a7963a3960c07c9e4caa1e7a4c0b69b800bd452 | 15:50 |
| locallycompact | certainly neater | 15:50 |
| locallycompact | can that merge? | 15:54 |
| * paulsher1ood is waiting to see what the issue is it's supposed to fix. or proof that it doesn't break anything | 15:55 | |
| * paulsher1ood is also interested in any help at all to understand recursive submodules, since ybd without it builds ci.morph. and without an example of the problem, it's hard to know what to fix | 15:57 | |
| tiagogomes_ | I tested and it worked. But there are no writing about artifact splitting that guarantees the way my patch solved the problem is the correct approach. | 15:58 |
| tiagogomes_ | paulsher1ood, what more possible explanation would you like besides the one that is the commit message? | 15:58 |
| locallycompact | can the approach to splitting be | 15:58 |
| locallycompact | don't write any meta files | 15:58 |
| locallycompact | buil everything | 15:58 |
| locallycompact | split on tar | 15:59 |
| locallycompact | (system tar) | 15:59 |
| paulsher1ood | locallycompact: yes, it can. and it will be, if i get around to it or someone beats me to it. | 15:59 |
| locallycompact | no assemblage metas no cache entries for assemblage | 15:59 |
| SotK | how does one reproduce a system with no meta files describing the things that went into it? | 15:59 |
| SotK | (many years in the future for example) | 16:00 |
| paulsher1ood | the metafiles would be present in the system | 16:00 |
| paulsher1ood | and also could be re-derived from the definitions | 16:00 |
| SotK | paulsher1ood: then you'd need to guarantee being able to know which commit of definitions a random system in the wild came from? | 16:01 |
| * SotK thought systems were assemblages too, and took "no assemblage metas" to mean there would be none at all | 16:01 | |
| locallycompact | I mean there isn't just an empty tarball in nowhere with a baserock meta in it | 16:01 |
| locallycompact | so it just goes in the one you're building | 16:01 |
| locallycompact | with the original assemblage info and all the metas | 16:02 |
| locallycompact | but there's no metas in kbas | 16:02 |
| * paulsher1ood disagrees - but probably locallycompact is not being precise | 16:02 | |
| locallycompact | which | 16:03 |
| SotK | there would be a meta in the system artifact in kbas? | 16:03 |
| paulsher1ood | kbas already only stores chunks | 16:03 |
| locallycompact | does it | 16:03 |
| locallycompact | ok | 16:03 |
| paulsher1ood | there are no strata or systems there | 16:03 |
| locallycompact | right | 16:03 |
| SotK | oh ok | 16:03 |
| paulsher1ood | any artifact should contain metadata describing itself. but chunk metadata does not need to describe potential splits for systems | 16:04 |
| locallycompact | right | 16:04 |
| SotK | paulsher1ood: +1 | 16:04 |
| paulsher1ood | hooray :) | 16:04 |
| paulsher1ood | so the problem is that ybd currently creates info about splits in chunks. | 16:05 |
| locallycompact | at the risk of dragging this out even further I'd like that to exist | 16:05 |
| locallycompact | so I might do it | 16:05 |
| paulsher1ood | like what to ex exist? | 16:06 |
| locallycompact | this behaviour | 16:06 |
| locallycompact | either in mainline or 010 | 16:06 |
| *** tiagogomes_ has quit IRC | 16:23 | |
| *** ctbruce has quit IRC | 16:26 | |
| *** tiagogomes has joined #baserock | 16:26 | |
| *** tiagogomes has quit IRC | 16:31 | |
| *** tiagogomes has joined #baserock | 16:36 | |
| *** gtristan has joined #baserock | 17:37 | |
| *** ctbruce has joined #baserock | 17:43 | |
| *** locallycompact has quit IRC | 17:44 | |
| *** tiagogomes has quit IRC | 17:54 | |
| *** aren has left #baserock | 18:03 | |
| *** toscalix has quit IRC | 18:45 | |
| *** locallycompact has joined #baserock | 19:32 | |
| *** paulw has joined #baserock | 20:04 | |
| *** locallycompact has quit IRC | 21:20 | |
| *** ctbruce has quit IRC | 21:38 | |
| *** ctbruce has joined #baserock | 21:50 | |
| *** ctbruce has quit IRC | 21:54 | |
| *** gtristan has quit IRC | 21:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!