IRC logs for #baserock for Thursday, 2016-10-27

*** gtristan has joined #baserock06:15
rjekaren: Most people in here are in Europe I think, and are probably not close to their computers yet after a night of slumber; hopefully they'll be around in an hour or three.06:34
paulsher1oodaren: can you identify an example, please?06:54
paulsher1oodthere's currently an issue with ybd and recursive submodules - https://gitlab.com/baserock/ybd/issues/24306:55
paulsher1oodi'm trying to figure out how to address that, and would be happy to know of examples with nested submodules06:56
*** paulw has joined #baserock07:21
*** ctbruce has joined #baserock07:29
*** toscalix has joined #baserock07:39
*** rdale has joined #baserock07:40
jjardonHi! Any idea why the install-files extension if failing with "env: /root/ybd/tmp/tmp2beYgl: No such file or directory" here ? https://gitlab.com/baserock/definitions/builds/557641608:10
*** franred has joined #baserock08:24
*** jude_ has joined #baserock08:26
*** CTtpollard has quit IRC08:29
*** CTtpollard has joined #baserock08:32
*** jude_ has quit IRC08:32
*** jude_ has joined #baserock08:36
jjardonwhat we need for this to happen? update lorry controller in our server? https://gerrit.baserock.org/#/c/2314/09:16
SotKyes09:16
SotKbut it has some missing dependencies in master of definitions nowadays09:17
SotKthat I haven't had time to integrate09:17
jjardonSotK: rigth, can you set a -2 to the patch so it doesn't get accidentally merged until that happen09:18
jjardon?09:18
SotKdone09:18
jjardonthanks!09:19
*** locallycompact has joined #baserock09:31
paulsher1oodgtristan: wrt your email just now... we can't resolve ref:=>commit for non-sha case without access to the repo09:54
paulsher1oodso no, i don't think you're resolving my concern yet :)09:55
gtristanpaulsher1ood, what is ref:=>commit09:56
gtristanpaulsher1ood, I think your definition of ref, in that case, is a tracking branch, not a concrete ref09:56
paulsher1oodfor case where ref: devel, say, or master09:56
paulsher1oodor even v5.309:56
gtristanright, thats the distinction we make09:56
gtristantag/branch etc is not the ref09:57
paulsher1oodref can be a sha or not in git-speak, iiuc09:57
gtristanin git speak, sure09:57
paulsher1oodand it can be a sha or not in definitions09:57
gtristanOk, so we have these two concepts: A BuildSource *must* be able to express an exact version09:58
gtristanA BuildSource *may* express a something that can be resolved, a symbolic tracking branch or tag or such09:58
gtristanThey are separate things09:58
paulsher1oodi'd rather understand this in our current vocabulary, before you go inventing new terms, if possible?09:59
gtristanI cannot09:59
gtristanBecause our current vocabulary revolves entirely around git, and only has one word to express the ref09:59
paulsher1oodplain english should be possible :)09:59
gtristanSure :)09:59
gtristanSo I thought that was plain english, ok so with any VCS one should be able to provide a way to express an "exact set of sources"10:00
gtristanin gitspeak that would be the commit sha or tree sha10:00
gtristanrepresents something exactly specific10:00
gtristanIn most, if not all VCSs, one can also represent a tag or branch10:01
gtristanWe treat these two things separately10:01
gtristanNot one "ref:" property10:01
paulsher1oodgtristan: so i was answering your original email from the perspective of what i know in ybd. making the modification in ybd would not lead to simplification, and i've tried to express why10:02
paulsher1oodi'm happy to consider a new solution, once you've published it... but some of the use-cases are subtle10:03
gtristanpaulsher1ood, right, my reply was not in context to what we can do with ybd in it's present form; for BuildStream we have to keep the door open to users who might not want to buy everything baserock all at once; which means they might not want to jump on the only-git bandwagon right away10:06
gtristanFor this we have a pretty simple abstraction I think which allows BuildSources to be many things, all of which can conform to this rule, that an explicit exact version can be expressed, or an abstract tracking branch can also be expressed10:06
paulsher1oodack10:08
gtristanpaulsher1ood, in any case; I guess the conversation has two sides to it; one predominant question left unanswered is the state of affairs with YBD whilst we develop BuildStream10:08
gtristanI did not answer that question, sorry10:09
gtristanheh10:09
* SotK wonders what BuildStream is10:09
paulsher1oodlocallycompact: am i correct in thinking that the submodules: approach doesn't work for nested/recursive submodules?10:09
gtristanI would suggest though, that ybd remain as is for Baserock at V810:09
locallycompactthere's no reason it can't10:09
tiagogomes_will BuildStream be another build tool?10:09
gtristanmy submodules proposal is garbage, solved in a different way with BuildStream, in any case people obviously hate submodules10:10
gtristantiagogomes_, yes.10:10
paulsher1oodlocallycompact: well, one reason, based on lots of previous experience, may be if it's never been tested :)10:10
locallycompactdo you mean does the implementation work right now10:11
paulsher1oodno. i'm asking if it was ever considered/tested?10:11
locallycompactI don't know10:12
paulsher1oodare there any actual examples of this anywhere?10:12
* paulsher1ood thought locallycompact implemented it10:12
SotKmore build tools? o.O10:12
* SotK gets back to work10:12
pedroalvarezSotK: :)10:12
paulsher1oodSotK: not necessarily. maybe less.10:12
pedroalvarezindeed, they will be less10:12
tiagogomes_yabt: yet another build tool10:12
pedroalvarezif it has tests, I'm all for it10:13
paulsher1oodtiagogomes_: that's only valid because build is overloaded. there are not so many things that do what morph, ybd etc do10:13
tiagogomes_No. But there are at least two. I would think that would be enough.10:16
gtristantiagogomes_, from what I understand, both of those tools are only useful to Baserock, only allow building from git, and only allow building an entire OS from the bootstrap up10:17
paulsher1oodbitbake, aboriginal, portage are three more that i'm aware of10:17
* paulsher1ood had hoped that ybd would be useful outside baserock systems, but agrees with gtristan's summary10:18
locallycompactthe second is true, I don't know what the first one means and the last is plainly false10:18
SotKgtristan: I thought morph and ybd supported building of just parts of systems, and also that the "only building from git" was a deliberate design choice to aid in reproducibility?10:19
gtristanlocallycompact, I guess the sysroot approach we took proves that last one false indeed10:19
gtristanalthough feels like a hack, it's true we did that10:19
gtristanSotK, a design choice which is good for the concept of Baserock, sure - not necessarily a good design choice for an abstract build tool usable in other scenarios10:20
locallycompactmmno, you can just declare an assemblage to build with the host tools10:20
paulsher1oodlocallycompact: in which build tool?10:21
locallycompactV10 ybd10:21
locallycompactif they're bootstrap components it uses the host tools10:21
locallycompactthe semantics are a little dry10:22
locallycompactbut you can mix assemblages whatever10:22
* tiagogomes_ will wait for more details about the new build tool and expanded scope of Baserock to further comment on.10:22
tiagogomes_I am with pedroalvarez, it has tests, go for it.10:23
paulsher1oodlocallycompact: ybd master doesn't support V10 yet... and i'm struggling to understand your implementation, as i think you are aware10:23
locallycompactwhich implementation10:23
paulsher1oodV1010:23
locallycompactto clarify:10:23
locallycompactyou think what's in staging/010 is *more confusing* than what's in ybd master10:24
paulsher1oodto clarify: your brevity and choice of vocabulary makes much of what you say difficult to understand10:27
paulsher1oodcase in point - were you asking, or stating, above? i should not have to guess.10:27
locallycompactbrevity, yes, if you don't want brevity I will give up. I want building to be "enrich cache keys, flatten assemblage, get remotes, build"10:29
locallycompactnot five files of side effecting loops10:29
paulsher1oodi'm fine with brevity of code. you just don't seem to care much whether your comments/explanations are understood or not10:29
locallycompactI do care, but you are far too sensitive to panic at any terminology you haven't heard10:32
locallycompactEspecially since it's all comp sci terminology and I gave up speaking maths months ago10:32
paulsher1oodgive up speaking comp sci too, and speak english.10:33
locallycompactso I'll just use words only from Sesame Street10:34
pedroalvarezplease guys, this isn't being constructive at all10:35
paulsher1oodpedroalvarez: noted, thanks10:38
jjardonpaulsher1ood: seems https://gitlab.com/baserock/definitions/builds/5592737 is not trying to upload the artifacts to the kbas server, any idea why?10:54
paulsher1oodjjardon: no, actually. are you sure the password is correct?11:06
jjardonlet me recheck11:06
gtristanpedroalvarez, so I see that you moved all the genivi stuff under a separate directory11:15
pedroalvarezgtristan: yep11:15
gtristanpedroalvarez, for the immediate future, in a world without unmaintained systems, should we follow that model ?11:16
gtristanpedroalvarez, and what systems are important enough to keep around ?11:16
gtristanI see we have awkwardly many systems defined for each arch it supports, with the lack of variants; so there are still at least much less than what appears to be there11:17
pedroalvarezthis was done to test our proposal for downstream consumers, to keep in a different folder their definitions11:17
pedroalvarezyes, I agree11:18
gtristanso should we do this for the gnome system ?11:18
gtristanbetter to have one homogeneous way of doing things at a time11:18
pedroalvarezput gnome things in gnome/ ?11:18
pedroalvarezyeah, I think it makes sense11:18
gtristanyeah, and only shared stuff in strata ? maybe we kee systems/ and clusters/ only for the minimal/devel basic systems based on generic bsps and coreutils ?11:19
pedroalvarezyeah11:21
gtristanpedroalvarez, what is build-system-* compared to devel-system-* ?11:21
pedroalvareznot sure if we will end up with loads of folders at the top level dir.. though..11:22
pedroalvarezdevel has more baserock development tools, like things you need to run the baserock-import-tool, and the tool itself11:22
paulsher1oodmaybe go for a baserock directory, with common strata etc under that?11:22
gtristanis there a reason for maintaining both devel- and build- ?11:23
pedroalvarezi wonder if there is a reason to maintain any of them11:23
pedroalvarezthe main reason was to trim the system used to just build, to be used in distbuild network11:24
pedroalvarezs11:24
pedroalvarezor to make lighter downloads for users11:24
pedroalvarezbut nowadays, nobody uses baserock systems to build, right?11:24
pedroalvarez(well, I do, but I'm an old school baserocker)11:24
paulsher1oodpedroalvarez: it's a valid usecase, and i would do it myself, but for some reason i had unresolvable kernel panics running with vbox11:25
leemingbaserock to build baserock you mean?11:25
paulsher1oodyup11:25
gtristanWell, I was thinking at the base of it, we have some minimal system all created with shared strata, and some development system that "has development tools" also stemming from those same basic strata11:26
gtristanseems to make sense to keep those simplistic things, capability to build a very basic system that has dev stuff on it11:26
gtristancompared to just a shell + coreutils11:26
paulsher1oodgtristan: it makes sense to keep others, too, for testing and 'historical reasons'11:27
paulsher1oodbut i know you'd like to see some de-cluttering11:27
gtristanpaulsher1ood, the goal of this exercise is to remove all of that; historical systems cannot work when based on an ever evolving trunk/master anyway11:28
* paulsher1ood does occasionally see bugs only show up building qt, openstack, gnome etc11:28
gtristanbut in the future with a more modular approach, systems which become historical artifacts still build against a consistent trunk and should actually be repeatable11:28
* gtristan finds himself doing svn-speak suddenly for some reason11:29
gtristanpedroalvarez, so anyway, the goal of starting this conversation was to clearly define what stays off the chopping block; for build-* and devel-* I think we should keep only devel-* (because build looks like a subset of devel-* anyway, as diff devel-system build-system | grep "name:" tells me11:36
gtristancross-bootstrap we should keep around I think for educational value, although I'm not sure anyone has tried that in a long time11:37
pedroalvarezI believe is broken, because of some bugs in GMP, but haven't tested it, no11:37
pedroalvarezbuild- is almost a subsystem of devel-11:38
pedroalvarezbut the reasons that make it different (mason CI, and distbuild nodes) might not be interesting for anyone here anymore11:38
gtristandevel includes nodejs and other things11:39
gtristanwhich build does not11:39
gtristanpedroalvarez, ok so you are saying that maybe the added sugar in "devel-" systems are no longer valuable ?11:43
gtristanpedroalvarez, I would contend that it's unimportant what they contain specifically; keeping the devel- systems with "more stuff" lets us keep more stuff around in a sensible way11:44
gtristanpedroalvarez, i.e. in other words... if you say that you dont want to have "devel-" systems in the common basic systems/ directory, and nothing else uses nodejs, I will certainly argue for removing nodejs completely from strata/11:46
gtristansome rampant lying around chunk that is not used is simply deadcode to me11:46
paulsher1oodgtristan: the downside of this is that although it's out of date, it has been a working integration example. if someone turns up in future and wants to untegrate nodejs, they'd have an easier time starting from that, than from scratch?11:48
gtristanpaulsher1ood, that's why I'm arguing that we keep the devel- system around which includes nodejs11:49
gtristani.e. basically this: At the toplevel directory we have strata, systems and clusters11:49
gtristanin the systems/ directory we have: A.) Minimal systems B.) Base systems C.) Devel Systems11:50
gtristanThe devel systems consume everything in the base strata/ directory if that's possible and makes sense11:50
gtristanStuff in genivi/ and gnome/ subdirectories contain only strata that is exclusively for those purposes11:51
gtristanNothing that is unused lies around11:51
gtristanpaulsher1ood, I have an interesting analogy: One day I spent hours and hours fixing a bug and wondering why the compiled result did not fix the bug11:52
pedroalvarezgtristan: I meant that build- has some extra sugar no longer valuable11:52
gtristanpaulsher1ood, until someone let me in on the secret that the #if directive I was playing under, which contained thousands of lines of deadcode, was never reachable11:52
gtristanand was maintained only for historical purposes11:52
pedroalvarezso yes, keeping devel- and killing build- is good for me11:52
gtristanpaulsher1ood, even though we had a perfectly working VCS, people wanted to keep deadcode floating around, wasting peoples time looking at files that did not contribute to the build in any way11:53
gtristanpedroalvarez, I did not notice that when diffing the x86 ones, I thought only devel- had the extra sugar :-/11:54
gtristanpedroalvarez, in that case: make sure devel includes both added sugarings and lose less ?11:54
pedroalvarezas I said, I don't think we care about that anymore11:55
gtristanOk.11:55
gtristanpedroalvarez, what is ivi-system, we have genivi so drop it, right ?11:55
pedroalvarezthat question is for jjardon11:55
paulsher1oodjjardon: ^^11:55
pedroalvarezi'd say that they are different things, but almost the same11:56
pedroalvarezmaybe in the future genivi could be ivi+extra things11:57
gtristanI thought there was a lorry system, i.e. a system to run a trove no ?11:57
gtristandid that get blasted away ?11:57
pedroalvareztrove-system-11:57
* gtristan was thinking that was one of the things we'd want to keep11:58
gtristanoh11:58
gtristanpedroalvarez, so even though there's not much, I think that merits a separate directory no ?11:58
gtristani.e. it targets something real11:58
pedroalvarezi'm not sure about that one11:58
pedroalvarezyes, that's true11:59
gtristangenivi, gnome, lorry-or-trove11:59
pedroalvarezopenstack11:59
pedroalvarezI don't like lorry-or-trove11:59
gtristanIs that something that is maintained and we care about ?11:59
gtristanpedroalvarez, no, but either lorry or trove, maybe trove is better ?11:59
gtristangenivi, gnome, trove, openstack ?12:00
pedroalvarezlooks to me that the name will imply only one system, I'd go for a more generic name12:00
gtristanwhich, for trove ?12:00
pedroalvarezbut I don't know which  one12:00
pedroalvarezyes12:00
gtristanisnt there only one trove system ?12:00
gtristanI dont think thats a bad thing, honestly12:01
gtristanthere could be more for each arch that trove eventually supports12:01
gtristangnome is also only one system12:01
gtristanit just has a "lotta stuff"12:01
pedroalvarez<pedroalvarez> not sure if we will end up with loads of folders at the top level dir.. though..12:01
gtristanright, you hate folders ?12:02
gtristanhehe12:02
pedroalvarezhehe12:02
gtristanpedroalvarez, I think no because... first of all, I think we've already discussed all the systems we're planning to keep right ?12:02
pedroalvarezyou are right, others are only one system too12:02
pedroalvarezone or two systems12:02
gtristanI mean, instead of running down the list, what other systems are maintained ?12:02
gtristanjjardon likes weston-system I think, I'm not sure why though, maybe it should be a kind of gnome system ? If I understand weston is a system to test the weston wayland compositor ?12:03
pedroalvarezwhat about creating a folder for "everything else" ?12:03
paulsher1oodi would just note that we need to continue with multi-arch examples, including be-arm12:03
paulsher1ood+1 to the everything else idea12:04
pedroalvarezto put things that we don't maintain, but are useful to keep?12:04
pedroalvarezor things that are done by contributors, that we don't garantee that are 100% working?12:04
pedroalvarezcontrib/12:04
paulsher1oodyup, but i thin gtristan wants a cleaner house than that :)12:04
pedroalvarezyes, I know12:05
gtristanSigh, I dont like it, but I can bend for that one if you guys like that12:05
gtristanThe end result will be that modules will be split up anyway12:05
pedroalvarezi don't really mind, tbh12:05
pedroalvarezit just makes it easier to discover done things than digging in git log12:05
gtristanOk let me put it this way: If there is stuff in "everything else" (which we will call something like "unmaintained"); then it also means that ostree work such like Ana is doing, does not have to touch those directories at all in order to pass review.12:06
gtristanSee what I mean ?12:06
pedroalvarezyep12:06
pedroalvarezno need to make things from that folder work for reviewing12:07
paulsher1oodright... but conversely if we do accept a contrib there... it crops up on other people's diffs?12:07
* paulsher1ood is not clear on how ostree works12:07
gtristanpaulsher1ood, only until we split up definitions modules12:07
pedroalvarezi did't understand that, paulsher1ood12:08
gtristaniiuc, what paulsher1ood is saying is that if people make contributions to unmaintained/ it makes an otherwise sensible git log dirty with unrelated garbage we dont care about12:08
paulsher1oodnot quite as harsh as that, but ok :)12:09
gtristanhehe12:09
pedroalvarezso, we are going to close the project to contributions if they aren't going to be maintained?12:10
paulsher1oodiiuc you're trying to get to a situation where downstream users can consume maintained stuff as tidily as possible12:10
paulsher1oodpedroalvarez: i think gtristan is trying to separate things out, so downstreams can consume only what they care about12:10
pedroalvarez(not against, any of these ideas, just wanted to clarify given that this has happened a lot in previous years)12:10
pedroalvarezright12:11
pedroalvarezI take we will have places to hold not maintained contributions12:11
* gtristan returns from restroom12:11
* paulsher1ood had thought that separate directories was enough, but now it seems that there are effects in the git log, and possible ostree implications,12:11
gtristanOk, to clarify; and it's all in the email regarding the module splitting thing (which will not be done with submodules anyway)...12:12
pedroalvarez( \o/ )12:12
gtristanpedroalvarez, there are a few issues, and one of the most importance is that flow of changes is backwards, making it impossible for anyone to maintain a stable system in a collaborative way12:13
gtristanstable systems can only be maintained by forking/duplicating everything12:13
locallycompactin V812:13
gtristanpedroalvarez, when someone makes a commit to core strata, it should not be forced upon every maintainer of every system12:13
pedroalvarezI agree there, gtristan12:14
pedroalvarezand other systems update whenever maintainers can12:14
gtristaninstead, the maintainer of the gnome system should at times upgrade core, they do this by taking the latest stable branch of core and pointing to a new ref of core12:14
gtristanthen, their shit breaks, and they report the bug, and a patch *if they have time* to the core maintainers12:15
gtristantheir release timeline is not blocked by the whims of the maintainers of core.morph12:15
paulsher1oodisn't a likely/logical implication of this that every 'stratum' or 'assemblage' will end up in its own repo?12:15
gtristanpaulsher1ood, yes, although they will not be joined by submodules or git subrepos12:16
gtristanpaulsher1ood, actually no12:16
gtristansorry12:16
gtristanI misread your question12:16
gtristanthat would be one way to organize it, but I would not organize it as such12:16
gtristanI would instead have one module for bootstrap, and another module for everything GNU/Linux including bsps, foundation strata and core strata12:17
gtristanand then have modules like gnome, which consume strata from those modules12:17
paulsher1oodack12:17
gtristanotherwise there would be a huge amount of modules, I dont think it's necessary to split *everything* like that12:18
paulsher1oodacck - that was my worry12:18
gtristannot even sure it makes sense to split out the bootstrap, but it could12:18
paulsher1oodworth thinking about for sure12:20
locallycompactgnome strata in and of themselves shouldn't need or have knowledge of core strata12:20
locallycompactcontaining assemblages do12:20
locallycompactbc if I want to build gnome on top of jibberish-root, I don't want gnome strata module to reference every conceivable core/root12:22
jjardongtristan: ivi and weston systems are important and maintained, please do not remove them12:33
jjardongtristan: build system should be keep; its the system that can be used to build any other baserock system; also Its the system we generate the baserock chroots from12:38
tiagogomes_Is there a case for the build systems anymore, ybd is capable of building every system as long as the dependencies are installed12:42
jjardonpaulsher1ood: ok to merge https://gitlab.com/baserock/ybd/merge_requests/258 ?13:28
jjardonpaulsher1ood: maybe you prefer to build a single stratum (or a chunk), instead a whole system?13:29
jjardonactually, I think build build-essential should be enough, I will update the patch13:33
gtristanjjardon, cant we build another baserock system with the -devel system ?13:42
gtristanjjardon, if not, why do we have 2 ?13:42
jjardongtristan: I think -devel is like "include everything" system, and Im not sure anyone is using it, but I can be wrong13:43
gtristanjjardon, right, so whats wrong with using that to build "another baserock system" ?13:43
pedroalvarezsize13:44
gtristanUmmm, okaaaay13:44
pedroalvarez(just anticipating his answer)13:44
gtristanSo anyway... "Any system with bash, posix compliant coreutils and a compiler can be used to build a baserock system"13:44
pedroalvarez(still waiting for it, though)13:44
gtristanThat has to be true at all times13:45
gtristanOr well, +python13:45
pedroalvarezand build tool and dependencies13:45
gtristanRight, yaml and git13:46
jjardongtristan: AFAIK thats what the build system is13:46
gtristanjjardon, yeah, is there an excuse we can come up with to cut some fat, please ?13:47
tiagogomes_Why would someone use the build-system if they can built with the distro tools?13:47
pedroalvareztiagogomes_: because of some distros, people sometimes use chroots13:48
pedroalvarezdeployments are very tied to the host13:49
gtristanOk ok host tooling is a bit of a problem we need to solve, the bootstrap itself could be improved as well with inspiration from the aboriginal stuff13:49
jjardontiagogomes_: I can not use ybd or moprh without being root: I need a chroot, and the build system give me a minimal chroot system to build wherever I want13:49
jjardonalso reproducibility13:49
pedroalvarezI'd like to have a very minimal build system13:50
pedroalvarezless than 200M13:50
jjardonme too13:50
pedroalvarezbut i thought the goal was to make everything run outside any kind of chroot13:51
jjardonlinux-user-root and bwrap should allow that13:52
gtristanthose both have issues presently, some of which could be circumvented by placing constraints on the definitions13:54
gtristan(i.e. no creating device nodes or such)13:55
jjardonok, lets do that then?13:55
jjardonI think we need to do that to support ostree anyway13:55
gtristanplace restrictions/constraints ?13:55
jjardonno creating device nodes13:55
gtristanI am hopeful that we can get away with something else13:56
jjardonmmm, problems again cloning repos: https://gitlab.com/baserock/definitions/builds/559898113:56
gtristanmaybe a fakeroot implementation with ptrace() could do it13:56
gtristanon the other hand, yocto also does not support that13:57
tiagogomes_I have been looking at that :P13:57
*** mdunford has joined #baserock13:57
gtristantiagogomes_, at what ? yocto ? ptrace() ?13:57
gtristanor the gitlab hehe13:57
tiagogomes_attempting to do a build without being a root. There is already a fakeroot implementation that uses ptrace, Unfortunately, it doesn't support all the arches we care about.13:59
gtristanThere is ?13:59
gtristancool !13:59
tiagogomes_fakeroot-ng13:59
gtristanHmmm, can it be made to ?13:59
gtristanOr are there restrictions / roadblocks to that ?14:00
tiagogomes_yes, some one that knows kernel stuff should be able to do it in a day, couple of days14:00
gtristanOh cool14:00
pedroalvarezthat sounds good14:00
gtristanSo that's an alternative14:00
tiagogomes_The whole project is 5000 lines of code. The platform specific code could be 20014:00
gtristanI may have to be a part of bubblewrap proper though14:00
gtristantiagogomes_, would you be able to tell me if that would work recursively ? I.e. try running fakeroot-ng, and inside run bwrap to create a child namespace, and in that process... syscalls are still trapped by fakeroot-ng ?14:02
tiagogomes_No, you have to do it the other way around I think. bbwrap -> fakeroot-ng14:03
gtristanhmmm, a sandbox could still be fashioned to do that14:03
gtristanwell14:03
gtristantiagogomes_, are you sure ?14:04
gtristantiagogomes_, if you cannot, then... I worry that it will still not work, i.e. when running a build, there are many child processes going on14:04
gtristanfakeroot-ng would have to act recursively on *those* at least for the mknod/chown etc commands launched from recursive build scripts to get caught14:05
tiagogomes_tiago@debian:~$ fakeroot-ng bwrap --bind / /  sh14:06
tiagogomes_No permissions to creating new namespace, likely because the kernel does not allow non-privileged user namespaces. On e.g. debian this can be enabled with 'sysctl kernel.unprivileged_userns_clone=1'.14:06
tiagogomes_tiago@debian:~$ bwrap --bind / / fakeroot-ng echo hello world14:06
tiagogomes_hello world14:06
gtristanthe fakeroot-ng inside bubblewrap sandbox we could potentially handle (but seems more obvious the other way around)14:06
tiagogomes_I was still researching how this could work, but I don't have time to complete my research at the moment14:07
tiagogomes_There is also pseudo to consider.14:08
gtristantiagogomes_, does fakeroot-ng sport the -s option ? And if you try to make it mknod, can you see what it looks like after ?14:08
tiagogomes_gtristan it does. But on my approach I chose to get rid of device nodes entirely.14:09
* gtristan starts to think that's the only sensible option, but reluctantly14:09
gtristanin *any* case, current fhs-dirs cannot work14:09
tiagogomes_I can't see much advantages in keeping device nodes tbh14:10
gtristanbecause you want/need /dev to be full of real devices14:10
gtristanat build time14:10
gtristantiagogomes_, the ability to create tarballs with device nodes, and deploy systems which already have device nodes, without having to specify any additional metadata about them, or expecting that your system is something so specific as say, systemd based14:11
tiagogomes_gtristan, solved with `bwrap --bind / / --dev /dev` . And because you are doing `--dev /dev`  instead of `--dev-bind /dev /dev` , you get a sanitized list of device nodes at build time.14:11
gtristanright, I know that as a regular user, you cannot use fhs-dirs the way they currently are14:12
gtristanyou cannot stage them and expect to redirect output to /dev/null and have a real /dev/null, or use any of that at build time14:12
gtristanbut I dont like that you cannot distribute a system with device nodes14:13
tiagogomes_what metadata about device nodes?14:15
gtristanI mean, in say an rpm based system, one would normally specify stuff in the package that says "when you install with root privileges on a system, install this device"14:16
gtristanthats how they solve the problem14:16
gtristanit's yuck, nicer to have something virtual you just tar up the results, even if it's virtual in the sandbox; in the tarball it can be a real device node (the tarball belongs to the user, not the device node represented in the tarball after all)14:17
tiagogomes_that is what fakeroot would be useful. But like I said, I don't think we need to build or deliver device nodes14:19
radiofreethere's also the yocto fakeroot replacement https://www.yoctoproject.org/tools-resources/projects/pseudo14:23
jjardontiagogomes_: you mean remove the "devices" section from fhs-dirs ?14:24
radiofreesome things will fail to build if you do that14:26
radiofreewithout replacing it with something else14:26
radiofreee.g webkit14:26
pedroalvarezOf course the idea would be to replace it with something else?14:27
tiagogomes_I mentioned pseudo above14:30
tiagogomes_The idea is to bind-mount them from the host14:31
*** brlogger has joined #baserock14:56
*** brlogger` has quit IRC14:57
locallycompactI rebased on ybd master and now I get https://gitlab.com/baserock/ybd/builds/560312814:58
locallycompactbut not locally15:00
locallycompactno yes locally15:01
locallycompactare we sure master bootstraps?15:03
leemingwell... a patch I added for bwrap brakes on a chroot bootstrap.. so i guess it does :15:07
leeming:)15:07
jjardonlocallycompact: to be sure in the future, that's why I submitted https://gitlab.com/baserock/ybd/merge_requests/25815:11
jjardonlocallycompact: (I built a minimal system earlier today with current master with no problem)15:11
locallycompactcertainly without that bubblewrap commit it's working, with it's broke15:12
leemingyes, this was raised before. Non bubblewrap backends fail with that patch15:13
leemingthe PR was merged without testing15:13
locallycompactwerp15:18
leemingno one confirmed if it broke with linux-user-chroot though15:23
leemingso not sure if it is a broken implementation of chroot in the sandbox, or something else15:23
leeminglocallycompact, re S.I commands. have you tried with bwrap backend?15:29
leemingthat mounts things as rw15:29
leeminginstead of just ro15:29
locallycompactis that so15:29
leemingwell15:29
leeminggiven some spec from ybd yes15:29
locallycompactguide me15:30
leemingif ybd tells it a mount is rw then it makes it so15:30
leemingor for a hack, you can set writable_paths:'all'15:30
leeminghttps://gitlab.com/baserock/ybd/blob/staging/010/ybd/sandbox.py#L14115:32
leemingthat is the config that specifies if mounts are ro/rw15:32
locallycompactah that's what I missed15:32
* locallycompact slaps15:32
leemingnot sure how linux-user-chroot/chroot behave15:33
locallycompact'system' keyword15:33
leemingoh right :)15:33
locallycompacttiagogomes_, I seem to recall you had a branch that tidied up splitting logic, did that not make it in?15:47
*** gtristan has quit IRC15:48
tiagogomes_I wouldn't call it a clean up. It was more about fixing a particular bug. I never created the PR though15:50
tiagogomes_https://gitlab.com/baserock/ybd/commit/2a7963a3960c07c9e4caa1e7a4c0b69b800bd45215:50
locallycompactcertainly neater15:50
locallycompactcan that merge?15:54
* paulsher1ood is waiting to see what the issue is it's supposed to fix. or proof that it doesn't break anything15:55
* paulsher1ood is also interested in any help at all to understand recursive submodules, since ybd without it builds ci.morph. and without an example of the problem, it's hard to know what to fix15:57
tiagogomes_I tested and it worked. But there are no writing about artifact splitting that guarantees the way my patch solved the problem is the correct approach.15:58
tiagogomes_paulsher1ood, what more possible explanation would you like besides the one that is the commit message?15:58
locallycompactcan the approach to splitting be15:58
locallycompactdon't write any meta files15:58
locallycompactbuil everything15:58
locallycompactsplit on tar15:59
locallycompact(system tar)15:59
paulsher1oodlocallycompact: yes, it can. and it will be, if i get around to it or someone beats me to it.15:59
locallycompactno assemblage metas no cache entries for assemblage15:59
SotKhow does one reproduce a system with no meta files describing the things that went into it?15:59
SotK(many years in the future for example)16:00
paulsher1oodthe metafiles would be present in the system16:00
paulsher1oodand also could be re-derived from the definitions16:00
SotKpaulsher1ood: then you'd need to guarantee being able to know which commit of definitions a random system in the wild came from?16:01
* SotK thought systems were assemblages too, and took "no assemblage metas" to mean there would be none at all16:01
locallycompactI mean there isn't just an empty tarball in nowhere with a baserock meta in it16:01
locallycompactso it just goes in the one you're building16:01
locallycompactwith the original assemblage info and all the metas16:02
locallycompactbut there's no metas in kbas16:02
* paulsher1ood disagrees - but probably locallycompact is not being precise16:02
locallycompactwhich16:03
SotKthere would be a meta in the system artifact in kbas?16:03
paulsher1oodkbas already only stores chunks16:03
locallycompactdoes it16:03
locallycompactok16:03
paulsher1oodthere are no strata or systems there16:03
locallycompactright16:03
SotKoh ok16:03
paulsher1oodany artifact should contain metadata describing itself. but chunk metadata does not need to describe potential splits for systems16:04
locallycompactright16:04
SotKpaulsher1ood: +116:04
paulsher1oodhooray :)16:04
paulsher1oodso the problem is that ybd currently creates info about splits in chunks.16:05
locallycompactat the risk of dragging this out even further I'd like that to exist16:05
locallycompactso I might do it16:05
paulsher1oodlike what to ex exist?16:06
locallycompactthis behaviour16:06
locallycompacteither in mainline or 01016:06
*** tiagogomes_ has quit IRC16:23
*** ctbruce has quit IRC16:26
*** tiagogomes has joined #baserock16:26
*** tiagogomes has quit IRC16:31
*** tiagogomes has joined #baserock16:36
*** gtristan has joined #baserock17:37
*** ctbruce has joined #baserock17:43
*** locallycompact has quit IRC17:44
*** tiagogomes has quit IRC17:54
*** aren has left #baserock18:03
*** toscalix has quit IRC18:45
*** locallycompact has joined #baserock19:32
*** paulw has joined #baserock20:04
*** locallycompact has quit IRC21:20
*** ctbruce has quit IRC21:38
*** ctbruce has joined #baserock21:50
*** ctbruce has quit IRC21:54
*** gtristan has quit IRC21:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!