IRC logs for #baserock for Wednesday, 2016-08-10

*** toscalix has quit IRC02:27
*** gtristan has joined #baserock05:16
gtristanwhat is git-minimal really required for in core.morph ?05:44
gtristani.e. bison and util-linux both build-depend on git-minimal, any reason they should ?05:44
gtristanmaybe cruft from the olden days when git submodules were checked out in the sandbox ?05:45
paulsherwoodgtristan: i think it's more cruft from when baserock was all self-contained, and we wanted git on all systems06:43
*** fay has joined #baserock07:02
*** fay is now known as Guest7749807:03
*** Guest77498 is now known as faybrocklebank07:03
gtristanpaulsherwood, I was more wondering why specific chunks were directly depending on it07:03
paulsherwooduser error, i think07:04
paulsherwoodactually, i think i'm wrong07:06
paulsherwoodgtristan: ^^07:07
gtristanpaulsherwood, "chunks such as libtool depend on git" that sounds bogus :-/07:30
gtristanunless there is another reason, i.e. libtool has gnulib submodule maybe ?07:30
gtristan"libxml2 depends on git" :-/07:31
gtristananyway, not so important07:31
*** jonathanmaw has joined #baserock08:05
pedroalvarezi don't think it's cruft. Some chunks require git to build08:09
pedroalvarezIf git is in core, it's because is needed08:09
gtristanpedroalvarez, you dont think it's because at some time, it was needed to checkout submodules ?08:11
jjardonpedroalvarez is rigth08:11
pedroalvarezgtristan: nope08:11
gtristanpedroalvarez, I dont think there is any way at all, that libxml2 actually requires or uses git in any way08:11
pedroalvarezsome chunks, require it just to set the version number of the tool you are building from git, with a git-describe or somethin like that08:12
pedroalvarezthat might be the case with some of the chunks you are mentioning08:12
pedroalvarezI believe we looked at removing those dependencies quite recently, so i believe we made the right decision08:13
pedroalvarezthanks jjardon08:15
gtristanjjardon, I see08:17
pedroalvarezregarding ybd failing when pointing to a morph file tha doesn't exist, I think that would be good. Although I wonder when was this broken in definitions.git and why we never fixed it when using morph08:17
gtristanthat is arguably a bug in the configure script, to make the assumption that because you have a .git directory, that git is installed, but on the other hand it's usually a relatively safe assumption08:18
paulsherwoodpedroalvarez: it was a bug in definitions a long time ago, has been fixed some time ago. morph didn't spot it because morph only parses what it needs for a given run, whereas ybd parses all definitions every time08:19
pedroalvarezoh, I see08:26
*** locallycompact has joined #baserock08:53
*** edcragg has joined #baserock08:53
pedroalvarezpaulsherwood: cannot you mark the definition as faulty, and only fail if that definition is being used?08:54
pedroalvarezmy point of view in definitions versions is that it helps to keep the build tools free of old code08:55
paulsherwoodpedroalvarez: that's a good idea, thanks!08:56
paulsherwoodpedroalvarez: wrt definitions versions - i don't think your theory is correct :)08:58
pedroalvarezheh I wouldn't call it a theory08:59
pedroalvarezwe removed loads of code when we stopped supporting some old versions08:59
paulsherwoodat the expense of breaking morph for users, iiuc09:00
* paulsherwood is happy at the idea of removing loads of code, though09:01
pedroalvarezwell, you could call them "flag-days", I would call them change the definitons version09:01
pedroalvarezwhat I mean is that you were suggesting doing something similar in your "Downstream metadata" email. Breaking ybd for users at a given day09:03
paulsherwoodpedroalvarez: as i tried to explain in there are too many moving parts. if upgrading morph causes users to have to upgrade definitions, or vice-versa, while also wresting with their version of 'baserock system' it's unworkable09:03
pedroalvarezwhat if the build tool itself says: "Hey, this is an old version of definitions, do you want me to upgrade them?"09:04
pedroalvarezthat would be more user friendly09:04
pedroalvarezbut yes, still users would have to upgrade definitions09:05
*** tiagogomes has joined #baserock09:05
paulsherwoodit's an interesting idea. doesn't sit well with our stated philosophy of very long term reproducibility09:05
locallycompactif the build tool is capable of detecting that it's capable of parsing it and using it.09:06
pedroalvarezI remember using a build tool that would upgrade/downgrade itself to the version needed to parse the  "definitions"09:06
locallycompactbut still nice09:06
pedroalvarezanyway, I still think that versioning definitions could help us moving forward without creating spaghetti code09:08
paulsherwoodpedroalvarez: i agree versioning is a helpful part of the puzzle, but i don't think we've got the approach 'right' yet09:09
* pedroalvarez nods09:10
pedroalvarezI have the same feeling, maybe a bit more decided that is useful right now09:10
paulsherwoodis thera a way to delete from
pedroalvareznot from the UI, I could try to go to the database if this is important09:14
paulsherwoodit's not important, i overreacted :)09:16
paulsherwoodthanks though09:16
paulsherwoodi'll know for next time09:16
*** gtristan has quit IRC09:52
*** brlogger has joined #baserock09:55
*** gary_perkins has quit IRC09:58
*** gary_perkins has joined #baserock09:59
paulsherwoodwould this be of interest?
paulsherwoodbackground, i'm trying to create an unusual dir layout, want to re-use fhs-dirs by feeding it parameters instead of hacking the script10:11
* locallycompact likes10:12
persiaPersonally, I think you ought use a replacement chunk or something.  FHS implies "Standard"10:12
persiaNote that I like the code, just not the name :)10:14
paulsherwoodpersia: well, to be clear, the default data there would deliver FHS 'standard' (although i need to test)10:14
paulsherwoodso it would only do something non-standard if user feeds it nonstandard data10:14
persiaYes, but if I use something called file-hierarchy-standard, I'll be surprised if I can get nonstandard results.10:14
persiaWhereas, if I used something like make-file-hierarchy that can take or, I'm less surprised.10:15
paulsherwoodok, maybe i'll create a new thing to achieve that then10:15
richard_mawsystemd's tmpfiles stuff does this kind of thing10:16
richard_mawit's designed for making the directory layout on boot, but it can also be used at other times10:16
persiarichard_maw: Would it work sensibly in a chroot, where we may not have full access to host tooling (or daemons)?10:18
paulsherwoodrichard_maw: my use-case can not assume that systemd is present10:19
paulsherwoodbut it's interesting there are other ways to do this10:19
paulsherwoodis the ln /proc/mounts /etc/mtab part of FHS also?10:20
richard_mawpersia: Fairly sure it doesn't talk to any daemons, it works by taking config files or reading its default paths10:20
richard_mawpaulsherwood: not sure, but modern linux systems don't work so well if you don't do it10:21
paulsherwoodrichard_maw: tvm10:21
persia is the official reference10:21
paulsherwoodpersia: so 'make-file-hierarchy' is better than 'create-directories' ?10:22
persiaHaving /etc/mtab is required by FHS, but how it gets there can be implemented lots of different ways.  Using /proc/mounts means the kernel becomes responsible.10:22
persiapaulsherwood: I'm fine with "create-directories".  I was only opposed to "*standard*" that could generate non-standard results.10:23
paulsherwoodok thanks10:23
*** vgrade12 has joined #baserock10:26
*** jonathanmaw has quit IRC10:27
*** jonathanmaw has joined #baserock10:27
* paulsherwood needs to stop yak-shaving, now, and do the thing10:34
pedroalvarezpaulsherwood: btw, maybe a bit messy all in one commit, but this would be moving all genivi-related definitions to a subfolder:
*** edcragg has quit IRC10:46
paulsherwoodwhy not juss s/definitions-genivi/genivi/ ?10:53
pedroalvarezwanted to go with definitions-genivi because it will be easier to go to "genivi" if wanted than the other way around10:54
pedroalvarezcan be just "genivi", but not sure if it will be clear that they contain definitions as well10:55
persiaThat the repo is definitions.git is a strong hint11:05
* pedroalvarez calls it meta-genivi11:06
persiaThe reduction to absurdity argument has as an example /root-usr/root-usr-local/root-usr-local-share/11:06
paulsherwoodpedroalvarez: shame on you :)11:10
*** CTtpollard has quit IRC11:18
*** CTtpollard has joined #baserock11:19
pedroalvarezNew Changes:11:19
pedroalvarez Move GENIVI definitions to 'genivi' subfolder11:19
paulsherwood+1, i can maybe try a build later11:20
pedroalvarezrebuild may be no-op if it was already cached11:32
paulsherwoodpedroalvarez: sadly not. paths are part of the cache-key11:35
paulsherwood(for ybd at least)11:38
pedroalvarezany reason for that?11:39
paulsherwoodactually, maybe i'm wrong11:40
*** gtristan has joined #baserock11:47
persiaIf path-of-definition-in-repo is a cache-key element, why?  What part of definitions breaks as a result of renaming a file?  How would this be expected to modify a build (assuming same definition inside the file)?11:57
paulsherwoodyes, i agree. i'm trying to reinsert the logic in my brain11:59
paulsherwoodpath is used as unique key for each definition in ybd12:00
paulsherwoodcan't use names, because names are missing in some cases, and unique in others12:00
paulsherwoodeach build-depend is referred to by its path, also12:01
paulsherwoodhence, paths of build-dpends of foo are in cache-key for foo12:02
persiaI see the logic, and have to agree that, given the implementation, it is sensible.  I still believe it to be wrong, at a deep level.12:03
persia(but something to fix in yet-another-respin, rather than something to address with current definitions, or tooling expected to process it, especially considering the defintions-versioning discussion earlier)12:04
paulsherwoodwell, in the absence of a unique names, i couldn't see how else to key definitions12:04
persiaYes.  The correct solution is to have, and enforce unique names.12:04
persiaThat requires a change of such significance, that it is not consistent with long-term support of existing defintions.12:04
pedroalvarezby the contents of the thing and not by the name/path?12:04
paulsherwoodi gave up arguing on that.12:04
persiaBut it is only by exploring this space that we learn such rules, so have a chance of actually making something that works.12:05
paulsherwoodpedroalvarez: that might be quite neat actually12:05
* pedroalvarez bows12:06
paulsherwoodbut i think it would require an extra pass.. since we can't hash the contents of the thing until we've collected it all12:06
paulsherwoodie thing  may be a chunk, in which case its contents are  (probably but not always) split across two files12:08
*** toscalix has joined #baserock12:08
paulsherwoodsimilarly stratum has its contents spread across the chunk morph files too, etc12:08
paulsherwoodand my head hurts now :)12:09
persiaI don't think hashing is safe for that: I think we'd want to compare definitons content either as text or as some other well-known representation.12:09
paulsherwoodlike yaml? :)12:09
persiaThat would be text12:10
*** tiagogomes has left #baserock12:10
*** tiagogomes has joined #baserock12:11
persiaIf there were other advantages to using a non-text serialisation format, comparing that might work also.12:11
persiaOne of the risks with YAML is that one can construct two different text files with YAML that have the same logical content, as applied for a given definition.12:11
persiaBut if one uses a fairly simple serialisation output mechanism, one is likely to get the same output YAML from both inputs.12:12
persiaso validation of identity by textual comparison of generated YAML should be safe.12:12
jjardonpersia: mtab can only be a symlink to /process/self/mounts . At least if you use systemd12:13
persiajjardon: systemd breaks if it detects it is not a symlink, or does it just expect the same content?12:13
persiaAny reason I couldn't (assuming I had too many CPU cycles), use inotify to see changes in the kernel-view of mounted filesystems, and write a text file to /etc/mtab on each change?12:14
* persia thinks this would be a poor implementation, fragile, and prone to error, but wants to understand if systemd is unable to deal with systems that are broken by design)12:14
jjardonAFAIK it breaks in recent versions12:15
* persia is insufficiently motivated to test this and file a FHS-compliance bug against systemd if it is true12:17
richard_mawsystemd uses libmount and systemd's NEWS file claims libmount breaks without it, and the default configuration of systemd-tmpfiles will remove /etc/mtab and turn it into a symlink to /proc/self/mounts12:23
richard_maw might be of interest12:24
persiaOh, heh.  Yes, pitti doesn't really care about FHS, and routinely breaks it (causing the rest of the world to play catch-up).  There still exists the bug, and someone should extend the /etc/mtab exception in FHS to make the file optional.12:35
*** toscalix_ has joined #baserock12:51
*** toscalix has quit IRC12:53
*** toscalix_ has quit IRC12:57
*** edcragg has joined #baserock14:10
*** CTtpollard has quit IRC14:48
*** CTtpollard has joined #baserock14:52
*** gtristan has quit IRC15:00
*** jonathanmaw has quit IRC15:17
*** locallycompact has quit IRC16:10
*** rdale has quit IRC16:11
*** rdale has joined #baserock16:14
*** faybrocklebank has quit IRC16:25
*** tiagogomes has quit IRC17:23
*** tiagogomes has joined #baserock17:26
*** edcragg has quit IRC17:27
*** edcragg has joined #baserock17:30
*** tiagogomes has quit IRC17:33
paulsherwoodcan anyone help me understand how to support `mkdir -p foo/{bar,bip,bop}` in install-commands?17:35
paulsherwoodie allwing the { } to be handled properly by mkdir17:35
*** edcragg has quit IRC17:42
rjekThe shell expands that17:50
rjekSo if you're using exec it probably won't work17:50
rjek(ie, execing mkdir)17:50
paulsherwoodwell ybd passes it via an extended sandboxed shell command, and yes, it doesn't work17:56
paulsherwoodno way round this?17:56
paulsherwood(short of expanding all of the examples longhand)17:56
persiaNot easily.  You could force the end-shell as bash, but that probably takes some doing.18:30
persiaGenerally speaking, outside of an explicit bash script, it is best to stay with POSIX semantics, despite them being more verbose.18:31
*** gtristan has joined #baserock19:09
*** toscalix has joined #baserock20:09
*** toscalix has quit IRC22:00
*** toscalix has joined #baserock23:05

Generated by 2.15.3 by Marius Gedminas - find it at!