IRC logs for #baserock for Wednesday, 2015-04-08

*** paulw has quit IRC00:40
*** paulsherwood has quit IRC00:49
*** paulsherwood has joined #baserock00:49
*** jjardon has quit IRC02:01
*** jjardon has joined #baserock02:05
*** zoli__ has joined #baserock04:57
*** zoli__ has quit IRC04:59
*** zoli__ has joined #baserock05:01
*** zoli__ has quit IRC05:53
*** zoli__ has joined #baserock06:20
*** paulw has joined #baserock06:47
*** a1exhughe5 has joined #baserock07:04
*** perryl has joined #baserock07:15
*** Albert has joined #baserock07:30
*** Albert has joined #baserock07:30
*** bashrc_ has joined #baserock08:03
*** jonathanmaw has joined #baserock08:03
jonathanmaw*sigh*. AudioManager PoC is a project that builds separately from AudioManager (it's a qt build system, while AudioManager is CMake) and exists entirely in a subdirectory of the AudioManager repo.08:08
jonathanmawso the most sensible thing for it seems to have been to have a separate chunk for the PoC that is built after qt5-tools, and has a completely separate chunk morphology08:09
jonathanmawit also tries to install things to /opt08:09
jjardonjonathanmaw: what PoC?08:11
jonathanmawjjardon: AudioManager PoC, part of the GENIVI Demo Platform08:11
jonathanmawbut annoyingly, hasn't been merged yet08:11
jonathanmawrather than try and make a patch file out of the gigantic E-mail that E-mail links to, I stole meta-genivi-demo's patches, instead.08:12
pedroalvarez2 different chunk definitions using the same repo makes sense to me, and also sound like the right thing to do08:15
jjardonjonathanmaw: right, put those things in a separate stratum on top of Qt seems sensible08:15
jonathanmawalso, the stuff installed into /opt is hard-coded to that path :¬|08:15
*** persia_ has quit IRC08:16
*** rdale has joined #baserock08:16
*** persia_ has joined #baserock08:17
*** persia_ has joined #baserock08:17
*** fay_ has quit IRC08:17
*** fay has joined #baserock08:18
*** fay is now known as fay_08:18
*** tiagogomes has quit IRC08:19
*** mike has joined #baserock08:19
*** tiagogomes_ has joined #baserock08:19
*** mike is now known as Guest9799008:19
pedroalvarez:/ sed?08:27
pedroalvarezbut that can go really wrong hehe08:28
*** edcragg has joined #baserock08:28
*** gary_perkins has joined #baserock08:29
*** benbrown_ has joined #baserock08:35
*** gary_perkins_ has joined #baserock08:49
*** gary_perkins has quit IRC08:49
*** gary_perkins_ has quit IRC08:52
*** gary_perkins has joined #baserock08:52
*** jmacs has joined #baserock09:02
persiajonathanmaw: For /opt installation: is the upstream build system written in a way that DESTDIR=/opt would achieve the current goal, if it had proper DESTDIR support?  If so, submitting a build-system patch upstream might be the easiest way around the problem.09:14
jonathanmawpersia: nope, hard-coded paths in the qt project file for where to install the files, and hard-coded paths in the source code to where some things should be.09:18
persiaUgh.  The former is usually sortable, but the latter indicates poor practice.  It ought use one of the many available dynamic schemes for this sort of thing.09:19
persiaWhat sort of things does it try to find?09:19
*** franred has joined #baserock09:21
jonathanmawpersia: it installs scripts and audio files to /opt/audiomanager-poc/{scripts,audio-files} respectively09:21
jonathanmawthe scripts look for the hard-coded paths to those audio files09:21
jonathanmawand the source code has the hard-coded paths to the scripts09:21
persiaDoes it leave anything in /etc ?09:22
jonathanmawpersia: I don't think so.09:22
jonathanmawnope, nothing in /etc09:23
persiaHrm.  You might introduce an /etc/defaults/audiomanager-poc that contains scripts and audio-files directives, but that gets more invasive.09:23
jjardonWhat's the problem with installing in /opt ?09:23
jonathanmawjjardon: from my experience, the files don't appear, due to deployment things09:24
*** kejiahu has joined #baserock09:25
jjardonSo its a morph bug then?09:26
jonathanmawjjardon: I hadn't considered whether it was intentional09:28
persiaWhether Baserock-generated systems should have and/or use /opt is a complicated question.09:29
persiaIt's probably a bug in morph that there is no way to do this, but it is also arguably a bug in any software that forces doing this, as it cannot be part of the distributed system and remain in compliance with FHS.09:30
persia /opt is generally for software added to a system, post deployment.09:32
persia(that line should have appeared above the other two, but was initially interpreted as a command: apologies for confusion)09:33
jjardonAFAIK /opt is part of the FHS. But even if not, i think morph should support installing components there (we create /opt in baserock BTW, see the fhs-dirs chunk)09:41
richard_mawjjardon: yep, nobody is suggesting that it isn't a bug that we can't install stuff into /opt as part of a baserock build09:42
franredgood morning, this week and the next, the Openstack in Baserock team is going to be upstreaming their branch, we will be very grateful if some other baserock member can have a look at the patches we will be posting here, thank you in advance!!09:43
persiajjardon: Yes, /opt is part of FHS, but it's reserved for stuff being installed later.09:43
franred <<-- Kmod fixes for working with iSCSI modules and userland09:44
persiaSpecifically, from FHS 2.3, section 3.13.1: "/opt is reserved for the installation of add-on application software packages", which is not generally interpreted to include anything that is part of a deployed system.09:44
persiaAnd 3.13.2 indicates that the child directories of /opt are "reserved for local system administrator use".09:45
persiaGiven the blurring between system developers and system administrators implied by Baserock, this isn't as simple as it looks, but still a strong indicator that there is reason for controversy.09:46
pedroalvarezATM one can install things in /opt on a running system, and they will be there after a system upgrade09:47
persiaThat is definitely correct behaviour, and should be retained.09:51
persiaBut I believe it to be orthogonal to installing software in /opt as part of deployment.09:52
pedroalvarezIt is a side effect. Upgrades will be more complex if we allow installing things in /opt09:55
pedroalvarezSomething similar happens with /var, which contains things installed at build time, and things that are added later. (things that we want to keep after upgraing a system)09:56
pedroalvarezNow upgrades will override /var with the new /var created when upgrading.09:57
pedroalvarezIf that is sensible for /opt too, we could do the same.09:57
richard_maw /var ought to be a bit simpler, software ought to not require any setup at build or deployment time, we ought to be able to leave /var as-is, because a system ought to be able to boot with an empty /var09:58
jjardonpedroalvarez: : there is some WIP trying to tackle that problem:
persiaMy main issue with overwriting /var is the potential to lose logs or critical historical data from /var/lib10:04
*** persia_ has quit IRC10:04
*** persia_ has joined #baserock10:05
*** persia_ has joined #baserock10:05
straycatDoes anyone think it would be useful to put .gitreview files into all the repos that are now on gerrit?10:11
bashrc_I did start doing that on gerrit10:12
*** DavePage has joined #baserock10:12
persiastraycat: +110:13
* straycat updates change
straycatit's also quicker to send subsequent versions with this10:19
* richard_maw needs to sit down and have a go at `git review` at some point10:28
straycatit's pretty much just, git review -s (to setup), git review -R (to submit the current branch without rebasing), and git review -d <change number> to download the change. You also want a .gitreview file in the repo telling git-review where to get its git review, but if we push those into all gerrit repos then it should be very convenient10:32
*** wschaller has joined #baserock10:36
bashrc_is there any existing code for determining the dependency order of a set of morph files?10:45
nowsterpresumably there is within morphlib...10:45
bashrc_might save me from rewriting something10:46
straycatfranred, richard_maw, i'm slightly confused by the addition of $PREFIX to the kmod morph?10:47
franredstraycat, what are your concerns?10:48
straycati thought it was supposed to be installed into /bin ?10:48
*** sambishop has quit IRC10:56
franredstraycat, iscsi kernel modules suppose to be installed in /sbin10:56
franredstraycat, also we want to move every binary to be installed in /usr/bin, /usr/sbin  to be able to merge /bin /sbin to /usr/bin /usr/sbin, so I moved it. Why did you think that kmod has to be installed in /bin? what are your reasons?10:58
* pedroalvarez appreciates that10:59
straycatoh i didn't even know we were doing that10:59
straycatand just that my debian box has it in /bin, and that's for "essential" binaries, which i guess might include kmod? so that's why i asked11:00
pedroalvarezstraycat: we sould like to do that afaik. Problems like "foo is not overriding busybox-foo" will disappear11:00
* bashrc_ finds ArtifactResolver11:01
straycatmoving all the binaries into /usr might be interesting, given that some things in build essential use hardcoded paths to binaries in /bin11:02
straycati guess you can leave symlinks for those11:02
pedroalvarez /bin /sbin and /usr/sbin will be symlinks to /usr/bin, yes11:03
* persia actually liked the separation, and used to use environments where it was important, but acknowledges that in modern systems with massive available secondary storage (measured in multiple megabytes), it doesn't matter.11:06
*** sambishop has joined #baserock11:11
*** Krin has quit IRC11:25
*** Krin has joined #baserock11:29
*** mdunford has quit IRC11:33
richard_mawAIUI we expect that the initramfs is the one to make /usr available now11:34
franredstraycat, richard_maw, <-- I've added richard_maw's comment as a warning11:35
* paulsherwood is up to his ears, but would like to understand whether morph *really* ignores names, or not.11:35
* paulsherwood notices that foo chunk can have many build-depends... aren't those are only referred to by name?11:36
paulsherwoodrichard_maw, straycat ^^ ?11:36
richard_mawI'm not sure I follow the question, or the source of your confusion as to whether morph ignores names. the build-depends refer to other components by name, but currently I'm a little fuzzy about whether it's the "name" field in the stratum or in the chunk that is being discussed11:38
*** mdunford has joined #baserock11:49
*** wschaller has quit IRC11:56
paulsherwoodrichard_maw: eg stage1-gcc build-depends on stage1-binutils, or more crucially,
paulsherwoodi assume hoe depends on ruby the chunk, based on its name12:26
richard_mawit must, since it cannot be depending on its stratum, or it would fail to build because of a dependency loop12:27
paulsherwoodcorrect. so i believe this establishes that morph *does* use the name field?12:28
richard_mawdo you mean the name field in the chunk definition or in the chunk list in the stratum definition?12:29
paulsherwoodeither or both, for my purposes i'm just trying to establish that morph relies on the name - removing name: would break morph12:30
richard_mawremoving the name completely would yes, since it would have no other way to represent build dependencies12:30
paulsherwoodand trying to rely on path instead would not work unless *all* details for ruby (eg repo ref as well as *commands) would need to be in strata/ruby/ruby.morph12:31
richard_mawpaulsherwood: yep, that plus you couldn't ever depend on a split artifact without us changing how they are defined completely12:32
paulsherwoodwhich is a much more invasive change, than (say) dropping path and making name:s unique? :-)12:33
paulsherwood(which would improve clarity imo)12:33
*** Guest97990 has quit IRC12:33
richard_mawit would be a more invasive change, but it may also make split artifacts more intuitive12:33
richard_mawsince currently the rules are half built-in and half complicatedly defined, but if we split them out into a flie, then it can be made more obvious how they are defined12:34
richard_mawI like the idea of file paths, because it forces the structure to have no duplicates, and we don't need to load every definition file in the repository to make sense of the definitions, but it is a wider change to how splitting rules work, and prevents us ever doing in-line chunks12:36
straycatthere's no reason logically to enforce the definitions of a name in the strata if it's defined in the chunk that's being referenced from the stratum12:36
richard_mawstraycat: well, I'd prefer it went the other way around, since you need to use the name in the stratum12:37
richard_mawyou don't need to use the name in the chunk definition12:37
straycati don't see why we can't have them defined in the chunk definition and optionally inlined in the stratum12:38
straycatbut i was just suggesting random ideas that have been suggested before, which has brought new attention to12:39
richard_mawoh you could, I just think it's extra congnitive load when trying to reason about a stratum if you have to constantly flick between two files to see what's going on12:39
* paulsherwood notes believes the claim that morph doesn't use name has been refuted. and that therefore morph is currently relying on path *and* name12:40
straycati don't see where it got refuted12:40
paulsherwoodmorph ties hoe's dependency to ruby based on the name: field for the ruby chunk12:42
straycati think i've misunderstood what we mean by 'name' then, i was talking about the name field defined in the stratum, which doesn't add any information12:42
*** gary_perkins has quit IRC12:42
richard_mawwe're currently arguing about the duplication of 'name' in both the stratum and the chunk12:43
*** gary_perkins has joined #baserock12:43
paulsherwoodi'm interested in making this generic, as you know - a definition can contain definitions. special handling for 'chunk' and 'strata' shouldn't be necessary imo12:44
paulsherwoodanyways, i'm labouring my point and should let folks get on with more useful stuff :)12:45
* richard_maw is as-yet unconvinced that bringing it down to one kind of definition will work, but we ought to be able to merge systems and strata12:57
*** mdunford has quit IRC13:00
*** Krin has quit IRC13:01
* richard_maw actually feels like we should introduce new definition types such as 'build-system', 'split-rule' and 'split-artifact'13:02
paulsherwoodrichard_maw: what would convince you? if we make things unique i can already show build working, while dropping the meaning of the kind: field ('chunk' 'strata' 'system') entirely13:04
paulsherwood(for a certain value of working.. i've not deployed anything yet)13:06
bashrc_the three layer structure is probably ok. I think of it as programs, groups of programs and systems. It could all be defined in a single file, but that be quite messy.13:06
bashrc_the fieldnames of things should be unique, for machine readability13:06
paulsherwoodthe suggestion is more nesting, not less bashrc_13:06
paulsherwoodwe already have it, in effect, with clusters13:07
paulsherwood(but that's me digressing)13:07
bashrc_merging systems and strata might simplify things, but also might result in more duplicated maintenance. A big factor is the ability to maintain and add new things to a system. If the complexity is too great then users will just go elsewhere.13:10
*** franred has quit IRC13:12
paulsherwoodi'm not suggesting merging them13:15
paulsherwoodi'm suggesting possibility of a stratum containing other strata, or systems containing subsystems which contain strata containing chunks. but all those nouns go away and we have a definition can contain definitions, where definition replaces system, subsystem, stratum, chunk13:17
pedroalvarezso, one can define everything in one definition, but is not mandatory :)13:18
pedroalvarezI like the idea13:18
paulsherwoodpedroalvarez: yes, that's my hope. one or many, in one directory or many, user can choose13:19
*** franred has joined #baserock13:21
*** gary_perkins has quit IRC13:23
*** gary_perkins has joined #baserock13:26
*** wschaller has joined #baserock13:29
straycatcan anyone review this quick change:  ?13:40
*** franred has quit IRC13:45
*** edcragg has quit IRC13:48
*** zoli__ has quit IRC13:53
*** simonh_ has quit IRC13:57
*** franred has joined #baserock13:57
*** Krin has joined #baserock13:57
*** simonh_ has joined #baserock13:57
*** edcragg has joined #baserock13:59
*** Zara has joined #baserock14:00
*** gary_perkins_ has joined #baserock14:06
*** mdunford has joined #baserock14:06
*** gary_perkins has quit IRC14:06
*** tiagogomes_ has quit IRC14:08
richard_mawpaulsherwood, bashrc_, jonathanmaw, persia: any chance any of you have time to review ^ or ? I'd like someone who isn't working on OpenStack to be aware of the changes involved, rather than having the slightly suspect practise of approving it internally.14:09
*** tiagogomes_ has joined #baserock14:09
jonathanmawrichard_maw: done14:20
richard_mawjonathanmaw: and ?14:22
jonathanmawrichard_maw: more than I have time to think about that the moment, sorry.14:23
*** zoli__ has joined #baserock14:23
richard_mawthanks for reviewing #22714:23
* straycat discovers gerrit themes14:26
bashrc_can chunks contain build-depends?14:30
straycat14:40  straycat$ can anyone review this quick change:  ?14:41
richard_mawjjardon maybe? ^14:46
*** petefoth has quit IRC14:49
richard_mawI've pinged everyone active today. straycat: how much is this change blocking you?14:50
straycateh, i'll live14:52
franredin any case I vote +1 - so you have 2 +1 anyone external to Openstack project can also review it but if this does not happened it can be merged14:55
franredrichard_maw, straycat ^^14:55
straycatok cool thanks14:55
*** paulw has quit IRC15:03
*** paulw has joined #baserock15:05
* straycat wonders if it'd be nice to allow the use of multiple artifact cache servers15:08
pedroalvarezit would be really really nice15:08
pedroalvarezso that a distbuild I have running here locally can reuse artifacts15:09
jjardonHi, I heard baserock has a import tool, where is it?15:48
*** mike has joined #baserock15:49
jjardonstraycat: thanks!15:49
pedroalvarezand some docs:
*** mike is now known as Guest5310615:49
pedroalvarezwell, this one better:
jjardonpedroalvarez: amazing, thanks15:49
*** simonh_ has quit IRC15:57
*** simonh_ has joined #baserock15:58
franrednew changes and
*** a1exhughe5 has quit IRC16:02
*** DavePage has quit IRC16:03
straycatwhy do we want to move that to python-common?16:11
franredstraycat, we need it for openstack-services.morph for cinder16:11
straycatif it's solely for openstack can it not stay in one of the openstack strata?16:12
franredstraycat, it is cluod-init16:12
franredis in*16:12
* franred sight....16:13
franredstraycat, it is in cloudinit-support.morph and we will need it in openstack-services.morph16:13
franred(I can't do English today...)16:13
straycatoh i see16:14
*** DavePage has joined #baserock16:14
straycatthat seems reasonable then16:17
pedroalvarezbase systems don't have python-common/python-core?16:21
straycatoh dear16:22
straycatthat might be a problem then16:22
straycatwait, why do base systems need cloud-init?16:22
pedroalvarezgit blame will blame me, since I added cloud-init to baserock16:23
richard_mawstraycat: because they might be deployed into an openstack16:23
franredpedroalvarez, only x86 base-systems have cloud-init configuration extension16:24
straycatsince we've been having abstract debates about definitions formats, i'll add that having this flat name space is perhaps not the best, generally people are working on a subset of definitions and maybe it's unreasonable to expect them to reason about the whole thing16:24
pedroalvarezrichard_maw: I agree with that, but I wouldn't mind if we remove it from base systems16:24
straycati mean here we are making changes to openstack and we've somehow almost broken the base system16:26
straycatwere it not for pedroalvarez spotting that :)16:26
franredjjardon, could you have a look at ?16:26
franredstraycat, the basesystem were broken before this patches arrived16:26
*** flatmush has quit IRC16:27
straycatfranred, oh?16:27
franredand I have to blame me for not realizing about that when I moved things from cloud init16:27
richard_mawruntime dependencies!16:27
straycathow are they broken?16:27
pedroalvarezrichard_maw: +1!16:27
pedroalvarezstraycat: well, cloud init stratum needs python-core and python-whatever to run I guess16:28
jjardonfranred: sure16:28
pedroalvarezI'm sure that we will end up adding support for runtime dependencies, but is taking time for other people to see that need :)16:29
franredpedroalvarez, python-core and python-common16:29
straycatpedroalvarez, sorry i still don't understand why the base system is broken?16:29
franredI can make a fast patch to add these to base-systems if we want to continue supporting them to deploy to openstack16:30
franredpedroalvarez, ^^16:30
straycati'm talking about current HEAD sans franred's patches16:30
pedroalvarezstraycat: base system is missing runtime depencencies for cloud-init16:30
straycatthe base system includes cloudinit-support?16:31
straycatwell the x86 ones do16:31
pedroalvarezis not really broken, but cloud-init won't work16:31
straycatwhy not?16:31
pedroalvarezif cloud-inits is missing runtime dependencies, I can't see how it might work16:31
pedroalvarezsome things were moved from cloudinit-support to python-*, and those strata weren't added to the base systems16:32
pedroalvarezThat's why I'm saying that it might be missing runtime depenencies16:33
straycatahh okay16:33
* franred creates a patch to fix that16:34
pedroalvarezfranred: thanks16:35
straycatwhat fix are you going to apply?16:35
franredstraycat, I will add python-common and python-core to the systems which have cloudinit-support.morph16:36
jjardonId create a new system: base-system-opensatack os something like that instead change our current base systems16:36
franredpedroalvarez, I don't understand why cloudinit-support.morph needs build-essential and foundation as build dependencies... is not the former inside the latest?16:37
richard_mawyes, but there was a push for more explicit build dependencies16:37
pedroalvarezwe are not being consistent anyway :/16:37
franredrichard_maw, can I not remove build-essential as a build dependency then?16:38
straycatoh damn, i thought people were only going to do that for chunk dependencies16:38
richard_mawstraycat: that's only the latest push, we've had pushes to do so before16:38
* jjardon prefer everything explicit16:39
jjardonIts a mess when you create/change strata and you have to move things around16:40
richard_mawwhat bites us when moving things around is how you need to list everything in the system, but it includes recursively in the stratum, we could fix this by having some form of implicit dependencies in the system16:43
*** flatmush has joined #baserock16:43
*** Guest53106 has quit IRC16:46
straycati also think what bites us is we expect the user to reason about an entire graph of components where changes in one section can propagate to changes in another completely unrelated section. i keep thinking it'd be nice to allow subgraphs to override components inherited from other sections, but that might be stupid/crazy16:46
straycati'll quiet down now and go do something useful :)16:47
straycatbut other things do it, so it's obviously not that stupid16:51
*** Albert has quit IRC16:58
*** bashrc_ has quit IRC17:02
franredstraycat, ^^17:10
straycatthat makes the x86 base systems inconsistent with all other base systems?17:12
franredstraycat, the other base systems doesn't have cloudinit-support17:18
straycati know, one base system having python and another not having python seems like a substantial difference for something that's supposedly the same system17:19
straycatcould you not create a common stratum that cloud-init and openstack could depend on?17:21
franredstraycat, what you are describing is python-common17:22
franredstraycat, also the base-systems are already differents because some of them support cloudinit and other no17:23
franredI can't ses your point sorry17:23
pedroalvarezstraycat: so you are suggesting to remove cloudinit from those systems?17:24
pedroalvarezor adding them also to the others?17:24
straycatthat or create a common stratum for configobj17:24
pedroalvarezstraycat: but they were more things in common between cloud-init and openstack17:25
pedroalvarezthat's when we decided to create python-common17:25
*** edcragg has quit IRC17:25
pedroalvarezstraycat: so you are suggesting to create another stratum for configobj?17:32
straycateither that or remove cloud-init from base-system, or if you need python-common then add it to all base systems, or if you need python-{common,core} add it to all base systems, but i doubt people would want python in the base system17:34
pedroalvarezjust answered in gerrit about "python in the base system"17:36
pedroalvarezI see your point though.17:38
straycati replied :)17:38
pedroalvarezI'm starting to think that we could remove cloud-init from there17:40
pedroalvarezone can deploy them to opentack anyway17:40
straycatthat would probably be ok then17:41
straycati wish we could just define these things in multiple places it'd be much simpler17:42
pedroalvarez"The set of strata required to have a minimal system"17:43
pedroalvarezmaybe makes sense to remove it from the base systems17:43
pedroalvarezmaybe creating a cloud-base system?17:43
*** franred has quit IRC17:52
*** franred has joined #baserock17:54
*** franred has quit IRC17:55
*** wschaller has quit IRC18:12
*** zoli__ has quit IRC18:29
jjardonThat's what I suggested before:17:36 <jjardon> Id create a new system: base-system-opensatack os something like that instead change our current base systems18:40
*** gary_perkins_ has quit IRC20:24
*** flatmush has quit IRC20:26
paulsherwoodi was arguing for explicit dependencies for chunks, not for strata. for strata they nest anyway. if strata-c depends on strata-b which depends on strata-a, no need to specify strata-a in c20:55
paulsherwood(and only to be explicit about depencies on sibling chunks in the same stratum, since the chunks in build-depend strata are clearly implied)20:56
*** pedroalvarez has quit IRC23:50
*** pedroalvarez has joined #baserock23:51
*** ChanServ sets mode: +v pedroalvarez23:51
*** juergbi has quit IRC23:57
*** juergbi has joined #baserock23:57

Generated by 2.15.3 by Marius Gedminas - find it at!