IRC logs for #baserock for Friday, 2015-06-26

*** pacon has joined #baserock00:51
*** petefoth has quit IRC01:00
*** pacon has quit IRC01:07
*** petefoth has joined #baserock01:09
*** pacon has joined #baserock01:59
*** pacon has quit IRC02:12
*** pacon has joined #baserock02:14
*** pacon has quit IRC03:53
*** pacon has joined #baserock03:55
*** pacon has quit IRC04:41
*** pacon has joined #baserock04:41
*** zoli__ has joined #baserock04:51
*** zoli__ has joined #baserock04:53
*** zoli__ has quit IRC06:43
*** zoli__ has joined #baserock06:44
*** zoli__ has joined #baserock07:04
*** bjdooks has joined #baserock08:00
*** bashrc_ has joined #baserock08:00
*** bashrc_ has quit IRC08:03
*** bashrc_ has joined #baserock08:03
*** gary_perkins has joined #baserock08:28
*** jonathanmaw has joined #baserock08:34
*** mariaderidder has joined #baserock08:35
perrylis there anything defined for how morph utilises parameters set in morph.conf? is it via cliapp and['parameter']?08:36
*** mwilliams_ct has left #baserock08:36
*** ssam2 has joined #baserock08:37
*** ChanServ sets mode: +v ssam208:37
Kinnisonperryl: yes, the config file (and command line parsing) is all handled by cliapp08:44
*** edcragg has joined #baserock09:12
*** lachlanmackenzie has joined #baserock09:18
paulsherwoodjonathanmaw: would it make sense for you to join #automotive?09:44
paulsherwoodi think that's where agl folks hang out09:44
jonathanmawcouldn't hurt09:44
paulsherwoodand i don't want to distract #baserockers with too much exciting yocto discussion09:44
rjekDid somebody say Yocto?09:44
* rjek gets excited.09:44
paulsherwoodrjek: it's all happening in #automotive :-)09:45
*** zoli__ has quit IRC10:52
*** zoli__ has joined #baserock11:15
*** zoli__ has quit IRC11:21
*** zoli__ has joined #baserock11:21
persiaThinking about the tarball problem, I wonder if it makes sense to have some way to describe *how* to generate the tarballs.12:12
persiaThe idea being that given a reference build system artifact, we can build a reference development system.  Then, given a reference development system, we can generate the bootstrap tarballs from git repos.12:12
persiaThen we can use the bootstrap tarballs from an arbitrary system to bootstrap the build system.12:12
*** tiagogomes_ has quit IRC12:13
persiaThe reason I think this would be nice is that we don't always know precisely what happens in the upstream release process, some of which is sometimes done by humans, meaning that it may be tricky to ensure that we can perfectly replicate something starting from a tarball (as a source change may potentially cause a generated tarball to change, which would differ from that same source change being done against a completed tarball tree, without12:14
persiaregenerating tarball artifacts).12:14
richard_mawpersia: That's in `prep-commands` with different dependencies for each of the set of commands territory as I understand the problem.12:14
richard_mawunless you want to prep your source trees in bootstrap mode12:15
persiaI don't understand your terminology, but I'm willing to trust your description.12:15
persiaWhat I mostly want is to go from upstream source repo to blob in some way that allows me to be certain I can repeat it.12:16
persiaWhen I start from upstream tarball, while I may be able to document cryptographic trust that upstream blessed the tarball, I have no view of the magic by which this is done, making certain sorts of source transformations unsafe.12:16
persiaIf this is already a solved problem, then I'm happy.12:17
persiaIn the distro model, it gets solved by publishing the constructed artifacts in such a way that one never needs to bootstrap.12:17
persiaBut since Baserock considers most artifacts to be disposable, with caching only as optimisation, I want to make sure that for anything that differs from upstream VCS that isn't a patch-under-review, we have a means to bootstrap *including* those modifications, even if it is painful, or requires starting with a binary blob (e.g. the build system artifact).12:18
persiaActually, that's not the example: that's the *only* artifact I'm comfortable having be a blob.12:19
persiaBut, as I understand it, for that to work, we need to have a way to be able to generate the bootstrap trees from a complete system, so that we can build the build system.12:20
*** tiagogomes has joined #baserock12:20
persiaAnd if this is `prep-commands`, then this is a solved problem.12:20
radiofreeis the cache server stuff working? if you build that with morph it's cached on gbo12:23
radiofrees/cache server stuff/cache server stuff in ybd/12:23
paulsherwoodsadly no... i've not seen any interest in morph changing its naming or cache code, so i fear i'll need to set up ybd cache server separately12:24
richard_mawpersia: we don't have prep-commands, it's just an idea I'd had previously to solve the problem. We either need to trust that the bootstrap-mode prep-commands don't leak too much stuff from the host environment, or that a trustworthy person has done the prep externally to the build and checked the result in, which AIUI is the usual process for release tarballs.12:25
persiarichard_maw: Right.  I don't trust humans, making me like the former solution.  That said, there are other things that need sorting before it becomes too important.12:26
radiofreepaulsherwood: the code to request is already there right?12:26
paulsherwoodyes, and to upload (but not hammered in a serious environment so far)12:26
persiaI like `prep-commands`, but I think we also need something to automate the process of processing `prep-commands`, such that given a build system and a definitions working tree, if one has push access to a git repo, one can commit the bootstrap trees.12:27
paulsherwoodi've been wondering about git-lfs, before settling on a soln12:27
radiofreenot sure it needs that, wouldn't the cache key in the filename be good enough?12:28
paulsherwoodyes, it would. i'm just not sure what the simplest soln is12:28
* richard_maw doesn't like soln as the contraction of solution12:30
richard_mawthat's pretty off topic12:30
richard_mawbut I'd be amused if people started contracting it to s6n12:30
* persia was very tempted to provide a half-complete set of vowels and a 't' in which to carry then upon reading 'soln'12:31
persiaI was just thinking about cache servers: is there anything special about the cache server that is unsupported by traditional object storage systems like Ceph or Swift?13:15
richard_mawpersia: I don't believe the Artifact Cache server needs anything, but the git cache server requires some methods that we haven't recently seen if we can't implemement on top of a regular git operation.13:17
persiaThat makes sense.  My thought was to reduce the amount of code we need to maintain to provide infrastructure.  For now, I think we need specialised git servers (it would be nice to only have one, rather than two), but if we can depend on another project for the cache server, it may reduce our maintenance overhead.13:18
persiaI suspect I'm thinking about this because of the mention of developing a new cache server for ybd ,which seems to add more to be maintained without adding much value over what we have.13:19
ssam2the current morph-cache-server is an almost trivial amount of code13:24
ssam2exposing the same functionality on top of an existing object store would gain us better storage management, but probably not less code13:25
persiaWhat functionality do we have beyond "store this object", "get this object", and "show me metadata for this object"?13:26
ssam2not much13:26
ssam2that handles both the Git cache and the artifact cache13:27
ssam2it doesn't summarise the API in one place, it's spread thru the code13:27
ssam2but the entire server is 400 lines13:27
ssam2it's basically 'glue' code on top of the filesystem13:28
ssam2i'm not saying we shouldn't change it, just that I don't think it'd make anything simpler if we did13:28
persiaI'd like to be able to separate git and object functions more, but yeah, on reading that code, I think I agree with you for now.13:29
richard_mawWe could do the ls_files functionality by turning on git-archive support on the git server and requesting a tarball of all the files we usually look at to see which build-system to attempt. We could do the cat-file with it too, for reading morphologies out of the repo (though we should migrate away from being able to do that), but the resolve_ref stuff is more difficult.13:32
richard_mawWe can do ref -> commit sha1 easily by inspecting the result of ls-remote.13:32
richard_mawHowever to get the tree sha1 we'd have to do the first part of the fetch protocol like ls-remote does, plus request a shallow copy of the commit, parse the pack for the commit object, read the tree sha1 out of it, then abort the rest of the fetch.13:34
richard_mawat which point we may as well just always cache locally, but only request the refs we are known to care about, and request shallow trees.13:35
persiaI don't want to change the local caching stuff more than necessary: that seems to work well.  I'm only thinking about remote artifact caches to optimise the case where multiple developers are working on similar trees.13:36
*** pacon has quit IRC14:33
ssam2some of us are doing research into Cloud Foundry and BOSH and how they can work with Baserock15:05
ssam2i've created a couple of wiki pages for this project...15:06
paulsherwoodssam2: cool!15:15
rdale"ERROR: Components build are not in systems/minimal-system-x86-64-openwrt.morph." - shouldn't that be 'to be built'?15:30
paulsherwoodrdale: is that morph? i can't find that string in the codebase?15:32
paulsherwoodssam2: nise bosh looks interesting!15:32
radiofreebosh is pretty big15:33
paulsherwoodnise is tiny15:33
SotKrdale: what was the command you used?15:34
rdalejust morph build15:34
SotKso `morph build systems/minimal-system-x86-64-openwrt.morph`?15:34
SotKif so that looks really odd, one sec15:35
rdalemay 13th morph15:36
paulsherwooddodgy commandline15:36
SotKaha, its not weird then15:36
rdaleah right :)15:37
SotKits looking for a component called `build`, because of the extra build on the commandline15:37
SotKthat error should probably print the components in quotes to avoid that confusion15:37
*** CTtpollard has quit IRC15:47
*** ssam2 has quit IRC16:01
paulsherwoodradiofree: is there anything for the wiki regarding how to build/deploy GENIVI Demo Platform yet?16:07
radiofreepaulsherwood: there's
radiofree even16:11
radiofreeuse the -rebase branch though16:11
*** jonathanmaw has quit IRC16:35
*** zoli__ has quit IRC16:37
*** bashrc_ has quit IRC16:55
*** mariaderidder has quit IRC16:56
*** zoli__ has joined #baserock17:14
*** zoli__ has quit IRC17:23
*** gary_perkins has quit IRC17:25
*** zoli__ has joined #baserock17:31
*** zoli__ has quit IRC17:36
*** edcragg has quit IRC17:49
*** lachlanmackenzie has quit IRC18:09
*** edcragg has joined #baserock19:48
*** zoli__ has joined #baserock20:26
*** zoli___ has joined #baserock20:52
*** zoli__ has quit IRC20:54
*** zoli___ has quit IRC22:10
*** edcragg has quit IRC23:50

Generated by 2.15.3 by Marius Gedminas - find it at!