| *** pedroalvarez_ [~quassel@ec2-54-191-112-126.us-west-2.compute.amazonaws.com] has joined #baserock | 12:42 | |
| Mode #baserock +cnt by verne.freenode.net | 12:42 | |
| radiofree | what's the correct thing to do for this particular horror http://git.jdl.com/gitweb/?p=dtc.git;a=blob;f=Makefile;h=962f94eba661c2ef7254ea725af76a0d9cc0fd99;hb=65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf | 12:42 |
|---|---|---|
| radiofree | HOME=$PREFIX make && HOME=$PREFIX make install DESTDIR=$DESTDIR ? | 12:42 |
| radiofree | on line 28 they set PREFIX=$(HOME), i just tried it like that and it seemed to be ok | 12:43 |
| radiofree | (DESTDIR had usr/bin/dtc in it) | 12:43 |
| richard_maw | you could also do `make PREFIX="$PREFIX"`, since assignments on the command-line override assignments in the Makefile | 12:44 |
| richard_maw | don't we already have a device tree compiler chunk? | 12:44 |
| pedroalvarez | radiofree: you can use the other device-tree-compiler repo | 12:44 |
| pedroalvarez | which already has a morphology | 12:44 |
| radiofree | :D | 12:45 |
| radiofree | i was searching for dtc | 12:45 |
| radiofree | for u-boot i had to do http://git.baserock.org/cgi-bin/cgit.cgi/delta/u-boot.git/commit/?h=baserock/james/tegra-uboot-btrfs&id=8ac347db3af19bcaa14a9e4c365c87f31f35a6de | 12:46 |
| radiofree | not sure if the ARCH=arm is needed though, but certainly you have to set CROSS_COMPILE there (it will complain about not being able to find arm-linux-gcc if you don't) | 12:46 |
| *** fay_ [~fay@82-68-191-81.dsl.posilan.com] has quit [Read error: Connection reset by peer] | 12:53 | |
| *** fay__ [~fay@82-68-191-81.dsl.posilan.com] has joined #baserock | 12:53 | |
| *** pedroalvarez [~pedroalva@82-68-191-81.dsl.posilan.com] has quit [Quit: Leaving] | 12:58 | |
| pedroalvarez is now known as pedroalvarez | 12:58 | |
| persia | Thinking about releases: I notice that the list of release systems appears to be documented in *three* places: release.morph in definitions, guides/release-process.mdwn on the wiki, and download.mdwn on the wiki. | 13:12 |
| persia | Does anyone have any suggestions about how we might reduce this duplication, or cause some of it to be the result of automation? | 13:12 |
| richard_maw | persia: we could change the wiki page to say that we only build systems in release.morph in definitions.git | 13:14 |
| persia | The release-process page? That sounds sane. | 13:14 |
| * persia commences editing | 13:14 | |
| persia | Would it be possible to have download.mdwn generated somehow during the release process, do you think? | 13:14 |
| richard_maw | and we could have the script that uploads artifacts generate download.mdwn, or put the files in a place that download.mdwn can generate itself | 13:15 |
| * persia isn't quite sure how autogeneration should be indicated on the page to discourage manual edits though | 13:15 | |
| persia | I think I like download.mdwn generating itself better, but I don't know the syntax well enough to know how we might do that. | 13:15 |
| richard_maw | I'd like if there were some ikiwiki magic to generate download pages | 13:15 |
| persia | Is there not? | 13:15 |
| tlsa | release-process on the wiki already says: "Update release.morph to contain the systems that are being released and no others." | 13:15 |
| persia | tlsa: Yes, which *encourages* duplication. I'm changing it to work the other way, so that the release team determines which images are to be released *from* release.morph, rather than the other way about. | 13:16 |
| persia | (as we already have a review process for changes to release.morph which ought handle any considerations that apply) | 13:16 |
| * persia decides to edit harder, finding a number of other oddities | 13:17 | |
| tlsa | persia: I think it depends on whether it's a genivi release or not | 13:18 |
| richard_maw | if we can get releases push-button enough, it doesn't matter if we make GENIVI images when we're not doing a GENIVI release | 13:18 |
| persia | So, release-process has a list of requirements for stuff needed: if folk add to release.morph in ways that increase hardware/infrastructure requirements, should we encourage wiki update on merge in the review process? | 13:18 |
| persia | tlsa: Ah, good point. I think it would make more sense to split things in definitions, so have genivi-release.morph and release.morph or similar. | 13:19 |
| tlsa | then you have to do the do-release script twice | 13:19 |
| SotK | is there a reason for genivi releases to be coupled to baserock releases? | 13:20 |
| persia | I certainly hope not. | 13:20 |
| pedroalvarez | SotK: no | 13:21 |
| persia | Although that's mostly because I think Baserock "releases" are a temporary workaround for lack of cache population infrastructure, but don't think I can convince GENIVI that their releases are similarly useless. | 13:21 |
| tlsa | beyond wanting to avoid having to do 2 releases in a week, I'm not sure | 13:21 |
| persia | So, this document claims I need a powerful x86 device for emulation for genivi releases. Can't I use a versatile layout on an FPGA, or similar? Does something actually depend on using qemu-on-x86? | 13:22 |
| pedroalvarez | the document assumes you dont have a versatile hardware | 13:23 |
| richard_maw | I think it's because we provided scripts and instructions for how to do it with qemu. If we had such hardware available to test releases we could do that | 13:23 |
| richard_maw | but that would require a change in release process anyway | 13:23 |
| persia | If there are scripts to assist automation with qemu-on-x86, that's a good enough reason for me. | 13:24 |
| persia | And I don't think I can actually make the change I want to make here, until we sort out the GENIVI/non-GENIVI bit. | 13:24 |
| persia | Because otherwise things will be wrong, which is worse than being hard to maintain. | 13:24 |
| tlsa | i think it's best if the updates to the release-process document are done by the people doing a release | 13:26 |
| robtaylor | a good read for anyone who hasn't seen it already: | 13:34 |
| robtaylor | http://continuousdelivery.com/2012/10/continuous-delivery-the-case-of-apple/ | 13:34 |
| *** motters [~motters@82-68-191-81.dsl.posilan.com] has joined #baserock | 13:36 | |
| persia | tlsa: if updates to that page are restricted to people doing releases, by what means should people not doing releases adjust the release process? | 13:37 |
| radiofree | if you merge something in into baserock/morph in some repo, should you update the ref: in definitions? | 13:42 |
| persia | You shouldn't merge something into baserock/morph | 13:43 |
| persia | Because it enables things to be garbage collected. | 13:43 |
| persia | Or is messes up history. | 13:43 |
| persia | Create a new branch. | 13:43 |
| radiofree | eh? | 13:43 |
| persia | Whether you update definitions.git is a matter of whether you think your new branch is a more sensible default. I think doing so is nicer, because it lets others use your work. | 13:43 |
| radiofree | oh right | 13:44 |
| radiofree | so after i merge my changes into tbdiff master | 13:44 |
| Kinnison | If you update definitions then your commit must be in the history of unpetrify-ref | 13:44 |
| Kinnison | that's the dependency | 13:44 |
| radiofree | i should submit a patch to update the ref in definitions? | 13:44 |
| persia | what? | 13:44 |
| persia | radiofree: If you think others would benefit from your work, yes. If you don't think others would benefit from your work, I encourage you to ask someone about it, as they will surely disabuse you :) | 13:45 |
| persia | Kinnison: Could you explain further about this unpetrify-ref dependency? | 13:46 |
| tlsa | persia: I don't know, but when I did a release with Daniel, the process document was well out of sync with what we needed to do, which made the process of releasing take much longer than it should have | 13:46 |
| radiofree | well updates won't work on a jetson unless you use it | 13:46 |
| radiofree | that's about it | 13:46 |
| persia | tlsa: Ah, yes, I entirely agree that the process doc needs to be precisely what should be done. I don't agree that the best way to achieve this is to restrict edits. | 13:46 |
| tlsa | and changes to the release process that aren't done as part of a release presumably aren't tested | 13:46 |
| persia | radiofree: RIght, so everyone who uses a Jetson probably wants your update, so please submit to definitions.git so Jetson users can be happy. | 13:47 |
| Kinnison | persia: read "unpetrify-ref" as "anchor-ref" and it should make more sense | 13:48 |
| persia | tlsa: It depends on the change. For example, there's a claim that "Baserock maintainers" have a certain level of access, but that term doesn't mean anything, and there's no correlation between folk changing code and folk with the right access, so it's clearly wrong (but trivally so, so I didn't fix it). | 13:48 |
| persia | I don't beleive that fixing that would be something that wouldn't be tested. | 13:48 |
| persia | Similarly, if some set of folk (not necessarily release folk) figured out a clean way to represent the set of release images in only one place, without significantly changing the set of commands run, I'd suggest this would fall into the same category. | 13:49 |
| persia | Kinnison: Not really. From prior discussion on the matter, I had the impression that any ref that was used needed to be preserved from garbage collection in the soruce repo permanently, and that unpetrify-ref was irrelevant. I'm not sure I understand how it becomes relevant, even if I call it "anchor-ref". | 13:50 |
| Kinnison | persia: I think of unpetrify-ref as somewhere to merge the commits *such that* they are considered preserved against GC | 13:52 |
| persia | Doesn't that enforce modification of source repos in all cases, rather than using upstream? | 13:53 |
| Kinnison | IMO, if the upstream commit isn't part of the history of an annotated tag (ideally signed) then it needs to be part of the history of a managed ref | 13:54 |
| Kinnison | and the manged ref should be noted in the morphologies | 13:54 |
| persia | Ah, OK. That makes sense, although in the last discussion I had on the topic (with robtaylor and richard_maw, if I recall correctly), I understood the conclusion to be that a branch ref wasn't actually a useful candidate for the managed ref. That said, I agree it's better than nothing, and serves that purpose (but still think anyone performing merge on such an object does a disservice to the git repo in question). | 13:56 |
| richard_maw | persia, robtaylor: I replied to Emmet's mail about what you were discussing yesterday. I include my arguments as to why I think your conclusion that we're insufficiently cacheable is incorrect. | 13:57 |
| persia | If we're incorrect, all is good :) | 13:58 |
| richard_maw | I just wish I could have kept up in real-time, so I could have saved you a day of arguing. | 13:58 |
| richard_maw | re: anchor refs, there's a distinction between the ref the commit comes from, and where we need to push changes to if we need a delta | 13:59 |
| richard_maw | as in we can have an anchor ref, but be unable to push to it | 13:59 |
| persia | Absolutely. Overloading these isn't useful. | 13:59 |
| persia | But as long as the user entirely ignores morph edit, this doesn't actually get in the way. | 14:00 |
| richard_maw | except that if they do need to make changes, then they need to come up with their own way to determine this | 14:03 |
| persia | Could you rephrase? I can't parse that in a way that makes sense to me. | 14:04 |
| richard_maw | if they need to make a change to the source of a chunk in definitions.git i.e. patch the kernel, then they'd need to manually clone the kernel repo, make their own branch, apply the change, push that, change the repo/ref of the chunk in the stratum to be able to even build it | 14:05 |
| persia | Actually, it's just clone, adjust defintions repo&ref, but yes, I agree. | 14:07 |
| richard_maw | ah, you use file:/// then | 14:08 |
| persia | Until I'm done testing, then I push somewhere, and put that repo. | 14:08 |
| persia | Pushing untested stuff goes against the grain, because I don't like force-pushing, and I like clean commit histories. | 14:09 |
| richard_maw | well, after you've gotten it to build, you still need to know which branch your changes should be on. If it's been always from an upstream branch before, you need to make something up | 14:09 |
| richard_maw | persia: re: pushing untested stuff: that's what `morph edit` was for, it just decides to change the ref fields and have `morph build` handle that rather than changing it to a file:// ref | 14:10 |
| persia | Generally speaking, I'd want to use the suggested upstream feture branch nomenclature for that, as I'd expect to have sent my change for review. | 14:10 |
| persia | It still makes commits that cause my git visiualisation tools to have conniptions. | 14:11 |
| * persia spent a lot of time trying to clean up a git repo after morph build once, and then had it messed up again on the next invocation and gave up | 14:11 | |
| richard_maw | persia: the temporary build branches give your tooling problems? | 14:11 |
| persia | Ys. | 14:12 |
| persia | Err, Yes. | 14:12 |
| richard_maw | I was not aware of this | 14:12 |
| richard_maw | I'm starting to understand your objections to the workflow better now. | 14:12 |
| persia | They show up as extra branches that have interwoven commit histories, and tend to have complicaed enough tree relationships that they stop having calculable anestry. | 14:12 |
| persia | richard_maw: Carefully reading your mail response, I find I have the opposite feeling to that you expressed yesterday. I'm going to need to set aside some significant time to really understand your mail to reply, rather than being able to give a response within what I typically consider appropriate response time. | 14:15 |
| richard_maw | having re-read what I replied with, I can see why, I used a lot of unclarified morph terminology | 14:27 |
| persia | But it's more comprehensive. Not only did the main participants in the conversation miss critical bits, but some of the other folk who joined in for a bit missed other bits. | 14:31 |
| persia | So although some of the terminology needs a bit of thought, the more relevant bit is that the understanding I developed yesterday needs to be almost entirely unlearned before I can understand correctly. | 14:32 |
| liw-orc | I'd like to update release-process.mdwn with the following patch: http://pastebin.com/1R1aQNBg -- anyone object? (I'm asking for review, since it's a pretty big change) | 15:16 |
| persia | liw-orc: Is it really restricted to Codethink staff, or is that just coincidence? | 15:17 |
| persia | Also, how are you resolving the GENIVI/non-GENIVI release sets in definitions.git? | 15:19 |
| liw-orc | persia, it is currently restricted to CT staff; there's no inherent requirement for that, of course | 15:19 |
| persia | My question is really, is it restricted to CT staff by policy, or is it restricted to an opaque set of which all members are coincidentally CT staff today? | 15:20 |
| liw-orc | for the GENIVI/non-GENIVI: if it's needed, edit release.morph in the release branch | 15:20 |
| persia | Manually for each release? | 15:20 |
| liw-orc | yes | 15:21 |
| persia | I suppose that works, although I'm not sure it's better than having described it in release-process (which is why I didn't delete that section earlier today). | 15:21 |
| persia | tlsa: You had some thoughts on GENIVI/non-GENIVI earlier: what do you think? | 15:22 |
| persia | For the staffing, might I suggest "Currently, only members of the release team have enough access to complete this process. The opaque nature of the release team is to be considered a bug that needs fixing.", unless there are policy reasons to limit to CodeThink staff? | 15:23 |
| liw-orc | there isn't a release team, though | 15:24 |
| persia | The release team is the set of folk who do releases. | 15:24 |
| persia | It'S an opaque team. We should fix that. | 15:25 |
| persia | But unless there's some reason it's restricted to Codethink (e.g. requires Codethink-only authentication tokens to access some bit of infrastructure), then I think it's not ideal to describe it as "Codethink only". | 15:29 |
| liw-orc | I've dropped that hunk, which is irrelevant to most of the changes, and will let others deal with that wording | 15:32 |
| persia | To be fair, I wasn't happy with the prior wording either :) | 15:32 |
| persia | And I do agree that it's a bug that we don't know who releases, and even more of a bug if there's some limitation that restricts it to staff of a specific organisation. | 15:33 |
| ssam2 | liw-orc: the release process says "...promote on your local Trove to get it lorried quickly", but it's not obvious how to do that | 15:36 |
| ssam2 | it'd be useful to create and link to documentation in the Trove reference manual, or for now just add a note in the release process about how to do it | 15:36 |
| tlsa | ssam2: when we did the release, Daniel used curl to poke some urls to promote things in trove | 15:37 |
| ssam2 | exactly | 15:37 |
| ssam2 | other than that, +1 for the changes | 15:39 |
| ssam2 | the "Announcing the release" should remind about updating the -current symlinks, probably, as it's quite important | 15:40 |
| * persia thought ripsum wrote a nifty script to automation promotion poking | 15:41 | |
| tlsa | we tried it, but there was no information on how to use it, and we didn't git it to work | 15:42 |
| tlsa | so we used curl | 15:42 |
| persia | heh | 15:45 |
| liw-orc | ssam2, the symlink bit should be, IMHO, implemented in the release-upload script (isn't there yet, but shouldn't be a big amount of code) | 15:50 |
| ssam2 | I fully agree, but so far it isn't, so the process should note that in a place where it won't be missed | 15:50 |
| ssam2 | the current wording implies that following the 'upload-release' script you can announce the release | 15:51 |
| ssam2 | making it likely that someone dumb like me would forget to update the symlinks, even though it does mention that earlier | 15:51 |
| liw-orc | "Manually create the "current" symlinks pointing at the new release." is there | 15:51 |
| ssam2 | sure, I mean in the "announcing the release" part | 15:51 |
| ssam2 | anyway, no big deal if you disagree | 15:51 |
| liw-orc | why should it be in two places? | 15:52 |
| ssam2 | well, why is "announcing the release" separated from the rest of the process? | 15:53 |
| ssam2 | if it followed on directly from the list that contains "manually create the "current" symlinks" then it would be clear | 15:53 |
| liw-orc | I don't think setting the symlinks is part of announcing the release | 15:53 |
| ssam2 | nor do I | 15:53 |
| ssam2 | but it should precede announcing the release | 15:54 |
| liw-orc | it is? | 15:54 |
| ssam2 | oh, actually I've been confused by reading a diff | 15:55 |
| ssam2 | I thought there was more content between 'release steps' and 'announcing the release' | 15:55 |
| persia | That particular diff is fairly comprehensive :) | 15:55 |
| liw-orc | the GENIVI release section is there | 15:55 |
| liw-orc | http://pastebin.com/V7Evw1Km is the whole page | 15:56 |
| ssam2 | my tiny worry was the line "Once the `release-upload` script completes successfully: [announce the release]", when "Run `scripts/release-upload` to publish release artifacts." actually has 3 steps following it in the "Release steps" section | 15:56 |
| ssam2 | so it should probably say "After you have completed all release steps: [announce the release]" | 15:57 |
| ssam2 | but like I said, it's tiny and I will now stop talking about it | 15:57 |
| liw-orc | I'll see about fixing that | 15:57 |
| *** fay__ [~fay@82-68-191-81.dsl.posilan.com] has quit [Remote host closed the connection] | 16:02 | |
| *** jonathanmaw [~jonathanm@82-68-191-81.dsl.posilan.com] has quit [Quit: Leaving] | 16:07 | |
| *** motters [~motters@82-68-191-81.dsl.posilan.com] has quit [Quit: Lost terminal] | 16:14 | |
| pedroalvarez | persia, richard_maw: hey! both of you commented on the thread to improve the OpenStack deployment, but none of you have voted, could you? | 16:35 |
| persia | pedroalvarez: The one about the OS_FOO and OPENSTACK_FOO variables? | 16:35 |
| pedroalvarez | yup | 16:35 |
| persia | Oh, if richard_maw is correct that the environment is passed through, then I think there's no point to the patch, and in fact, that the code section being modified should have all the other OS-related stuff removed. | 16:36 |
| persia | I can -1 it if you like, but think there is scope for useful discussion regarding what should be done. | 16:37 |
| richard_maw | OTOH, if the deployment commands prefer the command line options to the environment variables, then OPENSTACK_FOO provides a disambiguation mechanism for colliding namespaces, so it would have value | 16:37 |
| persia | Indeed, although whether that value is worth possible user confusion requires some understanding of under what circumstances such a collision might occur. | 16:42 |
| pedroalvarez | do you think that supporting both would be useful? | 16:42 |
| persia | Not unless we understand why we are doing it. | 16:42 |
| liw-orc | if I see a cluster morphology that has config variables I don't know about, I'd like to have an inclination of what they are related to (so I like prefixes like OPENSTACK_); if I see cluster morphologies that assume that non-standard environment variables are set, I'm going to have a harder time using them | 16:44 |
| persia | That's the key bit of the discussion. | 16:46 |
| persia | Are OS_FOO non-standard in an openstack environment? | 16:47 |
| pedroalvarez | they are | 16:52 |
| liw-orc | they definitely non-standard in the Unix environment, at least | 16:52 |
| pedroalvarez | if you have an openstack cloud, and you go to your access details, you can download something like this to set up your environment: http://pastebin.com/tsTezxMG | 16:56 |
| persia | pedroalvarez: Right. There are any number of scripts that set that set of things, which is why I think it's safe to consider them standard for an openstack client. | 16:58 |
| persia | In large part because attempting to use any of the openstack tools manually without that is a recipe for pain. | 16:58 |
| persia | However, there is the potential that there exist users who want to set OS_FOO to one set of values, and OPENSTACK_FOO to another set of values, so that morph operates on a non-default cloud. | 16:59 |
| persia | I'm not convinced these users exist, or that they wouldn't be surprised when morph didn't notice if they changed their OS_FOO setttings to point to a different cloud, but it is because they may exist that I'm not arguing more heatedly that the patch be removed, and the other OPENSTACK_FOO stuff be excised from morph. | 17:00 |
| pedroalvarez | well, I think that my openstack cloud is not going to need soon this patch, but I thought it was good to support these deployment variables | 17:03 |
| *** tiagogomes [~tiagogome@access.ducie-dc1.codethink.co.uk] has quit [Ping timeout: 260 seconds] | 17:08 | |
| *** ssam2 [~ssam2@82-68-191-81.dsl.posilan.com] has quit [Remote host closed the connection] | 17:12 | |
| pedroalvarez | I wonder if makes sense to add the capability of specifying the protocol used to lorry from the upstream trove in the cluster morphology. | 21:33 |
| pedroalvarez | Something like: TROVE_LORRY_PROTOCOL, defaulting to http or ssh | 21:34 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!