IRC logs for #baserock for Tuesday, 2015-11-24

*** locallycompact has quit IRC00:50
*** tiagogomes_ has joined #baserock08:53
*** ctbruce has joined #baserock08:53
*** rdale has joined #baserock08:56
*** bashrc_ has joined #baserock09:09
*** franred has joined #baserock09:46
*** gtristan has joined #baserock09:51
*** jmacs has quit IRC09:57
*** jmacs has joined #baserock09:58
tiagogomes_perryl, do you have a trove to test
perryltiagogomes_: i've got a personal trove set up but nothing public currently10:05
*** straycat is now known as VLetrmx10:08
*** ssam2 has joined #baserock10:11
*** ChanServ sets mode: +v ssam210:11
*** edcragg has joined #baserock10:17
*** jonathanmaw has joined #baserock10:18
rdalei'm following the 'quick start' instructions on the baserock wiki to set up a baserock-current-build-system-x86_64 image. when i get to the step where i checkout branch baserock-15.34 of defintions and try to build it i get 'ERROR: Definitions format version 5 is not supported10:32
rdale'. If i run 'morph --version' i get: fad8048de66fe6aeaa0478864914bb773f51ed3e. I assume that morph is either too old, or too new to build the definitions - anyone know how i fix this?10:32
ssam2ah, 15.34 is not the latest release10:34
ssam215.47 is the latest release10:34
rdaleah, ok i'll try that10:34
ssam2wiki must be out of date10:34
*** toscalix has joined #baserock10:35
VLetrmxssam2, the backend gerrit uses is on a separate box (Which is a separate system that provides mariadb) ?10:35
ssam2yes, we have a separate system providing a MariaDB database10:36
ssam2which is also used by Storyboard and the OpenID provider10:36
richard_mawhi VLetrmx, I didn't recognise you in your new nick, looks swish10:37
VLetrmxrdale, should be fixed now10:40
rdaleah ok, thanks - i was about to try and fix the wiki myself10:40
tiagogomes_shouldn't an entry for the last release of Baserock be added to
pedroalvareztiagogomes_: hm.. not sure, that was an investigation to see if we could rebuild all the releases we have done10:50
*** locallycompact has joined #baserock10:51
VLetrmxgiven we're about to remove branch-from-image and that the page reports that it's broken in the last 3 releases, the page seems a bit obsolete10:52
tiagogomes_yeah, I asking because of the branch-from-image command10:53
*** toscalix has quit IRC10:55
VLetrmxssam2, okay i'm guessing this uses fedora because we don't have mariadb in baserock?10:57
pedroalvarezVLetrmx: yes10:58
tiagogomes_docs are boring11:10
VLetrmxi'm slightly confused, our README says storyboard only supports mariadb, storyboard's docs say it only supports mysql, clearly you have an instance running it with mariadb, so is wrong?11:10
pedroalvarezno, our instance is broken11:10
pedroalvarezsome features present in mysql 5.6 are not present in mariadb yet11:11
tiagogomes_would anyone mind if I removed this page ?11:11
tiagogomes_mm, something is missing,
VLetrmxokay, so if our only reason for using fedora is that we didn't have time to package mariadb for baserock, we could switch the infradb over to baserock running mysql?11:11
pedroalvarezthat would solve my problem with deploying storyboard11:12
SotKpedroalvarez: what version of mariadb are we using?11:14
VLetrmxtiagogomes_, i don't mind if that's removed, it won't work soon anyway11:14
ssam2it's also a lot simpler to keep Fedora machines up to date, currently11:16
ssam2'ssh, dnf update' vs. 'ssh to remote devel machine, update morph, update definitions, morph build, morph upgrade'11:17
ssam2i've no objection if someone fancies switching more infrastructure systems to Baserock, but i don't have any plans to do it11:17
*** tiagogomes_ has quit IRC11:18
*** tiagogomes has joined #baserock11:18
SotKpedroalvarez: also, what feature does storyboard need that isn't there?11:20
pedroalvarezSotK: it needs fulltext indexes11:28
pedroalvarezI'll check now the version of mariadb11:28
SotKhmm, I thought they were supported in new-ish versions of mariadb?11:29
pedroalvarezI think I managed to get it working once, but then tried to upgrade our database to that  version (with a snapshot of it, not production one) and failed11:31
pedroalvarezI now just want to stop thinking about possible workarounds, and just put mysql in the storyboard instance11:32
VLetrmxthinking about this, i think that storyboard is broken for relying on mysql, i don't think fixing this is relevant enough for what i'm trying to do, though i do think it might be worth packaging mariadb in baserock11:33
VLetrmxi can probably just the existing fedora image for my purposes11:33
SotKpedroalvarez: sounds like you hit a place where the migrations expect mysql... there are a few places it does that which I noticed when trying to use postgresql with storyboard one time11:35
pedroalvarezSotK: yes, in the migration 022 and migration 042 :)  and IIRC new versions of mariadb was reporting also 5.5 as version number on those migrations..11:58
pedroalvarezanyway, no more databases conversations, I just hate them all now11:58
pedroalvarezput the data in git!11:59
persiaIn progress for gerrit, but I don't know that anyone has come up with a good way to store Storyboard info in git.12:13
ssam2but that is abandoned for a while12:16
*** Lachlan1975 has joined #baserock12:24
SotKheh, ssam2 beat me to linking consonant12:27
*** paulwaters_ has joined #baserock12:30
persiaReading the consonant spec, it might do the job, but the lack of sqlalchemy support means the entire data interaction layer would need to be redesigned substantially.12:39
*** petefotheringham has left #baserock12:55
*** locallycompact has quit IRC13:03
*** locallycompact has joined #baserock13:04
*** tiagogomes has quit IRC13:04
*** jonathanmaw has quit IRC13:04
*** bashrc_ has quit IRC13:04
*** tiagogomes has joined #baserock13:04
*** bashrc_ has joined #baserock13:05
*** jonathanmaw has joined #baserock13:06
*** rdale has quit IRC13:19
*** petefotheringham has joined #baserock13:21
*** rdale has joined #baserock13:22
*** petefotheringham has quit IRC13:27
*** ppeetteerr has joined #baserock13:37
*** ppeetteerr has quit IRC14:00
gtristanoh, btw the latest gnome patches are up !15:20
gtristan,, followed by
pedroalvarezhm.. I spotted that some 'unpetrify-ref's don't exist..15:31
pedroalvarezbut also agree that "we are not using them"15:32
tiagogomesit is a documentation field15:32
pedroalvarezand actually having that info would be more informative than "master"15:32
tiagogomesit probably should be renamed to something else15:32
ssam2i think we need a 'tracking-ref' field, for keeping track of the actual branch we care about15:41
ssam2and we need to sort out a way to use tag names instead of SHA1s in the 'ref' field15:41
richard_mawssam2: tags don't always resolve to the same sha1, so if you use a tag name instead of the sha1 you can't reproduce that definition in future15:43
ssam2yes, this has been discussed at length on the mailing list15:43
* gtristan has been thinking about that too, and mentioned something like that in his CI related email15:43
richard_mawmany times, haven't we jjardon?15:43
gtristanfor CI, or anything which wants to automatically pull a new version, we need some declarative way to specify where we get a new version from15:44
richard_mawyep, for firehose this was defined in a separate config file15:44
persiaIn my head, definitions always tracks some specific commit.15:46
persiaAnd CI of systems is done by submitting new definitions candidates.15:47
gtristanAgree that the SHA1 is ultimately the most reliable, on the other hand one could always be more lax, and if a tag-or-branch were specified, the ref could be derived from it - a snapshot of all the relevant SHA1s could always be generated at any given time15:47
persiaThe new definitions candidates should be constructed using some automated integration tool, which cares about branch names, or tags, or some other indicator.15:47
persiagtristan: We used to have that functionality, with "petrify" and "unpetrify": in practice, this generally meant that anything unpetrified failed to build.15:48
persiaThe lesson was that we only want to advance a small subset of things in concert.15:48
persia(or many parallel subsets, with integration)15:48
gtristancertainly agree15:48
gtristanany CI would want to select which chunks/strata should be updated in concert15:48
persiaRight, so that it would be wrong for *definitions* to impose the set of branches of concern.  That config belongs in the automated integrator that feeds the CI.15:50
gtristanperhaps allowing missing refs is not great, but I do think that specifying the active branch in the chunk morph itself would be the right place, even if ignored during a regular build15:51
persiaThe problem is that it only has meaning in a small subset of cases.15:51
gtristanI just think that it would be difficult to maintain a set of configs in parallel15:51
persiaMost users (whether human or automation) need to have separate config to decide whether to care about it.15:51
persiaAnd since that config is there anyway, why duplicate it?15:51
gtristanOn the other hand most refs exist on some active branch, having the branch encoded in the chunk allows a CI to build/test/propose a new ref from the same branch15:53
gtristanprobably a human prefers first to decide on which stable branch they build, and have the actual ref derived, well... probably15:54
ssam2persia: definitions doesn't need to "impose" branches of concern. It can usefully *suggest* them15:54
ssam2any sensible CI system would allow overriding them, but if the definitions suggested the branch to follow (which I think in 99% of cases would be correct) then CI users wouldn't need an enourmous separate config file just to achieve anything15:55
persiassam2: My contention is with your use of the word "usefully": to me the suggestions are going to be superfluous or wrong for most users.15:55
ssam2really? in 99% of cases it will be 'master'15:55
ssam2most projects don't even have any other branches15:56
ssam2in the context of the current definitions15:56
persiaThose are the boring cases, and for them the suggestion has little value.15:56
persiaIt is the *rest* of the cases that are interesting.15:56
gtristanand I think 99% of cases will be stable-of-release-Major.Minor15:56
gtristanwhile only the real risk takers will want master ;-)15:56
persiaWhere non-master is selected, the user (whether human or not) needs to determine whether they want master, or some non-master, or which.15:56
ssam2true, stable branches rather than master15:57
persiaThis decision is, to my mind, unlikely to be influenced by the content of definitions, and more likely to be influenced by that users' goals.15:57
ssam2I still think that info would be useful in the definitions. People make use of the current 'unpetrify-ref' field, that's why it's still there15:57
persiaPeople annotate it when they make changes.15:57
persiaDo people consume it to make decisions?15:57
ssam2i do15:57
ssam2to know which branch to update from15:58
gtristanMy thoughts on this is from the point of view of a consumer; I want to build a safe and reliable system, and I want to pull latest-stable of the packages I want to include in my system15:58
gtristanand I would like to automate some of that via some CI15:58
gtristanat the end of the day, I will enjoy the safe and reproducible builds which the SHA1s provide15:58
* franred too thinks that unpetrify-ref is nice to have to know where to look for the sha115:58
persiagtristan: CI is entirely orthogonal to this.15:58
gtristanpersia, well, perhaps it is, let's rephrase that: I want to build and test latest stable upstreams with all security bugfixes and important recent crasher fixes15:59
* persia uses tig to find sha1s: `tig --all`, '/', ${SHA1}15:59
gtristanwhether the testing is automated or not, is probably orthogonal15:59
persiagtristan: Yes, but my assertion is that *you* want to do that, and that this may not be the same as what other users want.16:00
gtristanbut what interests me as a systems builder/maintainer, is to be able to migrate to a latest-stable automatically16:00
persiaMany users will want to pull from their local forks of things, rather than upstream stables.16:00
persiaSome users will want to pull from upstream master, as part of validation of the master.16:00
gtristanpersia, I think its very far fetched to think that most users fork "most" of the packages which go into a system16:00
persiaSome users will want to pull from pre-merge candidates to see how a code change would integrate with a system16:00
persiagtristan: I can only speak from my experience of embedded systems.  Yours may differ.16:01
gtristanmost anyone building any embedded product may have a fork of one or two repos16:01
gtristanbut all of them ?16:01
franredpersia, imagine you need to do that for ~400 chucks... Also, I though we were pushing to make definitions more explicit16:01
persiaMost of the ones I've encountered start with vendor forks including a selection of changes all over the place.16:01
gtristanmy experience is that I encounter enormous stop energy when desiring to update *anything* in the platform, honestly16:02
persiafranred: Have you ever done that for 400 chunks?  Do you imagine doing that with or without the field?16:02
persiafranred: If I needed to process that many chunks, I'd use a script, which would show branches that have SHA1 as ancestor for each of the chunks.16:02
gtristanthat is a problem, pulling a security bugfix from a stable upstream may mean that the QA team is set back one month, and all of their testing is invalidated by one bugfix16:02
persiaYes.  I share that experience.16:03
franredpersia, I have to update openstack... yes, it was nice to read in definitions where the sha1 came from. So you know if it is a custom or a general branch16:03
gtristanthat is of course, insane, but from some paranoid viewpoints, valid as well :-/16:03
persiaI don't think fixing that problem has anything to do with a notation field in defintions about the branch name assigned to a SHA1 at the time the last person touched it, if they remembered to update the ref.16:03
persiafranred: How do you mean "custom" vs, "general"?16:04
franredpersia, custom: branches which are only in gbo, general: branches which we lorried16:05
persiaIf any of the former lasted any time, it indicates a failure to engage with upstream.16:05
persiaAnyway, I've complained about this enough for today.  My complaints aren't any different than they have been since I first sent email ridiculing this field.  I'll stop now.16:06
pedroalvarezok, there might be different opinions from: (a) baserock users (b) baserock developers (c) people just following the project16:06
gtristanI think that if we are going to be a venue for hosting stable reference builds, we need to have branch tracking in mind16:07
persiagtristan: I agree, but I think that belongs in the configuration of the tool that submits candidate patches to definitions, rather than definitions.16:09
gtristanperhaps, but I wonder how it would be to work with that16:09
gtristanpersia, what workflow do you envision, personally I see the entire definitions repository branched for a collective stable branch, and each chunk declaring which branch they track16:10
gtristanI think that's rather logical and easy to understand16:10
persiaI think the definitions repository should be branched for a stable model, if one wants that.16:11
persiaBut I think that any policy for branch tracking belongs in the automation that submits candidate changes to those definitions, rarther than in the definitions.16:11
gtristanSo baserock-1.0 is released and a baserock-1.0 branch is created, with a BASEROCK_1_0_0 release tag16:12
persiaBecause the value for stable differs from the value for non-stable in ways that mean one cannot simply branch and label definitions, but needs to re-annotate all the notes.16:12
gtristanhow do we move to BASEROCK_1_0_1 ?16:12
ssam2let's bear in mind that we've succeeded in creating the definitions format, but have failed 3 times now to create said CI tooling16:12
persiaSince I'm opposed to even the rudimentary baserock releases done now, as I think them counterproductive, I have trouble answering that without conflating it with other issues.16:12
* gtristan is digesting this...16:12
ssam2let's add the field to the definitions format for now, and if $amazing CI tool eventually exists and obsoletes it, we can remove it again16:13
persiassam2: The field is already there.  It's just a matter of not removing it.16:13
persiaThe reason I raised complaint again is that I don't want someone trying to use it for CI, as I think that is another recipe for failure.16:14
pedroalvarezso going back to the problem, what do we do :)16:19
persiaWhich problem?16:19
persiaThat some chunks don't have unpetrify refs?16:19
gtristansome people have nothing to do around here huh...16:20
*** ctbruce has quit IRC16:20
gtristanpedroalvarez, my benevolent dictator, want me to change it ?16:20
pedroalvarezit has been merged16:21
persiagtristan: Do you know what ref that was tracking?16:22
* gtristan checks this16:22
* gtristan is quite sure he just did 'git describe'16:23
persiagit describe doesn't always generate a ref that can be resolved.16:24
pedroalvarezthe ref used is tip of master16:24
pedroalvarezlooks like it's git describe, yes16:25
gtristanverified, indeed this is just the output of git describe at the tip of gnome-keyring master, which has not branched out for 3.18 yet16:25
pedroalvarezto me, git describe is more useful than master there16:25
* persia is actually curious how this field is consumed16:26
gtristanI may be guilty of having pushed some commits with unpetrify-ref pointing to an unresolvable git describe output16:26
gtristanyeah, me too16:26
pedroalvarezyou open the definition, see ref, don't understand a thing, and then see unpetrify-ref16:26
* tiagogomes always uses the branch where the ref came from for unpetrify-ref16:26
gtristanI consume it that way also16:26
gtristanbut I assume it has some deeper purpose :)16:27
pedroalvarezif "master" in unpetrify-ref, no useful information16:27
persiapedroalvarez: So, how is `git describe` output more useful than master?  It gives no information on how to advance to the "current" state from whatever state was used then.16:28
gtristanpedroalvarez, if it's cruft and not read by the machine, it may be better replaced by a per-chunk comment16:28
persiagtristan: We had issues with comments, which is part of why there aren't many16:28
pedroalvarezwe had, not sure if that is still the case16:28
tiagogomespedroalvarez, I woudn't say it is useless. It tells you that the commit came from the development branch instead of stable branch16:28
tiagogomes`git describe` seems useless to me though, as it is a middle term between the ref and a possible stable branch or tag16:29
gtristantiagogomes, sometimes that is all you have, though16:30
gtristantiagogomes, typically when the stable release is recent and no branch is created yet, because no API changing work needs to land in master just yet16:30
pedroalvarezpersia: I was just saying that for my human brain, 3.18.3-5-geb16c03 has more information than "master"16:32
persiapedroalvarez: I agree that it contains more information.  I'm just wondering how this information would be used?  What decisions might get taken based on this information, etc.16:33
tiagogomestbh, I don't care much about what to put there, but I'd like consistency16:33
pedroalvarezI don't have answers for those questions. We should probably ponder, and investigate if we need them or not16:33
persiapedroalvarez: I'd be very curious if you ever think of answers for them :)16:34
gtristanthis is an odd conversation; a more productive path may be: "Do we all agree that there is no tool which actually uses unpetrify-ref and that it is as such, obsolete ?" and then "Lets define a policy for what we actually do need instead and then enforce it"16:34
persiaI fear most folk are like tiagogomes: using this field because it is there, rather than for some purpose.16:35
pedroalvarezyes, but I fear that if we remove it people will miss it16:35
persiagtristan: We agreed to the first part of that last year: we just never managed to get an answer for the second part, and some people are attached to the text in ways that are hard to explain in terms of policy.16:36
persiapedroalvarez: I *know* people will miss it, as every time I've suggested removing it, people have told me we should keep it.16:36
richard_mawerm, I think it needs a better name16:37
richard_mawthat is all16:37
* richard_maw hides again16:37
gtristanRight, for machines, a stable/feature branch can be useful to pull the latest of that branch, for humans - "at least some information that is more comprehensible than an SHA1" would be nice, and I think for the latter a comment would suffice16:37
tiagogomesrichard_maw +116:38
tiagogomesAnd a policy of what to write on it16:38
gtristanSHA1s are the new binary, Klingon programmers can now claim to understand SHA1 keys16:38
* gtristan just wanted that branch so he wrote in a52bcff0844.... by hand !16:39
pedroalvarezshould we discuss the policy now that we are at it?16:39
ssam2I think there are two uses: readable (tag/branch) name for sha1, and suggested-branch-for-ci-to-follow16:39
ssam2we don't always need both16:39
pedroalvarezI think we can forget about the latter16:39
ssam2that's fine. we can add it if we need it16:40
* persia concurs about ignoring the latter16:40
ssam2so unpetrify-ref becomes ref-name, basically?16:40
* tiagogomes thinks the same16:40
persiaNext question: can we trivially convert from SHA to name in some programmatic way?16:40
* gtristan also agrees that would be a good rename16:40
persiaSo that we don't need to remember to update the field on every commit?16:40
ssam2persia: no, because one sha1 can be in multiple branches16:41
gtristana branch-tracking thing should be ignored at least until such time that tooling would make use of it, otherwise it's bloat and not worth discussing16:41
ssam2persia: if it appears in 'master', 'v1.3.0', and 'freds-feature-branch', it's hard for software to know which is the useful one16:41
ssam2where a human knows right away that v1.3.0 is the meaningful namne16:41
pedroalvarezI like fred16:41
pedroalvarezanyway, what we should do in the case I posted before this discussion16:42
gtristanrrright... ssam2 so basically I think persia is talking about some meaningful migration path16:42
pedroalvareza) create a branch with a meaningful name ?16:43
pedroalvarezb) put master in ref-name?16:43
gtristanssam2, so to put it another way, if we just ran a script right now which created ref-name from the SHA1, would it be "good enough for now" and follow a better policy in the future ?16:43
pedroalvarezc) ... more sugestions?16:43
ssam2gtristan: do we need that? the info is there in 'unpetrify-ref'16:43
ssam2pedroalvarez: (a)16:43
gtristanssam2, or maybe we have to play rock paper scissors, and the loser has to spend their evenings converting them one by one :D16:43
pedroalvarez(is part of baserock)16:44
ssam2pedroalvarez: as if someone upstream force pushes master, that commit would lost16:44
pedroalvarez(reproducible wins)16:44
persiaNo, I think I like what ssam2 is saying: that we need to put user-mediated information in the field to capture *intent*.16:44
persiaJust because a commit is on some given branch doesn't mean that is the branch we want to be using.16:44
pedroalvarezssam2: then, we shouldn't use tags either?16:45
gtristanwe certainly dont alway have a tag to go on16:45
persiaAnd tags aren't any more fixed than branches, except by social convention16:45
gtristanand when we intend to use stable, that stable is often still unbranched and on master16:45
persia(well, and that `git commit` doesn't auto-advance the tag)16:45
ssam2pedroalvarez: no, we should tag everything locally using `morph anchor`16:46
ssam2if we actually care about this stuff16:46
ssam2"locally" meaning "in our mirrored Git repos"16:46
* gtristan throws a stick in the front wheel:
gtristanBeware, gzip lorry ahead !16:49
persiaYes, or simply cloning a repo from, and adding any commit to any branch could also delete a commit we are about during the automatic `git gc` that runs as part of `git commit`.16:49
pedroalvarezI like the morph anchor idea, but.. are we going to do that for every single change? just for releases?16:55
pedroalvarezI want a baserock desing summit.. In a place that nobody can scape until we have answers for everything16:57
persiaDepends on the release model.16:57
persiaFor the every-merge-is-a-release story, it would be something the gatekeeper should do on merge.16:57
persiapedroalvarez: The last time we did one of those nobody worked on anything afterwards :)  We could do it again, but ...16:58
persiaI don't know if it is too late, but maybe we could have a BoF at FOSDEM?16:58
pedroalvarezthat's true.. I guess only people that can work on things should attend16:59
pedroalvarezhm.. no, there are users too17:00
ssam2well, and also people who actually have a use for them17:00
ssam2I think one problem we have is we do a lot of discussion without actual use cases17:00
persiaI think that if we meet and have agreement, that makes it easier for people to have confidence that they are doing the right thing between meetings.17:00
persiaWhether people actually have time to do it, etc. is different.17:00
ssam2and my actual use cases, I mean people who are using the tooling and definitions to do things with purposes that are not "improve Baserock"17:00
persiaBut by writing down the decisions and putting them on the wiki, there's a better chance that we end up building what we decided to build.17:01
persiassam2: Absolutely.  Non-developer users are key to successful creation of software.17:02
*** ssam2 has quit IRC17:05
*** ssam2 has joined #baserock17:05
*** ChanServ sets mode: +v ssam217:05
*** franred has quit IRC17:09
* pedroalvarez sends:
pedroalvarez(trying to get the gzip importer working on g.b.o)17:17
gtristanah, I missed that file17:22
pedroalvarezand its friend:
*** jonathanmaw has quit IRC17:59
*** jonathanmaw has joined #baserock17:59
*** bashrc_ has quit IRC18:06
*** edcragg has quit IRC18:20
*** locallycompact has quit IRC18:20
*** gtristan has quit IRC18:22
*** Lachlan1975 has quit IRC18:23
*** ssam2 has quit IRC18:44
*** gary_perkins has quit IRC18:57
*** rdale has quit IRC18:58
*** SotK has quit IRC22:16
*** SotK has joined #baserock22:17

Generated by 2.14.0 by Marius Gedminas - find it at!