IRC logs for #trustable for Tuesday, 2018-06-26

*** willbarnard has joined #trustable07:11
willbarnardWe could add a new t.changeset definition which is mutable, but I am not sure we gain much. Currently we consider the review/merge entity as a series of immutable t.changes which is in effect a mutable type.07:20
persiaWhat is the term we use for a review/merge entity?11:31
paulsherwoodwillbarnard: ^^11:45
willbarnardwe haven't defined a term as such12:27
paulsherwoodlet's do that12:27
paulsherwoodpersia: what do openstack call it/12:27
persia"change"12:28
willbarnardthey call it a Change12:28
persiaI'm fine with t.changeset being that thing.12:28
persiaKey is actually having semantics for everything that matters.12:28
willbarnardI agree with that12:28
paulsherwoodok here is there a need for t.changeset, distinct from t.change?12:29
persiaSo, if we have a t.changeset, which is composed of a set of t.changes, and during the review process, it is rejected, I think we need to be able to replace the set of t.changes in the t.changeset, while keeping some unique identifier for the t.changeset, so that we can see the history of review.12:29
willbarnardBut as the t.change itself is an arbitrary collection of modifications, I am wondering if we really need to define another term12:29
persiawillbarnard: The problem comes in the case where a t.change can never be merged, but we need to track the rationale for the changes that make it mergeable.12:30
persiaThe submitter of the final t.change that is merged may not be responsible for having decided to do something in a partciular manner, if that was required in order for the t.changeset to be merged.12:30
willbarnarddo you mean t.change or t.changeset12:30
persiaboth?12:30
persiaI mean t.change as the immutable component of history and t.changeset as the review/merge entity.12:31
willbarnardthere are some other clashes in the terminology that open stack use12:31
willbarnardthey refer to submission of a change to mean merge of a set of revisions12:31
persiaSadly, they run the conferences about CI/CD :)  But for now, let's concentrate on having a consistent vocabulary, rather than worrying about what words others use.12:32
willbarnardwhereas we use the same term to mean push of a revision12:32
persiaWhat?  I strongly disagree with the statement that openstack means "merge" with "submission of a change".  "Submission of a change" in openstack only means pushing a branch to the SCM.12:32
willbarnardwell if we are going to move towards the notedb standard, it might be wise to move to their terminology12:33
persiaBut I'm getting distracted :)12:33
willbarnardhttps://gerrit-documentation.storage.googleapis.com/Documentation/2.15.2/intro-user.html#submit12:33
persiaI don't think OpenStack is a good example of NoteDB use.  It is one example of gerrit use, but with a forked gerrit: probably better to harvest from one of the Linux Foundation, Google, or Mediawiki projects.12:34
willbarnardin the gerrit docs that is the meaning they define at least12:34
willbarnardperhaps the different implementations do not use a common vocabulary12:34
persiaYes.  Gerrit defines "submit a change" as causing to be merged by the gerrit internal merger.12:34
persiaYes.  Almost nobody uses consistent vocabulary in this space.12:35
persiaHence me being happy with t.change as immutable and t.changeset as tracking the review history.12:35
persiaYears ago, I would have used t.candidate for the review history, and t.patch for the immutable item.12:35
willbarnardI am still not convinced of the need for t.changeset, the head t.change can serve that purpose12:36
persiaIf t.change is immutable, and the review process requires modification to be accepted, and a new (immutable) t.change is submitted with the modification, against which unique identifier are the review comments tracked?12:37
persia(remember, most code is going to be unmergeable the first time, unless developers replicate the entire validation and review stack in their heads, which doesn't help velocity)12:38
willbarnardin the gitect implementation they are saved at each commit, then the final commit is used on merge12:38
persiaPlease express that using t.change.12:38
willbarnardevery t.change that is submitted triggers the Gate and results in the production and storage of t.evidence12:39
persiaOK.  Now, if a t.change produces t.evidence indicating it is unacceptable, and another t.change is submitted that addresses the issue, what tracks the linkage between the two t.change items?12:40
willbarnardthe change-review-evidence holds a list of SHAs representing the t.changes in the merge/review entity12:41
persiaThat you have to express that using nouns not prefaced with 't' is my strongest argument for t.changeset12:42
persiaSo: t.changeset holds identifiers for all t.changes that have been considered for merge12:43
persiaOr considered for a single merge operation, perhaps.12:43
willbarnardThere are certainly some advantages to following the gerrit approach of defined the changsets with a footer, but for that to work well you probably need a local githook12:43
* Kinnison finds that approach to be horrific because it involves the server potentially changing the commits upon submission12:44
willbarnardwell we have left the mechanism for defining the changset up to the policy, in the gitlab case in comes down to branching strategy12:44
willbarnardwhich server?12:45
persiaGerrit has a truly unfortunate implementation of tracking UUIDs for t.changeset12:46
persiaSpecifically, when one submits a branch to the magic "make me a uniquely named feature branch" API, gerrit mangles the commits to insert additional information.12:47
persiaOne *can* do that on client side, which a better implementation would do.12:47
paulsherwoodlet's not **accidentally** go the gerrit route... we should only return to that tunnel with our best spells12:47
persiaThe problem with gerrit's implementation is that the developer never has the same commits as the server, and needs to pull the mangled commits from the server to stay in sync.12:48
persiapaulsherwood: Yes :)12:48
persiaMind you, I think the git footer happens to be an excellent place to put a UUID for t.changeset, so as to track it consistently over a potentially large number of t.changes, but that should be under the control of the developer.12:49
willbarnardit could also easily be added in a note12:50
persiaI don't think so.  The data model between notes and commits is a many-to-one relationship.  This digs into implementation details, but I feel very strongly that each t.change only has a single t.changeset identifier.12:51
willbarnardnot really, for any notes ref there is a one-to-one relationship with the commit12:52
persiaAnd for any commit ref, there is a many-to-one relationship with potential notes refs12:52
persiaNotes have a commit ref, but each commit ref may be used by multiple notes.12:52
persiaAnyway, let's step back from implementation details.  Do we agree for the need for both t.change and t.changeset?12:53
willbarnardso if we has a notes/changesetid ref wouldn't that have a one-to-one relationship with the commit12:53
persiaNo.  One t.changeset has multiple t.changes, representing revisions of the t.change that will be merged.12:54
persiaThis is the inverse relationship to commits and notes.12:54
willbarnardif you mean there is redundancy that is true, but equally true if using the footer12:55
willbarnardI am almost convinced of the value of adding the extra definition for t.changeset12:56
persiaCould you restate that?  I don't mean anything about redundancy, but I think I don't understand your statement.12:56
willbarnardeach commit only has one note on the specified ref so I am unsure why you think that represents a many-to-one relationship12:58
persiaI'm confused.  I thought `git notes list` output a list of the set of all notes that annotate a specific commit.  This conflicts with my understanding of your statement.13:00
persiaErr, rather `git notes list SHA`13:00
willbarnardI just double checked this. If you omit the SHA it gives a list of notes for all SHAs. If you include the SHA it gives the single note associated with the SHA13:09
willbarnardYou can of course append notes but it is still considered a single note13:11
persiaWhat happens when you add two notes to a single commit?13:11
willbarnardYou can not add a second note, it will give an error message13:12
willbarnardYou can append or replace13:12
persiaThis is a case of the porcelain and the plumbing providing different feature sets :/13:13
willbarnardthat is possible13:13
persiaMy experimentation with the commonly-available `git notes` implementation mirrors yours, which surprises me given the nature of the underlying representation.13:13
persiaSo yeah, as long as folk are using this git-notes, I agree that there is no useful distinction in the data model between notes and footer.13:13
persiaAnyway, back to t.changeset13:14
willbarnardI think that notesdb is probably doing append, which is why they give the byte length of the payload, but I need to read more to be sure13:15
persiaLet's not be distracted by implementations until we have a vocabulary.  Most of the systems I've used for this in the past ended up being annoying because the implementation either conflated essentially different concepts or differentiated essentially similar things in ways that made it hard to explain to new contributors.13:16
willbarnardso if we were to add a t.changeset, what are the essential properties to define?13:17
persiaTo my mind, a t.changeset contains a list of t.changes that represent revisions to the goal.  I'm not sure we need anything else, as t.evidence probably properly accrues to the individual t.changes, and merge decisions would be taken based on the t.evidence associated with the most recent t.change in t.changeset13:19
persiaAlso, as we discuss it more, I think I don't like the orthography very much.  My naive interpretation of "changeset" is a set of changes that are to be merged together, rather than a set of changes representing one of the changes to be merged.13:20
willbarnardwe should say an *ordered* list of t.changes13:29
persiaExcellent point.  It is essential to know the sequence of the t.changes so that one can properly interpret how the t.evidence led to selection of the t.change that landed.13:30
willbarnardthe definitions have nothing about branches, merges, etc. so I am not sure we can add too much more to the definitions13:30
* paulsherwood looks forward to the merge requests distilling this conversation13:33
persiaI think the lack of branch mention is intentional and good.  Whether there are branches, or different servers, or just patches ends up being variable over workflow.13:35
persiaAnd I don't think it matters in terms of whether the workflow can be trustable.13:35
persiapaulsherwood: Don't get too excited.  As of now, we have two things: a) we're considering a string replacement of "t.update" to "t.release", and b) we may be adding a "t.changeset" (but we're not really done with (b) yet)13:36
* persia rereads the definitions to have a stronger opinion about mention of merge13:37
* paulsherwood is happy that at least we are logging this conversation13:37
willbarnardpersia: there is one other issue that informed our decision to keep the t.changes definition less precise. While we might advise that Git is used to to store all t.changes, in practice organizations might use other tools, especially for tracking requirements for example. In those cases, the distinction between changes and changsets may not be applicable.13:37
persiaIf we want that, then we should give up on t.change being immutable, in which case we don't need t.changeset.13:38
persiaAnd the history of review and revision that goes into t.change may be represented by a series of annotated git commits or something else (doesn't matter).13:39
persiawillbarnard: It seems the Product Policy Handler deals with the merges, so they don't need extra mention.  We got distracted in our discussion of Product Policy Handler, so I suggest we postpone discussion of "merge" and how it relates to t.change or t.changeset until we've gone back up the stack and dealt with the Product Policy Handler.13:41
willbarnardThe view we took was that it is only at the policy definition stage that we have actual mapping of tools to the roles which then determines whether we are working in a branch/merge based domain13:41
persiaDo you not want to postpone that discussion?13:42
willbarnardExactly, we tried to keep things more generic and have the Policy Handler as the placeholder to implement that13:43
persiaSo, shall we retain the current definition of t.change, and discard the assertion that t.change is immutable, to support a wider range of tooling?13:43
paulsherwooddiscard immutable13:44
paulsherwoodwith some summary of this reasoning in the commit message13:45
willbarnardwe do not actually state that it is immutable, though that was certainly the thinking as we moved to a concrete example with the tolling13:45
paulsherwood(maybe even a referral to the irclog)13:45
paulsherwoodis tolling when tooling becomes expensive?13:45
persiapaulsherwood: We got here as we were discussing something involving changes, and whether t.change should be immutable came up, ehnce the protracted discussion on how to track review history.13:46
paulsherwoodack13:46
persiaHaving done that, we seem to have consensus that the easier solution is declare t.change to be mutable, which allows us to return to our previous discussion where the immutability was raised as a point of confusion.13:47
* persia hunts backscroll to remember which that was13:47
persiaAha.  It was about Product Policy Handler.  The specific point of confusion was on whether the Product Policy Handler ran on every t.change or ran on every modification of every t.change.13:49
willbarnardregarding the Policy Handler, in our gitect implementation we define that as being implemented by a combination of the gitect tool and the git server (gitlab)13:49
persiaAgain, let's avoid discussing implementations until we have semantics.  Any given implementation may map perfectly to the semantics, or may not, but if we consider implmentaitons in too much detail, we'll get the semantics wrong.13:50
persiaI believe I was arguing that we should run the Product Policy Handler on every modification of t.change (including linkage of t.evidence), expecting the vast majority of runs to fail.13:53
persiaAnd I had suggested this to be the sensible model, as I did not understand how we could decide when to run the Product Policy Handler with a different model.13:53
willbarnardif the Gate gets that far13:53
persiaHeh, and then we get to another point of contention.13:54
persiaI don't think the Gate should be serialised.13:54
persiaI think that slows reporting, which slows development.13:54
willbarnardfor example if the t.sourcecode fails during construction, we cannot do validation13:54
persiaAnd we should resolve the Gate thing before we get back to Product Policy Handler.13:54
willbarnardbtw we dropped the Product it is just Policy Handler now13:55
persiaNot according to https://gitlab.com/trustable/workflow/blob/master/markdown/roles.md13:55
persiaHow can we know if it is safe to do validation?13:55
willbarnardno, but there is a MR awaiting approval with that fix13:55
persiaThat's fine.  When it is approved, I'll type fewer words.13:55
persia(and if the point I was making yesterday is agreed, it may never be approved)13:56
persiaSo, how can we know if is safe to do validation?13:57
willbarnardthat is the the decision of the Gate13:57
persiaYes.  How does it decide?13:57
willbarnardthis is the wording that I changed following Kinnison's feedback13:58
persiaYes, but that is only a t.change that is not yet policy compliant.  It is not safe for me to base my discussion upon it, as I have no means to know if it will be approved.13:59
willbarnardThe new test reads "Responsible for checking the result of the construction of artefacts13:59
willbarnard(**t.artefact**) and triggering the Orchestrator.13:59
willbarnard"13:59
persiaMy general experience with Kinnison's feedback is that it is excellent.  It probably should be approved.  I am not an approver, so my opinion doesn't matter.13:59
persiaSo, I assert that this is an invalid way to check.14:00
persiaValidation explicity includes static analysis, which can be done without a construction phase, and which feedback is valuable to a developer regardless of whether construction can be successful.14:00
willbarnardYes, that is correct14:01
persiaTherefore, we should not delay validation based on construction.14:01
persiaSo, is there another counterexample that suggests the gate should be serialised?14:02
willbarnardI am not sure we are saying anywhere that it should be serialized, only that there are some dependencies14:03
persiaThe statement "if the gate gets that far" suggests serialisation.14:04
persiaIf it is not serialised, then there is no "that far" to get.14:04
persiaIf you meant something else, my apologies.  Please rephrase.14:04
willbarnardif all the dependencies for policy compliance check are met14:05
persiaAnd how do we check the dependencies for policy compliance check?14:05
willbarnardthat is policy dependent14:05
persiaSo, if only the Product Policy Handler deals with the Policy, and we check the policy dependencies before we run the Product Policy Handler, how is it ever run?14:06
willbarnardOK, so we can define that the Policy Handler itself checks whether it is ready to proceed. In practice, the Policy Handling is actually distributed across several tools.14:09
persiaThat works for me.14:09
persiaAre you then comfortable with running the Product Policy Handler on every modification to a t.change?  Most of the time it fails the basic dependency, so does almost nothing.  Once in a while if fails after doing more.  Very rarely it succeeds.14:10
willbarnardYes, I only expect it to succeed when we are ready to merge14:11
persiaPerfect.  Now that we are in agreement, how is the Product Policy Handler different from the Validator?14:12
willbarnardIt is a completely different beast14:13
persiaIt is something than runs on every modification to a t.change, performs some tests, and supplies t.evidence.  I do not see the difference.14:14
willbarnardthe validator may execute t.tests, the policy handler does not14:15
persiaWhich part of the definition of t.test varies from what the Product Policy Handler does?14:16
willbarnardt.tests validate that the t.changes meet the t.requirements14:17
persiaSo the set of approvers is not part of t.requirements?14:17
willbarnardpolicy and requirements are not the same thing14:17
persiaHow do you define t.policy?14:18
willbarnardt.requirements define the functionality of t.software14:18
persiaFrom my perspective, it is simply the set of requirements imposed by the organisatoin about how the sofdtware is developed.14:18
willbarnardpolicy defines the operation of the software development process14:18
willbarnardyes, that is a very important distinction14:19
persiaDo we need to add a "functionality" statement to the t.requirments definition?  It isn't there now.14:19
willbarnardtwo completely different policies can product exactly the same software to meet the same requirements14:19
persiaI assert that a t.intent could be "the person approving a merge should not be the person who wrote the change", which would result in a t.requirement that could be validated.14:19
persiaYou still haven't defined "policy" in this context.14:20
willbarnardI do not believe that is a software requirement14:20
paulsherwoodi'd like to see it defined, even so14:20
persiaThat's fine, if we're using English, but do you have an objection to it being a t.requirement?  If so, which part of t.requirement is unmet?14:20
willbarnardWe have not defined policy, I recall there are a few statements that talk about process14:21
* persia would prefer not to have a definition of t.policy, but is unable to argue effectively against it without a strawman to use to show equivalence to another type.14:21
persiaSo, let's define policy.14:21
persiaAnd I'll argue about how it is the same as t.requirements14:21
persiaOr let's identify which part of t.requirements is wrong, such that it automatically excludes "policy"14:22
willbarnardpolicy is about how the the software is developed not what the software does14:22
persiaThat isn't inconsitent with policy being in t.requirements14:22
willbarnardthe end user does not care about the policy14:22
persiaHrm?  Why not?  I'm more likely to use software that has been certified to be compliant with various policies.14:23
persiaAm I particularly odd in that regard?  Does nobody else look for the labels in the back that make sure the police don't come around to ask why one is using a radio on the wrong frequencies?14:23
persia(as one example)14:23
willbarnardthat is a requirement not a policy14:23
persiaNo, it's a policy.  Most regulations are.  If you want a narrower definition for "policy", please supply one.14:24
willbarnardstandards are explicitly defined as a type of requirement14:24
willbarnardpolicy here is software development policy14:24
persia(side note: t.requirements MUST be both complete and consistent: this is provably impossible)14:25
persiaWhat is "software development policy" in terms similar to what we have in the document for t.requirements?14:25
willbarnardthe end user of thesoftware care what git branching strategy was used, he doesn't even know what git is...14:25
persia(and most importantly, in what way does "software development policy" differ from the current definition of t.requirements)14:25
persiaAgain, let's ignore implemnentations until we have consistent semantics.14:26
willbarnardpolicy has nothing to do with requirements14:27
willbarnardthey are completely different things14:27
persiaPlease explain how they differ.14:28
willbarnarda policy might be we merge branches on thursdays, a requirement might be that the software gives an audible warning beep14:29
persiaThose are fine examples, but if we are to refine our definition of t.requirements to exclude what you call "policy", and create a t.policy, we need to describe the structural ways in which they differ.14:30
persiaAs I do not see a difference, I am ill-placed to provide such a description.14:30
willbarnardI don't think the t.requirement definition needs to be changed to exclude policy, but it may well need improvements for other reasons14:33
persiaIt clearly does: my aside above indicates that the current definition cannot ever be met :)14:33
persiaSo, if t.requirement is not changed to exclude policy, are you willing to stipulate that t.requirement includes both "functional requirements" and "software development policies"?14:34
willbarnardNo14:34
persiaThen why doesn't t.requirement need to change to exclude policy?14:34
willbarnardit only includes requirements14:35
persiaIt *defines* requirements.  We can't use random english words to guide this.14:35
persiaWould it be easier if we spelled t.requirments t.14 ?14:35
willbarnardI am not sure why we would need to explicitly exclude things which are unrelated14:36
persiaYou assert a lack of relation.  I assert identity.14:36
persiaMy assertion of identity is based on policy being associated with intent, policy being unambiguous, policy being verifiable, policy being cohesive, policy being traceable from evidence, and policy requiring tests.14:37
persiaI disagree with the other parts of t.requirement (but we can discuss those separately).14:38
persia(note that my disagreement is general, and unrelated to this particular subpoint)14:38
persiaSo, what is the basis of your assertion?14:38
* paulsherwood *really* hopes persia can achieve a gitlab account, to offer patches directly :)14:38
persiapaulsherwood: In some ways, I think conversations like this are more valuable than just patches, as I think the process of discussion is important.  It may seem odd to have spent 2-3 hours discussing whether t.change is mutable, but I think we all have a deeper understanding of what we mean as a result.  I'm not sure that would have happened with a one-line patch asserting mutability (which isn't even required, as there is no assertion of14:40
persiaimmutability).14:40
paulsherwoodpersia: i do agree with the value of the conversations14:45
paulsherwoodso long as we retain the gist, and do patch appropriately14:46
persiaI believe we shall14:46
persiaSo far, we only have one potential change, being "t.update" to "t.release", as a pure string change, with no semantic value.14:49
willbarnardI have a change for that14:49
persiaCool :)14:49
persiaAny progress on the argument for t.policy?  Do you want to take some time, and we can reconvene?14:50
persiaAlternately, do you want to explore the bits of t.requirement with which I disagree, and then come back to how "software development policy" is not t.requirement?14:51
willbarnardI am thinking if there is a better way to explain this. I see a fundamental difference between them, one relates to the software itself and one the software development process14:52
persiaIn my mind, t.software encapsulates (either directly or by reference) all t.intent, t.requirements, t.tests, etc.  This would include any policy related to the development.14:52
persiaIt may be that the policy for changing policy differs from the policy for changing, say, intent, but that is covered by inclusion by reference (and being more specific digs into implementation details).14:53
willbarnardI also disagree with those statements, I didn't remove them in my earlier pruning as I wanted first to understand what the rationale was to include them originally14:53
persiaMy suspicion is that someone captured a discussion about ideas without subjecting them to rigorous analysis.14:54
persia"true" and "correct" are very alluring concepts, as indefensible as they end up being under scrutiny.14:54
willbarnardAgreed14:55
persiaI feel like the distinction between requirements for process and requirements for functionality is similar in that way.  Both end up being represented by code snippets that provide an output in some range that can be consumed to trigger further action.14:58
persiaBut in some sense the same functionality can be exposed regardless of the process.14:58
persiaThis is why I asked if it would work better if we changed "t.requirements" to "t.14" : that then becomes the conflation of the two classes of requirement.14:59
willbarnardI don't think it is useful to conflate requirements and policy14:59
persiaOn the other hand, we can accept the conflation without the string change, which implies we accept the case that different sets of t.requirements may result in the same t.artefact15:00
persiaI don't understand how it is useful to disambiguate them.15:00
persiaTo my mind, the main impact of disambiguation is to make the analysis of compliance more difficult and require more different pieces of software.15:00
persiaThat doesn't strike me as beneficial to either development velocity or confidence in the final result.15:01
persiaAm I missing something?  Is there a benefit I don't see?15:15
willbarnardLets come back to this one, I don't see that we will resolve it today15:27
persiaOK.15:31
persiaSadly, that also blocks going back to Product Policy Handler.15:32
persiaBefore that, we were discussing the role of the Gate.  We revisited that briefly today, but only in terms of serialisation.15:34
persiaI can't quite tell from backscroll: did we agree that there was no need to serialise the gate?15:34
persiaI think we did in terms of Product Policy Handler and Validator, but I don't think we touched on the other gate functions.15:35
persia(side nit: we have "Policy Compliance Checker" in the gate defintion and "Product Policy Handler" in the main role list.  This appears to be addressed in 9fbb7cc6f2a3afae4db2df55bbe116f2a9ea942a)15:37
*** willbarnard has quit IRC16:55
*** chrispolin has quit IRC18:03
*** ltu has quit IRC18:03
*** milloni has quit IRC18:03
*** chrispolin has joined #trustable18:06
*** paulsherwood has quit IRC18:07
*** laurence- has joined #trustable18:09
*** milloni has joined #trustable18:11
*** traveltissues has quit IRC19:24
*** traveltissues has joined #trustable19:25
*** paulsherwood has joined #trustable21:03

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!