IRC logs for #trustable for Wednesday, 2016-12-21

*** AlisonChaiken has joined #trustable03:51
*** AlisonChaiken has quit IRC07:12
*** AlisonChaiken has joined #trustable07:13
*** ctbruce has joined #trustable08:27
*** tiagogomes has joined #trustable09:26
*** ctbruce has quit IRC09:56
*** ctbruce has joined #trustable10:04
*** laurenceurhegyi has joined #trustable10:44
*** persia has quit IRC11:07
*** persia has joined #trustable11:09
*** ctbruce has quit IRC11:52
*** ctbruce has joined #trustable12:04
ChrisPolinI have updated the wiki entry to incorporate the deployment methods into the Trustable workflow. (
ChrisPolinI've still to update the UML (components -> Activities), its next on the list.14:03
ChrisPolinLet me know what you think of the updated model.14:03
ChrisPolin(nipping out for half an hour to grab lunch, so I'll be afk for a bit.)14:04
persiaTo my mind, an automerger usually operates on source code repos, consuming patches from a patch tracker, and injecting them into some consumable mainline.14:05
persiaFor the text, I don't understand the phrase "reports what downstream changes it is responsible for".14:05
ChrisPolinPerhaps that arrow should come from the CI component, rather than the test artifacts one.14:06
persiaThis is part of the problem with conflating component and activity diagrams :)14:06
ChrisPolinNoted. I was referring to what behavioural changes the patch may invoke, although perhaps this may be unnecessary.14:07
persiaThe "thing-that-is-done" is not necessarily easily associated with the "thing-that-does-things".14:07
ChrisPolinMm, I'm beginning to learn that!14:07
persiaHeh, anyway, enjoy your lunch.  I'll read deeper, and see if I can provide a more useful critique this afternoon.14:08
ChrisPolinThanks, let me know your thoughts.14:08
paulsherwoodChrisPolin: it's starting to look more like an elephant :)14:43
paulsherwoodtitle is still wrong... s/Trustablesystemstructure/something better/ please14:44
paulsherwoodChrisPolin: istm your Code box is not really code, it should be system, or instance or target or something?14:45
paulsherwoodCode to me implies source, and should be somewhere below Patch Review, above CI14:46
persiaAs I try to diagram things, I wonder if I'm thinking about the same problem as others.  Does
persialook sensible?14:46
persiaNote that when I'm trying to think in terms of "thing-that-is-done", none of CI system, CD system, Patch Tracker, Repo, etc. exist.14:48
persia(as those are nouns)14:48
paulsherwoodbefore i think about that, did you generate it via an api at
persiaBut it's not a very interesting API.  One loads a given page, submits text in a form, processes the form, and then screen-scrapes a URL.14:53
persiaIt is likely one could cache the initial form, so just submit something, then process the response for a URL, if you're thinking something javascripty to embed somewhere.14:54
persiaThe URL itself is another API, although I haven't found documentation for it: it compresses the plantUML text into the querystring, so that the plantUML server can generate it on demand.14:55
persiaActually, seems PlantUML publish the javascript to generate the magic URL in the source of the editor page, as the "compress" function.14:58
persia(the core bit being `encode64(zip_deflate(string));`: if one replaces "string" with the textual data, one ends up with the magic string for the generator URL.15:00
*** ctbruce has quit IRC15:01
jmacsIf we could put arbitrary javascript into our pages, we might as well render the UML in javascript ourselves15:01
persiaIf you cannot, almost no API will help, sadly.15:04
jmacsI don't think it'll be possible with the public instance of gitlab.15:06
persiaI can believe that for gitlab-hosted pages.  Would it make sense to have another service consume a git repo containing the content, and render it there?15:07
jmacsYou could theoretically repeatedly pull the git repo and render images out of it15:07
jmacsAssuming gitlab allows remote URLs for images, which I might test now15:08
persiaAssuming remote URLs for images is permitted, if the PlantUML is in the same repo as the content showing the diagram, you could also set up a pre-commit hook that generated the right URL string and didn't allow the commit unless the image reference matched.15:09
persiaThat makes the results all static, although it might be a little frustrating for committers.15:09
jmacsYes, you can embed remote images in a gitlab wiki.15:11
jmacsShall I try and set that up?15:12
persiaThe git hook?15:12
jmacsThe whole rendering service15:13
persiaI was hand-waving a bit when I described it.  There would need to be a known set of sources that were supposed to be pre-processed, and some means of tagging what URLs needed to be generated, etc.15:13
persiaOh, if you think you can do a full rendering service without it being as kludgey as I just described, I suspect I might use it if I wanted to submit UML to the wiki.15:13
jmacsI think I can do that.15:14
persiaMy ideal interface would be some mechanism to reference the filename of the UML in an <img> tag, but the only way that I would know how to do that is with arbitrary javascript, which complicates things.15:14
persia is the base page of the plantUML site, from which one is redirected to a session-aware page with pre-populated UML, the source of which contains the javascript functions they use.15:16
jmacsIdeally I'd like inline UML in wiki pages, which means hiding the UML from the browser15:16
persiaFor that, you need pre-processing.15:17
persiaActually, you could also do javascript post-processing with DOM tricks and hidden DIVs, but that's more likely to run into issues if there are limitations for arbitrary javascript.15:18
jmacsChrisPolin: Could you send me the UML source for your diagram?15:29
ChrisPolinpaulsherwood, if it's big and grey etc. I was going to change the page name but I can't see a straightforward way of doing so on gitlab without just creating a new page and removing the old one. If you're happy enough to lose the edit history then I'll go ahead and do that now.15:30
persiajmacs: It's in the repo, as TSS?UML15:30
persiaErr, TSS_UML15:30
ChrisPolinOop, I haven't updated the repo, hang on.15:30
ChrisPolinAnd ack re: 'code', I'll fix that too.15:31
persiaThat's the strongest argument for inline UML in the wiki :)15:31
jmacspersia, ChrisPolin: Got it, thanks15:31
persiaUpdates to the draft above, after which I'll stop until/unless someone tells me that I'm solving the same problem:
ChrisPolinThat's the updated version now (pre-changes as per Paul's suggestion)15:37
ChrisPolinOops, link breaker! Sorry persia!15:37
persiaLink breaker?15:37
persiaDid my text run over your buffer?15:38
ChrisPolinAh it did, and I posted in the middle of it lol.15:38
persiaOops.  I can paste again, if that would help.15:38
ChrisPolinGot it now.15:39
persiaIf that is the right path, there are more loops needed, more parallelism, and some of the labels probably have to change, but I worry that it moves far enough away from concrete systems to be confusing.15:49
ChrisPolinpersia: I can see the distinction between activity and component that you're making. I would argue that, given the approach that you've taken with yours, the diagram I made should be made *more* component-oriented.16:36
paulsherwoodChrisPolin: nearly there. can/should there be a box for code between patch review and continuous integration?16:38
ChrisPolinYep, can do that. Although should the code go before or after patch review?16:38
ChrisPolinTo my mind, the code is submitted to review.16:38
paulsherwoodto my mind code is a collection of patch reviews16:39
ChrisPolinMm, fair enough. Will add now.16:40
paulsherwoods/patch reviews/patches, preferably reviewed/16:40
ChrisPolinThe elephant is getting taller.16:42
persiaI'm not seeing the latest update (wiki claims it was updated 2 hours ago), but I'm not sure I understand the intent of the proposed change.16:43
persiaChrisPolin: Ah, so you're thinking that we don't want an activity diagram, but instead want a component diagram?16:44
ChrisPolinI think it gives the reader a bit more to bite into. The activity-oriented approach is, to my mind, a little too abstract.16:46
persiaThat was my concern, but paulsherwood seems to be encouraging more use of verbs, and less description of infrastructure.16:46
persiaI am not convinced that both activities and structure can be described in the same UML diagram, if that diagram is intended to be machine-parseable.16:47
ChrisPolinLol I was just typing the same thing. They tell the same story, but I'm thinking more from a reader perspective.16:48
persiaAnyway, I promised you a detailed review of the text, which I never sent.16:48
ChrisPolinPerhaps my approach is not the correct way to look at it.16:48
persiaFirst paragram is clear.16:48
persiaSecond Paragraph:16:48
persia1) Should G/W/T be an example?  Is there an expectation that UML will also be acceptable (assuming it can be machine-tested)?16:49
persia2) Same applies to "BDD tools", really.16:50
persia3) The last sentence doesn't appear to have any referent (specifically "this metadata" doesn't seem to attach to any previous use of the word "metadata")16:50
persiaThird Paragraph:16:50
persia1) How does the patch review system trace to a developer?  Signed Commits?  Authenticated-push?  Trust of the VCS assertions?16:51
persia2) How is a scenario linked to a line of code?  Is this developer discretion + human review?  (consider the example where code that implements something doesn't happen to be in the same commit as code that enables that codepath)16:52
persia3) Why are test results a standalone artifact?  I would expect to find them linked from the patch review system.16:52
persiaFourth paragraph is sensibly, modulo the G/W/T note from the second paragraph16:52
*** ctbruce has joined #trustable16:53
persiaThe fifth paragraph seems to have some assumptions I don't understand.  I'd need to be walked through it to be able to critique it beyond "I don't understand this".16:53
ChrisPolinDetailed indeed, thank you! To take your points in turn:16:54
ChrisPolinFirst paragraph: Good start! =)16:54
ChrisPolinSecond paragraph: 1) and 2) - This is not my area of expertise (yet) so I'm very open to suggestion. From my reading, G/W/T and BDD seem to be a very convenient way to link requirements and tests relatively directly. I should perhaps include a caveat that the overall trustable model does not exclusively function on the BDD approach.16:55
ChrisPolin3) Noted. I was referring to the requirements in the yaml file referenced in the first paragraph, but I will clarify this.16:56
persiaI agree that they are convenient, but from our discussion Monday, I had the impression that it made sense to step back one layer of abstraction, so that Trustable was only asserting that requirements be A) human comprehensible and B) machine testable, without specifying the workflow for the project, any specific formats, etc.16:57
persiaAh, that's the difference then.  I consider "requirements" to be core data, preferably in a repo (ideally not the same repo as the code repo), so not metadata.16:57
jmacsWhy not in the same repo?16:58
persiajmacs: Mostly to make it easier to review/stage changes to requirements separately from changes to code.  This may be an implementation concern.16:58
persiaMy past experience with projects that have requirements & code in the same repo has been that they will sometimes change in concert without anyone noticing.16:59
persiaMy experience with projects that separate the requirements from the code is that it becomes easy to have additional roles that might be part of the reviewers for the requirements changes, but are not trusted to be reviewers for the code changes.16:59
jmacsInteresting, thanks.16:59
persiaWith separated repos, it also becomes easy to flag that a given change is a requirements change without delicate regexes, etc.  This can be important if one has a waterfall model (even iterative waterfall with multi-month cycles) and needs to enforce change control in order to ensure schedules are met.17:00
persiaBut the more I think about this, the more I realise it is just an implementation detail.17:01
ChrisPolinWRT stepping back one layer of abstraction; my approach with these sorts of conceptual introductions is to keep it very abstract in the diagram and offer examples in the explanatory text to compliment. Perhaps the text is a little too implementation-based however.17:04
ChrisPolinIt does assert BDD in a 'this is the way it is done' fashion, which I can soften a bit.17:06
persiaThat's really my concern.  There's a lot of environments where the results should be trusted that don't want to work in sprints with ad-hoc requirements development, and a first read of BDD stuff may cause one to think this is mandated.17:08
ChrisPolinDo you suggest taking all mention of BDD out of the explanation then? Or caveating that it isn't the only game in town?17:09
persiaI also suspect there are a fair number of environments where they use XMI or similar to describe some requirements, for which there exist compliance verification systems.  I don't see any reason these folk have to change the entire way they think about requirements just to be trustable.17:09
persiaI'd suggest avoiding the term "BDD", as I don't know of any results one gets from a BDD search that don't expect some derivative of Agile.17:10
persiaMaybe I'm being too careful, but I think it is more useful to use yarn or behave as examples of test framework tooling that can be used, skipping the entire discussion about development methodology.17:10
persiaIf folk research those, decide they like BDD, and want to go that way (or with a modification thereof), that becomes a safe (and possibly even recommended) path, but I think not mandated.17:11
ChrisPolinA separate 'example implementation' page might be useful here.17:12
ChrisPolinIt does raise the same concern slightly though; if those that don't like that approach then associate trustable as a whole with it, they may decided against it.17:12
persiaMaybe it doesn't need a separate page.  Perhaps, if the system is described as a component diagram, potential software that might implement a given component could be listed at the bottom of that page.17:13
ChrisPolinSure, that can be done.17:13
persiaI really don't think there's any need to mention "BDD" anywhere.  BDD is a method for generating software, not a toolset.  I very strongly believe that some of the tools used for BDD are tools that should be used for trustable, but that doesn't necessarily imply that the folk using these tools must be doing BDD.17:14
persiaTo take this to another domain, there are lots of situations in which a roofing hammer is the right tool, even if one isn't working on a roof.17:14
ChrisPolinHaha fair enough.17:14
ChrisPolinOk, I'll take it out.17:15
persiaMind you, taking it out probably means being a bit more verbose about the important bits, making the use of G/W/T as an example more important.17:15
ChrisPolinI can work around that.17:20
ChrisPolinHappy to move on to the next point?17:21
ChrisPolin1) I had left that deliberately vague. I had signed commits in my head, but of course this harks back to the conversation on the mailing list re: sharing passwords etc (how much can you trust that the person on the signature is the person between the screen and the chair?)17:24
ChrisPolinSo that's an open question as it stands.17:25
persiaOK.  As long as it is deliberately vague, I see no issues with it.  I just wanted to make sure it was well-understood that this was wishful thinking without careful implementation.17:25
persia(in that most patch tracking systems only loosely enforce this.  The more common safe model I see is using ssh keys to authenticate pushes, but that model could also be debated at length)17:26
ChrisPolinIt's safer, but again falls at the same hurdle.17:27
ChrisPolin2) In a sense, yes it is human review in that the link is in the commit message. What I'm thinking here is ease-of-audit more than machine-enforcible.17:29
persiaSome aspects can be handled by machine.  For example, any commit that doesn't indicate why it exists could be rejected, etc.17:29
ChrisPolinYes, true.17:30
persiaBut I think it is important to indicate that the ultimate responsibility for linking code to requirements belongs to humans, as human activities require a greater level of review/audit than machine activities (because it is that much less possible to prove correct behaviour)17:31
ChrisPolinI can make that clarification, I think you're right.17:31
ChrisPolin3) They don't have to be a standalone artifact, I was thinking that as they are consumed by a few different components, it would be easier to have them in a dedicated file, but again I'm open to correction here.17:34
persiaMost of the patch tracking systems I've used that support CI provide a means to show the CI results (including executed tests) in the patch tracker.  One of the results might be generated artifacts.17:36
persiaAs an example, for one project I watch, one of the build artifacts is a javascript application that loads in a browser.  One of the "tests" just generates this application and posts it as a "result" to the patch tracker, so that human reviewers can load the application in their browser easily if they want to check the behaviour of some bit of code.17:37
persiaBut I'm not sure I understand how any test artifacts would be consumed outside the CI system.  Aren't they only relevant for the patch review?17:37
ChrisPolinMy thought process might be explained better with my explanation of paragraph 5.17:38
persiaHappy to table this point until after that discussion :)17:38
ChrisPolinWhich it appears we have reached!17:39
persiaWell, unless you want to revisit the notes on paragraph 2 for paragraph 4, but I think that would be redundant :)17:39
ChrisPolinThis is again a bit of a 'high-level abstraction vs implementation' debate.17:39
ChrisPolinI'm only aware of one approach to automatically produce the SSP, although I'm sure there are several.17:40
persiaTo make sure we're talking about the same thing, could you expand "SSP"?17:41
ChrisPolinBut the OpenControl project is the one we have researched for this purpose, are you familiar with it? I'll make a note to link our opencontrol page here.17:42
ChrisPolinSSP is the System Security Plan, it's the documentation that you must produce to an auditor which details compliance to a standard.17:42
persiaI'm loosely familiar with Open Control.  For the purposes of this discussion, would you agree that an "SSP" is a representative of the class of compliance documentation produced by comparing requirements to implementation and generating a report matching some predefined template?17:44
ChrisPolinYes, although I think it is specific to the standard compliance domain.17:45
persiaMaybe?  When I worked in the finance industry, projects had to submit compliance reports to the information security office, which didn't have that acronym, were likely in a different format, but I think serve the same purpose.17:46
ChrisPolinI see! Like the man in the orthopaedic shoes, I stand corrected.17:47
persiaNote that industry is very wide.  There may be vast sectors for which a standardised SSD is normal.17:47
persiaErr, SSP.17:48
ChrisPolinI can generalise the mention of SSPs in which case.17:48
ChrisPolinThe initial point however, is that the compliance documentation is produced is created using some software (in this case, Compliance Masonry - I'm not aware of others) which consumes a yaml file with a narrative detailing how each control is satisfied.17:50
persiaI am reminded of an old standard for electronic document interchange (I recall X.25, but my search engine disagrees with the number).  It specified how to format invoices, shipping labels, etc. for electronic transmission between sites.  It was used by a vast number of companies, but also entirely alien to other sectors, to the degree that when SOAP became a thing, everyone invented new ways of sharing documents, as if there had been no standard.17:50
ChrisPolinLol this is a large elephant indeed, in which case.17:51
persiaRight.  I'm presuming that Compliance Masonry requires a special input format, which could be generated by one of the "tests" in the CI system.17:51
persiaAnd perhaps again when things merge, so that one has up-to-date compliance information.17:52
ChrisPolinIt can also be converted to from a Mustard yaml.17:52
persia(this was "Produce Development Compliance Report" and "Produce SSP" in my draft diagram, with the former representing something that could be in production, and the latter representing what was in production)17:52
persiaConverted from a Mustard yaml?  That worries me.17:53
persiaIn my mind, the Mustard documents the requirements, so doesn't change except when the requirements change.  As a result of this, it becomes hard for this to reflect when or how a given requirement is implemented.17:53
persiaAn alternate view is that there exists some requirements repository in some format, and that a Mustard containing the current mapping is produced at the time of each merge to the development branch.17:54
persiaBut that breaks the assumption of the first paragraph.17:54
*** AlisonChaiken has quit IRC17:57
persiaAha.  Found the old EDI format.  "ANSI X12"18:00
persiaRe-reading the last paragraph, I suppose the source of my confusion is the uncertainty about how the metadata is stored, combined with the subtle distinctions between CI and CD that mean that compliance documentation ends up being generated multiple times: at push time (so reviewers can know if a patch breaks or adds compliance), at merge time (so everyone can check the compliance status of the development branch), and at deploy time (so that the18:07
persiaproduction system compliance is documented properly).18:07
ChrisPolinIn the case of compliance masonry, it reads in an 'opencontrol.yaml' file, which is separate from the mustard yaml. jmacs has created a script to convert from one schema to the other, but there is no reason why the conversion can't be influenced by external information from the CD/CI18:08
ChrisPolin'influenced' is very vague here, but I'm thinking out loud.18:08
persiaAh, yes, that makes sense.  A script that consumes the Mustard YAML, is informed by info available from the patch tracking/testing systems (driven by CI/CD), and then produces output that can be consumed to generate compliance information.18:08
persiaAnd now the model makes sense, if not the text of the last paragraph.  I'm still not sure how to critique that in detail :)18:09
ChrisPolinI'll link this page in the narrative, but for your information:
persiaand, going back to point 3 of paragraph 3, I think the test artifacts end up getting the same sort of treatment as the compliance artifacts, in the sense that there may be multiple ways to produce them (ideally from the same scripts), called at multiple times during the process, for consumption by multiple entities.18:10
ChrisPolinYes, I believe so.18:10
persiaWell, I hope that critique was useful.  Please let me know if you want me to look at it again at that level of detail.18:11
ChrisPolinAbsolutely, thank you. I'll revise the text and see how it reads from there.18:12
*** AlisonChaiken has joined #trustable18:16
*** tiagogomes has quit IRC18:33
*** laurenceurhegyi has quit IRC18:43
persiaOh, I thought of another reason to separate requirements and code repos: in a complex deployment situation (e.g. devices), one may end up receiving a requirements update, and want to be able to demonstrate that those requirements are already met without any code changes (to avoid a renewed depoyment).  This is eased by being able to demonstrate compliance with top-level VCS metadata, rather than direct diff of the code.  Although, again, an18:47
persiaimplementation detail.18:47
*** ctbruce has quit IRC19:17
*** ctbruce has joined #trustable19:17
jmacsIt's very much a prototype at the moment, but could people look at and see if that's what they had in mind for PlantUML rendering?19:39
jmacs(No rush, I'm heading home now)19:39
persiaThat renders as expected for me.  Nice work.19:49
persiaFrustratingly, when attempting to update the local git repo to see the source, I get "fatal: not valid: is this a git repository?"19:51
persia(for both `git pull` in an existing local repo, and when trying to clone a new repo)19:52
jmacsI think I've seen that error recently but I can't remember where right now21:02
*** ctbruce has quit IRC23:53

Generated by 2.14.0 by Marius Gedminas - find it at!