IRC logs for #baserock for Tuesday, 2017-03-07

*** persia has quit IRC02:09
*** persia has joined #baserock02:12
*** gtristan has joined #baserock03:07
*** jude_ has joined #baserock08:15
*** ctbruce has joined #baserock08:27
*** gtristan has quit IRC08:38
*** toscalix has joined #baserock09:05
*** noisecell has joined #baserock09:12
*** jonathanmaw has joined #baserock09:15
paulsher1oodanyone here know anything about artifactory?09:32
* paulsher1ood is trying to hack ybd to to handle binary inputs...09:32
paulsher1oodand i'm imagining that we could just have an extra field for a chunk - 'url:' which specifies a place to wget from09:33
paulsher1oodi'm wondering if/how to support authentication for generic urls and for artifactory09:34
tiagogomespaulsher1ood, Sam T might know something09:37
paulsher1oodah, ok thanks09:38
*** rdale has joined #baserock09:39
*** rdale has quit IRC09:41
*** gtristan has joined #baserock09:42
*** rdale has joined #baserock09:43
*** CTtpollard has joined #baserock10:10
*** ssam2 has joined #baserock10:23
*** ChanServ sets mode: +v ssam210:23
paulsher1oodssam2: do you know anything about artifactory authentication?10:30
ssam2I once did10:30
paulsher1oodi'm trying to adapt ybd so it can handle binary *inputs* from wget or attifacctory10:30
ssam2I think Artifactory is basically a web server, which can also pretend to be one of various types of package repository10:31
paulsher1oodbasically my thought is just to extend definitions to have an optional  'url:' field for chunks - if present, wget from it instead of git checkout10:31
ssam2seems workable10:32
paulsher1oodand then follow *commands as normal10:32
ssam2Artifactory can work the same way, everything it stores can be accessed via HTTP I think10:32
ssam2modulo permissions10:32
ssam2question is whether you assume the remote file never changes (thus opening a reproducibility hole) or whether you redownload it and rechecksum it on each build10:35
ironfootmaybe the md5 is exposed somewhere, so you can check that before fetching the file?10:58
ironfoot(or any other kind of checksum)11:00
ironfootit looks like in this instance of artifactory i'm looking at, md5 and sha1 files are being generated for all the files11:01
* paulsher1ood would rather not get into artifactory-specific handling11:04
paulsher1oodso if 'url:' pull the file, and if 'md5:' specified, check that the downloaded file matches it11:05
rjekI was under the impression you could ask Artifactory for a URL to a specific artefact that was ever-living and unchanging, and you could pop that into your definition11:05
ironfootpaulsher1ood: yep, that's what I was thinking11:06
rjek(You may need to also support basic auth or something)11:06
ironfootpaulsher1ood: maybe 'checksum' instead of 'md5'11:06
ssam2I think it depends on the permissions of the Artifactory instance whether that's actually true, but it seems reasonable to trust that people don't overwrite existing artifacts with new ones and expect nothing to break11:06
paulsher1oodironfoot: aren't there other ways of  checksumming? md5 is explicit, at least?11:06
paulsher1oodis it reasonable to assume that if any binary input requires auth, the same auth applies for all binaries?11:09
ironfootyeah, makes sense11:09
ironfoot^ regarding other ways of checksumming and using 'md511:10
* rjek wonders where MD5 is coming from for this11:11
CTtpollardfor what it's worth it's common for a yocto recipe to check before md5sum & sha256sum11:11
paulsher1oodrjek: user puts it in definitions?11:11
rjekMD5 should not be used for new designs, even ones that are not security sensitive11:12
rjek(And I'd suggest that there *is* a security aspect to this)11:12
paulsher1oodrjek: what would you prefer?11:12
rjekMD5 is broken in that a mobile phone can find collisions in an hour, SHA1 is broken in that $500,000 will find you a collision (and that price will only come down).  SHA2 isn't broken *yet* but is the same fundamental design as SHA1.11:13
ironfootpaulsher1ood: sha1 is also autogenerated by default in artifactory11:14
rjekYou could of course define the hash algorithm being used in the hash: field11:14
paulsher1oodthe request for this is in a situation to assemble 3rd party blobs in a (presumed) controlled environment11:14
ironfootno other sha's are generated by default11:14
rjekhash: md5$d41d8cd98f00b204e9800998ecf8427e11:15
rjekhash: sha256$eb4bd64f7014f7d42e9d358035802242741b974e8dfcd37c59f9c21ce29d781e11:15
rjekor whatever11:15
ironfootpaulsher1ood:  regarding authentication, using `wget` with urls like:  https://<user>:<password> works11:15
paulsher1oodi'm not sure that ybd is the right place to be checking *security* in this use case...11:15
rjekironfoot: Fortunately it's easy to calculate the hash :)11:15
rjekpaulsher1ood: I think ybd should aim to check for reproducability and flag errors where that reproducability is at risk11:16
ironfootrjek: yeah, what we wanted to avoid was to re-download something11:16
rjekironfoot: It's not that much of a chore to download the artefact when creating or changing the definition?11:16
paulsher1oodrjek: yup. but i think md5 or sha1 is sufficient for that11:17
ironfoot"i have just built this a minute ago, why am I building it again?? I just want to deploy it"11:17
rjekWell, it means it's trivial to change a build to inject new code without it being detected11:18
rjekPlease at least use SHA256 if possible11:18
ironfootstop using sha1's in git also?11:18
rjekI would like to see that, but it's less urgent as git doesn't only hash the content of objects11:19
CTtpollardVerifying what I mentioned, it's a requirement now for a yocto recipe when handling fetches of upstream 'files' to have MD5 & SHA25611:23
rjekHaving two hashes makes it much more difficult to attack and make mistakes.11:23
rjekI imagine MD5 also provides some level of backwards compatibility11:24
ironfootpaulsher1ood: regarding Concourse, it just looks unmaintained11:27
paulsher1oodironfoot: ack11:27
* paulsher1ood needs to find some time11:28
ironfootyep, no rush :)11:28
paulsher1oodalso waiting for clarity on buildstream progress11:28
rjekpaulsher1ood: That doesn't seem to mandate a hash, or am I behing blind?11:29
ironfootrjek: "WIP"11:31
*** CTtpollard has quit IRC11:39
paulsher1oodrjek: it doesn't *mandate* a hash... it checks them if they are provided11:42
* rjek would suggest mandating them otherwise people will be lazy11:42
paulsher1oodas i said before, ybd can't be responsible for security... it's a build tool11:45
rjekOn the subject: Enjoy this animated GIF that shows its own MD5:
paulsher1oodi'm ok to warn, not error11:45
*** CTtpollard has joined #baserock11:45
rjekpaulsher1ood: I thought ybd was about being able to demonstrate provenance in builds?11:45
rjekThe instant you don't have the hash there you lose that11:46
paulsher1oodrjek: it is. but then there's the use-case where someone just wants 'whatever is the latest version of blob'11:46
rjekDoesn't that totally defeat provenance?11:47
SotKand also long-term reproducibility?11:47
* paulsher1ood is not the customer here11:47
ironfoota warning is ok if we want this functionality asap11:49
noisecellor an argument to free that functionality that it will error if that argument is not used i.e. --I-want-to-do-my-thing11:50
ironfootor --ignore-rjek11:51
ironfootI would want my name included in an option of a cmd line tool!11:52
noisecellironfoot, there were some kind of petrify before... so it is next era...11:53
noisecellstone -> bronze -> iron ;-)11:53
ironfootis there a tool called `forge` ?11:54
ssam2*sourceforge even11:58
CTtpollardIt's also useful to override checksum checking when internally testing12:17
paulsher1ood+1 :)12:18
CTtpollardsame goes for enforcing license checking, if ybd enforces that12:20
paulsher1oodnot so far, but agreed :)12:24
CTtpollardThere use to be a per recipe 'BB_STRICT_CHECKSUM' override in yocto, I can only presume it's still a thing. I used it when locally pulling in gfx blobs that were constantly changing from a private source12:28
CTtpollardit would warn about not verifying the hashes but wouldn't halt12:28
*** CTtpollard has quit IRC12:52
*** CTtpollard has joined #baserock13:02
paulsher1oodybd has check-definitions: which can be 'ignore', 'warn', 'exit'13:06
paulsher1oodmy latest wip aims to use that do decide wether to exit or not
jjardonpaulsher1ood:  ok to merge ?13:10
*** rdale has quit IRC13:16
*** rdale has joined #baserock14:02
*** gtristan has quit IRC15:22
*** gtristan has joined #baserock15:36
*** paulwaters_ has joined #baserock15:45
*** ctbruce has quit IRC16:14
*** noisecell has quit IRC16:56
*** toscalix has quit IRC17:48
*** jonathanmaw has quit IRC17:59
*** ssam2 has quit IRC18:14
*** jude_ has quit IRC18:38
*** jude_ has joined #baserock19:38
*** gtristan has quit IRC20:49
*** jude_ has quit IRC22:39
*** rdale has quit IRC23:11

Generated by 2.15.3 by Marius Gedminas - find it at!