| *** persia has quit IRC | 02:09 | |
| *** persia has joined #baserock | 02:12 | |
| *** gtristan has joined #baserock | 03:07 | |
| *** jude_ has joined #baserock | 08:15 | |
| *** ctbruce has joined #baserock | 08:27 | |
| *** gtristan has quit IRC | 08:38 | |
| *** toscalix has joined #baserock | 09:05 | |
| *** noisecell has joined #baserock | 09:12 | |
| *** jonathanmaw has joined #baserock | 09:15 | |
| paulsher1ood | anyone here know anything about artifactory? | 09:32 |
|---|---|---|
| * paulsher1ood is trying to hack ybd to to handle binary inputs... | 09:32 | |
| paulsher1ood | and i'm imagining that we could just have an extra field for a chunk - 'url:' which specifies a place to wget from | 09:33 |
| paulsher1ood | i'm wondering if/how to support authentication for generic urls and for artifactory | 09:34 |
| tiagogomes | paulsher1ood, Sam T might know something | 09:37 |
| paulsher1ood | ah, ok thanks | 09:38 |
| *** rdale has joined #baserock | 09:39 | |
| *** rdale has quit IRC | 09:41 | |
| *** gtristan has joined #baserock | 09:42 | |
| *** rdale has joined #baserock | 09:43 | |
| *** CTtpollard has joined #baserock | 10:10 | |
| *** ssam2 has joined #baserock | 10:23 | |
| *** ChanServ sets mode: +v ssam2 | 10:23 | |
| paulsher1ood | ssam2: do you know anything about artifactory authentication? | 10:30 |
| ssam2 | I once did | 10:30 |
| paulsher1ood | i'm trying to adapt ybd so it can handle binary *inputs* from wget or attifacctory | 10:30 |
| ssam2 | I think Artifactory is basically a web server, which can also pretend to be one of various types of package repository | 10:31 |
| paulsher1ood | basically my thought is just to extend definitions to have an optional 'url:' field for chunks - if present, wget from it instead of git checkout | 10:31 |
| ssam2 | seems workable | 10:32 |
| paulsher1ood | and then follow *commands as normal | 10:32 |
| ssam2 | Artifactory can work the same way, everything it stores can be accessed via HTTP I think | 10:32 |
| ssam2 | modulo permissions | 10:32 |
| ssam2 | question is whether you assume the remote file never changes (thus opening a reproducibility hole) or whether you redownload it and rechecksum it on each build | 10:35 |
| paulsher1ood | ack | 10:38 |
| ironfoot | maybe the md5 is exposed somewhere, so you can check that before fetching the file? | 10:58 |
| ironfoot | (or any other kind of checksum) | 11:00 |
| ironfoot | it looks like in this instance of artifactory i'm looking at, md5 and sha1 files are being generated for all the files | 11:01 |
| ssam2 | nice | 11:03 |
| * paulsher1ood would rather not get into artifactory-specific handling | 11:04 | |
| paulsher1ood | so if 'url:' pull the file, and if 'md5:' specified, check that the downloaded file matches it | 11:05 |
| rjek | I was under the impression you could ask Artifactory for a URL to a specific artefact that was ever-living and unchanging, and you could pop that into your definition | 11:05 |
| paulsher1ood | ack | 11:06 |
| ironfoot | paulsher1ood: yep, that's what I was thinking | 11:06 |
| rjek | (You may need to also support basic auth or something) | 11:06 |
| ironfoot | paulsher1ood: maybe 'checksum' instead of 'md5' | 11:06 |
| ssam2 | I think it depends on the permissions of the Artifactory instance whether that's actually true, but it seems reasonable to trust that people don't overwrite existing artifacts with new ones and expect nothing to break | 11:06 |
| paulsher1ood | ironfoot: aren't there other ways of checksumming? md5 is explicit, at least? | 11:06 |
| paulsher1ood | is it reasonable to assume that if any binary input requires auth, the same auth applies for all binaries? | 11:09 |
| ironfoot | yeah, makes sense | 11:09 |
| ironfoot | ^ regarding other ways of checksumming and using 'md5 | 11:10 |
| paulsher1ood | ack | 11:10 |
| * rjek wonders where MD5 is coming from for this | 11:11 | |
| CTtpollard | for what it's worth it's common for a yocto recipe to check before md5sum & sha256sum | 11:11 |
| CTtpollard | *both | 11:11 |
| paulsher1ood | rjek: user puts it in definitions? | 11:11 |
| rjek | MD5 should not be used for new designs, even ones that are not security sensitive | 11:12 |
| rjek | (And I'd suggest that there *is* a security aspect to this) | 11:12 |
| paulsher1ood | rjek: what would you prefer? | 11:12 |
| rjek | SHA3 | 11:12 |
| rjek | MD5 is broken in that a mobile phone can find collisions in an hour, SHA1 is broken in that $500,000 will find you a collision (and that price will only come down). SHA2 isn't broken *yet* but is the same fundamental design as SHA1. | 11:13 |
| paulsher1ood | hmmm. | 11:14 |
| ironfoot | paulsher1ood: sha1 is also autogenerated by default in artifactory | 11:14 |
| rjek | You could of course define the hash algorithm being used in the hash: field | 11:14 |
| paulsher1ood | the request for this is in a situation to assemble 3rd party blobs in a (presumed) controlled environment | 11:14 |
| ironfoot | no other sha's are generated by default | 11:14 |
| rjek | hash: md5$d41d8cd98f00b204e9800998ecf8427e | 11:15 |
| rjek | hash: sha256$eb4bd64f7014f7d42e9d358035802242741b974e8dfcd37c59f9c21ce29d781e | 11:15 |
| rjek | or whatever | 11:15 |
| ironfoot | paulsher1ood: regarding authentication, using `wget` with urls like: https://<user>:<password>@artifactory.example.com/artifactory/Something/foo.zip works | 11:15 |
| paulsher1ood | i'm not sure that ybd is the right place to be checking *security* in this use case... | 11:15 |
| rjek | ironfoot: Fortunately it's easy to calculate the hash :) | 11:15 |
| rjek | paulsher1ood: I think ybd should aim to check for reproducability and flag errors where that reproducability is at risk | 11:16 |
| ironfoot | rjek: yeah, what we wanted to avoid was to re-download something | 11:16 |
| rjek | ironfoot: It's not that much of a chore to download the artefact when creating or changing the definition? | 11:16 |
| paulsher1ood | rjek: yup. but i think md5 or sha1 is sufficient for that | 11:17 |
| ironfoot | "i have just built this a minute ago, why am I building it again?? I just want to deploy it" | 11:17 |
| rjek | Well, it means it's trivial to change a build to inject new code without it being detected | 11:18 |
| rjek | Please at least use SHA256 if possible | 11:18 |
| ironfoot | stop using sha1's in git also? | 11:18 |
| ironfoot | :P | 11:18 |
| rjek | I would like to see that, but it's less urgent as git doesn't only hash the content of objects | 11:19 |
| CTtpollard | Verifying what I mentioned, it's a requirement now for a yocto recipe when handling fetches of upstream 'files' to have MD5 & SHA256 | 11:23 |
| rjek | Having two hashes makes it much more difficult to attack and make mistakes. | 11:23 |
| rjek | I imagine MD5 also provides some level of backwards compatibility | 11:24 |
| paulsher1ood | okhttps://gitlab.com/baserock/ybd/commit/883478f4e56dc4e502d4fe7195bcd9932af46360 | 11:27 |
| ironfoot | paulsher1ood: regarding Concourse, it just looks unmaintained | 11:27 |
| paulsher1ood | ironfoot: ack | 11:27 |
| * paulsher1ood needs to find some time | 11:28 | |
| ironfoot | yep, no rush :) | 11:28 |
| paulsher1ood | also waiting for clarity on buildstream progress | 11:28 |
| rjek | paulsher1ood: That doesn't seem to mandate a hash, or am I behing blind? | 11:29 |
| ironfoot | rjek: "WIP" | 11:31 |
| rjek | heh | 11:32 |
| *** CTtpollard has quit IRC | 11:39 | |
| paulsher1ood | rjek: it doesn't *mandate* a hash... it checks them if they are provided | 11:42 |
| * rjek would suggest mandating them otherwise people will be lazy | 11:42 | |
| paulsher1ood | as i said before, ybd can't be responsible for security... it's a build tool | 11:45 |
| rjek | On the subject: Enjoy this animated GIF that shows its own MD5: https://shells.aachen.ccc.de/~spq/md5.gif | 11:45 |
| paulsher1ood | i'm ok to warn, not error | 11:45 |
| *** CTtpollard has joined #baserock | 11:45 | |
| rjek | paulsher1ood: I thought ybd was about being able to demonstrate provenance in builds? | 11:45 |
| rjek | The instant you don't have the hash there you lose that | 11:46 |
| paulsher1ood | rjek: it is. but then there's the use-case where someone just wants 'whatever is the latest version of blob' | 11:46 |
| rjek | Doesn't that totally defeat provenance? | 11:47 |
| SotK | and also long-term reproducibility? | 11:47 |
| rjek | ^ | 11:47 |
| * paulsher1ood is not the customer here | 11:47 | |
| paulsher1ood | s|customer|customer/user| | 11:48 |
| ironfoot | a warning is ok if we want this functionality asap | 11:49 |
| noisecell | or an argument to free that functionality that it will error if that argument is not used i.e. --I-want-to-do-my-thing | 11:50 |
| rjek | --yes-I-know-just-do-it | 11:50 |
| ironfoot | or --ignore-rjek | 11:51 |
| rjek | :( | 11:51 |
| ironfoot | hah | 11:51 |
| ironfoot | I would want my name included in an option of a cmd line tool! | 11:52 |
| noisecell | ironfoot, there were some kind of petrify before... so it is next era... | 11:53 |
| noisecell | stone -> bronze -> iron ;-) | 11:53 |
| ironfoot | is there a tool called `forge` ? | 11:54 |
| ssam2 | sourcecorge | 11:58 |
| ssam2 | *sourceforge even | 11:58 |
| CTtpollard | It's also useful to override checksum checking when internally testing | 12:17 |
| paulsher1ood | +1 :) | 12:18 |
| CTtpollard | same goes for enforcing license checking, if ybd enforces that | 12:20 |
| paulsher1ood | not so far, but agreed :) | 12:24 |
| CTtpollard | There use to be a per recipe 'BB_STRICT_CHECKSUM' override in yocto, I can only presume it's still a thing. I used it when locally pulling in gfx blobs that were constantly changing from a private source | 12:28 |
| CTtpollard | it would warn about not verifying the hashes but wouldn't halt | 12:28 |
| *** CTtpollard has quit IRC | 12:52 | |
| *** CTtpollard has joined #baserock | 13:02 | |
| paulsher1ood | ybd has check-definitions: which can be 'ignore', 'warn', 'exit' | 13:06 |
| paulsher1ood | my latest wip aims to use that do decide wether to exit or not https://gitlab.com/baserock/ybd/commit/7cd25e1c2f3a104dbcfd4fd42ebc4f2b4cb34216 | 13:07 |
| jjardon | paulsher1ood: ok to merge https://gitlab.com/baserock/ybd/merge_requests/320 ? | 13:10 |
| paulsher1ood | yup | 13:14 |
| *** rdale has quit IRC | 13:16 | |
| *** rdale has joined #baserock | 14:02 | |
| *** gtristan has quit IRC | 15:22 | |
| *** gtristan has joined #baserock | 15:36 | |
| *** paulwaters_ has joined #baserock | 15:45 | |
| *** ctbruce has quit IRC | 16:14 | |
| *** noisecell has quit IRC | 16:56 | |
| *** toscalix has quit IRC | 17:48 | |
| *** jonathanmaw has quit IRC | 17:59 | |
| *** ssam2 has quit IRC | 18:14 | |
| *** jude_ has quit IRC | 18:38 | |
| *** jude_ has joined #baserock | 19:38 | |
| *** gtristan has quit IRC | 20:49 | |
| *** jude_ has quit IRC | 22:39 | |
| *** rdale has quit IRC | 23:11 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
