| *** bjdooks has quit IRC | 01:40 | |
| *** bjdooks has joined #baserock | 01:41 | |
| *** gtristan has joined #baserock | 05:38 | |
| *** franred has joined #baserock | 08:32 | |
| *** toscalix has joined #baserock | 08:43 | |
| *** paulw has joined #baserock | 09:00 | |
| *** ctbruce has joined #baserock | 09:07 | |
| *** fay has joined #baserock | 09:10 | |
| *** fay is now known as Guest99468 | 09:10 | |
| *** Guest99468 is now known as faybrocklebank | 09:10 | |
| *** ctbruce has quit IRC | 10:03 | |
| *** ctbruce has joined #baserock | 10:03 | |
| *** ctbruce has quit IRC | 10:29 | |
| *** ctbruce has joined #baserock | 10:31 | |
| *** lchlan has quit IRC | 10:43 | |
| *** faybrocklebank has quit IRC | 10:44 | |
| *** faybrocklebank has joined #baserock | 10:44 | |
| *** lchlan has joined #baserock | 10:45 | |
| pedroalvarez | meh, there was an email from paulsherwood waiting to be moderated since sunday, but I only received the moderator request yesterday | 11:36 |
|---|---|---|
| pedroalvarez | (which I missed due other email client problems) | 11:37 |
| paulsherwood | pedroalvarez: never mind, i re-sent it anyway | 11:52 |
| paulsherwood | by all means delete that mail | 11:52 |
| pedroalvarez | :/ I approved it before coming here | 12:02 |
| pedroalvarez | jjardon: I left some comments in the list re. migration to gitlab | 12:03 |
| *** paulw has joined #baserock | 12:21 | |
| *** gtristan has quit IRC | 13:20 | |
| *** gtristan has joined #baserock | 13:26 | |
| rjek | Hello. I'm looking into adding hierarchical caching into kbas such that your local kbas can query a shared/internet-facing kbas for things it doesn't have. I've been thinking about how to get artefacts into an upstream cache. I see some problems, such as DoSing the upstream because of nonsense uploads, as well as the possibility of pushing secret stuff. | 13:33 |
| rjek | I have two ideas: first make upstreams only pushable to with credentials (as is true now, ish), and some sort of configurability in kbas or signal from ybd about the confidentiality of artefacts (perhaps a list of regexps in a white/black list) | 13:34 |
| rjek | Any thoughts or ideas are will be welcomed. | 13:34 |
| leeming | rjek, does kbas have other information other than the hash? e.g name of the chunk/thing | 13:39 |
| leeming | just wondering if it has access to software name level things, to compare against the master definitions and only cache things that exist there? as a basic whitelist | 13:40 |
| leeming | unless, there are proprietary patches :\ | 13:41 |
| rjek | Current protocol includes the chunk name when querying or submitting, that would have to change too | 13:41 |
| leeming | short of introducing a field in the definitions that states cacheable, im not sure... accidents may happen | 13:41 |
| leeming | just name? | 13:42 |
| rjek | I'm think about this from a "public ubercache" point of view, which may not be possible to do securely, in which case that idea can be shelved | 13:42 |
| leeming | surely there are multiple versions of the artifact, based on what filesystem exists below it | 13:42 |
| rjek | But it'd still be useful to have an office-wide cache server each developer can make use of, for example | 13:42 |
| leeming | yes, i am thinking of a pure public side whitelist too | 13:42 |
| rjek | Or for each runner in a CI system to have a local cache that can query a data centre-wide cache | 13:42 |
| leeming | private, clearly you wont have the same security issues for caching proprietary things | 13:43 |
| rjek | Nod | 13:43 |
| leeming | i can only think that if the hash was the key, then chunks marked as private, could either a) not be uploaded to the public cache, b) include some key for access | 13:44 |
| rjek | I'm already planning on having credential processing for read as well as write | 13:45 |
| leeming | other than whitelisting on the incoming, whitelisting on the private cache for out going is probs safer? | 13:46 |
| leeming | therefore making it a responsibility of the kbas local server.. | 13:47 |
| leeming | no change to definitions | 13:47 |
| leeming | client -> tmp cache -> local kbas -> globla kbas | 13:47 |
| leeming | correct? | 13:47 |
| rjek | yeah | 13:49 |
| rjek | tbh it sounds too fraught to consider having public upstream caches at all | 13:49 |
| leeming | yeah, so a simple 'fix' would be to apply a whitelist on the kbas UPloading | 13:49 |
| leeming | whitelist = in public definitions | 13:50 |
| paulsherwood | rjek: and yet, we already do? :) | 13:50 |
| leeming | additional things to control private patches tho.. not sure without thinking about it | 13:50 |
| rjek | paulsherwood: We don't have anything that anyone can push to | 13:50 |
| rjek | My concern is accidental uploading | 13:50 |
| leeming | kbas should have in + out rules then | 13:51 |
| leeming | out the out rules are more important | 13:51 |
| rjek | Where my aim with this is to improve cache hits among teams rather than just per-developer or per-runner | 13:51 |
| leeming | kbas -> kbas | 13:51 |
| leeming | agreed. caching should happen at multiple levels/ network distance matters | 13:52 |
| radiofree | rjek: i think the main kbas (i.e the one everyone can use) should only be pushed to by the CI, so password protected | 13:53 |
| radiofree | i don't trust random artifacts from random users | 13:53 |
| radiofree | i think that's what we currently do? | 13:53 |
| rjek | Well, you probably won't ever fetch random artefacts from random users | 13:55 |
| rjek | Unless you'd be building the same random thing | 13:55 |
| rjek | In which case, even if the binary is broken some how, at least you found out sooner | 13:55 |
| rjek | But yes, one where only CI can push to would be a perfectly sensible approach if that's what a team wanted | 13:56 |
| rjek | I need a cup of tea. | 13:56 |
| paulsherwood | the 'only ci can push' makes sense to me too. bonus points if (for any public kbas), the uploads can only happen via i private network with the ci runners | 13:58 |
| rjek | paulsherwood: Why a private network? | 14:00 |
| rjek | Because credentials are involved and it's over-the-internet, I was just going to mandate TLS | 14:00 |
| *** edcragg has quit IRC | 14:17 | |
| *** edcragg has joined #baserock | 14:31 | |
| paulsherwood | rjek: because physical security is better than not? if uploads can only happen via private network, wouldn't that be better than TLS? | 14:53 |
| rjek | Tricky | 14:55 |
| rjek | There's no way to have a private network into AWS, for example. | 14:55 |
| rjek | You can VPN, but that's just the different wrapper around cryptography and the upshot being the same as using TLS | 14:55 |
| paulsherwood | rjek: oh? i thought it's possible to have a private network shared between aws machines? | 14:56 |
| rjek | paulsherwood: That's a VLAN, plus the data still needs to get there over the internet. | 14:56 |
| paulsherwood | not if all the runners are already on the vlan? | 14:58 |
| rjek | Any secret, sensitive, or propriatary data that they have has to get there somehow | 14:58 |
| paulsherwood | true... but not via kbas? | 15:01 |
| paulsherwood | jjardon: is https://gitlab.com/baserock/ybd/issues/244 reproducible with master? is there something special about the branch you're using? | 15:06 |
| rjek | Gah now I've got something which is essentially ACLs, which is probably too complicated for this | 15:43 |
| *** anahuelamo has quit IRC | 15:56 | |
| *** anahuelamo has joined #baserock | 15:58 | |
| *** faybrocklebank has quit IRC | 16:19 | |
| *** franred has quit IRC | 16:23 | |
| *** ctbruce has quit IRC | 16:23 | |
| *** toscalix_ has joined #baserock | 17:26 | |
| *** toscalix has quit IRC | 17:27 | |
| *** toscalix__ has joined #baserock | 17:27 | |
| *** toscalix_ has quit IRC | 17:31 | |
| *** CTtpollard has quit IRC | 18:10 | |
| *** pedroalvarez has quit IRC | 18:28 | |
| *** pedroalvarez has joined #baserock | 18:29 | |
| *** ChanServ sets mode: +v pedroalvarez | 18:29 | |
| *** toscalix__ has quit IRC | 18:33 | |
| *** tiagogomes has quit IRC | 19:10 | |
| *** gtristan has quit IRC | 20:10 | |
| *** bjdooks_ has joined #baserock | 20:45 | |
| *** parag0n has joined #baserock | 20:48 | |
| *** thinkl33t has quit IRC | 20:48 | |
| *** bjdooks has quit IRC | 20:48 | |
| *** SotK has quit IRC | 20:48 | |
| *** SotK has joined #baserock | 20:48 | |
| *** parag0n is now known as thinkl33t | 20:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!