Distributing Credentials (Secrets) to Containers

Talk on distributing safely distributing secrets to containers

Transcript

1. http://AssimilationSystems.com/ 1/16 Distributing Credentials (Secrets) Distributing Credentials (Secrets) Securely to Containers Securely to Containers Alan Robertson Charter Communications NaaS Project alan.robertson@charter.com @OSSAlanR Founder: Linux-HA (Pacemaker) project Assimilation Project
2. http://AssimilationSystems.com/ 2/16 Agenda Agenda • What’s a secret? • What does it mean to protect a secret? • So what’s the problem? • Two solutions: – Custodia – Custom Code
3. http://AssimilationSystems.com/ 3/16 What’s a Secret? What’s a Secret? • These kinds of things are often secrets: – Passwords, Application Tokens, Secret keys, AWS keys, etc. • You’re storing them so machines (and humans) have access to them • It might be something small and internal • It might be something a bit more important • It might be the “Keys To The Kingdom”
4. http://AssimilationSystems.com/ 4/16 What does protecting a secret What does protecting a secret mean? mean? • Don’t store them in plain text on disk • Don’t pass them as command line parameters, or environment variables • Don’t store them in GitHub... • This is harder than it sounds...
5. http://AssimilationSystems.com/ 5/16 So what’s the problem? So what’s the problem? • Microservices are by definition network-facing • If you cannot protect your Vault key, then your secrets are at risk • If an attacker compromises your application, they probably have access to your vault key... • If you could protect your Vault key, you could probably protect your secrets • Vault software helps but is not enough • We need to be able to protect our vault keys!
6. http://AssimilationSystems.com/ 6/16 What to do? What to do? • Don’t store vault keys in your containers • Identify and validate your application without using keys they need to protect • How to do this?
7. http://AssimilationSystems.com/ 7/16 UNIX sockets to the rescue! UNIX sockets to the rescue! • UNIX domain sockets allow authentication of the application that’s connecting – User ID – Group ID – Process ID – Security Context (SE Linux)
8. http://AssimilationSystems.com/ 8/16 UNIX Sockets downside UNIX Sockets downside • This information comes from the process that initially connected • There is an attack based on this “freezing” • It involves forking lots of processes and wrapping around pids • It will likely fail many, many times before succeeding • It’s much harder to carry out in a container environment • The shorter the interval between connection and authentication, the harder this is to carry out • In the worst case, it’s still much harder to subvert than an environment variable or a file • It cannot subvert the security context information
9. http://AssimilationSystems.com/ 9/16 This helps you get even more This helps you get even more information from information from • /proc • Docker inspect This gives you something similar to the DNA of the caller – the cyber analog of biometrics Authenticate your caller with just this information
10. http://AssimilationSystems.com/ 10/16 /proc interesting things /proc interesting things • /proc/<pid>/cgroup – identifies the container • /proc/<pid>/exe - full pathname client binary • /proc/<pid>/uid_map (and gid_map) • /proc/<pid>/status – uid/gid info, ppid • /proc/<pid>/cmdline - client command line – can be modified by the application
11. http://AssimilationSystems.com/ 11/16 Docker inspect interesting things Docker inspect interesting things • Name - the name of the image • Path - full pathname of the container's "init" process • Args - the arguments given to the init process in the container • State.Pid - the process id of the container's "init" process • Config.Hostname - hostname of this container • Config.Image - the image name that client is running in • Image - the SHA256 sum of "Config.Image"
12. http://AssimilationSystems.com/ 12/16 Red Hat’s Custodia Red Hat’s Custodia
13. http://AssimilationSystems.com/ 13/16 Could it be simpler? Could it be simpler? • Custodia: – Proxies the entire secret management infrastructure – Duplicate configuration with Vault – Larger attack surface (even though it’s good code) • What if you just provided vault keys? – Each client just gets a vault key when authenticated – Clients use key to connect directly to vault
14. http://AssimilationSystems.com/ 14/16 An alternative approach An alternative approach
15. http://AssimilationSystems.com/ 15/16 What’s Interesting? What’s Interesting? • AuthProxy does all its work immediately – short interval – makes the attack harder • AuthProxy doesn’t read any data from client – This eliminates several classes of attacks • Total AuthProxy code is small ~ 300 lines • Much harder to subvert than environment variables or files to store keys in
16. http://AssimilationSystems.com/ 16/16 References References • Custodia article: – https://peerlyst.com/posts/sharing-secrets-with- containers-using-custodia-alan-robertson • Authproxy article: – https://peerlyst.com/posts/the-authproxy-method- of-sharing-secrets-safely-with-containers-alan- robertson