Kara de la Marck

GitOps, Kubernetes, and Secret Management

GitOps uses Git as the “single source of truth” for declarative infrastructure and enables developers to manage infrastructure with the same Git-based workflows they use to manage a codebase. Having all configuration files version-controlled by Git has many advantages, b
ut best practices for securely managing secrets with GitOps remain contested. Join us in this presentation about GitOps and secret management. Attendees will learn about the pros and cons of various approaches and why the Jenkins X project has chosen to standardize on Kubernetes External Secrets for secret management.

Transcript

1. GitOps, Kubernetes, and Secret Management: Don’t Bake in a Tent! @KaraMarck
2. Kubernetes GitOps: Git @KaraMarck Git is the single source of truth for the desired state of the whole system. Declarative specification for each environment, in Git.
3. Kubernetes @KaraMarck Kubernetes deployments: Automation — automates the process of applying changes Convergence — continually, until success Idempotence — repeated convergence actions have same result Flux jx-git-operator
4. Kubernetes Secret Management What is a secret? • data to protect • username/password • SSH keys, API keys • TLS certificates @KaraMarck
5. Kubernetes Kubernetes Secrets Built-in Kubernetes object • Name of secret • Type of secret (optional) • Map of field names to sensitive data (base64 encoded) @KaraMarck
6. Kubernetes Kubernetes Secrets @KaraMarck
7. Kubernetes @KaraMarck Git was not built to hold secrets Collaborative tool, makes it easy to view and review each other’s code No granular, file-level access controls Distributed git repos
8. Kubernetes @KaraMarck Rules for GitOps, Kubernetes, and Secret Management: Don’t store Kubernetes Secrets in Git
9. Kubernetes @KaraMarck Comedic Interlude: Determining Guidelines is a Difficult Task https://www.youtube.com/watch?v=JKhOFKCOH5M Political Skit from The Great British Bake Off, starring Matt Lucas.
10. Kubernetes @KaraMarck Rules: Don’t store Kubernetes Secrets in Git Don’t Bake in a Tent
11. Kubernetes @KaraMarck If you do really want to store secrets in Git, use Sealed Secrets Secret is encrypted into a Sealed Secret. One challenge is the encryption key. bitnami-labs/sealed- secrets
12. Kubernetes Bake Secrets into Container Image @KaraMarck Secret data directly copied into container image as part of the build process. Pros: git no longer a security concern. Image can be run anywhere. Cons: Container image sensitive. Secret not flexible; can’t be rotated.
13. Kubernetes @KaraMarck Rules: #1 Don’t store raw Kubernetes Secrets in Git Don’t Bake in a Tent Don’t Bake in a Tent #2 Don’t “bake” sensitive data right into container images
14. Kubernetes @KaraMarck Rules: #1 Don’t store raw Kubernetes Secrets in Git Don’t Bake in a Tent Don’t Bake in a Tent #2 Don’t “bake” sensitive data right into container images
15. Kubernetes @KaraMarck External Secret Management Systems • HashiCorp Vault • Google Cloud Secret Manager • AWS Secrets Manager • Microsoft Azure Key Vault • Alibaba Cloud KMS Secret Manager
16. Kubernetes External Secret Management Systems @KaraMarck
17. Kubernetes @KaraMarck Open Sourced by GoDaddy. Enables secure retrieval of secrets stored in external secret management systems and securely adds secrets to your cluster. Extends the Kubernetes API by adding an External Secrets object, which declares how to fetch external secret data, and a controller, which converts all External Secrets to Secrets. Kubernetes External Secrets
18. Kubernetes Kubernetes External Secrets @KaraMarck Source: https://github.com/godaddy/kubernetes-external- secrets#system-architecture
19. Kubernetes Kubernetes External Secrets @KaraMarck Source: https://github.com/godaddy/kubernetes-external-secrets 1.Controller fetches ExternalSecrets using the Kubernetes API 2. Controller uses ExternalSecrets to fetch secret data from external providers 3. Controller upsert Secrets 4. Pods can access Secrets normally
20. Kubernetes @KaraMarck Kubernetes External Secrets A single way to manage secrets which supports major secret management systems. jx-secret, command line tool for working with Kubernetes External Secrets.
21. Kubernetes @KaraMarck 3 Alpha Docs: Intro Blog Post Overview and architecture Getting Started Blog Post on Octant Community: Office Hours Slack Discourse Hacktoberfest
22. Kubernetes @KaraMarck
23. Kubernetes @KaraMarck
24. Kubernetes @KaraMarck External Secret Management Systems • HashiCorp Vault • Google Cloud Secret Manager • AWS Secrets Manager • Microsoft Azure Key Vault • Alibaba Cloud KMS Secret Manager
25. Kubernetes @KaraMarck Kubernetes Secrets
26. Kubernetes @KaraMarck
27. Kubernetes @KaraMarck Kubernetes External Secrets
28. Kubernetes @KaraMarck Rules: #1 Don’t store raw Kubernetes Secrets in Git Don’t Bake in a Tent Don’t Bake in a Tent
29. Kubernetes @KaraMarck Kara de la Marck Open Source Community Manager, Cloudbees Director, codebar GitHub: MarckK LinkedIn: Kara de la Marck Twitter: KaraMarck Thank you!