Helm Summit PDX 2018 - Securing Helm

Transcript

2. Lachlan Evenson Principal Program Manager – Azure Containers @LachlanEvenson Securing Helm
3. “The only truly secure system is one that is powered off, cast in a block of concrete and sealed in a lead-lined room with armed guards.” – Gene Spafford
4. Helm security in a nutshell • What does Kubernetes security entail? • RBAC • PodSecurityPolicy • NetworkPolicy • Secrets (storage and access) • Certificate Management • User Management • Kubeconfig Management • Role/RoleBinding Management • Source to container trust • Container signing • Secured container runtime • Seccomp • Secure node bootstrapping • Admission controllers • Auditing Helm is only as secure as the cluster as it’s installed on (mostly)
5. Helm security in a nutshell cont.. So when it comes to Helm security let’s not Source: https://www.phrases.org.uk/meanings/put-the-cart-before-the-horse.html
6. Helm security in a nutshell cont.. Helm Architecture Review • The Helm Client • Command-line client for end users. • The Tiller Server • An in-cluster server that interacts with the Helm client, and interfaces with the Kubernetes API server.
7. Helm Architecture
8. Helm security in a nutshell cont.. • Tiller is just an app running on Kubernetes • Ideal – Tiller performing actions using the rights of the client, instead of the rights of Tiller • Contingent upon the outcome of the Pod Identity Working Group, which has taken on the task of solving the problem in a general way • Solving Helm security is a community effort • Here’s what we can do to secure Helm
9. Kubernetes RBAC • Turn RBAC on • By default Tiller uses the default service account in a namespace $ helm init --service-account <service-account> • The command does not create the associated ServiceAccount/Roles/RoleBindings
10. Kubernetes RBAC cont.. • Be very careful when doing the following $ kubectl -n kube-system create sa tiller $ kubectl create clusterrolebinding tiller -- clusterrole cluster-admin -- serviceaccount=kube-system:tiller $ helm init --service-account tiller • Carefully create your roles • Roles/ClusterRoles • RoleBindings/ClusterRoleBindings
11. Kubernetes RBAC cont.. • Multi-Tiller installations • Per developer, per team, per functional environment
12. Kubernetes RBAC cont.. • Helm Secure Tiller plugin • https://github.com/michelleN/helm-secure-tiller $ helm secure-tiller --namespace dev-team example-profiles/dev-team/ serviceaccount "dev-team-rbac-profile" created rolebinding "dev-team-tiller-binding" created role "dev-team" created
13. Release Information • By default, Tiller stores release information in configMaps $ helm init --override 'spec.template.spec.containers[0].comma nd'='{/tiller,--storage=secret}'
14. Chart Repos • Use HTTPS • Publish signed charts • Helm client supports repo client certs $ helm repo add --help • Chartmuseum supports basic auth • https://github.com/kubernetes-helm/chartmuseum#basic- auth • Quay app registry plugin supports auth • https://coreos.com/blog/quay-application-registry-for- kubernetes.html
15. gRPC • Tiller supports TLS on gRPC $ helm init \ –tiller-tls \ –tiller-tls-verify \ –tiller-tls-ca-cert=ca.pem \ –tiller-tls-cert=cert.pem \ –tiller-tls-key=key.pem \ –service-account=accountname $ helm ls --tls --tls-ca-cert ca.cert.pem --tls-cert helm.cert.pem --tls-key helm.key.pem
16. Secured Helm Architecture
17. Secure By Default • What would need to happen to make Helm secure “by default”? • Make sure RBAC is enabled • Allow only HTTPS, signed charts with at least basic auth or mTLS • Enforce mTLS on gRPC • Default to secrets for release information • Are these really Helm’s decisions to make? • Is the a more Kubernetes “native” way to tackle this problem? • Could we make Helm more Kubernetes ”native”?
18. Other efforts • Optional check to restrict charts to target namespace • https://github.com/kubernetes/helm/pull/3212 • When enabled, this will reject charts that attempt to install resources outside the target namespace • Helm CRD (Helm 3.0?) • Use RBAC to determine weather a user can perform CRUD operations on a “HelmRelease” custom type • https://engineering.bitnami.com/articles/helm-security.html • https://github.com/wpengine/lostromos
19. Other efforts cont.. • Kube RBAC proxy • https://github.com/brancz/kube-rbac-proxy @fredbrancz • This can ensure that a request is truly authorized, instead of being able to access an application simply because an entity has network access to it. (In the case of Tiller) • https://github.com/brancz/kube-rbac-proxy/issues/1 @mumoshu • Adds gRPC support
20. Kube RBAC proxy Architecture • Experimental
21. Other efforts cont.. • Helm Local • https://github.com/adamreese/helm-local • Runs Tiller locally which uses the local kubeconfig to access the Kubernetes API $ helm plugin install https://github.com/adamreese/helm-local $ helm local start $ helm local COMMAND
22. Other efforts cont.. • Tiller-less Helm (Lostromos) • https://github.com/wpengine/lostromos/pull/59 • Tiller functionality client-side
23. What’s next? • This is a community problem. Come help solve it • Join the Helm development calls • Plugins are a great way to start vetting/solving problem before trying to get a PR into core • Helm 3.0 proposals tomorrow
24. References • Securing Helm • https://docs.helm.sh/using_helm/#securing-your-helm-installation • App Registry for Quay • https://coreos.com/blog/quay-application-registry-for-kubernetes.html • Secure Deployment of Helm Repositories through Client Side Certificates • https://www.twistlock.com/2017/09/05/secure-deployment-helm- repositories-client-side-certificates/ • Exploring The Security Of Helm • https://engineering.bitnami.com/articles/helm-security.html