Upgrade to Pro — share decks privately, control downloads, hide ads and more …

To Helm Or Not To Helm

To Helm Or Not To Helm

Alex Khaerov

January 17, 2019
Tweet

More Decks by Alex Khaerov

Other Decks in Technology

Transcript

  1. @hayorov I’m Alex Khaerov $ Development Lead at Chainstack About

    Me ∞ blockchains We ❤ Helm and K8s ∞ cloud providers
  2. @hayorov “Batteries included” Complexity mngt Packaging Application lifecycle Application metadata

    Repositories Package dependancies Helm addresses several needs Parametrization Templating
  3. @hayorov “Batteries included” Complexity mngt Packaging Application lifecycle Application metadata

    Repositories Package dependancies Helm addresses several needs Parametrization Templating Deploy/config revisions Rollbacks Hooks Application probes
  4. @hayorov “Batteries included” Complexity mngt Packaging Application lifecycle Application metadata

    Repositories Package dependancies Helm addresses several needs Parametrization Templating Deploy/config revisions Rollbacks Hooks Application probes Command plugins
  5. @hayorov My Cluster Cloud Provider Open Service Broker API integration

    What are Service Brokers? Managed MySQL manually
  6. @hayorov My Cluster Cloud Provider Open Service Broker API integration

    What are Service Brokers? Managed MySQL Service Broker
  7. @hayorov My Cluster Cloud Provider Open Service Broker API integration

    What are Service Brokers? Managed MySQL Service Broker OSBA
  8. @hayorov My Cluster Cloud Provider Open Service Broker API integration

    What are Service Brokers? Managed MySQL Tier: Basic Service Broker OSBA
  9. @hayorov My Cluster Cloud Provider Open Service Broker API integration

    What are Service Brokers? Managed MySQL Tier: Basic Service Broker OSBA DB credentials
  10. @hayorov My Cluster Cloud Provider Open Service Broker API integration

    What are Service Brokers? Managed MySQL Tier: Basic Service Broker OSBA DB credentials
  11. @hayorov My Cluster Cloud Provider Open Service Broker API integration

    What are Service Brokers? Managed MySQL Tier: Basic Service Broker OSBA DB credentials Helm glues Service Broker and charts that consume OSB resources
  12. @hayorov Repositories on GCS ChartMuseum de-facto is a standard helm-gcs

    is a plugin that allows to manage private repos on GCS Authentification using: • application default credentials • service account 
 (via the global flag)

  13. @hayorov Helm architecture Helm client Chart Repo K8s cluster kube-system

    ns microservice ns Kube-api
 server Tiller-deploy svc Release configmap Microservice pod release ns(s) Release deployment Tiller-deploy pod Service Account Release svc Release configmap fetch Kubeconfig HTTPS gRPC gRPC RestAPI
  14. @hayorov Helm architecture Helm client Chart Repo K8s cluster kube-system

    ns microservice ns Kube-api
 server Tiller-deploy svc Release configmap Microservice pod release ns(s) Release deployment Tiller-deploy pod Service Account Release svc Release configmap fetch Kubeconfig HTTPS gRPC gRPC RestAPI
  15. @hayorov • Turn RBAC on • Tiller uses the default

    service account 
 in a namespace Kubernetes RBAC • helm init does not create the associated
 ServiceAccount/Roles/RoleBindings ☝ Tiller-deploy svc Tiller-deploy pod Service Account Release configmap RBAC Default Service Account RestAPI
  16. @hayorov • Turn RBAC on • Tiller uses the default

    service account 
 in a namespace Kubernetes RBAC • helm init does not create the associated
 ServiceAccount/Roles/RoleBindings ☝ Tiller-deploy svc Tiller-deploy pod Service Account Release configmap RBAC RestAPI
  17. @hayorov Role RoleBinding • Turn RBAC on • Tiller uses

    the default service account 
 in a namespace Kubernetes RBAC • helm init does not create the associated
 ServiceAccount/Roles/RoleBindings ☝ Tiller-deploy svc Tiller-deploy pod Service Account Release configmap RBAC RestAPI
  18. @hayorov Kubernetes RBAC Helm client Kubeconfig gRPC K8s cluster kube-system

    ns Kube-api
 server Tiller-deploy svc Release configmap Tiller-deploy pod Service Account Helm client Kubeconfig Cluster Admin X Team Kube-api
 server Tiller-deploy svc Release configmap Tiller-deploy pod Service Account Release deployment Release svc Release configmap RestAPI RBAC gRPC RestAPI RBAC • No multi-tenancy support • Solution - Multi-Tiller installations • Per developer, per team, per environment X Team ns ?
  19. @hayorov Kubernetes RBAC Helm client Kubeconfig gRPC K8s cluster kube-system

    ns Kube-api
 server Tiller-deploy svc Release configmap Tiller-deploy pod Service Account Helm client Kubeconfig Cluster Admin X Team Kube-api
 server Tiller-deploy svc Release configmap Tiller-deploy pod Service Account Release deployment Release svc Release configmap RestAPI RBAC gRPC RestAPI RBAC • No multi-tenancy support • Solution - Multi-Tiller installations • Per developer, per team, per environment X Team ns
  20. @hayorov • By default Tiller stores releases
 information in ConfigMaps

    Release information Tiller-deploy svc Tiller-deploy pod Service Account Role RoleBinding Release
 secrets RBAC Release
 secrets Release
 configmap • Secrets are still limited to 1MB in size
  21. @hayorov • By default Tiller stores releases
 information in ConfigMaps

    Release information Tiller-deploy svc Tiller-deploy pod Service Account Role RoleBinding Release
 secrets RBAC Release
 secrets • Secrets are still limited to 1MB in size
  22. @hayorov • Use HTTPS always • Publish signed charts •

    Helm client supports repo certs • Chartmuseum supports basic auth • helm-gcs plugin with GCP auth Chart Repos Helm client Signed Chart Repo fetch Chart Repo
  23. @hayorov • Use HTTPS always • Publish signed charts •

    Helm client supports repo certs • Chartmuseum supports basic auth • helm-gcs plugin with GCP auth Chart Repos Helm client Signed Chart Repo fetch HTTPS mTLS or basic auth
  24. @hayorov gRPC Helm client Kubeconfig gRPC • Tiller supports TLS

    on gRPC Tiller-deploy svc Tiller-deploy pod Service Account Role RoleBinding Release configmap RBAC Tiller-deploy svc
  25. @hayorov gRPC Helm client Kubeconfig gRPC • Tiller supports TLS

    on gRPC Tiller-deploy svc Tiller-deploy pod Service Account Role RoleBinding Release configmap RBAC TLS
  26. @hayorov Secured Helm Helm client Chart Repo K8s cluster kube-system

    ns microservice ns Kube-api
 server Tiller-deploy svc Release secret Microservice pod release ns(s) Release deployment Release svc Release configmap fetch Kubeconfig HTTPS gRPC gRPC Service Account HTTPS TLS or basic auth mTLS RBAC mTLS Tiller-deploy pod
  27. @hayorov Helm 3 is the next big thing • Single-server

    architecture (no more Tiller) • State storage based of Release object (CRD) • Embedded Lua engine for scripting • Schematized values files (values.schema.yaml) • Single event-driven model Current status: a design proposal document Future of Helm and alternatives 3
  28. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives
  29. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives more control more k8s native
  30. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives more control more k8s native v2
  31. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives more control more k8s native v3 v2
  32. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives more control more k8s native v3 v2 K Ksonnet, Metaparticle
  33. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives more control more k8s native v3 Y Your Cfg
 management v2 K Ksonnet, Metaparticle
  34. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives more control more k8s native v3 Operator Framework Y Your Cfg
 management v2 K Ksonnet, Metaparticle
  35. @hayorov Any alternatives? • Ksonnet, Metaparticle • Your configuration mngt

    system 
 (ansible, terraform, chef …) • CoreOS Operator Framework
 (1k+ ⭐) Helm add-ons: • Draft, Scaffold Future of Helm and alternatives more control more k8s native v3 Operator Framework Y Your Cfg
 management v2 A Draft, Skaffold K Ksonnet, Metaparticle