Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kustomizing your Kubernetes Deployments

Kustomizing your Kubernetes Deployments

How to parametrize your Kubernetes object definitions using kustomize, a template-free customization tool.

Talk presented at the Cloud Native Computing Meetup November 2018.

David Schweikert

November 22, 2018
Tweet

More Decks by David Schweikert

Other Decks in Technology

Transcript

  1. Kustomizing your Kubernetes Deployments Cloud Native Computing Switzerland Meetup, 22

    November 2018 David Schweikert @dschweikert AdNovum Informatik AG
  2. “Configuration management” in Kubernetes?

  3. We don’t need it! $ kubectl apply -f <directory>/

  4. But… how to parametrize?

  5. None
  6. Parametrization is important! § Minimize differences between environments § Test

    what you deploy in prod! § Avoid code duplication
  7. Me in 2017: !

  8. Me in 2017: ! § OpenShift Templates § Helm §

    Self-made? Jinja2? § Even thinking of using Ansible…
  9. Me in 2018: ! § Kustomize

  10. What are the options?

  11. Ansible Forge Helm K8comp KPM KY Kapitan Kdeploy Kedge Kenv

    Kexpand Kit-Deploymentizer Kompose Konfd Kontemplate Ksonnet Ktmpl Kubecfg Kubegen Kubernetes-deploy Kubetpl Kustomize Mortar OpenShift templates Psykube Spread Terraform …
  12. Why Kustomize?

  13. Reason #1: Embrace Kubernetes API Object Descriptions (YAML files)

  14. kind: Service apiVersion: v1 metadata: name: my-service spec: selector: app:

    MyApp ports: - protocol: TCP port: 80 targetPort: 9376 my-service.yaml:
  15. local params = std.extVar("__ksonnet/params").components.demo; local k = import "k.libsonnet"; local

    service = k.core.v1.service; local servicePort = k.core.v1.service.mixin.spec.portsType; local targetPort = params.containerPort; local labels = {app: params.name}; local appService = service .new( params.name, labels, servicePort.new(params.servicePort, targetPort)) .withType(params.type); k.core.v1.list.new([appService]) Ksonnet:
  16. keep your YAML files the way they are

  17. Reason #2: Keep using kubectl

  18. $ kustomize build . | kubectl apply -f - Typical

    workflow:
  19. Helm: § forget about kubectl, now you need to always

    use “helm install”, “helm ls”, “helm status”
  20. Reason #3: Declarative and Templates-free

  21. YAML can be painful sometimes

  22. None
  23. Templated YAML is much worse…

  24. apiVersion: v1 kind: Service metadata: name: {{ template "grafana.fullname" .

    }} labels: app: {{ template "grafana.name" . }} chart: {{ template "grafana.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} {{- if .Values.service.labels }} {{ toYaml .Values.service.labels | indent 4 }} {{- end }} {{- with .Values.service.annotations }} annotations: {{ toYaml . | indent 4 }} {{- end }} spec: {{- if (or (eq .Values.service.type "ClusterIP") (empty .Values.service.type)) }} type: ClusterIP {{- if .Values.service.clusterIP }} clusterIP: {{ .Values.service.clusterIP }} {{end}} {{- else if eq .Values.service.type "LoadBalancer" }} …
  25. Kustomize’s cure: § No templating at all!

  26. Kustomize’s cure: § Overlays § Transformations § Generators § Patches

  27. Overlays

  28. Overlays myApp | ├── base │ ├── deployment.yaml │ ├──

    kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml resources: - deployment.yaml - service.yaml
  29. Overlays myApp | ├── base │ ├── deployment.yaml │ ├──

    kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml bases: - ../base resources: - ingress.yaml bases: - ../base resources: - ingress.yaml
  30. $ kustomize build development apiVersion: v1 kind: Service metadata: …

    --- apiVersion: apps/v1 kind: Deployment metadata: … --- apiVersion: apps/v1 kind: Ingress metadata: …
  31. Transformations

  32. Transformations myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development │ └── kustomization.yaml bases: - ../base namePrefix: dev- § All resource names are now prefixed with “dev-”
  33. Transformations It’s what makes kustomize so powerful: § Because it

    knows Kubernetes semantics § A single line, use-case specific (e.g. namePrefix) causes big changes § All references are preserved
  34. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile -

    configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents
  35. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile -

    configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents !!!
  36. Patches

  37. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml bases: - ../base patches: - deployment.patch.yaml
  38. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: replicas: 1
  39. Patches Same syntax as “kubectl patch” § Strategic merge patches

    § JSON patches (RFC 6902)
  40. Challenges

  41. Challenges § Documentation is (currently) not super great

  42. Challenges § Things that Kustomize doesn’t know about § OpenShift

    objects § CRDs
  43. Challenges § Things that Kustomize doesn’t know about § OpenShift

    objects § CRDs It is now possible to extend Kustomize knowledge about Kubernetes objects see also: https://github.com/adnovum/kustomize-openshift
  44. Summary When to use kustomize § It’s the perfect tool

    to parametrize your own application When not to use kustomize § Packaging an application for the general public (use Helm for that)
  45. Questions? More about this topic: § Declarative application management in

    Kubernetes August 2017, by Brian Grant § Introducing kustomize; Template-free Configuration Customization for Kubernetes May 2018, by Jeff Regan and Phil Wittrock Contacting me: § david@schweikert.ch, @dschweikert