Pro Yearly is on sale from $80 to $50! »

Kustomizing your Kubernetes Deployments

Kustomizing your Kubernetes Deployments

How to parametrize your Kubernetes object definitions using kustomize, a template-free customization tool.

Talk presented at the Cloud Native Computing Meetup November 2018.

B41604b29277208b65682d098be6e2fc?s=128

David Schweikert

November 22, 2018
Tweet

Transcript

  1. Kustomizing your Kubernetes Deployments Cloud Native Computing Switzerland Meetup, 22

    November 2018 David Schweikert @dschweikert AdNovum Informatik AG
  2. “Configuration management” in Kubernetes?

  3. We don’t need it! $ kubectl apply -f <directory>/

  4. But… how to parametrize?

  5. None
  6. Parametrization is important! § Minimize differences between environments § Test

    what you deploy in prod! § Avoid code duplication
  7. Me in 2017: !

  8. Me in 2017: ! § OpenShift Templates § Helm §

    Self-made? Jinja2? § Even thinking of using Ansible…
  9. Me in 2018: ! § Kustomize

  10. What are the options?

  11. Ansible Forge Helm K8comp KPM KY Kapitan Kdeploy Kedge Kenv

    Kexpand Kit-Deploymentizer Kompose Konfd Kontemplate Ksonnet Ktmpl Kubecfg Kubegen Kubernetes-deploy Kubetpl Kustomize Mortar OpenShift templates Psykube Spread Terraform …
  12. Why Kustomize?

  13. Reason #1: Embrace Kubernetes API Object Descriptions (YAML files)

  14. kind: Service apiVersion: v1 metadata: name: my-service spec: selector: app:

    MyApp ports: - protocol: TCP port: 80 targetPort: 9376 my-service.yaml:
  15. local params = std.extVar("__ksonnet/params").components.demo; local k = import "k.libsonnet"; local

    service = k.core.v1.service; local servicePort = k.core.v1.service.mixin.spec.portsType; local targetPort = params.containerPort; local labels = {app: params.name}; local appService = service .new( params.name, labels, servicePort.new(params.servicePort, targetPort)) .withType(params.type); k.core.v1.list.new([appService]) Ksonnet:
  16. keep your YAML files the way they are

  17. Reason #2: Keep using kubectl

  18. $ kustomize build . | kubectl apply -f - Typical

    workflow:
  19. Helm: § forget about kubectl, now you need to always

    use “helm install”, “helm ls”, “helm status”
  20. Reason #3: Declarative and Templates-free

  21. YAML can be painful sometimes

  22. None
  23. Templated YAML is much worse…

  24. apiVersion: v1 kind: Service metadata: name: {{ template "grafana.fullname" .

    }} labels: app: {{ template "grafana.name" . }} chart: {{ template "grafana.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} {{- if .Values.service.labels }} {{ toYaml .Values.service.labels | indent 4 }} {{- end }} {{- with .Values.service.annotations }} annotations: {{ toYaml . | indent 4 }} {{- end }} spec: {{- if (or (eq .Values.service.type "ClusterIP") (empty .Values.service.type)) }} type: ClusterIP {{- if .Values.service.clusterIP }} clusterIP: {{ .Values.service.clusterIP }} {{end}} {{- else if eq .Values.service.type "LoadBalancer" }} …
  25. Kustomize’s cure: § No templating at all!

  26. Kustomize’s cure: § Overlays § Transformations § Generators § Patches

  27. Overlays

  28. Overlays myApp | ├── base │ ├── deployment.yaml │ ├──

    kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml resources: - deployment.yaml - service.yaml
  29. Overlays myApp | ├── base │ ├── deployment.yaml │ ├──

    kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml bases: - ../base resources: - ingress.yaml bases: - ../base resources: - ingress.yaml
  30. $ kustomize build development apiVersion: v1 kind: Service metadata: …

    --- apiVersion: apps/v1 kind: Deployment metadata: … --- apiVersion: apps/v1 kind: Ingress metadata: …
  31. Transformations

  32. Transformations myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development │ └── kustomization.yaml bases: - ../base namePrefix: dev- § All resource names are now prefixed with “dev-”
  33. Transformations It’s what makes kustomize so powerful: § Because it

    knows Kubernetes semantics § A single line, use-case specific (e.g. namePrefix) causes big changes § All references are preserved
  34. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile -

    configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents
  35. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile -

    configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents !!!
  36. Patches

  37. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml bases: - ../base patches: - deployment.patch.yaml
  38. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: replicas: 1
  39. Patches Same syntax as “kubectl patch” § Strategic merge patches

    § JSON patches (RFC 6902)
  40. Challenges

  41. Challenges § Documentation is (currently) not super great

  42. Challenges § Things that Kustomize doesn’t know about § OpenShift

    objects § CRDs
  43. Challenges § Things that Kustomize doesn’t know about § OpenShift

    objects § CRDs It is now possible to extend Kustomize knowledge about Kubernetes objects see also: https://github.com/adnovum/kustomize-openshift
  44. Summary When to use kustomize § It’s the perfect tool

    to parametrize your own application When not to use kustomize § Packaging an application for the general public (use Helm for that)
  45. Questions? More about this topic: § Declarative application management in

    Kubernetes August 2017, by Brian Grant § Introducing kustomize; Template-free Configuration Customization for Kubernetes May 2018, by Jeff Regan and Phil Wittrock Contacting me: § david@schweikert.ch, @dschweikert