Kustomizing your Kubernetes Deployments

How to parametrize your Kubernetes object definitions using kustomize, a template-free customization tool.

Talk presented at the Cloud Native Computing Meetup November 2018.

Transcript

1. Kustomizing your Kubernetes Deployments Cloud Native Computing Switzerland Meetup, 22 November 2018 David Schweikert @dschweikert AdNovum Informatik AG
2. “Configuration management” in Kubernetes?
3. We don’t need it! $ kubectl apply -f <directory>/
4. But… how to parametrize?
6. Parametrization is important! § Minimize differences between environments § Test what you deploy in prod! § Avoid code duplication
7. Me in 2017: !
8. Me in 2017: ! § OpenShift Templates § Helm § Self-made? Jinja2? § Even thinking of using Ansible…
9. Me in 2018: ! § Kustomize
10. What are the options?
11. Ansible Forge Helm K8comp KPM KY Kapitan Kdeploy Kedge Kenv Kexpand Kit-Deploymentizer Kompose Konfd Kontemplate Ksonnet Ktmpl Kubecfg Kubegen Kubernetes-deploy Kubetpl Kustomize Mortar OpenShift templates Psykube Spread Terraform …
12. Why Kustomize?
13. Reason #1: Embrace Kubernetes API Object Descriptions (YAML files)
14. kind: Service apiVersion: v1 metadata: name: my-service spec: selector: app: MyApp ports: - protocol: TCP port: 80 targetPort: 9376 my-service.yaml:
15. local params = std.extVar("__ksonnet/params").components.demo; local k = import "k.libsonnet"; local service = k.core.v1.service; local servicePort = k.core.v1.service.mixin.spec.portsType; local targetPort = params.containerPort; local labels = {app: params.name}; local appService = service .new( params.name, labels, servicePort.new(params.servicePort, targetPort)) .withType(params.type); k.core.v1.list.new([appService]) Ksonnet:
16. keep your YAML files the way they are
17. Reason #2: Keep using kubectl
18. $ kustomize build . | kubectl apply -f - Typical workflow:
19. Helm: § forget about kubectl, now you need to always use “helm install”, “helm ls”, “helm status”
20. Reason #3: Declarative and Templates-free
21. YAML can be painful sometimes
23. Templated YAML is much worse…
24. apiVersion: v1 kind: Service metadata: name: {{ template "grafana.fullname" . }} labels: app: {{ template "grafana.name" . }} chart: {{ template "grafana.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} {{- if .Values.service.labels }} {{ toYaml .Values.service.labels | indent 4 }} {{- end }} {{- with .Values.service.annotations }} annotations: {{ toYaml . | indent 4 }} {{- end }} spec: {{- if (or (eq .Values.service.type "ClusterIP") (empty .Values.service.type)) }} type: ClusterIP {{- if .Values.service.clusterIP }} clusterIP: {{ .Values.service.clusterIP }} {{end}} {{- else if eq .Values.service.type "LoadBalancer" }} …
25. Kustomize’s cure: § No templating at all!
26. Kustomize’s cure: § Overlays § Transformations § Generators § Patches
27. Overlays
28. Overlays myApp | ├── base │ ├── deployment.yaml │ ├── kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml resources: - deployment.yaml - service.yaml
29. Overlays myApp | ├── base │ ├── deployment.yaml │ ├── kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml bases: - ../base resources: - ingress.yaml bases: - ../base resources: - ingress.yaml
30. $ kustomize build development apiVersion: v1 kind: Service metadata: … --- apiVersion: apps/v1 kind: Deployment metadata: … --- apiVersion: apps/v1 kind: Ingress metadata: …
31. Transformations
32. Transformations myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml | ├── development │ └── kustomization.yaml bases: - ../base namePrefix: dev- § All resource names are now prefixed with “dev-”
33. Transformations It’s what makes kustomize so powerful: § Because it knows Kubernetes semantics § A single line, use-case specific (e.g. namePrefix) causes big changes § All references are preserved
34. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile - configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents
35. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile - configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents !!!
36. Patches
37. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml bases: - ../base patches: - deployment.patch.yaml
38. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: replicas: 1
39. Patches Same syntax as “kubectl patch” § Strategic merge patches § JSON patches (RFC 6902)
40. Challenges
41. Challenges § Documentation is (currently) not super great
42. Challenges § Things that Kustomize doesn’t know about § OpenShift objects § CRDs
43. Challenges § Things that Kustomize doesn’t know about § OpenShift objects § CRDs It is now possible to extend Kustomize knowledge about Kubernetes objects see also: https://github.com/adnovum/kustomize-openshift
44. Summary When to use kustomize § It’s the perfect tool to parametrize your own application When not to use kustomize § Packaging an application for the general public (use Helm for that)
45. Questions? More about this topic: § Declarative application management in Kubernetes August 2017, by Brian Grant § Introducing kustomize; Template-free Configuration Customization for Kubernetes May 2018, by Jeff Regan and Phil Wittrock Contacting me: § david@schweikert.ch, @dschweikert