Kustomizing your Kubernetes Deployments

Transcript

1. Kustomizing your Kubernetes Deployments Cloud Native Computing Switzerland Meetup, 22

   November 2018 David Schweikert @dschweikert AdNovum Informatik AG

2. “Configuration management” in Kubernetes?

3. We don’t need it! $ kubectl apply -f <directory>/

4. But… how to parametrize?

5. None

6. Parametrization is important! § Minimize differences between environments § Test

   what you deploy in prod! § Avoid code duplication

7. Me in 2017: !

8. Me in 2017: ! § OpenShift Templates § Helm §

   Self-made? Jinja2? § Even thinking of using Ansible…

9. Me in 2018: ! § Kustomize

10. What are the options?

11. Ansible Forge Helm K8comp KPM KY Kapitan Kdeploy Kedge Kenv

    Kexpand Kit-Deploymentizer Kompose Konfd Kontemplate Ksonnet Ktmpl Kubecfg Kubegen Kubernetes-deploy Kubetpl Kustomize Mortar OpenShift templates Psykube Spread Terraform …

12. Why Kustomize?

13. Reason #1: Embrace Kubernetes API Object Descriptions (YAML files)

14. kind: Service apiVersion: v1 metadata: name: my-service spec: selector: app:

    MyApp ports: - protocol: TCP port: 80 targetPort: 9376 my-service.yaml:

15. local params = std.extVar("__ksonnet/params").components.demo; local k = import "k.libsonnet"; local

    service = k.core.v1.service; local servicePort = k.core.v1.service.mixin.spec.portsType; local targetPort = params.containerPort; local labels = {app: params.name}; local appService = service .new( params.name, labels, servicePort.new(params.servicePort, targetPort)) .withType(params.type); k.core.v1.list.new([appService]) Ksonnet:

16. keep your YAML files the way they are

17. Reason #2: Keep using kubectl

18. $ kustomize build . | kubectl apply -f - Typical

    workflow:

19. Helm: § forget about kubectl, now you need to always

    use “helm install”, “helm ls”, “helm status”

20. Reason #3: Declarative and Templates-free

21. YAML can be painful sometimes

22. None

23. Templated YAML is much worse…

24. apiVersion: v1 kind: Service metadata: name: {{ template "grafana.fullname" .

    }} labels: app: {{ template "grafana.name" . }} chart: {{ template "grafana.chart" . }} release: {{ .Release.Name }} heritage: {{ .Release.Service }} {{- if .Values.service.labels }} {{ toYaml .Values.service.labels | indent 4 }} {{- end }} {{- with .Values.service.annotations }} annotations: {{ toYaml . | indent 4 }} {{- end }} spec: {{- if (or (eq .Values.service.type "ClusterIP") (empty .Values.service.type)) }} type: ClusterIP {{- if .Values.service.clusterIP }} clusterIP: {{ .Values.service.clusterIP }} {{end}} {{- else if eq .Values.service.type "LoadBalancer" }} …

25. Kustomize’s cure: § No templating at all!

26. Kustomize’s cure: § Overlays § Transformations § Generators § Patches

27. Overlays

28. Overlays myApp | ├── base │ ├── deployment.yaml │ ├──

    kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml resources: - deployment.yaml - service.yaml

29. Overlays myApp | ├── base │ ├── deployment.yaml │ ├──

    kustomization.yaml │ └── service.yaml | ├── development │ ├── ingress.yaml │ └── kustomization.yaml | └── production ├── ingress.yaml └── kustomization.yaml bases: - ../base resources: - ingress.yaml bases: - ../base resources: - ingress.yaml

30. $ kustomize build development apiVersion: v1 kind: Service metadata: …

    --- apiVersion: apps/v1 kind: Deployment metadata: … --- apiVersion: apps/v1 kind: Ingress metadata: …

31. Transformations

32. Transformations myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development │ └── kustomization.yaml bases: - ../base namePrefix: dev- § All resource names are now prefixed with “dev-”

33. Transformations It’s what makes kustomize so powerful: § Because it

    knows Kubernetes semantics § A single line, use-case specific (e.g. namePrefix) causes big changes § All references are preserved

34. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile -

    configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents

35. Generators kustomization.yaml: configMapGenerator: - name: myconfig files: - configs/configfile -

    configs/another_configfile § generates: myconfig-b62k6t7g8f (and fixes all references to it) § b62k6t7g8f is a hash of the contents !!!

36. Patches

37. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml bases: - ../base patches: - deployment.patch.yaml

38. Patches myApp| ├── base │ ├── deployment.yaml │ └── kustomization.yaml

    | ├── development | ├── deployment.patch.yaml │ └── kustomization.yaml apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: replicas: 1

39. Patches Same syntax as “kubectl patch” § Strategic merge patches

    § JSON patches (RFC 6902)

40. Challenges

41. Challenges § Documentation is (currently) not super great

42. Challenges § Things that Kustomize doesn’t know about § OpenShift

    objects § CRDs

43. Challenges § Things that Kustomize doesn’t know about § OpenShift

    objects § CRDs It is now possible to extend Kustomize knowledge about Kubernetes objects see also: https://github.com/adnovum/kustomize-openshift

44. Summary When to use kustomize § It’s the perfect tool

    to parametrize your own application When not to use kustomize § Packaging an application for the general public (use Helm for that)

45. Questions? More about this topic: § Declarative application management in

    Kubernetes August 2017, by Brian Grant § Introducing kustomize; Template-free Configuration Customization for Kubernetes May 2018, by Jeff Regan and Phil Wittrock Contacting me: § david@schweikert.ch, @dschweikert