Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kustomizing your Kubernetes Deployments

Kustomizing your Kubernetes Deployments

How to parametrize your Kubernetes object definitions using kustomize, a template-free customization tool.

Talk presented at the Cloud Native Computing Meetup November 2018.

David Schweikert

November 22, 2018
Tweet

More Decks by David Schweikert

Other Decks in Technology

Transcript

  1. Kustomizing your Kubernetes Deployments
    Cloud Native Computing Switzerland Meetup, 22 November 2018
    David Schweikert @dschweikert
    AdNovum Informatik AG

    View Slide

  2. “Configuration management” in Kubernetes?

    View Slide

  3. We don’t need it!
    $ kubectl apply -f /

    View Slide

  4. But… how to parametrize?

    View Slide

  5. View Slide

  6. Parametrization is important!
    § Minimize differences between environments
    § Test what you deploy in prod!
    § Avoid code duplication

    View Slide

  7. Me in 2017:
    !

    View Slide

  8. Me in 2017:
    !
    § OpenShift Templates
    § Helm
    § Self-made? Jinja2?
    § Even thinking of using Ansible…

    View Slide

  9. Me in 2018:
    !
    § Kustomize

    View Slide

  10. What are the options?

    View Slide

  11. Ansible
    Forge
    Helm
    K8comp
    KPM
    KY
    Kapitan
    Kdeploy
    Kedge
    Kenv
    Kexpand
    Kit-Deploymentizer
    Kompose
    Konfd
    Kontemplate
    Ksonnet
    Ktmpl
    Kubecfg
    Kubegen
    Kubernetes-deploy
    Kubetpl
    Kustomize
    Mortar
    OpenShift
    templates
    Psykube
    Spread
    Terraform

    View Slide

  12. Why Kustomize?

    View Slide

  13. Reason #1: Embrace Kubernetes API Object Descriptions (YAML files)

    View Slide

  14. kind: Service
    apiVersion: v1
    metadata:
    name: my-service
    spec:
    selector:
    app: MyApp
    ports:
    - protocol: TCP
    port: 80
    targetPort: 9376
    my-service.yaml:

    View Slide

  15. local params = std.extVar("__ksonnet/params").components.demo;
    local k = import "k.libsonnet";
    local service = k.core.v1.service;
    local servicePort = k.core.v1.service.mixin.spec.portsType;
    local targetPort = params.containerPort;
    local labels = {app: params.name};
    local appService = service
    .new(
    params.name,
    labels,
    servicePort.new(params.servicePort, targetPort))
    .withType(params.type);
    k.core.v1.list.new([appService])
    Ksonnet:

    View Slide

  16. keep your YAML files the way they are

    View Slide

  17. Reason #2: Keep using kubectl

    View Slide

  18. $ kustomize build . | kubectl apply -f -
    Typical workflow:

    View Slide

  19. Helm:
    § forget about kubectl, now you need to always use “helm install”, “helm
    ls”, “helm status”

    View Slide

  20. Reason #3: Declarative and Templates-free

    View Slide

  21. YAML can be painful sometimes

    View Slide

  22. View Slide

  23. Templated YAML is much worse…

    View Slide

  24. apiVersion: v1
    kind: Service
    metadata:
    name: {{ template "grafana.fullname" . }}
    labels:
    app: {{ template "grafana.name" . }}
    chart: {{ template "grafana.chart" . }}
    release: {{ .Release.Name }}
    heritage: {{ .Release.Service }}
    {{- if .Values.service.labels }}
    {{ toYaml .Values.service.labels | indent 4 }}
    {{- end }}
    {{- with .Values.service.annotations }}
    annotations:
    {{ toYaml . | indent 4 }}
    {{- end }}
    spec:
    {{- if (or (eq .Values.service.type "ClusterIP") (empty .Values.service.type)) }}
    type: ClusterIP
    {{- if .Values.service.clusterIP }}
    clusterIP: {{ .Values.service.clusterIP }}
    {{end}}
    {{- else if eq .Values.service.type "LoadBalancer" }}

    View Slide

  25. Kustomize’s cure:
    § No templating at all!

    View Slide

  26. Kustomize’s cure:
    § Overlays
    § Transformations
    § Generators
    § Patches

    View Slide

  27. Overlays

    View Slide

  28. Overlays
    myApp
    |
    ├── base
    │ ├── deployment.yaml
    │ ├── kustomization.yaml
    │ └── service.yaml
    |
    ├── development
    │ ├── ingress.yaml
    │ └── kustomization.yaml
    |
    └── production
    ├── ingress.yaml
    └── kustomization.yaml
    resources:
    - deployment.yaml
    - service.yaml

    View Slide

  29. Overlays
    myApp
    |
    ├── base
    │ ├── deployment.yaml
    │ ├── kustomization.yaml
    │ └── service.yaml
    |
    ├── development
    │ ├── ingress.yaml
    │ └── kustomization.yaml
    |
    └── production
    ├── ingress.yaml
    └── kustomization.yaml
    bases:
    - ../base
    resources:
    - ingress.yaml
    bases:
    - ../base
    resources:
    - ingress.yaml

    View Slide

  30. $ kustomize build development
    apiVersion: v1
    kind: Service
    metadata:

    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:

    ---
    apiVersion: apps/v1
    kind: Ingress
    metadata:

    View Slide

  31. Transformations

    View Slide

  32. Transformations
    myApp|
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    |
    ├── development
    │ └── kustomization.yaml
    bases:
    - ../base
    namePrefix: dev-
    § All resource names are now prefixed with “dev-”

    View Slide

  33. Transformations
    It’s what makes kustomize so powerful:
    § Because it knows Kubernetes semantics
    § A single line, use-case specific (e.g. namePrefix) causes big changes
    § All references are preserved

    View Slide

  34. Generators
    kustomization.yaml:
    configMapGenerator:
    - name: myconfig
    files:
    - configs/configfile
    - configs/another_configfile
    § generates: myconfig-b62k6t7g8f (and fixes all references to it)
    § b62k6t7g8f is a hash of the contents

    View Slide

  35. Generators
    kustomization.yaml:
    configMapGenerator:
    - name: myconfig
    files:
    - configs/configfile
    - configs/another_configfile
    § generates: myconfig-b62k6t7g8f (and fixes all references to it)
    § b62k6t7g8f is a hash of the contents !!!

    View Slide

  36. Patches

    View Slide

  37. Patches
    myApp|
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    |
    ├── development
    | ├── deployment.patch.yaml
    │ └── kustomization.yaml bases:
    - ../base
    patches:
    - deployment.patch.yaml

    View Slide

  38. Patches
    myApp|
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    |
    ├── development
    | ├── deployment.patch.yaml
    │ └── kustomization.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: myapp
    spec:
    replicas: 1

    View Slide

  39. Patches
    Same syntax as “kubectl patch”
    § Strategic merge patches
    § JSON patches (RFC 6902)

    View Slide

  40. Challenges

    View Slide

  41. Challenges
    § Documentation is (currently) not super great

    View Slide

  42. Challenges
    § Things that Kustomize doesn’t know about
    § OpenShift objects
    § CRDs

    View Slide

  43. Challenges
    § Things that Kustomize doesn’t know about
    § OpenShift objects
    § CRDs
    It is now possible to extend Kustomize knowledge about Kubernetes
    objects
    see also: https://github.com/adnovum/kustomize-openshift

    View Slide

  44. Summary
    When to use kustomize
    § It’s the perfect tool to parametrize your own application
    When not to use kustomize
    § Packaging an application for the general public (use Helm for that)

    View Slide

  45. Questions?
    More about this topic:
    § Declarative application management in Kubernetes
    August 2017, by Brian Grant
    § Introducing kustomize; Template-free Configuration Customization for
    Kubernetes
    May 2018, by Jeff Regan and Phil Wittrock
    Contacting me:
    § [email protected], @dschweikert

    View Slide