Introduction to kustomize

A talk at Kubernetes Meetup Tokyo #12
https://k8sjp.connpass.com/event/90631/

Transcript

1. Introduction to
   kustomize
   Kubernetes Meetup Tokyo #12, Jul 11, 2018
   

   View Slide

2. @spesnova
   SRE at Mercari, Inc. / Kubernetes Tokyo Community Organizer
   

   View Slide

3. Agenda
   

   View Slide

4. 1. Basics
   2. Features
   3. Keys
   

   View Slide

5. Tested with kustomize v1.0.3
   

   View Slide

6. Basics
   

   View Slide

7. What is kustomize?
   

   View Slide

8. kustomize is a command line tool
   

   View Slide

9. kustomize is a CLI for managing
   k8s style object with declarative way
   

   View Slide

10. Let’s learn a basic usage!
    

    View Slide

11. Basics / Hello World
    

    View Slide

12. • 3 environments (dev, stg, prod)
    • 1 deployment resource
    • different replicas by environments
    Example Requirements
    

    View Slide

13. hello-world/
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    └── overlays
    ├── production
    │ ├── replica_count.yaml
    │ └── kustomization.yaml
    └── staging
    ├── replica_count.yaml
    └── kustomization.yaml
    File Structure
    

    View Slide

14. hello-world/
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    └── overlays
    ├── production
    │ ├── replica_count.yaml
    │ └── kustomization.yaml
    └── staging
    ├── replica_count.yaml
    └── kustomization.yaml
    File Structure
    

    View Slide

15. hello-world/
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    └── overlays
    ├── production
    │ ├── replica_count.yaml
    │ └── kustomization.yaml
    └── staging
    ├── replica_count.yaml
    └── kustomization.yaml
    File Structure
    

    View Slide

16. hello-world/
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    └── overlays
    ├── production
    │ ├── replica_count.yaml
    │ └── kustomization.yaml
    └── staging
    ├── replica_count.yaml
    └── kustomization.yaml
    File Structure
    

    View Slide

17. hello-world/
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    └── overlays
    ├── production
    │ ├── replica_count.yaml
    │ └── kustomization.yaml
    └── staging
    ├── replica_count.yaml
    └── kustomization.yaml
    Base
    

    View Slide

18. # hello-world/base/deployment.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-world
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello-world
    template: ..
    Base
    

    View Slide

19. # hello-world/base/kustomization.yaml
    resources:
    - deployment.yaml
    Base
    

    View Slide

20. hello-world/
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    └── overlays
    ├── production
    │ ├── replica_count.yaml
    │ └── kustomization.yaml
    └── staging
    ├── replica_count.yaml
    └── kustomization.yaml
    Staging
    

    View Slide

21. # hello-world/staging/replica_count.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-world
    spec:
    replicas: 3
    Staging
    

    View Slide

22. # hello-world/staging/kustomization.yaml
    bases:
    - ../../base
    patches:
    - replica_count.yaml
    Staging
    

    View Slide

23. $ kustomize build -h
    Print current configuration per contents of kustomization.yaml
    Usage:
    kustomize build [path] [flags]
    $ kustomize build
    

    View Slide

24. $ kustomize build hello-world/overlays/staging/
    Print staging configuration
    

    View Slide

25. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-world
    spec:
    replicas: 3
    selector:
    matchLabels:
    app: hello-world
    template: ..
    Print staging configuration
    

    View Slide

26. template-free customization
    

    View Slide

27. overlay customization
    

    View Slide

28. base deployment (replicas 1)
    

    View Slide

29. staging deployment (replicas: 3)
    

    View Slide

30. overlayed staging deployment (replicas 3)
    

    View Slide

31. hello-world/
    ├── base
    │ ├── deployment.yaml
    │ └── kustomization.yaml
    └── overlays
    ├── production
    │ ├── replica_count.yaml
    │ └── kustomization.yaml
    └── staging
    ├── replica_count.yaml
    └── kustomization.yaml
    Production
    

    View Slide

32. # hello-world/production/replica_count.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-world
    spec:
    replicas: 7
    Production
    

    View Slide

33. # hello-world/production/kustomization.yaml
    bases:
    - ../../base
    patches:
    - replica_count.yaml
    Production
    

    View Slide

34. $ kustomize build hello-world/overlays/production/
    Print production configuration
    

    View Slide

35. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-world
    spec:
    replicas: 7
    selector:
    matchLabels:
    app: hello-world
    template: ..
    Print production configuration
    

    View Slide

36. $ kustomize build [PATH] | kubectl apply -f -
    Apply printed configuration
    

    View Slide

37. Basics / Motivation
    

    View Slide

38. Declarative specification
    is the recommended way
    

    View Slide

39. However…
    

    View Slide

40. It’s difficult to use only current kubectl
    to follow declarative way…
    

    View Slide

41. Then…
    

    View Slide

42. • Helm
    • Ksonnet
    • Kapitan
    • Forge
    • Ktmpl
    • etc…
    Another Tools are required
    

    View Slide

43. 1. I have to learn new tools…
    2. I have to learn new DSL… (complicated!)
    3. I have to teach new concepts to teams…
    Drawbacks of those tools
    

    View Slide

44. Features
    

    View Slide

45. Features / Name Prefix
    

    View Slide

46. # overlays/production/kustomization.yaml
    namePrefix: prod-
    bases:
    - ../../base
    patches:
    - replica_count.yaml
    Name Prefix
    

    View Slide

47. $ kustomize build hello-world/overlays/production/
    Name Prefix
    

    View Slide

48. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: prod-hello-world
    spec:
    replicas: 7
    selector:
    matchLabels:
    app: hello-world
    template: ..
    Name Prefix
    

    View Slide

49. Features / Common Labels
    

    View Slide

50. # base/kustomization.yaml
    commonLabels:
    owner: spesnova
    resources:
    - deployment.yaml
    Common Labels
    

    View Slide

51. $ kustomize build hello-world/overlays/production/
    Common Labels
    

    View Slide

52. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-world
    labels:
    owner: spesnova
    spec:
    replicas: 7
    selector:
    matchLabels:
    app: hello-world
    template: ..
    Common Labels
    

    View Slide

53. Features / Common Annotattion
    

    View Slide

54. # base/kustomization.yaml
    commonAnnotations:
    description: This is Hello World App
    resources:
    - deployment.yaml
    Common Annotations
    

    View Slide

55. $ kustomize build hello-world/overlays/production/
    Common Annotations
    

    View Slide

56. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-world
    annotations:
    description: This is Hello World App
    spec:
    replicas: 7
    selector:
    matchLabels:
    app: hello-world
    template: …
    Common Annotations
    

    View Slide

57. Features / ConfigMap Generator
    

    View Slide

58. # base/kustomization.yaml
    resources:
    - deployment.yaml
    configMapGenerator:
    - name: hello-config
    files:
    - hello.config
    ConfigMap Generator
    

    View Slide

59. # hello.config
    name=hello-world
    region=tokyo
    ConfigMap Generator
    

    View Slide

60. $ kustomize build hello-world/overlays/production/
    ConfigMap Generator
    

    View Slide

61. apiVersion: v1
    data:
    hello.config: |
    name=hello-world
    region=tokyo
    kind: ConfigMap
    metadata:
    creationTimestamp: null
    name: hello-config-4g5t58m8t5
    ---
    apiVersion: apps/v1
    kind: Deployment
    …
    ConfigMap Generator
    

    View Slide

62. apiVersion: v1
    data:
    hello.config: |
    name=hello-world
    region=tokyo
    kind: ConfigMap
    metadata:
    creationTimestamp: null
    name: hello-config-4g5t58m8t5
    ---
    apiVersion: apps/v1
    kind: Deployment
    …
    Hash suffix
    

    View Slide

63. # hello.config
    name=hello-world
    region=london
    Hash suffix
    

    View Slide

64. apiVersion: v1
    data:
    hello.config: |
    name=hello-world
    region=tokyo
    kind: ConfigMap
    metadata:
    creationTimestamp: null
    name: hello-config-bdmmkghm2m
    ---
    apiVersion: apps/v1
    kind: Deployment
    …
    Hash suffix
    

    View Slide

65. Features / Secrets Generator (skip)
    

    View Slide

66. Features / Diff
    

    View Slide

67. $ kustomize diff hello-world/overlays/production/
    $ kustomize diff
    

    View Slide

68. @@ -3,7 +3,7 @@
    metadata:
    name: hello-world
    spec:
    - replicas: 1
    + replicas: 7
    selector:
    matchLabels:
    app: hello-world
    $ kustomize diff
    

    View Slide

69. Features / Substitute (skip)
    

    View Slide

70. Workflows / Bespoke config
    

    View Slide

71. Bespoke config
    

    View Slide

72. Workflows / Off-the-shelf config
    

    View Slide

73. Off-the-shelf config
    

    View Slide

74. Keys
    

    View Slide

75. Keys / Overlay vs Template
    

    View Slide

76. 1. Can only override parameterized config
    2. DSL is too complicated for human
    3. Most tools can not read DSL
    Drawbacks of Templating
    

    View Slide

77. 1. I’m using official Redis Helm chart
    2. I want to add annotation
    3. Annotations are not defined in the chart…
    4. …Fork?
    Example
    

    View Slide

78. With kustomize
    You can override any part of config with kustomize
    

    View Slide

79. Keys / Single source of truth
    

    View Slide

80. 1. There is a config file “hello.config”
    2. Copy contents of the file
    3. Paste it into configMap
    4. … I have 2 config sources…
    Before kustomize
    

    View Slide

81. # base/kustomization.yaml
    resources:
    - deployment.yaml
    configMapGenerator:
    - name: hello-config
    files:
    - hello.config
    ConfigMap Generator
    

    View Slide

82. # hello.config
    name=hello-world
    region=tokyo
    ConfigMap Generator
    

    View Slide

83. $ kustomize build hello-world/overlays/production/
    ConfigMap Generator
    

    View Slide

84. apiVersion: v1
    data:
    hello.config: |
    name=hello-world
    region=tokyo
    kind: ConfigMap
    metadata:
    creationTimestamp: null
    name: hello-config-4g5t58m8t5
    ---
    apiVersion: apps/v1
    kind: Deployment
    …
    ConfigMap Generator
    

    View Slide

85. 1. There is a config file “hello.config”
    2. Run “kustomize build”
    3. kustomize generates configMap
    4. The config source is only “hello.config”
    After kustomize
    

    View Slide

86. Keys / Rolling ConfigMap Update
    

    View Slide

87. 1. Update contents of existing configMap
    2. Deployment itself is not changed…
    3. Deployment still reads old configMap…
    Updating existing configMap
    

    View Slide

88. apiVersion: v1
    data:
    hello.config: |
    name=hello-world
    region=tokyo
    kind: ConfigMap
    metadata:
    creationTimestamp: null
    name: hello-config-4g5t58m8t5
    ---
    apiVersion: apps/v1
    kind: Deployment
    …
    Hash suffix
    

    View Slide

89. 1. Update contents of configMap
    2. kustomize prints new configMap
    3. Update configMap name in deployment
    4. Deployment reads new configMap
    Rolling ConfigMap Update
    

    View Slide

90. Keys / Teaching native k8s APIs
    

    View Slide

91. kustomize exposes and teaches native k8s APIs,
    rather than hiding them.
    IUUQTHJUIVCDPNLVCFSOFUFTTJHTLVTUPNJ[FCMPCNBTUFSEPDTHMPTTBSZNE
    

    View Slide

92. Same as kubernetes manifest
    Using Native Kubernetes API
    

    View Slide

93. 1. Lower learning cost
    2. Deeper understanding about Kubernetes
    Using Native Kubernetes API
    

    View Slide

94. Keys / Rollback
    

    View Slide

95. $ git checkout XXXXXX
    $ kustomize build [PATH] | kubectl apply -f -
    Rollback
    

    View Slide

96. kustomize rollback is very good for GitOps.
    However, I also like heroku style rollback such as “helm
    status”, “helm history”, “helm rollback”.
    Helm provides us logical group of k8s resources as
    “application”. kustomize doesn’t.
    Rollback
    

    View Slide

97. Kubernetes Application proposal KEP
    Related issue
    https://github.com/kubernetes/community/pull/1629
    

    View Slide

98. Keys / might be moved to kubectl
    

    View Slide

99. Kustomize was initially developed as its own cli, however once it
    has matured, it should be published as a subcommand of
    kubectl or as a statically linked plugin.
    IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSLFQTTJHDMJLVTUPNJ[FNEJNQMFNFOUBUJPOEFUBJMTOPUFTDPOTUSBJOUTPQUJPOBM
    

    View Slide

100. Keys / See design doc!
     

     View Slide

101. https://github.com/kubernetes/community/blob/master/contributors/
     design-proposals/architecture/declarative-application-
     management.md
     IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSLFQTTJHDMJLVTUPNJ[FNEJNQMFNFOUBUJPOEFUBJMTOPUFTDPOTUSBJOUTPQUJPOBM
     It’s awesome!
     

     View Slide

102. It’s awesome!
     If kustomize looks easy to use for you,
     I think it comes from good design!
     

     View Slide

103. Questions
     

     View Slide

104. Can I delete labels with overlay?
     As far as I know, you can not for now
     

     View Slide

105. End
     

     View Slide