Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 1 HELM Build fashionable container systems with Kubernetes 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion HELM 2 Originally developed by Deis 2015 and 
 donated to Cloud Native Computing Foundation 2018 Second generation available 
 and third generation development is started at July 2018 Package manager for Kubernetes clusters 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion What is Helm? 3 •Helm is a Kubernetes Package Manager •Helm charts are build on top of Kubernetes manifests •Charts are stored in a registry called Helm Museum (hub.helm.sh) •It use a templating preprocessing and managed release of Kubernetes resources •Ability to consider scalability from the get-go •SRE’s and developer can search of charts and scratch it •You can reuse other charts 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 4 • Kubernetes is a container orchestrator. • It’s how to run containers at scale. • It’s a very active open-source platform with lots of contributors, start at 6. June 2014 • Originally developed by Google and 
 donated to Cloud Native Computing Foundation 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 5 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Kubernetes primitives 6 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion K8s Deployment Challenge 7 * Migrate to microservice or serverless architecture * Applications consists from multiple components each component has its own resources and can be scaled individually It’s hard to ... ... manage, edit and update multiple K8s configurations ... deploy multiple K8s configurations as a SINGLE application ... share and reuse K8s configurations and applications ... parametrize and support multiple environments ... manage application releases: rollout, rollback, diff, history ... define deployment lifecycle (control operations to be run in different phases) ... validate release state after deployment 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion What Helm can do for you? 8 •Create new charts, aka package K8s manifests, from scratch •Package charts into chart archive (tgz) file •Interact with chart repositories where charts are stored •Reuse exiting charts •Install and uninstall charts into an existing Kubernetes cluster •Manage the release cycle of charts that have been installed with Helm 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Chart 9 a package of Kubernetes manifests Release a chart instance is loaded into Kubernetes Repository a download area of published charts Template a Kubernetes configuration file mixed with Go/Sprig templates Concepts 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion How Helm 2 works? 10 client api-service tiller service(s) https grpc brew install kubernetes-helm curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash or 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion helm init 11 $ cat >tiller-rbac.yaml <

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Helm flow 12 client api-service tiller service(s) publish fetch install upgrade delete redis create application tomcat postgres 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Helm CLI Commands 13 completion Generate autocompletions script for the specified shell (bash or zsh) create create a new chart with the given name delete given a release name, delete the release from Kubernetes dependency manage a chart's dependencies fetch download a chart from a repository and (optionally) unpack it in local directory get download a named release history fetch release history home displays the location of HELM_HOME init initialize Helm on both client and server inspect inspect a chart install install a chart archive lint examines a chart for possible issues list list releases package package a chart directory into a chart archive plugin add, list, or remove Helm plugins repo add, list, remove, update, and index chart repositories reset uninstalls Tiller from a cluster rollback roll back a release to a previous revision search search for a keyword in charts serve start a local http web server status displays the status of the named release template locally render templates test test a release upgrade upgrade a release verify verify that a chart at the given path has been signed and is valid version print the client/server version information 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Example: Simple config map 14 $ helm create my-chart # review mychart templates $ rm -rf mychart/templates/*.* $ cat <mychart/templates/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: mychart-configmap data: myvalue: "Hello World" EOF $ helm install --dry-run --debug ./mychart $ helm install --name full-coral --namespace test ./mychart $ helm get manifest full-coral $ helm delete full-coral 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Example: Prometheus & Grafana 15 # pre init
 # ————————————————————————————————————————————————————————————————————————— $ helm init --client-only # install prometheus # ————————————————————————————————————————————————————————————————————————— $ helm install stable/prometheus \ --name prometheus \
 --namespace monitoring \ --set rbac.create=true,server.persistentVolume.enabled=false,\ alertmanager.enabled=false,pushgateway.enabled=false # install grafana
 # ————————————————————————————————————————————————————————————————————————— $ helm install ./grafana \ --name grafana \ --namespace monitoring \ --set server.persistentVolume.enabled=false,server.adminPassword=admin 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Example: Traefik 16 $ helm install stable/traefik \
 --name traefik --namespace kube-system \
 --set rbac.enabled=true,serviceType=NodePort $ kubectl run —image=bee42/whoami:2.0.0 \ --expose --port=80 --target-port=80 --name=whoami 
 $ cat >whoami-ingress.yaml <: 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Helm chart directory 17 ├── Chart.yaml ├── ├── .md ├── charts │ ├── .tgz │ ├── │ │ ├── Chart.yaml │ │ ├── templates │ │ │ ├── _helpers.tpl │ │ │ └── .yaml │ │ └── values.yaml ├── requirements.lock ├── requirements.yaml
 ├── templates │ ├── _helpers.tpl │ ├── NOTES.txt │ └── .yaml └── values.yaml 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Chart.yaml 18 apiVersion: v1 description: A Helm chart for Docker Voting App porting to k8s cluster name: voting-app version: 0.3.2 keywords: - demo - kubernetes - helm - voting-app home: https://bee42.com/ icon: https://bee42.com/images/logo.svg sources: - https://gitlab.bee42.com/kubernetes/examples/voting-app maintainers: - name: Peter Rossbach email: [email protected] 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion values.yaml 19 # default ingress port servicePort: 80 # configure subchart redis redis: master: persistence: enabled: false usePassword: false …
 imageCredentials: registry: r.gitlab.bee42.com username: gitlab+deploy-token-1 password: xxx-xxx # ingress ingress: enabled: true nginx: enabled: true traefik: enabled: false 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion requirements.yaml 20 dependencies: - name: postgresql version: 3.7.0 repository: https://kubernetes-charts.storage.googleapis.com/ alias: postgresql-1 - name: redis version: 5.1.3 repository: https://kubernetes-charts.storage.googleapis.com/ condition: postgresql-1.enabled,global.result.enabled tags: - back-end - subchart2 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion _helper.tpl 21 {{/* vim: set filetype=mustache: */}} {{/* Expand the name of the chart. */}} {{- define "name" -}} {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} {{- end -}} {{/* Create a default fully qualified app name. We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). */}} {{- define "fullname" -}} {{- $name := default .Chart.Name .Values.nameOverride -}} {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} {{- end -}} {{/* Create a registry image secret to pull voting app images */}} {{- define "imagePullSecret" }} {{- printf "{\"auths\": {\"%s\": {\"auth\": \"%s\"}}}" .Values.imageCredentials.registry 
 printf "%s:%s" .Values.imageCredentials.username .Values.imageCredentials.password | b64enc) | b64enc }} {{- end }} 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Flow Control 22 If/Else {{ if PIPELINE }} # Do something {{ else if OTHER PIPELINE }} # Do something else {{ else }} # Default case {{ end }} data: myvalue: "Hello World" drink: {{ .Values.favorite.drink | default "tea" | quote }} food: {{ .Values.favorite.food | upper | quote }} {{- if eq .Values.favorite.drink "lemonade" }} mug: true {{- end }} # notice the "-" in the left, if will help eliminate newline before variable 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Flow Control 23 With data: myvalue: "Hello World" {{- with .Values.favorite }} drink: {{ .drink | default "tea" | quote }} food: {{ .food | upper | quote }} {{- end }} # instead of writing ".Values.favorite.drink" 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Flow Control 24 Range # predefined variable pizzaToppings: - mushrooms - cheese - peppers - onions toppings: |- {{- range $i, $val := .Values.pizzaTopping }} - {{ . | title | quote }} # upper first character, then quote {{- end }} sizes: |- {{- range tuple "small" "medium" "large" }} - {{ . }} {{- end }} # make a quick list 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Flow Control 25 Variables data: myvalue: "Hello World" {{- $relname := .Release.Name -}} {{- with .Values.favorite }} drink: {{ .drink | default "tea" | quote }} food: {{ .food | upper | quote }} release: {{ $relname }} {{- end }} # use variable in range toppings: |- {{- range $index, $topping := .Values.pizzaToppings }} {{ $index }}: {{ $topping }} {{- end }} {{- range $key,$value := .Values.favorite }} {{ $key }}: {{ $value }} {{- end }} # instead of specify the key, we can actually loop through the values.yaml file and print values values.yaml
 pizzaToppings: - mushrooms - cheese - peppers - onions 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Flow Control 26 Nested Templates # _helpers.tpl {{/* Generate basic labels */}} {{- define "my_labels" }} labels: generator: helm date: {{ now | htmlDate }} version: {{ .Chart.Version }} name: {{ .Chart.Name }} {{- end }} # configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: {{ .Release.Name }}-configmap {{- template "my_labels" . }} # Notice the final dot, it will pass the global scope inside template file. Without it version & name will not be generated. {{- include "my_labels" . | indent 2 }} # similar to "template" directive, have the ability to control indentation 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Flow Control 27 Files inside Templates # file located at parent folder # config1.toml: |- # message = config 1 here # config2.toml: |- # message = config 2 here # config3.toml: |- # message = config 3 here data: {{- $file := .Files }} # set variable {{- range tuple "config1.toml" "config2.toml" "config3.toml" }} # create list {{ . }}: |- # config file name {{ $file.Get . }} # get file's content {{- end }} 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Flow Control Glob-patterns & encoding apiVersion: v1 kind: ConfigMap metadata: name: conf data: +{{ (.Files.Glob "foo/*").AsConfig | indent 2 }} --- apiVersion: v1 kind: Secret metadata: name: very-secret type: Opaque data: +{{ (.Files.Glob "bar/*").AsSecrets | indent 2 }} +token: |- + {{ .Files.Get "config1.toml" | b64enc }} 28 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion ImagePullSecrets 29 {{- define "imagePullSecret" }} {{- printf "{\"auths\": {\"%s\": {\"auth\": \"%s\"}}}" .Values.imageCredentials.registry (printf "%s: %s" .Values.imageCredentials.username .Values.imageCredentials.password | b64enc) | b64enc }} {{- end }} imageCredentials: registry: quay.io username: someone password: sillyness apiVersion: v1 kind: Secret metadata: name: imageSecretKey type: kubernetes.io/dockerconfigjson data: .dockerconfigjson: {{ template "imagePullSecret" . }} values.yaml templates/_helper.tpl templates/imageSecretKey.yaml 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Autodeploy (dependencies) 30 apiVersion: apps/v1 kind: Deployment spec: template: metadata: annotations: checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} deployment.yaml or
 $ helm upgrade --recreate-pods 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Helm hook flow 31 kubernetes manifests pre-install post-install pre-delete post-delete pre-upgrade post-upgrade pre-rollback post-rollback crd-install apiVersion: ... kind: ... metadata: annotations: "helm.sh/hook": "pre-install" # ... 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Helm: Hook reference 32 pre-install: Executes after templates are rendered, but before any resources are created in Kubernetes. post-install: Executes after all resources are loaded into Kubernetes pre-delete: Executes on a deletion request before any resources are deleted from Kubernetes. post-delete: Executes on a deletion request after all of the release’s resources have been deleted. pre-upgrade: Executes on an upgrade request after templates are rendered, 
 but before any resources are loaded into Kubernetes (e.g. before a Kubernetes apply operation). 
 post-upgrade: Executes on an upgrade after all resources have been upgraded. pre-rollback: Executes on a rollback request after templates are rendered, but before any resources have been rolled back. post-rollback: Executes on a rollback request after all resources have been modified. 
 crd-install: Adds CRD resources before any other checks a 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Helm hook flow 33 Chart load Chart verification Pre-install hooks Sorted by weight lowest first Wait hooks ready Load chart manifests Post-install hooks sorted by weight lowest first Wait hooks ready Return release name Helm client exists 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Test Hooks (templates/tests/*.yaml) 34 apiVersion: v1 kind: Pod metadata: name: "{{ include "whoami.fullname" . }}-test-connection" labels: app.kubernetes.io/name: {{ include "whoami.name" . }} helm.sh/chart: {{ include "whoami.chart" . }} app.kubernetes.io/instance: {{ .Release.Name }} app.kubernetes.io/managed-by: {{ .Release.Service }} annotations: "helm.sh/hook": test-success spec: containers: - name: wget image: busybox command: ['wget'] args: ['{{ include "whoami.fullname" . }}:{{ .Values.service.port }}'] restartPolicy: Never or „helm.sh/hook": test-failure helm test --cleanup whoami 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Helm unittest plugin 35 suite: test ingress templates: - ingress.yaml tests: - it: should render nothing if not enabled asserts: - hasDocuments: count: 0 - it: should set annotations if given set: ingress.enabled: true ingress.annotations: kubernetes.io/ingress.class: nginx kubernetes.io/tls-acme: "true" ingress.kubernetes.io/rewrite-target: / asserts: - equal: path: metadata.annotations value: kubernetes.io/ingress.class: nginx kubernetes.io/tls-acme: "true" ingress.kubernetes.io/rewrite-target: / https://github.com/lrills/helm-unittest 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 36 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 37 Text 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Voting app 38 Web Ingress Web Service Vote Vote Pod Vote Pod Redis Master Service Redis Pod Vote Result Pod Vote Result Pod PostgreSQL Service PostgreSQL Pod ReplicaSet Deployment ReplicaSet Deployment Web Service Vote Result Vote Worker Pod ReplicaSet Deployment ReplicaSet Deployment ReplicaSet Deployment DNS & LB /-vote /-result Redis Master Pod StatefulSet PersistenceVolumeClaim Redis Slave Service Volume (data) 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Example: voting app Dependencies Subcharts Templates voting-app-reference ├── Chart.yaml ├── charts │ ├── postgresql-3.7.0.tgz │ ├── redis-5.1.3.tgz │ ├── result │ │ ├── Chart.yaml │ │ ├── templates │ │ │ ├── _helpers.tpl │ │ │ ├── deployment.yaml │ │ │ └── service.yaml │ │ └── values.yaml │ ├── vote │ │ ├── Chart.yaml │ │ ├── templates │ │ │ ├── _helpers.tpl │ │ │ ├── deployment.yaml │ │ │ └── service.yaml │ │ └── values.yaml │ └── worker │ ├── Chart.yaml │ ├── templates │ │ ├── _helpers.tpl │ │ └── deployment.yaml │ └── values.yaml ├── requirements.lock ├── requirements.yaml
 ├── templates │ ├── _helpers.tpl │ ├── kcr-imageSecret.yaml │ ├── result-ingress.yaml │ └── vote-ingress.yaml └── values.yaml 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Demo 40 • Review app code • Chart and Subchart review • Insides • Tips and Tricks W o rk hard Dream big 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 41 http://voting-app-traefi-yujm3v.k8s.customer.bee42.cloud/dev-vote http://voting-app-traefi-yujm3v.k8s.customer.bee42.cloud/dev-result 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Kubernetes stacking 42 Your 
 multi tenant
 multi stage multi branches
 multi services SYSTEM Kubernetes Platform Application Services k8s Extensions Linux OS vm or bare metal k8s node CRI CNI CSI CLI wireguard operator controller alerts tracing RBAC Security Auth metrics dns lb/ingress ReplicaSet Deployment RBAC ServiceAccount PodSecurityPolicy NetWorkPolicy Limits Auth Pod container container Service Ingress ReplicaSet Deployment RBAC ServiceAccount PodSecurityPolicy NetWorkPolicy Limits Auth Namespace Dev Namespace PreProd Kubernets Backplane Pod container container Service Ingress logging clusterAPI Image repo Backup Vault SCM Pipelines Load Balancer / Router / DNS Artefact repo Service Backplane Extensions Volumes Network Kubernetes extensions 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Release a chart with security in mind 43 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion PodSecurityPolicy 44 apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: restricted annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default' apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default' seccomp.security.alpha.kubernetes.io/defaultProfileName: 'docker/default' apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default' spec: privileged: false # Required to prevent escalations to root. allowPrivilegeEscalation: false # This is redundant with non-root + disallow privilege escalation, # but we can provide it for defense in depth. requiredDropCapabilities: - ALL # Allow core volume types. volumes: - 'configMap' - 'emptyDir' - 'projected' - 'secret' - 'downwardAPI' # Assume that persistentVolumes set up by the cluster admin are safe to use. - 'persistentVolumeClaim' hostNetwork: false hostIPC: false hostPID: false runAsUser: # Require the container to run without root privileges. rule: 'MustRunAsNonRoot' seLinux: # This policy assumes the nodes are using AppArmor rather than SELinux. rule: 'RunAsAny' supplementalGroups: rule: 'MustRunAs' ranges: # Forbid adding the root group. - min: 1 max: 65535 fsGroup: rule: 'MustRunAs' ranges: # Forbid adding the root group. - min: 1 max: 65535 readOnlyRootFilesystem: false https://kubernetes.io/docs/concepts/policy/pod-security-policy/ 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Lessons learned 45 • Use the immutable deployment pattern • Only system pods need access to the Api Server • Add PodSecurityPolicy and limits of the resource usages • Limit the network access with a NetworkPolicy • Start with a Deny All Policy • Use multiple tiller with limit RBAC roles and track your K8s cluster users • Check your container images with CVE Checkers (Clair, NeuVector, AquaSec, … ) • Add more Know How to your teams: Teach your Ops and Dev colleagues • Share your Knowledge and go to events, conferences and meetups • Think about Chaos Engineering: Replace your system components every time and do that really! • Automate all what you can and never stop this doing! • Don`t allow manuell manipulation at your K8s Cluster 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 46 Container System Blueprint: Beehive Orchestration Provisioning Security Config Metrics Alarming Logging Traceing Artefact Registry Vault Auth Source Code CI/CD Pipeline Storage Network API Gateway Routing Loadbalancer, DNS and Firewall On Premise Cloud Operating Systems Backends VM machines Databases Bare metal Container machines Container machines Data Bastion 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 47 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion 48 We hiring :-) 
 https://bit.ly/2K8DtRu 
 [email protected]
 @bee42solutions 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion rethink IT Build fashionable container systems with Kubernetes 49 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion bee42 Trainings 50 https://bee42.com/de/trainings 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Kubernetes poster Get a K8s poster today 51 https://bee42.com/de/kubernetes-poster/ PREVIEW PREVIEW WE 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion Cloud Native System Architect & bee42 founder Peter Roßbach @PRossbach [email protected] https://bee42.com https://devops-gathering.io 52 

Copyright 2019 bee42 solutions gmbh @PRossbach rethink IT - We improve your systems with passion References 53 [Sprig library](https://godoc.org/github.com/Masterminds/sprig [Deis Workflow](https://github.com/deis/workflow/tree/master/charts/workflow)) [OpenStack chart](https://github.com/sapcc/openstack-helm) [Helm.sh](https://helm.sh) [Bitnami charts](https://github.com/bitnami/charts) [kubernetes charts](https://github.com/kubernetes/charts)