Helm: Build fashionable container systems with Kubernetes

There are thousands of developers and admins packaging their applications for use with Kubernetes.
This usually involves the creation of several different Kubernetes resource definitions
that configure the application runtime, as well as define the mechanism that users
and other apps use to communicate with the services.

For optimal approach such as these services
should be configured, we need a configuration management tool.
Helm is the native package manager for Kubernetes. The Helm ecosystem offers you many features to setup and manage complex cloud native container architectures.

In this presentation we will go into the basics of Helm,
Tiller and the Helm Charts. Beyond the basics we are introducing a strategy,
how to fully automate a complete microservice system.
Afterwards we show you some examples and tricks for the usage of helm and
will start a discussion.

[Docker Meetup Berlin](https://www.meetup.com/de-DE/Docker-Berlin/events/251861654/)

Transcript

1. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion 1 Helm:: Build fashionable container systems with Kubernetes
2. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Kubernetes cluster 2 • Kubernetes is container orchestration
 It enables container to scale • It’s a very active open-source project with lots of contributors, started at 6. June 2014 • Originally developed by Google and 
 donated to Cloud Native Computing Foundation
3. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion K8s cluster architecture 3 k8s master k8s minion node api server controller manager scheduler ETCD kubelet kubeproxy kubelet kubeproxy pod Containers Containers pod Containers Containers pod Containers Containers cri cri k8s minion node kubelet kubeproxy pod Containers Containers pod Containers Containers pod Containers Containers cri DevOps User
4. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion K8s Deployment Challenge 4 Move to microservice architecture application consists from multiple components each component has its own resources and can be scaled individually It’s hard to ... ... manage, edit and update multiple K8s configurations ... deploy multiple K8s configurations as a SINGLE application ... share and reuse K8s configurations and applications ... parametrize and support multiple environments ... manage application releases: rollout, rollback, diff, history ... define deployment lifecycle (control operations to be run in different phases) ... validate release state after deployment
5. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion What helm can do for you? 5 Create new charts, aka package k8s manifests, from scratch Package charts into chart archive (tgz) file Interact with chart repositories where charts are stored Install and uninstall charts into an existing kubernetes cluster Manage the release cycle of charts that have been installed with helm
6. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Chart 6 a package of kubernetes manifests Release a chart instance is loaded into kubernetes Repository a download area of published charts Template a kubernetes configuration file mixed with Go/Sprig templates
7. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion How helm works? 7 client api-service tiller service(s) https grpc brew install kubernetes-helm curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash or
8. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion helm init 8 $ kubectl create serviceaccount tiller --namespace kube-system $ cat >tiller-rbac.ymal <<EOF apiVersion: v1 kind: ServiceAccount metadata: name: tiller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: tiller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: tiller namespace: kube-system EOF $ kubectl create -f tiller-rbac.yaml $ helm init --service-account tiller —upgrade
9. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion helm cli commands 9 completion Generate autocompletions script for the specified shell (bash or zsh) create create a new chart with the given name delete given a release name, delete the release from Kubernetes dependency manage a chart's dependencies fetch download a chart from a repository and (optionally) unpack it in local directory get download a named release history fetch release history home displays the location of HELM_HOME init initialize Helm on both client and server inspect inspect a chart install install a chart archive lint examines a chart for possible issues list list releases package package a chart directory into a chart archive plugin add, list, or remove Helm plugins repo add, list, remove, update, and index chart repositories reset uninstalls Tiller from a cluster rollback roll back a release to a previous revision search search for a keyword in charts serve start a local http web server status displays the status of the named release template locally render templates test test a release upgrade upgrade a release verify verify that a chart at the given path has been signed and is valid version print the client/server version information
10. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Helm flow 10 client api-service tiller service(s) publish fetch install upgrade delete redis create application tomcat postgres
11. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Example: Simple config map 11 $ helm create my-chart # review mychart templates $ rm -rf mychart/templates/*.* $ cat <<EOF >mychart/templates/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: mychart-configmap data: myvalue: "Hello World" EOF $ helm install --dry-run --debug ./mychart $ helm install --name full-coral --namespace test ./mychart $ helm get manifest full-coral $ helm delete full-coral
12. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Example: Prometheus & Grafana 12 # pre init
 # ————————————————————————————————————————————————————————————————————————— $ helm init --client-only $ helm repo add stable \ https://kubernetes-charts.storage.googleapis.com/ $ helm repo add incubator \ https://kubernetes-charts-incubator.storage.googleapis.com/ $ helm repo update # install prometheus # ————————————————————————————————————————————————————————————————————————— $ helm install stable/prometheus \ --name prometheus \
 --namespace monitoring \ --set rbac.create=true,server.persistentVolume.enabled=false,\ alertmanager.enabled=false,pushgateway.enabled=false # install grafana
 # ————————————————————————————————————————————————————————————————————————— $ helm install ./grafana \ --name grafana \ --namespace monitoring \ --set server.persistentVolume.enabled=false,server.adminPassword=admin
13. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Flow Control 13 If/Else {{ if PIPELINE }} # Do something {{ else if OTHER PIPELINE }} # Do something else {{ else }} # Default case {{ end }} data: myvalue: "Hello World" drink: {{ .Values.favorite.drink | default "tea" | quote }} food: {{ .Values.favorite.food | upper | quote }} {{- if eq .Values.favorite.drink "lemonade" }} mug: true {{- end }} # notice the "-" in the left, if will help eliminate newline before variable
14. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Flow Control 14 With data: myvalue: "Hello World" {{- with .Values.favorite }} drink: {{ .drink | default "tea" | quote }} food: {{ .food | upper | quote }} {{- end }} # instead of writing ".Values.favorite.drink"
15. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Flow Control 15 Range # predefined variable pizzaToppings: - mushrooms - cheese - peppers - onions toppings: |- {{- range $i, $val := .Values.pizzaTopping }} - {{ . | title | quote }} # upper first character, then quote {{- end }} sizes: |- {{- range tuple "small" "medium" "large" }} - {{ . }} {{- end }} # make a quick list
16. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Flow Control 16 Variables data: myvalue: "Hello World" {{- $relname := .Release.Name -}} {{- with .Values.favorite }} drink: {{ .drink | default "tea" | quote }} food: {{ .food | upper | quote }} release: {{ $relname }} {{- end }} # use variable in range toppings: |- {{- range $index, $topping := .Values.pizzaToppings }} {{ $index }}: {{ $topping }} {{- end }} {{- range $key,$value := .Values.favorite }} {{ $key }}: {{ $value }} {{- end }} # instead of specify the key, we can actually loop through the values.yaml file and print values values.yaml
 pizzaToppings: - mushrooms - cheese - peppers - onions
17. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Flow Control 17 Nested Templates # _helpers.tpl {{/* Generate basic labels */}} {{- define "my_labels" }} labels: generator: helm date: {{ now | htmlDate }} version: {{ .Chart.Version }} name: {{ .Chart.Name }} {{- end }} # configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: {{ .Release.Name }}-configmap {{- template "my_labels" . }} # Notice the final dot, it will pass the global scope inside template file. Without it version & name will not be generated. {{- include "my_labels" . | indent 2 }} # similar to "template" directive, have the ability to control indentation
18. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Flow Control 18 Files inside Templates # file located at parent folder # config1.toml: |- # message = config 1 here # config2.toml: |- # message = config 2 here # config3.toml: |- # message = config 3 here data: {{- $file := .Files }} # set variable {{- range tuple "config1.toml" "config2.toml" "config3.toml" }} # create list {{ . }}: |- # config file name {{ $file.Get . }} # get file's content {{- end }}
19. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Flow Control Glob-patterns & encoding apiVersion: v1 kind: ConfigMap metadata: name: conf data: +{{ (.Files.Glob "foo/*").AsConfig | indent 2 }} --- apiVersion: v1 kind: Secret metadata: name: very-secret type: Opaque data: +{{ (.Files.Glob "bar/*").AsSecrets | indent 2 }} +token: |- + {{ .Files.Get "config1.toml" | b64enc }} 19
20. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion ImagePullSecrets 20 {{- define "imagePullSecret" }} {{- printf "{\"auths\": {\"%s\": {\"auth\": \"%s\"}}}" .Values.imageCredentials.registry (printf "%s: %s" .Values.imageCredentials.username .Values.imageCredentials.password | b64enc) | b64enc }} {{- end }} imageCredentials: registry: quay.io username: someone password: sillyness apiVersion: v1 kind: Secret metadata: name: imageSecretKey type: kubernetes.io/dockerconfigjson data: .dockerconfigjson: {{ template "imagePullSecret" . }} values.yaml templates/_helper.tpl templates/imageSecretKey.yaml
21. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Autodeploy 21 apiVersion: apps/v1 kind: Deployment spec: template: metadata: annotations: checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} deployment.yaml or
 $ helm upgrade --recreate-pods
22. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Helm hook flow 22 Chart load Chart verification Pre-install hooks Sorted by weight lowest first Wait hooks ready Load chart manifests Post-install hooks sorted by weight lowest first Wait hooks ready Return release name Helm client exists
23. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Helm hook flow 23 kubernetes manifests pre-install post-install pre-delete post-delete pre-upgrade post-upgrade pre-rollback post-rollback crd-install apiVersion: ... kind: ... metadata: annotations: "helm.sh/hook": "pre-install" # ...
24. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Microservice with k8s manifests 24 POD POD POD Deployment ReplicaSet Service Ingress Ingress Proxy Endpoints Loadbalancer watch service watch Req/Res selector manage manage Req/Res manage group of containers contaner contaner contaner contaner contaner contaner pod of whales manage :-)
25. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Voting app 25 Web Ingress Web Service Vote Vote Pod Vote Pod Redis Master Service Redis Pod Vote Result Pod Vote Result Pod PostgreSQL Service PostgreSQL Pod ReplicaSet Deployment ReplicaSet Deployment Web Service Vote Result Vote Worker Pod ReplicaSet Deployment ReplicaSet Deployment ReplicaSet Deployment DNS & LB /<release-name>-vote /<release-name>-result Redis Master Pod StatefulSet PersistenceVolumeClaim Redis Slave Service Volume (data)
26. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Example: voting app voting-app-reference ├── Chart.yaml ├── charts │ ├── postgresql-0.13.1.tgz │ ├── redis-3.3.6.tgz │ ├── result-0.1.6.tgz │ ├── vote-0.1.6.tgz │ └── worker-0.1.5.tgz ├── requirements.lock ├── requirements.yaml
 ├── components │ ├── result │ │ ├── Chart.yaml │ │ ├── templates │ │ │ ├── _helpers.tpl │ │ │ ├── deployment.yaml │ │ │ └── service.yaml │ │ └── values.yaml │ ├── vote │ │ ├── Chart.yaml │ │ ├── templates │ │ │ ├── _helpers.tpl │ │ │ ├── deployment.yaml │ │ │ └── service.yaml │ │ └── values.yaml │ └── worker │ ├── Chart.yaml │ ├── templates │ │ ├── _helpers.tpl │ │ └── deployment.yaml │ └── values.yaml ├── templates │ ├── _helpers.tpl │ ├── kcr-imageSecret.yaml │ ├── result-ingress.yaml │ └── vote-ingress.yaml └── values.yaml Dependencies Subcharts Templates
27. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Demo 27 • Review app code • Chart and Subchart review • Insides • Tips and Tricks W o rk hard Dream big
28. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Kubernets stacking 28 Application Services Kubernetes extensions Linux OS vm or bare metal Kubernetes minion CRI CNI CSI CLI wireguard operator controller clusterAPI logging tracing RBAC Security Auth network storage metrics DNS LB Pod contaner contaner Servvice Ingress ReplicaSet Deployment RBAC ServiceAccount PodSecurityPolicy NetWorkPolicy Limits Auth Pod contaner contaner Servvice Ingress ReplicaSet Deployment RBAC ServiceAccount PodSecurityPolicy NetWorkPolicy Limits Auth Namespace Dev Namespace PreProd Kubernets Backplane Your multi tenant, multistage or
 multi branch services Kubernetes Platform
29. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Release a chart with security in mind 29 PodSecurityPolicy NetworkPolicy Deployment ReplicaSet ClusterRole ClusterRoleBinding helm client kubeconfig api-server tiller service tiller deploy pod tiller release configmap Service Account Service Pod Ingress Release Namespace X Role RoleBinding Service Account Limits Endpoints Containers Release grpc https kube-system Namespace ClusterRole ClusterRoleBinding Service Account
30. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion PodSecurityPolicy 30 apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: restricted annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default' apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default' seccomp.security.alpha.kubernetes.io/defaultProfileName: 'docker/default' apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default' spec: privileged: false # Required to prevent escalations to root. allowPrivilegeEscalation: false # This is redundant with non-root + disallow privilege escalation, # but we can provide it for defense in depth. requiredDropCapabilities: - ALL # Allow core volume types. volumes: - 'configMap' - 'emptyDir' - 'projected' - 'secret' - 'downwardAPI' # Assume that persistentVolumes set up by the cluster admin are safe to use. - 'persistentVolumeClaim' hostNetwork: false hostIPC: false hostPID: false runAsUser: # Require the container to run without root privileges. rule: 'MustRunAsNonRoot' seLinux: # This policy assumes the nodes are using AppArmor rather than SELinux. rule: 'RunAsAny' supplementalGroups: rule: 'MustRunAs' ranges: # Forbid adding the root group. - min: 1 max: 65535 fsGroup: rule: 'MustRunAs' ranges: # Forbid adding the root group. - min: 1 max: 65535 readOnlyRootFilesystem: false https://kubernetes.io/docs/concepts/policy/pod-security-policy/
31. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Lessons learned 31 • Use the immutable deployment pattern • Only system pods need access to the api server • Add PodSecurityPolicy and limits of the resource usages • Limit the network access with a NetworkPolicy • Use multiple tiller with limit RBAC roles and track your k8s cluster users • Check your container images with CVE Checkers (Clair, NeuVector, AquaSec, … ) • Add more Know How to your teams: Teach your Ops and Dev colleagues • Share your Knowledge • Think about Chaos Engineering: Replace your system components every time • Automate all and never stop this process
32. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion 32 Container System Blueprint: beehive Orchestration Provisioning Security Config Metrics Alarming Logging Traceing Artefact Registry Vault Auth Source Code CI/CD Pipeline Storage Network API Gateway Routing Loadbalancer, DNS and Firewall On Premise Cloud Operating Systems Backends VM machines Databases Bare metal Container machines Container machines
33. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion CSSC: Container System Supply Chain 33 CI/CD Git DevOps commit trigger commit Jobs ci trigger build test tag push helm install CI Registry Runner K8s cluster push image deploy manifests pull jobs pull images K8s resources Monitor Tiller metrics, logs, tracing status Company Registry Hubs & Mirrors Package Repos pull image use packages execute
34. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion 34
35. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion 35 We hiring :-) 
 https://bit.ly/2K8DtRu 
 jobs@bee42.com
 @bee42solutions
36. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion bee42 Trainings 36 25.06.2018 in Berlin 04.09.2018 in Berlin 15.10.2018 in München 19.11.2018 in Essen 06.07.2018 in Essen 03.09.2018 in Essen 09.10.2018 in Berlin 17.09.2018 in Essen 22.10.2018 in Essen 19.09.2018 in Essen 22.11.2018 in Essen Rabatt code meetup-berlin-20 Date of Expiry 31/07/2018 03.08.2018 in Essen 24.10.2018 in Essen
37. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion K8s poster pre registration started https://bit.ly/2JN0hWI 37 We
38. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion Cloud Native System Architect & bee42 founder Peter Roßbach @PRossbach peter.rossbach@bee42.com https://bee42.com https://devops-gathering.io 38 Save the date… #DOG19 11.-13.3.2019 at Bochum
39. Copyright 2018 bee42 solutions gmbh <peter.rossbach@bee42.com> @PRossbach rethink IT - improve your systems with passion References 39 [Sprig library](https://godoc.org/github.com/Masterminds/sprig [Deis Workflow](https://github.com/deis/workflow/tree/master/charts/workflow)) [OpenStack chart](https://github.com/sapcc/openstack-helm) [helm.sh](https://helm.sh) [bitnami charts](https://github.com/bitnami/charts) [kubernetes charts](https://github.com/kubernetes/charts)