Tarmak, why do we need another Kubernetes provisioner?

This is a talk given at cfgmgmtcamp 2019

Tarmak is an open-source toolkit for Kubernetes cluster lifecycle management.
It is focused on best-practice cluster security, management and operation.
This talk will clarify the reasoning behind creating Tarmak, the architecture and technologies used.
With Tarmak we tried to not reinvent the full wheel, that is why we have chosen to depend on a lot open-source tools (Terraform, Puppet, Vault, …).
This talk will explain how all of this is tied together.

Transcript

1. jetstack.io Tarmak, why do we need another Kubernetes provisioner? Presented by Mattias Gees @MattiasGees
2. Mattias Gees jetstack.io @MattiasGees • Solutions Engineer @ Jetstack • Marathon runner
3. What is Tarmak? jetstack.io @MattiasGees Tarmak is an open-source toolkit for Kubernetes cluster lifecycle management. It is focused on best-practice cluster security, management and operation. https://github.com/jetstack/tarmak
4. jetstack.io Kubernetes kubectl storage (etcd) API server scheduler controller-manager Node kubelet proxy docker Node kubelet proxy docker Control Plane Other clients jetstack.io @MattiasGees
5. History jetstack.io @MattiasGees • 1st and 2nd generation (around k8s 1.0) ◦ CoreOS ◦ CloudFormation and Terraform ◦ Cloud-init Bash ◦ Ruby and Shell scripts
6. Lessons jetstack.io @MattiasGees • Immutable is not always needed • Testing / Debugging • Dependencies need to be versioned • PKI Management • Terraform abstraction
7. Motivations jetstack.io @MattiasGees • Shorter feedback loop • Reusability of code • Continuous roll-out of changes • Dry-run • Tried and tested tools
8. Motivations jetstack.io @MattiasGees • Immutable can be expensive and slow • Desired vs actual • Stateful application
9. jetstack.io Components jetstack.io @MattiasGees
10. jetstack.io Components jetstack.io @MattiasGees
11. jetstack.io @MattiasGees Architecture
12. Packer jetstack.io @MattiasGees • Prepare images • Faster start-up
13. Terraform jetstack.io @MattiasGees • Cloud Infrastructure • Previously: multi stack • Tarmak provider • Now: 1 stack
14. Vault jetstack.io @MattiasGees • PKI Management • Vault-unsealer • Backed by Consul • Short lived certificates • Automatic renew
15. Vault jetstack.io @MattiasGees
16. Puppet jetstack.io @MattiasGees • Config of servers • Desired vs actual state
17. Wing jetstack.io @MattiasGees • Wrapper • Installed with boot-init • Get Puppet manifests • Puppet apply • Report status
18. jetstack.io Wing tarmak storage (etcd) wing-server instance-a wing puppet instance-b wing puppet Wing server jetstack.io @MattiasGees
19. Go jetstack.io @MattiasGees • Glue • Validation • Verification • Customization
20. Tarmak config jetstack.io @MattiasGees
21. Tarmak config jetstack.io @MattiasGees
22. Demo jetstack.io @MattiasGees
23. Advantages jetstack.io @MattiasGees • Security • Encryption • Close to upstream Kubernetes • Cluster Autoscaler • Logging to Elasticsearch • Monitoring • Multiple instance pools
24. Downsides jetstack.io @MattiasGees • Rolling upgrades are a pain • Spot instances • Logging outputs • AWS only* • No public images *
25. mattias.gees@jetstack.io @MattiasGees @JetstackHQ Thank you. jetstack.io