Ahoi! Complex applications on Kubernetes

This is a talk I gave at the 60th Berlin Elixir UG on Jun 13

Transcript

1. Ahoi!
2. Your captain today is Lukas Rieder.
3. I run a data consultancy. https://www.hibase.co/
4. I give talks. https://speakerdeck.com/Overbryd
5. Complex applications.
6. What are complex applications?
7. We start out simple…
8. We extend…
9. and store…
10. and integrate.
11. The world is complex.
 Simply manage it.
12. Complex applications.
13. Introducing Kubernetes.
14. Imagine an Elixir Supervisor. children = [
 %{
 id: MyApplication,
 start: {MyApplication, :start_link, [[:hello]]}
 }
 ]
 
 {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one)
15. Imagine an Elixir Supervisor. children = [
 %{
 id: MyApplication,
 start: {MyApplication, :start_link, [[:hello]]}
 }
 ]
 
 {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one)
16. Imagine an Elixir Supervisor. children = [
 %{
 id: MyApplication,
 start: {MyApplication, :start_link, [[:hello]]}
 }
 ]
 
 {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one)
17. Imagine an Elixir Supervisor. Supervisor.count_children(pid)
 #=> %{active: 1, specs: 1, supervisors: 0, workers: 1}
18. A Deployment in Kubernetes. apiVersion: apps/v1
 kind: Deployment
 metadata:
 name: my-app
 labels:
 app: my-app
 spec:
 # ... snip ...
 template:
 spec:
 containers:
 - name: my-container
 image: my-app:1.7.9
 command: ["myapp", "start_link"]
 args: ["hello"]

19. Elixir Declarative programming. apiVersion: apps/v1
 kind: Deployment
 metadata:
 name: my-app
 labels:
 app: my-app
 spec:
 # ... snip ...
 template:
 spec:
 containers:
 - name: my-container
 image: my-app:1.7.9
 command: ["myapp", "start_link"]
 args: ["hello"]
 children = [
 %{
 id: MyApplication,
 start: {MyApplication, :start_link, [[:hello]]}
 }
 ]
 
 {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one) Kubernetes
20. Single-responsibility principle. Job Service Ingress Deployment Supervisor Processes Elixir Kubernetes
21. Organized in graphs. Elixir Kubernetes
22. State converges.
23. Extend Kubernetes with Elixir.
24. Meet the operator pattern.
25. Meet the operator pattern. Operators are a combination of K8 resources.
26. Meet the operator pattern. It allows developers to encode domain knowledge
 into an extension of the K8s API.
27. Custom
 Resource
 Definition Controller API Custom
 Resource Meet the operator pattern.
28. Custom
 Resource
 Definition Controller API Custom
 Resource The CRD defines your object through a schema.
29. Custom
 Resource
 Definition Controller API Custom
 Resource Resources are the common declarative language, to describe a desired state.
30. Custom
 Resource
 Definition Controller API Custom
 Resource apiVersion: apiextensions.k8s.io/v1beta1
 kind: CustomResourceDefinition
 metadata:
 name: crontabs.stable.example.com
 spec:
 group: stable.example.com
 versions:
 - name: v1
 served: true
 storage: true
 scope: Namespaced
 names:
 plural: crontabs
 singular: crontab
 kind: CronTab
 shortNames:
 - ct

31. Custom
 Resource
 Definition Controller API Custom
 Resource The API handles CRUD, storage, changes, etc.
32. Custom
 Resource
 Definition Controller API Custom
 Resource The controller implements domain specific knowledge.
33. Custom
 Resource
 Definition Controller API Custom
 Resource The controller manages K8s and Custom resources for the given application domain.
34. Why Elixir?
35. Excellent state handling. Process Process
36. Pattern matching. Deconstruction. %{ "apiVersion" => "apps/v1", "kind" => "Deployment", "metadata" => %{ "labels" => %{"app" => "nginx"}, "name" => "nginx-deployment", "namespace" => "default" }, "spec" => %{ "replicas" => 3, "selector" => %{"matchLabels" => %{"app" => "nginx"}}, "template" => %{ "metadata" => %{"labels" => %{"app" => "nginx"}}, "spec" => %{ "containers" => [ %{ "image" => "nginx:1.7.9", "name" => "nginx", "ports" => [%{"containerPort" => 80}] } ] } } } }
37. There are great libraries. https://github.com/coryodaniel/bonny https://github.com/coryodaniel/k8s
38. Inter-cluster communication. Solved. BeamVM BeamVM BeamVM BeamVM
39. Long running. High self sufficiency.
40. Real world examples.
41. hs-preview "As a customer, I want to see my campaigns live, to check if they meet my expectations."
42. hs-preview www.example.com staging.example.com preview-aefd23.example.com preview-b32bef.example.com git:master git:live git:feat/abc git:feat/xyz
43. hs-preview Controller Ingress Deployment Service www.example.com git:feat/xyz
44. hs-preview Controller Ingress Deployment Service www.example.com git:feat/xyz
45. hs-preview Controller Ingress Deployment Service www.example.com preview-b32bef.example.com git:feat/xyz
46. hs-preview Ingress Deployment Service www.example.com Ingress preview-b32bef.example.com Controller git:feat/xyz
47. hs-preview Ingress Deployment Service www.example.com Ingress Deployment preview-b32bef.example.com Controller git:feat/xyz
48. hs-preview Ingress Deployment Service www.example.com Ingress Deployment Service preview-b32bef.example.com Controller git:feat/xyz
49. hs-preview Ingress Deployment Service www.example.com Ingress Deployment Service preview-b32bef.example.com Domain Controller git:feat/xyz
50. hs-preview preview-b32bef.example.com git:feat/xyz
51. hs-preview https://tech.highsnobiety.com/optimizing-our-deployment- pipeline-part-1-3fb04e727013
52. Software multitenancy "As a customer, I want my workflows to run in a predictable and reliable manner."
53. Customer apiVersion: "internal.example.com/v1"
 kind: Customer
 metadata:
 name: big-client-0815
 spec:
 features:
 database:
 enabled: true
 storage: 256gb
 scheduler:
 enabled: true
 workers: 4
 webhooks:
 enabled: false
 Software multitenancy
54. Customer Database Scheduler Job
 (k8s) Namespace
 (k8s) Deployment
 (k8s) Customer
 Controller API Database
 Controller Scheduler
 Controller Software multitenancy
55. Software multitenancy Customer Database Scheduler Job
 (k8s) Namespace
 (k8s) Deployment
 (k8s) Customer Database Scheduler Job
 (k8s) Namespace
 (k8s) Deployment
 (k8s) Customer Database Scheduler Job
 (k8s) Namespace
 (k8s) Deployment
 (k8s)
56. Robots building robots.
57. Getting started
58. Getting started ✓ Docker / Docker for Mac ✓ minikube / Kubernetes ✓ Elixir, Erlang ✓ bonni / k8s
59. Docker for mac is a one-stop-shop.
60. Example: global connection pools "As a customer, I want my database to be running at all times, in order to make money."
61. Example: global connection pools App
62. Example: global connection pools App Worker 1
63. Example: global connection pools App Worker 1 Worker 2
64. Example: global connection pools App Worker 1 Worker 2
65. Example: global connection pools App Worker 1 Worker 2 pgbouncer
66. Example: global connection pools App Worker 1 Worker 2 pgbouncer pgbouncer Customer A Customer B
67. Example: global connection pools Terminal time
68. Thank you for having me.
69. Contact: lukas@hibase.co