Kubernetes - Beyond the Basics

Learn about automated deployments, blue/green and rolling updates, load balancing and networking in Kubernetes

Transcript

1. Kubernetes Beyond the basics @pbakker
2. Paul Bakker @pbakker
3. Automated, production ready Kubernetes cluster in steps 9
4. Step Understanding Kubernetes 0 Terminology, and concepts to build upon
5. Nodes, Pods, Controllers Docker container Docker container Docker container Docker container Pods Node Docker container Docker container Docker container Docker container Pods Node Docker container Docker container Docker container Replication Controller Master schedules schedules
6. Deployment 101 Push your Docker container Create a new replication controller JSON file kubectl  create  -­‐f  mycontroller.json   Replication Controller creates Pods
7. mycontroller.json  "spec":{              "replicas":3,              "selector":{                    "name":"frontend"              },              "template":{                    "metadata":{                          "labels":{                                "name":"frontend"                          }                    },                    "spec":{                          "containers":[                                {                                      "name":"php-­‐redis",                                      "image":"kubernetes/example-­‐guestbook-­‐php-­‐redis:v2",                                      "ports":[                                            {                                                  "containerPort":80                                            }                                      ]                                }                          ]                    }              }
8. Scaling kubectl  scale                                                                   —replicas=10  myreplication-­‐controller
9. Updating my app Create a new Replication Controller JSON file kubectl create -f my-new-rc.json Scale down and delete old RC
10. Step Automated deployment (simplistic) 1 This kubectl stuff seems a lot of typing!
11. The simplest Automated deployment Don’t use kubectl, use the API! Build server creates Replication Controller using REST Build server destroys old cluster using REST
12. Docker container Docker container Docker container Docker container Node Docker registry Build Server Docker container Docker container Docker container Docker container Node push Create RC Docker container Docker container Docker container Replication Controller Master schedules schedules API
13. Curl example curl  -­‐X  POST       http://k8-­‐master:8080/api/v1beta3/namespaces/default/replicationcontrollers  -­‐d  '{   #Pod  definition   }’
14. What about downtime? Not quite there yet
15. Step Load balancing 2 Our containers are running, but how do we access them!?
16. Pods come and go Pods have dynamic IP addresses First try - Kubernetes Services A service is a proxy to your Pods Fixed IP P O D S E
 R
 V
 I
 C E
17. Docker container Docker container Docker container Docker container Pods Node Docker container Docker container Docker container Docker container Pods Node MyService HTTP Virtual IP Virtual IP Fixed IP
18. What about SSL offloading? … better load balancing? … redirects, rewrites, etc? … and that “fixed” IP can’t be reached!? Services - Not quite right
19. Services are for communication within the k8 network (inter Pod communication) Services - A Hammer and screws…
20. Docker container Docker container Docker container Docker container Pods Node Docker container Docker container Docker container Docker container Pods Node Vulcan Proxy HTTP Virtual IP Virtual IP Fixed IP Custom load balancer etcd
21. Choosing a load balancer Vulcan uses etcd for all its config Can use Nginx / HA-proxy with templating
22. So you’re telling me… —link doesn’t work!? And now you’re telling me… —I can’t see my Pods!?
23. Step Weave 3 Each Pod gets its own IP Access Pods from outside k8
24. Docker container Docker container Docker container Docker container Node Docker container Docker container Docker container Docker container Pods Node Vulcan Proxy HTTP Virtual IP Virtual IP Fixed IP Weave network Pods
25. Step Load balancer registration 4 How does the load balancer keep track of pods?
26. Watch the Kubernetes API Kubernetes API Vulcan etcd Registrator Watch New Vulcan Config Uses new backends
27. Amdatu Vulcanized Open source tool connecting Kubernetes and Vulcan Written in Go Available as Docker container
28. Amdatu Vulcanized Kubernetes API Vulcan etcd Amdatu Vulcanized Watch New Vulcan Config Uses new backends
29. Step Blue / Green deployment 5 Auto deploy is great, but downtime not so much
30. Step 5 - Blue / Green Scale up new cluster Wait until healthy Switch backend in Load Balancer Dispose old cluster
31. How do we know a Pod is healthy? Its RUNNING status is not sufficient… Is the app fully started?
32. Introduce App level health checks Docker container Docker container Docker container Docker container Node Docker container Docker container Docker container Docker container Pods Node Deployer GET /health GET /health Pods Deploy Server
33. Running a Deployer This whole things starts be to complex! Our build server can’t access the Pods … how do we health check?
34. Kubernets API etcd Deployer Build Server Start deployment Kubernets API Kubernets API Kubernets API Pods GET /health Create RC Switch Vulcan Backend
35. Kubernets API Vulcan etcd Deployer Build Server Start deployment Kubernets API Kubernets API Kubernets API Pods GET /health Create RC Switch Vulcan Backend Amdatu Vulcanized Watch Create backends Read config
36. Deployment descriptor { "useHealthCheck": true, "newVersion": "${bamboo.deploy.version}", "appName": "todo", "replicas": 2, "vulcanFrontend": "rti-todo.amdatu.com", “podspec": { …. } }
37. "podspec": { "containers": [{ "image": “amdatu/mycontainer", "name": "todo", "ports": [{ "containerPort": 8080 }], "env": [ { "name": "version", "value": "${bamboo.deploy.version}" } ]}] }
38. One more thing… We need to tell Kubernetes replicas need to run on different machines! Docker container Docker container Docker container Docker container Node Docker container Docker container Docker container Docker container Node Create RC Docker container Docker container Docker container Replication Controller Master S E R V I C E
39. Deployment demo Demo
41. Step Canary deployment 6
42. Canary deployments Different strategy for the Deployer Add Replication Controller But don’t change the running cluster
43. K8 Node K8 Node K8 Node K8 Node Prod pod Canary Main Replication Controller K8 Node K8 Node K8 Node K8 Node Canary pod Canary Replication Controller Vulcan
44. Step Persistent data 7 How to deploy Mongo/MySQL/ElasticSearch in Kubernetes?
45. You don’t
46. Kubernetes is great for… Stateless containers Running lots of containers together Moving containers around
47. Datastores scaling mechanics Reactive scaling makes less sense Cluster should be tuned Scaling is expensive
48. Infra server(s) K8 Master K8 Node K8 Node K8 Node K8 Node K8 Node Vulcan Vulcanized Deployer Mongo Cluster ElasticSearch Cluster … Cluster Cluster topology
49. Step Logging 8 kubectl logs mypod?
50. Logging Centralised logging is key in a dynamic environment Assume you can’t access a pod ELK is very useful for this
51. Logging Docker container Docker container Docker container Docker container Docker container Docker container Docker container Docker container LogStash ElasticSearch Kibana
52. Logging example OSGi app OSGi LogService SLF4J Kafka
53. Logging example Kafka LogStash ElasticSearch Infra components Kibana Dashboard Developer
55. Step Configuration 9 Passing config to containers
56. Use environment variables dbName=todo-­‐app   host=${mongo} myconfig.cfg "podspec":  {                "env":  [                    {                      "name":  "mongo",                      "value":  "10.100.2.4"                      }, Deployment descriptor Approach 1
57. Use etcd etcd=localhost:2375 myconfig.cfg /apps/config/demo-­‐app etcd Approach 2 [    {      "name":  "mongo",      "value":  "10.100.2.4"    }   ]
58. And if you don’t want to do all this yourself…. RTI Fully managed Kubernetes based clusters Logging and Monitoring Automated deployments Not your standard PAAS
59. Thank you! Blog: http://paulbakker.io Twitter: @pbakker Mail: paul.bakker@luminis.eu