Kubernetes at DigitalOcean

This talk is an overview of the journey DigitalOcean has undertaken to adopt Kubernetes and build a platform for deploying our internal services on top.

Transcript

1. digitalocean.com Kubernetes at DigitalOcean: Building a Platform for the Future Joonas Bergius @joonas
2. digitalocean.com Let's be honest: Operationalizing software well is hard
4. digitalocean.com In the beginning...
5. digitalocean.com Evolving operations • Early 2013 What is operations, even? • Late 2013 Embedded operations, “devops” • Early 2014 Centralized operations • Early 2015 Operations as advisors • Mid 2015 You build it, you run it
6. digitalocean.com Evolving architecture • Early 2013 2 monoliths • Mid 2013 4 monoliths • Early/Mid 2014 Introduction of services • Late 2014/Early 2015 Service all the things!
7. digitalocean.com
8. digitalocean.com Weighing our options
9. digitalocean.com Possible strategies • Maintain status quo • Hire dedicated operations • Invest in existing tooling • Use different tooling
10. digitalocean.com Possible strategies • Maintain status quo • Hire dedicated operations • Invest in existing tooling • Use different tooling
11. digitalocean.com • Active community & ecosystem • API-driven for automation • First-class container support • Production-grade Defining requirements
12. digitalocean.com Candidates
13. digitalocean.com
14. digitalocean.com Refining our solution
15. digitalocean.com • Curate a simplified user experience • Focus on stateless services • Make deployments declarative Guiding principles
16. digitalocean.com docc - a tool for deploying containerized, stateless applications
17. • Describe application runtime requirements • Integrate external systems with application • Establish accountability through ownership digitalocean.com Driven with a manifest
18. digitalocean.com { "application": "kubecon-2017", "scale": 3, "containers": { "kubecon": { "image": "registry.corp.example.com/demo/kubecon" "image_tag": "2017" } }, "maintainer": "joonas@digitalocean.com" } KubeCon application
19. digitalocean.com Translated to Kubernetes Deployment ReplicaSet Pod Pod Pod Created by docc
20. digitalocean.com { "application": "kubecon-2017", "scale": 3, "containers": { "kubecon": { "image": "registry.corp.example.com/demo/kubecon" "image_tag": "2017", "ports": [8080] } }, "maintainer": "joonas@digitalocean.com" } Exposing a Service
21. digitalocean.com Translated to Kubernetes: Exposing a Service Deployment ReplicaSet Pod Pod Pod Service kubecon-2017:8080 Endpoint Created by docc
22. digitalocean.com { "application": "kubecon-2017", "scale": 3, "ingresses": { "kubecon-2017": { "scheme": "http", "container_port": 8080 } }, "containers": { "kubecon": { "image": "registry.corp.example.com/demo/kubecon" "image_tag": "2017", "ports": [8080] } }, "maintainer": "joonas@digitalocean.com" }
23. digitalocean.com Translated to Kubernetes: Ingress connectivity Deployment ReplicaSet Pod Pod Pod Service kubecon-2017:8080 Endpoint Ingress kubecon-2017 Created by docc
24. digitalocean.com { "application": "kubecon-2017", "scale": 3, "auto_tls": true, "ingresses": { "kubecon-2017": { "scheme": "tls", "container_port": 8080 } }, "containers": { "kubecon": { "image": "registry.corp.example.com/demo/kubecon" "image_tag": "2017", "ports": [8080] } }, "maintainer": "joonas@digitalocean.com" }
25. digitalocean.com Translated to Kubernetes: Adding TLS Deployment ReplicaSet Pod Pod Pod Service kubecon-2017:8080 Endpoint Ingress kubecon-2017 Certificate Secret /etc/docc/tls/... Created by docc
26. • 12-Factor-ish & containerized • Horizontally scalable • Stateless In practice: 90-95% of our services fit this bill. digitalocean.com Application requirements
27. • Allows for separation of responsibilities • Promotes decoupling from implementation • Provides a magical end-user experience digitalocean.com Why do this?
28. digitalocean.com docc in action
29. digitalocean.com $ docc --help docc is the CLI tool for the docc server Available Commands: completion Generate shell completions config Config modifies and manages the docc config delete Delete and purges the given app(s) deploy Deploy or update a app by filename events Events shows the most recent events init Init creates a ready to use manifest.json file in the current directory inspect Inspect returns detailed information about the given app(s) list Lists the current deployed apps logs Logs shows the most recent logs open Open the specific subcommand show Show the manifest of given app update Update the state of an application validate Validates and shows errors/warnings
30. digitalocean.com $ docc deploy manifest.json
31. digitalocean.com kube-apiserver kube-apiserver kube-apiserver controllers doccserver ingress docc kubectl kubelet json external services TPRs
32. • Translates between docc and kubernetes concepts • Tracks applications with annotations and groups by labels • Coordinates with external services digitalocean.com doccserver
35. digitalocean.com Quantifying impact
36. • Every new application is deployed with docc • Deployment no longer the bottleneck: ◦ New apps deployed in hours, not days ◦ Existing apps deployed in minutes, not hours ◦ Averaging 25 deploys a day • Seeing as many as 75 apps per cluster ◦ Migrating apps has become trivial • Most importantly: developers are much happier! digitalocean.com Results so far
37. digitalocean.com – Tommy Murphy “Reduce developer decision fatigue so they can make decisions that matter [to the business].”
38. Thank you!