Creating a fast Kubernetes Development Workflow

Kubernetes is very powerful in orchestrating where your services are running, but building docker images, writing and updating all the necessary YAML files and applying them to your cluster can still become an annoying maintenance overhead. In this talk you will learn how to create a fast, low friction development workflow with Kubernetes and tools like Telepresence and Forge which enables you to quickly and safely build, test and deploy your services.

Transcript

1. @BastianHofmann Creating a fast Kubernetes Development Workflow Bastian Hofmann
3. Container orchestration platform
4. Deploy, run and scale your services in isolated containers
5. No vendor lock in
6. Runs on
7. Your laptop
9. Bare metal
10. Cloud Providers
11. And if you don't want to install and maintain Kubernetes yourself
12. Managed Kubernetes
14. Standardized APIs
15. It works the same everywhere*
16. It works the same everywhere* *mostly
17. This talk is about how to use Kubernetes
18. Not only for production workloads
19. But in your development workflows
20. Agenda
21. Deployment of a micro-service application
22. Some tools to help with local development of this application on Kubernetes
23. Let's have a look at the sample application
25. OpenStack Cloud LoadBalancer NGINX Ingress Controller NGINX Ingress Controller NGINX Ingress Controller web-application web-application MySQL Master MySQL Slave quote-svc quote-svc hello-svc hello-svc
26. external-dns to create DNS entries automatically
27. cert-manager to retrieve Let's Encrypt certificates automatically
28. Database is managed by an Operator
29. MySQL Operator MySQLCluster MySQL pods MySQL statefulset Kubernetes controller manager Discovers Creates Creates Discovers Monitors and manages
30. LinkerD as a Service Mesh for Telemetry
32. If you are interested in the code and how to set it up: https:/ /github.com/syseleven/ golem-workshop
33. Demo
34. Writing this YAML files is tedious
35. YAML files are tied to a specific version and a specific environment
36. Production
37. Staging
38. Development
39. Per Development team
40. Per branch
41. Per developer
42. We need to maintain multiple, very similar YAML files with slightly different versions and configuration
43. "Templating"
44. Great tools because of standardized Kubernetes API
45. Helm
47. Allows to install applications
48. So called "charts"
49. $ helm install stable/wordpress \ --name my-blog \ --namespace blog
50. Charts can depend on other charts
51. Multiple deployments of one chart possible
52. Different release names
53. Different namespaces
54. Configuration with values
56. $ helm install stable/wordpress \ --name my-blog \ --namespace blog \ -f my-config-values.yaml
57. Writing your own charts is fairly easy
58. Scaffolding to get started
59. $ helm create quote-svc
60. $ helm install ./quote-svc \ --namespace dev-bastian \ --name dev-bastian-quote-svc \ --values dev.yaml --values bastian.yaml
61. Demo
62. Still, for development:
63. Make a code change
64. Build docker image
65. Push docker image
66. Run helm install/upgrade with new image version
67. Can this be quicker?
68. Run everything locally
69. docker-compose
70. Duplication of the definition of how to run a container
71. Inconsistencies
72. If you have a lot of services, you have to run a lot locally
73. Some services locally, some remote
74. Service Discovery
75. Not every service is exposed to the Internet
76. Shared resources with other developers?
77. Other options?
78. Tilt
79. $ tilt up
80. Watches for code changes
81. Rebuilds docker image
82. Deploys to Kubernetes
83. Sets up port-forwarding
84. Can sync changed files directly into a running container
85. Demo
86. Another approach
88. Creates a two-way proxy between the Kubernetes cluster and you
89. $ telepresence T: Starting proxy with method 'vpn-tcp'... @fhgbvx65xg|bash-3.2$ curl http://quote-svc/quote | jq '.' [ { "ID": 503, "title": "stefan sagmeister", "content": "<p>...</p>\n", "link": "https://quotesondesign.com/stefan- sagmeister-2/" } ]
90. Swap a running deployment in the cluster with a local process
91. ... or a locally running docker container
92. $ telepresence --swap-deployment quote-svc --namespace dev-flow-demo --expose 3000 --run npm run debug T: Starting proxy with method 'vpn-tcp',... T: Forwarding remote port 3000 to local port 3000.... > quote-svc@1.0.0 debug /Users/bhofmann/forge_test/quote- svc > nodemon --inspect quote-svc.js [nodemon] watching: *.* [nodemon] starting `node --inspect quote-svc.js` Debugger listening on ws://127.0.0.1:9229/83aa27ac- d879-4b50-a228-440354cca791 quote svc listening on port 3000!
93. Demo
94. Summary
95. Powerful
96. Great tooling because of common APIs
97. Especially great if you have multiple services and don't want to run everything locally
98. I just picked helm, tilt and telepresence. There is more for different use-cases.
99. mail@bastianhofmann.de https:/ /twitter.com/BastianHofmann http:/ /speakerdeck.com/u/bastianhofmann https:/ /github.com/syseleven/golem-workshop