Elastic scaling with Kubernetes in a (micro)service oriented architecture

Splitting an application up into multiple independent services can be a good way to keep it scaling and ensure stability and developer productivity in larger, growing teams. But just splitting the codebase, creating APIs and deploying the code on some servers is not enough, somehow your services need to know where and how other services are accessible. Classical approaches like hardcoding everything in every service or having a central load-balancer can quickly lead to problems in terms of scalability and maintainability. In this talk I'll show how we at ResearchGate tackled this challenge. With the help of tools like Consul, linkerd and Kubernetes we created a setup that allows us to quickly boot and shutdown services. This ensures that all servers are utilized optimally and load spikes can be reacted upon quickly and automatically.

Transcript

1. Scaling microservices with Kubernetes @BastianHofmann
2. Microservices
3. Monolith
4. http://blog.philipphauer.de/microservices-nutshell-pros-cons/ Monolith Microservices
5. Benefits
6. Stricter separation of concerns
7. Diverse technology stacks
8. Things that you don’t want to do in language X
9. Problems
10. Problems
11. Challenges
12. Performance
13. Latency
14. Stability
15. Reliability
16. Transparency
17. Monitoring
18. Learning Curves
19. Code Reuse
20. Maintenance
21. Technical scalability
22. Cultural scalability
23. This is the story how we are trying to solve these
27. 13 million users
28. 193 countries
29. ~1800 request/s
30. lots of data
31. ~ 140 components
32. ~ 400 repositories
33. https://www.flickr.com/photos/npobre/2601582256/
34. Key parts in a scalable micro service architecture
35. Deployment
36. Running the service
37. Configuration Management
38. Service Discovery
39. Diverse technology stacks
40. The same for every service
41. Deployment
42. How to get the services on our servers?
43. One Click Deployment
44. Availability
45. Zero Downtime Deployments
46. Server Server Server Server
47. Server Server Server Server
48. Server Server Server Server
49. Server Server Server Server
50. Server Server Server Server
51. Server Server Server Server
52. Stability
53. Canary environments
54. Server Server Server Server
55. Server Server Server Server Test with low amount of traffic
56. Fast deployments
57. Fast rollbacks
58. •Ansible •Capistrano •Saltstack •Custom •….
59. https://www.flickr.com/photos/40987321@N02/5580348753/
60. Different libraries, packages, web servers, configurations, versions
61. •Chef •Puppet •Ansible •Docker •….
62. https://www.docker.com/
63. Running the service
64. How do I stop and start a service and ensure it keeps running?
65. •Supervisord •Upstart •S6 •Ruine •Monit •Circus •Restartd •Docker •…
66. https://www.docker.com/
67. docker run my-service
68. Configuration Management
69. How do I synchronize configuration over services?
70. [ "db_user": "user", "db_pw": "pw", "serviceA": "serviceA.local:8018" ]
71. Config file on disk
72. Duplication
73. Inconsistencies
74. Consul https://www.consul.io/
75. Consul Server Consul Server Consul Server Consul Agent ver Consul Agent Server Consul Agent Server Co Ag Server
76. Key/Value Store
77. $kv->put('test/foo/bar', 'bazinga'); $kv->get('test/foo/bar', ['raw' => true]); $kv->delete('test/foo/bar');
78. Credentials
79. $kv->put('test/db/pw', 'secret_pw');
80. https://www.vaultproject.io/
81. Cycling of credentials
82. Service Discovery
83. How does one service know where another service is?
84. Server Service A Server Service B Service C Service C
85. Central load balancer
86. HAproxy http://www.haproxy.org/
87. Server Service A Server Service B Service C Service C Load balancer
88. Health checks
89. GET /health HTTP/1.1 Host: serviceA.local HTTP/1.1 200 OK
90. Scalability?
91. Load balancer
92. Load balancer
93. Consul https://www.consul.io/
94. Consul Server Consul Server Consul Server Consul Agent ver Consul Agent Server Consul Agent Server Co Ag Server
95. Consul for Service Discovery
96. Consul Agent Server Service A Registration Health check
98. Load balance directly in the client
99. Consul API
100. $ curl http://localhost:8500/v1/catalog/service/refind- service [ { "ServicePort": 10780, "ServiceAddress": "", "ServiceTags": [ "env:rg_dev", "protocol:http" ], "ServiceName": "refind-service", "ServiceID": "refind-service", "Address": "172.20.4.61", "Node": "refind-1.ipbl.rgoffice.net" }, { "ServicePort": 10780, "ServiceAddress": "", "ServiceTags": [ "env:rg_dev", "protocol:http"
101. DNS
102. $ dig -p 8600 @localhost refind- service.service.rgoffice.consul. ANY ; <<>> DiG 9.9.5-3ubuntu0.11-Ubuntu <<>> -p 8600 @localhost refind-service.service.rgoffice.consul. ANY ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19315 ;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;refind-service.service.rgoffice.consul. IN ANY ;; ANSWER SECTION: refind-service.service.rgoffice.consul. 0 IN A 172.20.4.61 refind-service.service.rgoffice.consul. 0 IN A 172.20.4.58
104. Metrics
105. Flexible routing options
106. Circuit breakers
107. Tracing
109. Server Service A Server Service B Service C Service C Linkerd Consul
110. Single Point of Failure
111. Server Service A Server Service B Service C Service C Linkerd Consul Linkerd Consul
112. So we found solutions to these challenges
113. Lots of manual work
114. Vendor lock in
116. Very Powerful
117. Learning curve
118. Solves a lot of the before mentioned challenges out of the box
119. Runs on
120. AWS
121. Azure
122. Google Cloud Platform
123. Bare metal
124. Your laptop
125. Kubernetes Cluster
126. Image • A docker image built from a Dockerfile that contains everything a service needs to run
127. • A container runs a docker image. • Only 1 process can run inside of a container Container
128. Pod • A group of 1 or more containers • Same port space • Ports are not accessible from outside of the pod
129. Replica Set • Defines and manages how many instances of a pod should run
130. Running our services
131. Deployment • Manages updates and rollbacks of replica sets
132. Deployments
133. Service • Makes a port of a pod accessible to other pods
134. Ingress • Makes a service accessible to the outside of Kubernetes
135. Service discovery
136. Node • A physical server • Containers get distributed automatically
137. ConfigMaps & Secrets • Configuration that can be mounted inside of a container
138. Volumes • Volumes can be mounted into a container to access a ConfigMap, Secret or a folder on the host
139. Configuration management
140. Namespaces • Dedicated environment to deploy services in
141. Example
142. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
143. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
144. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER CONFIG WEB :80 PHP Application POD PHP Application POD
145. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER CONFIG WEB :80 https://php-app.k8s.foo.com:443/ PHP Application POD PHP Application POD
146. A bit of code
147. FROM node:7 WORKDIR /opt/appmiral ADD . /opt/appmiral RUN apt-get install -y curl git && \ npm install bower@latest -g && npm install grunt@latest -g && \ npm install && bower install --allow- root && grunt build EXPOSE 9012 CMD node /opt/appmiral/dist/server.js
148. docker build -t your-registry/researchgate/ appmiral . docker push your-registry/researchgate/ apprmial
149. ApiVersion: extensions/v1beta1 kind: Deployment metadata: name: appmiral spec: replicas: 2 template: spec: containers: - name: appmiral image: your-registry/researchgate/appmiral resources: requests: cpu: 1 memory: 200Mi env: - name: NODE_ENV value: "production" ports: - containerPort: 9012 livenessProbe: httpGet: path: /health port: 9012
150. - name: appmiral image: your-registry/researchgate/appmiral resources: requests: cpu: 1 memory: 200Mi env: - name: NODE_ENV value: "production" ports: - containerPort: 9012 livenessProbe: httpGet: path: /health port: 9012
151. kind: Service apiVersion: v1 metadata: name: appmiral spec: ports: - name: http port: 9012 targetPort: 9012 protocol: TCP selector: app: appmiral
152. apiVersion: extensions/v1beta1 kind: Ingress metadata: name: appmiral-ing spec: rules: - host: appmiral.kluster-01.rgoffice.net http: paths: - path: / backend: serviceName: appmiral servicePort: 9012
153. kubectl create -f k8s_appmiral.yaml
154. Rolling Deployments
155. Service Discovery
156. Service Virtual IP address
157. Environment Variables
158. APPMIRAL_SERVICE_HOST=10.0.162.149 APPMIRAL_SERVICE_PORT=80
159. DNS
160. $ nslookup appmiral Server: 10.0.0.10 Address 1: 10.0.0.10 Name: appmiral Address 1: 10.0.162.149
161. LinkerD in Kubernetes
162. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
164. Tooling
165. kubectl
166. REST API
168. Helm The package manager for Kubernetes https://helm.sh/
169. Draft Smart scaffolding https://github.com/Azure/draft
171. Scaling
172. Manual Scaling
173. kubectl scale --replicas=3 deployment/my-app
174. AutoScaling
176. https://kubernetes.io/docs/user-guide/horizontal-pod- autoscaling/
177. https://www.flickr.com/photos/darkdwarf/19701555974/
178. http://speakerdeck.com/u/bastianhofmann
179. http://twitter.com/BastianHofmann http://lanyrd.com/people/BastianHofmann http://speakerdeck.com/u/bastianhofmann mail@bastianhofmann.de