Elastic scaling in a (micro)service oriented architecture

Splitting an application up into multiple independent services can be a good way to keep it scaling and ensure stability and developer productivity in larger, growing teams. But just splitting the codebase, creating APIs and deploying the code on some servers is not enough, somehow your services need to know where and how other services are accessible. Classical approaches like hardcoding everything in every service or having a central load-balancer can quickly lead to problems in terms of scalability and maintainability. In this talk I'll show how we at ResearchGate tackled this challenge. With the help of tools like Consul and haproxy we created a setup that allows us to quickly boot and shutdown services. This ensures that all servers are utilized optimally and load spikes can be reacted upon quickly and automatically.

Transcript

1. Scaling microservices with Kubernetes @BastianHofmann
2. Microservices
3. Monolith
4. http://blog.philipphauer.de/microservices-nutshell-pros-cons/ Monolith Microservices
5. Benefits
6. Stricter separation of concerns
7. Diverse technology stacks
8. Things that you don’t want to do in language X
9. Problems
10. Problems
11. Challenges
12. Performance
13. Latency
14. Stability
15. Reliability
16. Monitoring
17. Learning Curves
18. Code Reuse
19. Maintenance
20. Technical scalability
21. Cultural scalability
22. This is the story how we are trying to solve these
26. 13 million users
27. 193 countries
28. ~1800 request/s
29. lots of data
30. ~ 140 components
31. ~ 400 repositories
32. https://www.flickr.com/photos/npobre/2601582256/
33. Key parts in a scalable micro service architecture
34. Deployment
35. Configuration Management
36. Service Discovery
37. Diverse technology stacks
38. The same for every service
39. Deployment
40. How to get the services on our servers?
41. One Click Deployment
42. Availability
43. Zero Downtime Deployments
44. Server Server Server Server
45. Server Server Server Server
46. Server Server Server Server
47. Server Server Server Server
48. Server Server Server Server
49. Server Server Server Server
50. Fast deployments
51. Fast rollbacks
52. •Ansible •Capistrano •Saltstack •Custom •….
53. https://www.flickr.com/photos/40987321@N02/5580348753/
54. Different libraries, packages, web servers, configurations, versions
55. •Chef •Puppet •Ansible •Docker •….
56. https://www.docker.com/
57. Configuration Management
58. How do I synchronize configuration over services?
59. [ "db_user": "user", "db_pw": "pw", "serviceA": "serviceA.local:8018" ]
60. Config file on disk
61. Duplication
62. Inconsistencies
63. Maintenance
64. Consul https://www.consul.io/
65. Consul Server Consul Server Consul Server Consul Agent ver Consul Agent Server Consul Agent Server Co Ag Server
66. Key/Value Store
67. $kv->put('test/foo/bar', 'bazinga'); $kv->get('test/foo/bar', ['raw' => true]); $kv->delete('test/foo/bar');
68. Credentials
69. $kv->put('test/db/pw', 'secret_pw');
70. https://www.vaultproject.io/
71. Cycling of credentials
72. Service Discovery
73. How does one service know where another service is?
74. Server Service A Server Service B Service C Service C
75. Central load balancer
76. HAproxy http://www.haproxy.org/
77. Server Service A Server Service B Service C Service C Load balancer
78. Health checks
79. GET /health HTTP/1.1 Host: serviceA.local HTTP/1.1 200 OK
80. Scalability?
81. Load balancer
82. Load balancer
83. Consul https://www.consul.io/
84. Consul Server Consul Server Consul Server Consul Agent ver Consul Agent Server Consul Agent Server Co Ag Server
85. Consul for Service Discovery
86. Consul Agent Server Service A Registration Health check
88. Load balance directly in the client
89. Consul API
90. $ curl http://localhost:8500/v1/catalog/service/refind- service [ { "ServicePort": 10780, "ServiceAddress": "", "ServiceTags": [ "env:rg_dev", "protocol:http" ], "ServiceName": "refind-service", "ServiceID": "refind-service", "Address": "172.20.4.61", "Node": "refind-1.ipbl.rgoffice.net" }, { "ServicePort": 10780, "ServiceAddress": "", "ServiceTags": [ "env:rg_dev", "protocol:http"
91. DNS
92. $ dig -p 8600 @localhost refind- service.service.rgoffice.consul. ANY ; <<>> DiG 9.9.5-3ubuntu0.11-Ubuntu <<>> -p 8600 @localhost refind-service.service.rgoffice.consul. ANY ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19315 ;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; WARNING: recursion requested but not available ;; QUESTION SECTION: ;refind-service.service.rgoffice.consul. IN ANY ;; ANSWER SECTION: refind-service.service.rgoffice.consul. 0 IN A 172.20.4.61 refind-service.service.rgoffice.consul. 0 IN A 172.20.4.58
94. Metrics
95. Flexible routing options
96. Circuit breakers
97. Tracing
99. Server Service A Server Service B Service C Service C Linkerd Consul
100. Single Point of Failure
101. Server Service A Server Service B Service C Service C Linkerd Consul Linkerd Consul
102. So we found solutions to these challenges
103. Lots of manual work
104. Vendor lock in
106. Very Powerful
107. Learning curve
108. Solves a lot of the before mentioned challenges out of the box
109. Runs on
110. AWS
111. Azure
112. Google Cloud Platform
113. Bare metal
114. Your laptop
115. Kubernetes Cluster
116. Image • A docker image built from a Dockerfile that contains everything a service needs to run
117. • A container runs a docker image. • Only 1 process can run inside of a container Container
118. Pod • A group of 1 or more containers • Same port space • Ports are not accessible from outside of the pod
119. Replica Set • Defines and manages how many instances of a pod should run
120. Running our services
121. Deployment • Manages updates and rollbacks of replica sets
122. Deployments
123. Service • Makes a port of a pod accessible to other pods
124. Ingress • Makes a service accessible to the outside of Kubernetes
125. Service discovery
126. Node • A physical server • Containers get distributed automatically
127. ConfigMaps & Secrets • Configuration that can be mounted inside of a container
128. Volumes • Volumes can be mounted into a container to access a ConfigMap, Secret or a folder on the host
129. Configuration management
130. Namespaces • Dedicated environment to deploy services in
131. Example
132. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
133. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
134. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER CONFIG WEB :80 PHP Application POD PHP Application POD
135. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER CONFIG WEB :80 https://php-app.k8s.foo.com:443/ PHP Application POD PHP Application POD
136. A bit of code
137. FROM node:7 WORKDIR /opt/appmiral ADD . /opt/appmiral RUN apt-get install -y curl git && \ npm install bower@latest -g && npm install grunt@latest -g && \ npm install && bower install --allow- root && grunt build EXPOSE 9012 CMD node /opt/appmiral/dist/server.js
138. docker build -t your-registry/researchgate/ appmiral . docker push your-registry/researchgate/ apprmial
139. ApiVersion: extensions/v1beta1 kind: Deployment metadata: name: appmiral spec: replicas: 2 template: spec: containers: - name: appmiral image: your-registry/researchgate/appmiral resources: requests: cpu: 1 memory: 200Mi env: - name: NODE_ENV value: "production" ports: - containerPort: 9012 livenessProbe: httpGet: path: /health port: 9012
140. - name: appmiral image: your-registry/researchgate/appmiral resources: requests: cpu: 1 memory: 200Mi env: - name: NODE_ENV value: "production" ports: - containerPort: 9012 livenessProbe: httpGet: path: /health port: 9012
141. kind: Service apiVersion: v1 metadata: name: appmiral spec: ports: - name: http port: 9012 targetPort: 9012 protocol: TCP selector: app: appmiral
142. apiVersion: extensions/v1beta1 kind: Ingress metadata: name: appmiral-ing spec: rules: - host: appmiral.kluster-01.rgoffice.net http: paths: - path: / backend: serviceName: appmiral servicePort: 9012
143. kubectl create -f k8s_appmiral.yaml
144. Rolling Deployments
145. Service Discovery
146. Service Virtual IP address
147. Environment Variables
148. APPMIRAL_SERVICE_HOST=10.0.162.149 APPMIRAL_SERVICE_PORT=9012
149. DNS
150. $ nslookup appmiral Server: 10.0.0.10 Address 1: 10.0.0.10 Name: appmiral Address 1: 10.0.162.149
151. LinkerD in Kubernetes
152. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
154. Tooling
155. kubectl
156. REST API
158. Helm The package manager for Kubernetes https://helm.sh/
159. Draft Smart scaffolding https://github.com/Azure/draft
161. Scaling
162. Manual Scaling
163. kubectl scale --replicas=3 deployment/my-app
164. AutoScaling
166. https://kubernetes.io/docs/user-guide/horizontal-pod- autoscaling/
167. https://www.flickr.com/photos/darkdwarf/19701555974/
168. http://speakerdeck.com/u/bastianhofmann
169. http://twitter.com/BastianHofmann http://lanyrd.com/people/BastianHofmann http://speakerdeck.com/u/bastianhofmann mail@bastianhofmann.de