Elastic Scaling of (Micro)services with Kubernetes

Splitting an application up into multiple independent services can be a good way to keep it scaling and ensure stability and developer productivity in larger, growing teams. But just splitting the codebase, creating APIs and deploying the code on some servers is not enough, somehow your services need to know where and how other services are accessible. Classical approaches like hardcoding everything in every service or having a central load-balancer can quickly lead to problems in terms of scalability and maintainability. In this talk I’ll show how we at ResearchGate tackled this challenge. With the help of tools like Consul, linkerd and Kubernetes we created a setup that allows us to quickly boot and shutdown services. This ensures that all servers are utilised optimally and load spikes can be reacted upon quickly and automatically.

Transcript

1. Elastic Scaling of (Micro)services with Kubernetes @BastianHofmann
3. Microservices
4. Monolith
5. http://blog.philipphauer.de/microservices-nutshell-pros-cons/ Monolith Microservices
6. Benefits
7. Stricter separation of concerns
8. Diverse technology stacks
9. Things that you don’t want to do in language X
10. Problems
11. Problems
12. Challenges
13. Performance
14. Latency
15. Stability
16. Reliability
17. Transparency
18. Monitoring
19. Learning Curves
20. Knowledge silos
21. Code Reuse
22. Maintenance
23. To sum it up
24. Technical scalability
25. Cultural scalability
26. How to solve these
27. Diverse technology stacks
28. The same solution for every service
31. 11 million users
32. 193 countries
33. ~1800 request/s
34. lots of data
35. >100 million publications
36. ~ 140 components
37. ~ 400 repositories
38. https://www.flickr.com/photos/npobre/2601582256/
39. Key parts in a scalable micro service architecture
40. Deployment
41. Running the service
42. Configuration Management
43. Service Discovery
44. Deployment
45. How to get the services on our servers?
46. One Click Deployment
47. Availability
48. Zero Downtime Deployments
49. Server Server Server Server
50. Stability
51. Canary environments
52. Server Server Server Server Test with low amount of traffic
53. Fast deployments
54. Fast rollbacks
55. Diverse technology stacks
56. https://www.flickr.com/photos/40987321@N02/5580348753/
57. Different libraries, packages, web servers, configurations, versions
58. https://www.docker.com/
59. Running the service
60. How do I stop and start a service and ensure it keeps running?
61. Diverse technology stacks
62. Configuration Management
63. How do I synchronize configuration over services?
64. Diverse technology stacks
65. Duplication
66. Inconsistencies
67. Service Discovery
68. How does one service know where another service is?
69. Server Service A Server Service B Service C Service C
70. Configuration
71. $config = [ 'serviceA' => [ '192.168.0.1:8001', '192.168.0.2:8001', ], 'serviceB' => [ '192.168.0.1:8002', ], 'serviceC' => [ '192.168.0.2:8003', ] ];
72. Service/Server down?
73. $config = [ 'serviceA' => [ '192.168.0.1:8001', '192.168.0.2:8001', ], 'serviceB' => [ '192.168.0.1:8002', ], 'serviceC' => [ '192.168.0.2:8003', ] ];
75. Very Powerful
76. Learning curve
77. Kubernetes Cluster
78. Image • A docker image built from a Dockerfile that contains everything a service needs to run
79. • A container runs a docker image. • Only 1 process can run inside of a container Container
80. Pod • A group of 1 or more containers • Same port space • Ports are not accessible from outside of the pod
81. Replica Set • Defines and manages how many instances of a pod should run
82. Deployment • Manages updates and rollbacks of replica sets
83. Service • Makes a port of a pod accessible to other pods
84. Ingress • Makes a service accessible to the outside of Kubernetes
85. Node • A physical server • Containers get distributed automatically
86. ConfigMaps & Secrets • Configuration that can be mounted inside of a container
87. Volumes • Volumes can be mounted into a container to access a ConfigMap, Secret or a folder on the host
88. Namespaces • Dedicated environment to deploy services in
89. Example
90. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
91. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
92. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER CONFIG WEB :80 PHP Application POD PHP Application POD
93. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER ReplicaSet: 2 instances PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER CONFIG WEB :80 https://php-app.k8s.foo.com:443/ PHP Application POD PHP Application POD
94. FROM node:7 WORKDIR /opt/appmiral ADD . /opt/appmiral RUN apt-get install -y curl git && \ npm install bower@latest -g && npm install grunt@latest -g && \ npm install && bower install --allow- root && grunt build EXPOSE 9012 CMD node /opt/appmiral/dist/server.js
95. docker build -t appmiral . docker run appmiral
96. ApiVersion: extensions/v1beta1 kind: Deployment metadata: name: appmiral spec: replicas: 2 template: spec: containers: - name: appmiral image: your-registry/yourcompany/appmiral resources: requests: cpu: 1 memory: 200Mi env: - name: NODE_ENV value: "production" ports: - containerPort: 9012 livenessProbe: httpGet: path: /health port: 9012
97. - name: appmiral image: your-registry/researchgate/appmiral resources: requests: cpu: 1 memory: 200Mi env: - name: NODE_ENV value: "production" ports: - containerPort: 9012 livenessProbe: httpGet: path: /health port: 9012
98. kind: Service apiVersion: v1 metadata: name: appmiral spec: ports: - name: http port: 9012 targetPort: 9012 protocol: TCP selector: app: appmiral
99. apiVersion: extensions/v1beta1 kind: Ingress metadata: name: appmiral-ing spec: rules: - host: appmiral.kluster-01.rgoffice.net http: paths: - path: / backend: serviceName: appmiral servicePort: 9012
100. kubectl create -f k8s_appmiral.yaml
101. Rolling Deployments
102. kubectl
103. REST API
105. Helm The package manager for Kubernetes https://helm.sh/
107. Service Discovery
108. Service Virtual IP address
109. Environment Variables
110. APPMIRAL_SERVICE_HOST=10.0.162.149 APPMIRAL_SERVICE_PORT=80
111. DNS
112. $ nslookup appmiral Server: 10.0.0.10 Address 1: 10.0.0.10 Name: appmiral Address 1: 10.0.162.149
113. LinkerD in Kubernetes
115. Metrics
116. Tracing
117. Circuit breakers
118. Flexible routing options
120. PHP-FPM NGINX LINKERD STATSD MEM CACHED MONGO ROUTER PHP Application POD
122. Manual Scaling
123. kubectl scale --replicas=3 deployment/my-app
124. AutoScaling
126. https://kubernetes.io/docs/user-guide/horizontal-pod- autoscaling/
127. http://speakerdeck.com/u/bastianhofmann
128. http://twitter.com/BastianHofmann http://lanyrd.com/people/BastianHofmann http://speakerdeck.com/u/bastianhofmann mail@bastianhofmann.de