Scaling Microservices with Message Queues, Spring Boot and Kubernetes

How do you design a service that can handle hundreds of thousands of requests?
And how do you deploy an application that scales dynamically?

In this talk, you'll learn how to design, package and deploy applications to Kubernetes. You'll understand the many configuration options available to configure your cluster and how best to tweak them to scale your application. You will learn how you can expose metrics from your app and autoscale it as you receive more traffic.

Transcript

1. scaling microservices with message queues, spring boot and kubernetes @danielepolencic
5. scalability and robustness
7. 10:00 pm 08:00 pm 06:00 pm 05:00 pm 03:00 pm 01:00 pm 10k req/s 7k req/s 3k req/s
8. how do you deal with so much traffic? !"
9. backend + frontend
10. BACKEND REST API
11. BACKEND REST API
12. BACKEND REST API
13. BACKEND REST API
14. BACKEND REST API
15. BACKEND REST API
16. BACKEND REST API
17. planning for launch
18. User
19. User BACKEND
20. API User BACKEND BACKEND BACKEND BACKEND
21. API LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND
22. API LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND
23. performance tests
24. BACKEND
25. BACKEND • Stateless • Limit: 1000 rq/s
26. BACKEND • Stateless • Limit: 1000 rq/s • Stateless • Limit: 250 rq/s
27. BACKEND RATIO 1:4
28. ready…
29. 10:00 pm 08:00 pm 06:00 pm 05:00 pm 03:00 pm 01:00 pm 10k req/s 7k req/s 3k req/s NOW
30. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND
31. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND
32. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND
33. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND
34. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
35. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
36. too little, too late
37. 10:00 pm 08:00 pm 06:00 pm 05:00 pm 03:00 pm 01:00 pm 10k req/s 7k req/s 3k req/s NOW
38. LOAD BALANCER LOAD BALANCER User User User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
39. LOAD BALANCER LOAD BALANCER User User User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
40. LOAD BALANCER LOAD BALANCER User User User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
41. LOAD BALANCER LOAD BALANCER User User User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
42. LOAD BALANCER LOAD BALANCER User User User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
43. LOAD BALANCER LOAD BALANCER User User User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
44. LOAD BALANCER LOAD BALANCER User User User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
45. game over
46. nice story, but we know how to do scaling. — Smart guy
53. the retailer reportedly didn't have enough servers to handle the traffic surge for the day — fortune.com
54. amazon had to manually add servers to address the issue (and failed to add them fast enough) — fortune.com
55. what could you improve?
56. 1. automatic scaling?
57. 2. decoupling scaling?
58. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND RATIO 1:4
59. LOAD BALANCER LOAD BALANCER BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND User RATIO 1:4
60. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND RATIO 1:4
61. LOAD BALANCER LOAD BALANCER User RATIO 1:2 BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
62. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND RATIO 1:2 BACKEND BACKEND BACKEND BACKEND
63. 3. decoupling failures?
64. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
65. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
66. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
67. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
68. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
69. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
70. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
71. LOAD BALANCER LOAD BALANCER User BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND BACKEND
72. what if…
73. message broker
74. User BACKEND
75. User BACKEND
76. User BACKEND
77. User BACKEND
78. User BACKEND
79. User BACKEND
80. User BACKEND
81. User BACKEND
82. User BACKEND
83. User BACKEND
84. User BACKEND
85. User BACKEND
86. User BACKEND
87. scale idependently
88. User BACKEND
89. User BACKEND BACKEND BACKEND BACKEND
90. User BACKEND BACKEND BACKEND BACKEND
91. can handle failures
92. User BACKEND
93. what about automatic scaling?
94. Mesos
96. 1. open source
97. 1. open source 2. multi-cloud
98. 1. open source 2. multi-cloud 3. designed to scale
99. scale idependently can handle failures automatic scaling
100. scale idependently can handle failures automatic scaling
101. scale idependently can handle failures automatic scaling
102. scale idependently can handle failures automatic scaling
103. sounds like a plan
104. demo
105. the app
109. containers, containers, containers
111. JAR XML +
112. JAR XML JCE + +
113. JAR XML JCE + + CONTAINER
115. deploying apps in kubernetes
116. FRONTEND FRONTEND BACKEND Incoming traffic
117. FRONTEND FRONTEND BACKEND Incoming traffic
118. FRONTEND FRONTEND BACKEND ACTIVEMQ Incoming traffic
119. FRONTEND FRONTEND BACKEND ACTIVEMQ Incoming traffic Autoscaling Group
120. FRONTEND FRONTEND Front-end service BACKEND Backend service Queue service ACTIVEMQ Incoming traffic Autoscaling Group
121. FRONTEND Pod 1 FRONTEND Pod 2 Front-end service BACKEND Pod 3 Backend service ACTIVEMQ Pod 4 Incoming traffic Queue service Autoscaling Group
122. FRONTEND Pod 1 FRONTEND Pod 2 Front-end service Horizontal Pod Autoscaling BACKEND Pod 3 Backend service ACTIVEMQ Pod 4 Incoming traffic Queue service
124. more autoscaling
125. 1. cluster autoscaling
135. 2. many flavours
138. 3. keda
140. thanks
141. https://learnk8s.io/blog /scaling-spring-boot-microservices https://github.com/ learnk8s/spring-boot-k8s-hpa
146. questions?