Make Scalable MicroProfile Application

how to use MicroProfile Metrics as a threshold of scaling Kubernetes applications by using Kubernetes HorizontalPodAutoscaler and Prometheus Adapter

1. © 2022 Fujitsu Limited Kenji Kazumura @kkzr 2022.10.26 Make Scalable MicroProfile Application
2. Agenda © 2022 Fujitsu Limited Cloud Native and Scalability Kubernetes AutoScaler MicorProfile Metrics K8S HPA with MicroProfile Metrics Wrap Up 2
3. Who Am I © 2022 Fujitsu Limited Work for Fujitsu • FUJITSU Software Enterprise Application Platform • Launcher Member of Jakarta EE SC Member of JCP Executive Committee Board of Director of Eclipse Foundation 3
4. Agenda © 2022 Fujitsu Limited Cloud Native and Scalability Kubernetes AutoScaler MicorProfile Metrics K8S HPA with MicroProfile Metrics Wrap Up 4
5. Cloud Native (CNCF definition) © 2022 Fujitsu Limited https://github.com/cncf/toc/blob/master/DEFINITION.md Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. 5
6. Jevons Paradox © 2022 Fujitsu Limited https://upload.wikimedia.org/wikipedia/commons/b/b8/Jev onsParadoxA.png technological improvements that increased the efficiency of coal-use led to the increased consumption of coal in a wide range of industries -- Wikipedia https://en.wikipedia.org/wiki/Jevons_paradox 6
7. Step Define thresholds of metrics (When do you want to scale?) General Steps to scale © 2022 Fujitsu Limited Step (if needed) Export metrics (e.g. CPU usage, Memory usage, …) Step Monitor metrics Step Scale your VMs, Containers, Applications 7
8. Step Define thresholds of metrics (When do you want to scale?) General Steps to scale © 2022 Fujitsu Limited Step (if needed) Export metrics (e.g. CPU usage, Memory usage, …) Step Monitor metrics Step Scale your VMs, Containers, Applications 8 Step3 and Step4 are usually required to be automated. Nobody wants to monitor metrics with your own eyes. Nobody wants to scale manually.
9. Agenda © 2022 Fujitsu Limited Cloud Native and Scalability Kubernetes AutoScaler MicorProfile Metrics K8S HPA with MicroProfile Metrics Wrap Up 9
10. Kubernetes Kubernetes is a de fact standard container orchestration tool You can use it in your own infrastructures It is also available as managed service provided by cloud vendors Kubernetes provides automatic scaling methods © 2022 Fujitsu Limited 10
11. Step Define thresholds Steps to scale with Kubernetes © 2022 Fujitsu Limited Step Export metrics Step Monitor metrics Step Scale containers You need to do Kubernetes takes care of Kubernetes takes care of Kubernetes takes care of Iff you use only Node/Pod information e.g. CPU , Memory 11
12. Types of Scalability © 2022 Fujitsu Limited Pod CPU/Core Memory # of Pods Pod Pod Vertical Scale Horizontal Scale 12
13. Kubernetes Pod AutoScaler Vertical Pod AutoScaler (VPA) •need not consider special application design •less reliability Horizontal Pod AutoScaler (HPA) •need design your application to work with multi instances •easy to scale © 2022 Fujitsu Limited 13
14. HPA System Diagram © 2022 Fujitsu Limited Horizontal Pod AutoScaler poll /apis/metrics.k8s.io/v1beta1 Metrics API API Server Metrics Server push Pod poll 14
15. Agenda © 2022 Fujitsu Limited Cloud Native and Scalability Kubernetes AutoScaler MicorProfile Metrics K8S HPA with MicroProfile Metrics Wrap Up 15
16. Export monitoring data (aka Telemetry) • Access metrics endpoints (e.g. http://example.com/metrics) 3 types of metrics • Base metrics • Application metrics • Vendor Specific metrics Provide Java APIs • Programmers can export their data by API Will be integrate into MicroProfile OpenTelemtry ? MicroProfile Metrics © 2022 Fujitsu Limited https://download.eclipse.org/microprofile/microprofile-metrics-4.0/microprofile-metrics-spec-4.0.html 16
17. Code Example © 2022 Fujitsu Limited import org.eclipse.microprofile.metrics.Counter; import org.eclipse.microprofile.metrics.annotation.Metric; @Path(“eclipsecon") @RequestScoped public class LoginService { @Inject @Metric(name="LoginCounter", absolute=true) Counter counter; 17
18. Code Example © 2022 Fujitsu Limited @GET @Path("login") public String increment() { counter.inc(); return "number of login is " + counter.getCount() + "¥n"; } @GET @Path("logout") public String decrement() { if (counter.getCount() > 0) counter.inc(-1); return "number of login is " + counter.getCount() + "¥n"; } 18
19. Demo © 2022 Fujitsu Limited https://microprofile.io/compatible/5-0/ Access MP Metrics endpoint Use Launcher • MicroProfile 5.0 implementation • https://github.com/fujitsu/launcer 19
20. Agenda © 2022 Fujitsu Limited Cloud Native and Scalability Kubernetes AutoScaler MicorProfile Metrics K8S HPA with MicroProfile Metrics Wrap Up 20
21. HPA scenarios © 2022 Fujitsu Limited login do some work monitor CPU/memory usage scale login do some work monitor # of login users scale Use default API and Metrics server Use external API and MP Metrics 21
22. HPA /w MP Metrics © 2022 Fujitsu Limited Horizontal Pod AutoScaler poll /apis/external.metrics.k8s.io/v1beta1 External Metrics API API Server push Pod poll Prometheus Server Prometheus Adapter poll 22
23. Step • create your application pod and deployment of K8S • set Prometheus scraping information Steps to build HPA systems © 2022 Fujitsu Limited Step • create external API for your MP application metric using Prometheus Adapter Step • set HPA configuration # of replicas, metrics name, target threshold, and so on 23
24. Demo (before scale out) © 2022 Fujitsu Limited ・・・ Login Application (K8S cluster) Total # of Logins Pod #2 Pod #1 counter compare threshold (when scale?) http counter HPA 24
25. Demo (after scale out) © 2022 Fujitsu Limited ・・・ Login Application (K8S cluster) Total # of Logins Pod #2 Pod #1 counter compare threshold (when scale?) Pod #3 add extra pod http counter counter HPA 25
26. Agenda © 2022 Fujitsu Limited Cloud Native and Scalability Kubernetes AutoScaler MicorProfile Metrics K8S HPA with MicroProfile Metrics Wrap Up 26
27. ⚫ Scalability is one of important keys of Cloud Native applications ⚫ Kubernets provides scaling mechanism • pods and nodes metrics can be used as default • but they are not always enough ⚫ MP Metrics provides standard way to handle metrics ⚫ Collaboration of K8S and MP Metrics produces values • flexible scalability according to application metrics • easy to scale your MP applications Wrap Up © 2022 Fujitsu Limited 27
28. © 2022 Fujitsu Limited 28
29. Thank you © 2022 Fujitsu Limited