Make Scalable MicroProfile Application

how to use MicroProfile Metrics as a threshold of scaling Kubernetes applications by using Kubernetes HorizontalPodAutoscaler and Prometheus Adapter

Transcript

1. © 2022 Fujitsu Limited
   Kenji Kazumura
   @kkzr
   2022.10.26
   Make Scalable
   MicroProfile
   Application
   

   View Slide

2. Agenda
   © 2022 Fujitsu Limited
   Cloud Native and Scalability
   Kubernetes AutoScaler
   MicorProfile Metrics
   K8S HPA with MicroProfile Metrics
   Wrap Up
   2
   

   View Slide

3. Who Am I
   © 2022 Fujitsu Limited
   Work for Fujitsu
   • FUJITSU Software Enterprise Application Platform
   • Launcher
   Member of Jakarta EE SC
   Member of JCP Executive Committee
   Board of Director of Eclipse Foundation
   3
   

   View Slide

4. Agenda
   © 2022 Fujitsu Limited
   Cloud Native and Scalability
   Kubernetes AutoScaler
   MicorProfile Metrics
   K8S HPA with MicroProfile Metrics
   Wrap Up
   4
   

   View Slide

5. Cloud Native (CNCF definition)
   © 2022 Fujitsu Limited
   https://github.com/cncf/toc/blob/master/DEFINITION.md
   Cloud native technologies empower
   organizations to build and run scalable
   applications in modern, dynamic environments
   such as public, private, and hybrid clouds.
   Containers, service meshes, microservices,
   immutable infrastructure, and declarative APIs
   exemplify this approach.
   5
   

   View Slide

6. Jevons Paradox
   © 2022 Fujitsu Limited
   https://upload.wikimedia.org/wikipedia/commons/b/b8/Jev
   onsParadoxA.png
   technological improvements that increased the efficiency of
   coal-use led to the increased consumption of coal in a wide
   range of industries -- Wikipedia
   https://en.wikipedia.org/wiki/Jevons_paradox
   6
   

   View Slide

7. Step
   Define thresholds of metrics (When do you want to scale?)
   General Steps to scale
   © 2022 Fujitsu Limited
   Step (if needed)
   Export metrics (e.g. CPU usage, Memory usage, …)
   Step
   Monitor metrics
   Step
   Scale your VMs, Containers, Applications
   7
   

   View Slide

8. Step
   Define thresholds of metrics (When do you want to scale?)
   General Steps to scale
   © 2022 Fujitsu Limited
   Step (if needed)
   Export metrics (e.g. CPU usage, Memory usage, …)
   Step
   Monitor metrics
   Step
   Scale your VMs, Containers, Applications
   8
   Step3 and Step4 are usually required to
   be automated.
   Nobody wants to monitor metrics with
   your own eyes.
   Nobody wants to scale manually.
   

   View Slide

9. Agenda
   © 2022 Fujitsu Limited
   Cloud Native and Scalability
   Kubernetes AutoScaler
   MicorProfile Metrics
   K8S HPA with MicroProfile Metrics
   Wrap Up
   9
   

   View Slide

10. Kubernetes
    Kubernetes is a de fact standard container
    orchestration tool
    You can use it in your own infrastructures
    It is also available as managed service provided by
    cloud vendors
    Kubernetes provides automatic scaling methods
    © 2022 Fujitsu Limited
    10
    

    View Slide

11. Step
    Define thresholds
    Steps to scale with Kubernetes
    © 2022 Fujitsu Limited
    Step
    Export metrics
    Step
    Monitor metrics
    Step
    Scale containers
    You need to do
    Kubernetes takes care of
    Kubernetes takes care of
    Kubernetes takes care of
    Iff you use only Node/Pod information
    e.g. CPU , Memory
    11
    

    View Slide

12. Types of Scalability
    © 2022 Fujitsu Limited
    Pod
    CPU/Core
    Memory
    # of Pods
    Pod Pod
    Vertical Scale
    Horizontal Scale
    12
    

    View Slide

13. Kubernetes Pod AutoScaler
    Vertical Pod AutoScaler (VPA)
    •need not consider special application design
    •less reliability
    Horizontal Pod AutoScaler (HPA)
    •need design your application to work with
    multi instances
    •easy to scale
    © 2022 Fujitsu Limited
    13
    

    View Slide

14. HPA System Diagram
    © 2022 Fujitsu Limited
    Horizontal Pod AutoScaler
    poll
    /apis/metrics.k8s.io/v1beta1
    Metrics API
    API Server
    Metrics Server
    push
    Pod
    poll
    14
    

    View Slide

15. Agenda
    © 2022 Fujitsu Limited
    Cloud Native and Scalability
    Kubernetes AutoScaler
    MicorProfile Metrics
    K8S HPA with MicroProfile Metrics
    Wrap Up
    15
    

    View Slide

16. Export monitoring data (aka Telemetry)
    • Access metrics endpoints (e.g. http://example.com/metrics)
    3 types of metrics
    • Base metrics
    • Application metrics
    • Vendor Specific metrics
    Provide Java APIs
    • Programmers can export their data by API
    Will be integrate into MicroProfile OpenTelemtry ?
    MicroProfile Metrics
    © 2022 Fujitsu Limited
    https://download.eclipse.org/microprofile/microprofile-metrics-4.0/microprofile-metrics-spec-4.0.html
    16
    

    View Slide

17. Code Example
    © 2022 Fujitsu Limited
    import org.eclipse.microprofile.metrics.Counter;
    import org.eclipse.microprofile.metrics.annotation.Metric;
    @Path(“eclipsecon")
    @RequestScoped
    public class LoginService {
    @Inject
    @Metric(name="LoginCounter", absolute=true)
    Counter counter;
    17
    

    View Slide

18. Code Example
    © 2022 Fujitsu Limited
    @GET
    @Path("login")
    public String increment() {
    counter.inc();
    return "number of login is " + counter.getCount() + "¥n";
    }
    @GET
    @Path("logout")
    public String decrement() {
    if (counter.getCount() > 0)
    counter.inc(-1);
    return "number of login is " + counter.getCount() + "¥n";
    }
    18
    

    View Slide

19. Demo
    © 2022 Fujitsu Limited
    https://microprofile.io/compatible/5-0/
    Access MP Metrics endpoint
    Use Launcher
    • MicroProfile 5.0 implementation
    • https://github.com/fujitsu/launcer
    19
    

    View Slide

20. Agenda
    © 2022 Fujitsu Limited
    Cloud Native and Scalability
    Kubernetes AutoScaler
    MicorProfile Metrics
    K8S HPA with MicroProfile Metrics
    Wrap Up
    20
    

    View Slide

21. HPA scenarios
    © 2022 Fujitsu Limited
    login do some work
    monitor CPU/memory usage
    scale
    login do some work
    monitor # of login users
    scale
    Use default API and Metrics server
    Use external API and MP Metrics
    21
    

    View Slide

22. HPA /w MP Metrics
    © 2022 Fujitsu Limited
    Horizontal Pod AutoScaler
    poll
    /apis/external.metrics.k8s.io/v1beta1
    External Metrics API
    API Server
    push
    Pod
    poll
    Prometheus Server
    Prometheus Adapter
    poll
    22
    

    View Slide

23. Step
    • create your application pod and deployment of K8S
    • set Prometheus scraping information
    Steps to build HPA systems
    © 2022 Fujitsu Limited
    Step
    • create external API for your MP application metric
    using Prometheus Adapter
    Step
    • set HPA configuration
    # of replicas, metrics name, target threshold, and so on
    23
    

    View Slide

24. Demo (before scale out)
    © 2022 Fujitsu Limited
    ・・・
    Login Application (K8S cluster)
    Total # of Logins
    Pod #2
    Pod #1
    counter
    compare
    threshold
    (when scale?)
    http
    counter
    HPA
    24
    

    View Slide

25. Demo (after scale out)
    © 2022 Fujitsu Limited
    ・・・
    Login Application (K8S cluster)
    Total # of Logins
    Pod #2
    Pod #1
    counter
    compare
    threshold
    (when scale?)
    Pod #3
    add extra pod
    http
    counter
    counter
    HPA
    25
    

    View Slide

26. Agenda
    © 2022 Fujitsu Limited
    Cloud Native and Scalability
    Kubernetes AutoScaler
    MicorProfile Metrics
    K8S HPA with MicroProfile Metrics
    Wrap Up
    26
    

    View Slide

27. ⚫ Scalability is one of important keys of Cloud Native
    applications
    ⚫ Kubernets provides scaling mechanism
    • pods and nodes metrics can be used as default
    • but they are not always enough
    ⚫ MP Metrics provides standard way to handle metrics
    ⚫ Collaboration of K8S and MP Metrics produces values
    • flexible scalability according to application metrics
    • easy to scale your MP applications
    Wrap Up
    © 2022 Fujitsu Limited
    27
    

    View Slide

28. © 2022 Fujitsu Limited
    28
    

    View Slide

29. Thank you
    © 2022 Fujitsu Limited
    

    View Slide