JVM warm up

#성능

Java 어플리케이션은 시작 과정에서 성능을 보장하기 위한 warm up 과정이 필요합니다. 이 세션에서는 JVM warm up이 성능에 어떻게 도움이 되는지 그리고 이와 관련된 jit compiler의 내부 절차를 소개합니다. 또한 카카오 T 계정 서버를 배포하는 과정에서 발생한 초기 응답 지연 문제를 어떻게 해결하였는지를 함께 말씀드리려고 합니다.

발표자 : greg.1020
카카오모빌리티에서 백엔드엔지니어 역할을 맡고 있는 그렉입니다. 저는 주로 Python으로 개발하는 것을 좋아하는데요, 최근에는 Java를 활용한 백엔드 개발과 React를 활용한 프론트엔드 개발에도 관심이 많습니다.

Transcript

1. JVM warm up
   이형구 greg.1020
   
   
   카카오모빌리티
   
   
   Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public display is not permitted without written permission from Kakao.
   if(kakao)2022
   

   View Slide

2. 계정 서비스
   

   View Slide

3. Java Spring Boot
   11 2
   

   View Slide

4. Byte Code
   Source Code
   Java
   Compile
   

   View Slide

5. Byte Code
   Source Code
   Java
   JVM
   Interprete
   machine code
   Compile
   

   View Slide

6. Program
   
   
   (machine code)
   Compile language
   Process
   Compile with optimization
   Source Code
   

   View Slide

7. JIT (Just In Time) Compiler
   

   View Slide

8. JIT Compiler
   
   
   Hotspot
   

   View Slide

9. User
   JIT Compiler
   JVM(runtime)
   JIT
   
   
   Compiler
   Byte
   
   
   Code
   Machine Code
   cache, optimization
   

   View Slide

10. Warm up
    application
    
    
    start interprete byte code
    cache (X)
    

    View Slide

11. Warm up
    JVM warm up
    application
    
    
    start
    cache (O)
    interprete byte code
    

    View Slide

12. 계정 서비스 배포시 발생한
    
    
    Latency 이슈
    

    View Slide

13. latency 이슈
    

    View Slide

14. 병목을 찾자
    - CPU
    
    
    - Mem
    
    
    - Network bandwidth
    
    
    - TPS
    
    
    

    View Slide

15. 병목을 찾자
    0.01%
    0.04%
    99.95%
    REDIS
    SQL
    Others
    Ratio
    
    
    By Time
    

    View Slide

16. GW
    병목을 찾자
    계정 API
    
    
    (pod)
    Thread
    
    
    10 ~ 8192
    API
    

    View Slide

17. 병목을 찾자
    계정 API
    
    
    (pod)
    Connection pool
    
    
    20~50
    RDB
    in
    -
    memory
    Connection 1
    

    View Slide

18. 병목을 찾자
    계정 API
    

    View Slide

19. Application
    
    
    Ready
    
    
    (event)
    병목을 찾자 - API 시작 과정
    Traf
    fi
    c In
    Kubernetes
    
    
    Liveness/Readiness probes
    3 seconds
    
    
    delay
    

    View Slide

20. 병목을 찾자 - API 시작 과정
    계정 API
    RDB
    in
    -
    memory
    Kubernetes
    
    
    liveness/readiness
    
    
    probe
    

    View Slide

21. 병목을 찾자 - API 시작 과정
    @GetMapping({"liveness"})
    
    
    public String health() {
    
    
    warmer();
    
    
    return "OK";
    
    
    }
    
    
    @GetMapping({"readiness"})
    
    
    public String health() {
    
    
    if (isHealthy) {
    
    
    warmer();
    
    
    return "OK";
    
    
    } else {
    
    
    throw Exception....;
    
    
    }
    
    
    }
    private void warmer() {
    
    
    userRepository.xxxx();
    
    
    agreementCacheService.xxxx();
    
    
    ...
    
    
    }
    
    
    

    View Slide

22. 분석 요약
    as
    -
    is to
    -
    be
    tomcat
    
    
    (min
    -
    spare)
    10 256
    warm up database query localhost api call
    

    View Slide

23. Application
    
    
    Ready
    
    
    (event)
    대응 - API 시작 과정
    JVM
    
    
    Warm Up
    Traf
    fi
    c In
    Kubernetes
    
    
    Liveness/Readiness Probes
    

    View Slide

24. Application
    
    
    Ready
    
    
    (event)
    JVM
    
    
    Warm Up
    Traf
    fi
    c In
    400 Bad Request
    
    
    (readiness probe)
    대응 - API 시작 과정
    

    View Slide

25. Application
    
    
    Ready
    
    
    (event)
    JVM
    
    
    Warm Up
    Traf
    fi
    c In
    200 OK
    
    
    (readiness probe)
    대응 - API 시작 과정
    

    View Slide

26. 대응 - localhost API call
    계정 API
    
    
    (pod) localhost GET API
    localhost GET API
    
    
    .
    
    
    .
    
    
    .
    

    View Slide

27. 대응 결과
    warm up
    

    View Slide

28. 2달 뒤
    

    View Slide

29. latency 추가 이슈
    

    View Slide

30. - Graal JIT
    Ideation
    

    View Slide

31. - Graal JIT
    
    
    - AOT(Ahead of time) Compile
    
    
    Ideation
    

    View Slide

32. - Graal JIT
    
    
    - AOT(Ahead of time) Compile
    
    
    - Redis Connection pool
    
    
    Ideation
    

    View Slide

33. - Graal JIT
    
    
    - AOT(Ahead of time) Compile
    
    
    - Redis Connection pool
    
    
    - Warm up count
    
    
    Ideation
    

    View Slide

34. JIT Internals
    

    View Slide

35. - Method
    
    
    JIT(Just in time)
    

    View Slide

36. - Method
    
    
    - Pro
    fi
    ling
    JIT(Just in time)
    

    View Slide

37. - Method
    
    
    - Pro
    fi
    ling
    
    
    - Tiered compilation
    
    
    JIT(Just in time)
    

    View Slide

38. - Method
    
    
    - Pro
    fi
    ling
    
    
    - Tiered compilation
    
    
    - C1: optimization
    
    
    - C2: fully optimization
    JIT(Just in time)
    

    View Slide

39. \
    C1
    Interpreter
    Code
    
    
    Cache
    Tiered Compilation
    C2
    fully
    
    
    Optimization
    1 2 3
    Optimization
    

    View Slide

40. - level 0: interpreted code
    
    
    - level 1: simple C1 compiled code
    
    
    - level 2: limited C1 compiled code
    
    
    - level 3: full C1 compiled code
    
    
    - level 4: C2 compiled code
    Compilation Level
    

    View Slide

41. - level 0: interpreted code
    
    
    - level 1: simple C1 compiled code
    
    
    - level 2: limited C1 compiled code
    
    
    - level 3: full C1 compiled code
    
    
    - level 4: C2 compiled code
    Compilation Level
    

    View Slide

42. - level 0: interpreted code
    
    
    - level 1: simple C1 compiled code
    
    
    - level 2: limited C1 compiled code
    
    
    - level 3: full C1 compiled code
    
    
    - level 4: C2 compiled code
    Compilation Level
    

    View Slide

43. \
    C1
    
    
    (level1,2,3)
    Interpreter
    
    
    (level0)
    Code
    
    
    Cache
    Tiered Compilation
    C2
    
    
    (level4)
    Deoptimization
    1 2 3
    4
    fully
    
    
    Optimization
    Optimization
    

    View Slide

44. \
    C1
    
    
    (level1,2,3)
    Interpreter
    
    
    (level0)
    Code
    
    
    Cache
    Tiered Compilation
    C2
    
    
    (level4)
    Deoptimization
    1 2 3
    4
    fully
    
    
    Optimization
    Optimization
    

    View Slide

45. -
    XX:InitialCodeCacheSize=N
    
    
    -
    XX:ReservedCodeCacheSize=N
    
    
    Code Cache
    

    View Slide

46. messge:
    
    
    CodeHeap [NAME] is full. Compiler has been disabled.
    
    
    Try increasing the code heap size using
    -
    XX:ReservedCodeCacheSize=...
    
    
    Code Cache
    

    View Slide

47. $ java
    -
    XX:+PrintFlagsFinal -version | grep Threshold | grep Tier
    
    
    Tier3InvocationThreshold
    
    
    Tier3BackEdgeThreshold
    
    
    Tier3CompileThreshold
    
    
    Tier4InvocationThreshold
    
    
    Tier4BackEdgeThreshold
    
    
    Tier4CompileThreshold
    Compilation Level Threshold
    

    View Slide

48. $ java
    -
    XX:+PrintFlagsFinal -version | grep Threshold | grep Tier
    
    
    Tier3InvocationThreshold = (default) 200
    
    
    Tier3BackEdgeThreshold = (default) 60000
    
    
    Tier3CompileThreshold = (default) 2000
    
    
    Tier4InvocationThreshold = (default) 5000
    
    
    Tier4BackEdgeThreshold = (default) 40000
    
    
    Tier4CompileThreshold = (default) 15000
    
    
    Compilation Level Threshold
    

    View Slide

49. public class Hello {
    
    
    public int findMax(int[] arr) {
    
    
    int max = 0;
    
    
    for (int i: arr) {
    
    
    max = (max > i) ? max : i;
    
    
    }
    
    
    return max;
    
    
    }
    
    
    public static void main(String[] argv) {
    
    
    var hello = new Hello();
    
    
    int[] arr = {1,2,3,4,5,6,7,8,9,10};
    
    
    IntStream.rangeClosed(1, N).forEach(i -> {
    
    
    try {
    
    
    hello.findMax(arr);
    
    
    if (i % 100 == 0) {Thread.sleep(100);}
    
    
    } catch (Exception e) {}
    
    
    });
    
    
    }
    
    
    }
    Sample Code
    N
    ->
    1, 250, 500, 1000
    

    View Slide

50. VM Options
    
    
    -
    XX:+UnlockDiagnosticVMOptions
    -
    XX:+LogCompilation
    JIT(Just in time) Log
    

    View Slide

51. Run
    
    
    hotspot_pid[PID Number].log
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    OpenJDK 64
    -
    Bit Server VM
    
    
    
    
    
    ...
    
    
    JIT(Just in time) Log
    

    View Slide

52. backedge_count='2048' iicount='205' level='3' stamp='0.683' comment='tiered'
    hot_count='205'/>
    
    
    size='1168' address='0x0000000115846f10' ... method='Hello
    fi
    ndMax ([I)I' bytes='45'
    count='250' backedge_count='2500' iicount='250' stamp='0.683'/>
    Sample JIT log
    

    View Slide

53. backedge_count='18934' iicount='1894' stamp='2.323' comment='tiered'
    hot_count='1894'/>
    
    
    size='1016' address='0x00000001285e9090' ... method='Hello
    fi
    ndMax ([I)I' bytes='45'
    count='1900' backedge_count='19000' iicount='1900' stamp='2.327'/>
    Sample JIT log
    

    View Slide

54. C1/C2 Compile Count
    warm up - 0 warm up - 5 warm up - 250 warm up - 500
    c1
    35928 42313 64314 72546
    c2 19892 27327 45932 49265
    deoptimization 420 498 578 536
    

    View Slide

55. C1/C2 Compile Count
    warm up - 0 warm up - 5 warm up - 250 warm up - 500
    c1
    35928 42313 64314 72546
    c2 19892 27327 45932 49265
    deoptimization 420 498 578 536
    

    View Slide

56. 2차 대응
    warm up
    

    View Slide

57. - Java, JIT Compiler
    
    
    - Real traf
    fi
    c API Warm up
    
    
    - JIT C2 Optimization
    정리하기
    

    View Slide

58. - Java, JIT Compiler
    
    
    - Real traf
    fi
    c API Warm up
    
    
    - JIT C2 Optimization
    정리하기
    

    View Slide

59. - Java, JIT Compiler
    
    
    - Real traf
    fi
    c API Warm up
    
    
    - JIT C2 Optimization
    정리하기
    

    View Slide

60. 감사합니다
    

    View Slide

61. 참고 문헌
    1) 벤저민 J. 에번스, 제임스 고프, 크리스 뉴랜드, 『Optimizing Java』, 이일웅 옮김, 한빛미디어, 2019
    
    
    2) Scott Oaks, 『Java Performance』, 2nd Edition, 2020, Ch4, https:/
    /www.oreilly.com/library/view/java
    -
    performance-2nd/9781492056102/ch04.html
    
    
    3) Oracle, The Java HotSpot Performance Engine Architecture, https:/
    /www.oracle.com/java/technologies/whitepaper.html
    
    
    4) OpenJDK open source, https:/
    /github.com/openjdk/jdk11u
    
    
    

    View Slide