JVM warm up

#성능

Java 어플리케이션은 시작 과정에서 성능을 보장하기 위한 warm up 과정이 필요합니다. 이 세션에서는 JVM warm up이 성능에 어떻게 도움이 되는지 그리고 이와 관련된 jit compiler의 내부 절차를 소개합니다. 또한 카카오 T 계정 서버를 배포하는 과정에서 발생한 초기 응답 지연 문제를 어떻게 해결하였는지를 함께 말씀드리려고 합니다.

발표자 : greg.1020
카카오모빌리티에서 백엔드엔지니어 역할을 맡고 있는 그렉입니다. 저는 주로 Python으로 개발하는 것을 좋아하는데요, 최근에는 Java를 활용한 백엔드 개발과 React를 활용한 프론트엔드 개발에도 관심이 많습니다.

Transcript

1. JVM warm up 이형구 greg.1020 카카오모빌리티 Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public display is not permitted without written permission from Kakao. if(kakao)2022
2. 계정 서비스
3. Java Spring Boot 11 2
4. Byte Code Source Code Java Compile
5. Byte Code Source Code Java JVM Interprete machine code Compile
6. Program (machine code) Compile language Process Compile with optimization Source Code
7. JIT (Just In Time) Compiler
8. JIT Compiler Hotspot
9. User JIT Compiler JVM(runtime) JIT Compiler Byte Code Machine Code cache, optimization
10. Warm up application start interprete byte code cache (X)
11. Warm up JVM warm up application start cache (O) interprete byte code
12. 계정 서비스 배포시 발생한 Latency 이슈
13. latency 이슈
14. 병목을 찾자 - CPU - Mem - Network bandwidth - TPS
15. 병목을 찾자 0.01% 0.04% 99.95% REDIS SQL Others Ratio By Time
16. GW 병목을 찾자 계정 API (pod) Thread 10 ~ 8192 API
17. 병목을 찾자 계정 API (pod) Connection pool 20~50 RDB in - memory Connection 1
18. 병목을 찾자 계정 API
19. Application Ready (event) 병목을 찾자 - API 시작 과정 Traf fi c In Kubernetes Liveness/Readiness probes 3 seconds delay
20. 병목을 찾자 - API 시작 과정 계정 API RDB in - memory Kubernetes liveness/readiness probe
21. 병목을 찾자 - API 시작 과정 @GetMapping({"liveness"}) public String health() { warmer(); return "OK"; } @GetMapping({"readiness"}) public String health() { if (isHealthy) { warmer(); return "OK"; } else { throw Exception....; } } private void warmer() { userRepository.xxxx(); agreementCacheService.xxxx(); ... }
22. 분석 요약 as - is to - be tomcat (min - spare) 10 256 warm up database query localhost api call
23. Application Ready (event) 대응 - API 시작 과정 JVM Warm Up Traf fi c In Kubernetes Liveness/Readiness Probes
24. Application Ready (event) JVM Warm Up Traf fi c In 400 Bad Request (readiness probe) 대응 - API 시작 과정
25. Application Ready (event) JVM Warm Up Traf fi c In 200 OK (readiness probe) 대응 - API 시작 과정
26. 대응 - localhost API call 계정 API (pod) localhost GET API localhost GET API . . .
27. 대응 결과 warm up
28. 2달 뒤
29. latency 추가 이슈
30. - Graal JIT Ideation
31. - Graal JIT - AOT(Ahead of time) Compile Ideation
32. - Graal JIT - AOT(Ahead of time) Compile - Redis Connection pool Ideation
33. - Graal JIT - AOT(Ahead of time) Compile - Redis Connection pool - Warm up count Ideation
34. JIT Internals
35. - Method JIT(Just in time)
36. - Method - Pro fi ling JIT(Just in time)
37. - Method - Pro fi ling - Tiered compilation JIT(Just in time)
38. - Method - Pro fi ling - Tiered compilation - C1: optimization - C2: fully optimization JIT(Just in time)
39. \ C1 Interpreter Code Cache Tiered Compilation C2 fully Optimization 1 2 3 Optimization
40. - level 0: interpreted code - level 1: simple C1 compiled code - level 2: limited C1 compiled code - level 3: full C1 compiled code - level 4: C2 compiled code Compilation Level
41. - level 0: interpreted code - level 1: simple C1 compiled code - level 2: limited C1 compiled code - level 3: full C1 compiled code - level 4: C2 compiled code Compilation Level
42. - level 0: interpreted code - level 1: simple C1 compiled code - level 2: limited C1 compiled code - level 3: full C1 compiled code - level 4: C2 compiled code Compilation Level
43. \ C1 (level1,2,3) Interpreter (level0) Code Cache Tiered Compilation C2 (level4) Deoptimization 1 2 3 4 fully Optimization Optimization
44. \ C1 (level1,2,3) Interpreter (level0) Code Cache Tiered Compilation C2 (level4) Deoptimization 1 2 3 4 fully Optimization Optimization
45. - XX:InitialCodeCacheSize=N - XX:ReservedCodeCacheSize=N Code Cache
46. messge: CodeHeap [NAME] is full. Compiler has been disabled. Try increasing the code heap size using - XX:ReservedCodeCacheSize=... Code Cache
47. $ java - XX:+PrintFlagsFinal -version | grep Threshold | grep Tier Tier3InvocationThreshold Tier3BackEdgeThreshold Tier3CompileThreshold Tier4InvocationThreshold Tier4BackEdgeThreshold Tier4CompileThreshold Compilation Level Threshold
48. $ java - XX:+PrintFlagsFinal -version | grep Threshold | grep Tier Tier3InvocationThreshold = (default) 200 Tier3BackEdgeThreshold = (default) 60000 Tier3CompileThreshold = (default) 2000 Tier4InvocationThreshold = (default) 5000 Tier4BackEdgeThreshold = (default) 40000 Tier4CompileThreshold = (default) 15000 Compilation Level Threshold
49. public class Hello { public int findMax(int[] arr) { int max = 0; for (int i: arr) { max = (max > i) ? max : i; } return max; } public static void main(String[] argv) { var hello = new Hello(); int[] arr = {1,2,3,4,5,6,7,8,9,10}; IntStream.rangeClosed(1, N).forEach(i -> { try { hello.findMax(arr); if (i % 100 == 0) {Thread.sleep(100);} } catch (Exception e) {} }); } } Sample Code N -> 1, 250, 500, 1000
50. VM Options - XX:+UnlockDiagnosticVMOptions - XX:+LogCompilation JIT(Just in time) Log
51. Run hotspot_pid[PID Number].log <?xml version='1.0' encoding='UTF-8'?> <hotspot_log version='160 1' process='50425' time_ms='1661499123369'> <vm_version> <name> OpenJDK 64 - Bit Server VM </name> ... JIT(Just in time) Log
52. <task_queued compile_id='1049' method='Hello fi ndMax ([I)I' bytes='45' count='205' backedge_count='2048' iicount='205' level='3' stamp='0.683' comment='tiered' hot_count='205'/> <nmethod compile_id='1049' compiler='c1' level='3' entry='0x00000001158470c0' size='1168' address='0x0000000115846f10' ... method='Hello fi ndMax ([I)I' bytes='45' count='250' backedge_count='2500' iicount='250' stamp='0.683'/> Sample JIT log
53. <task_queued compile_id='1041' method='Hello fi ndMax ([I)I' bytes='45' count='1894' backedge_count='18934' iicount='1894' stamp='2.323' comment='tiered' hot_count='1894'/> <nmethod compile_id='1041' compiler='c2' level='4' entry='0x00000001285e9220' size='1016' address='0x00000001285e9090' ... method='Hello fi ndMax ([I)I' bytes='45' count='1900' backedge_count='19000' iicount='1900' stamp='2.327'/> Sample JIT log
54. C1/C2 Compile Count warm up - 0 warm up - 5 warm up - 250 warm up - 500 c1 35928 42313 64314 72546 c2 19892 27327 45932 49265 deoptimization 420 498 578 536
55. C1/C2 Compile Count warm up - 0 warm up - 5 warm up - 250 warm up - 500 c1 35928 42313 64314 72546 c2 19892 27327 45932 49265 deoptimization 420 498 578 536
56. 2차 대응 warm up
57. - Java, JIT Compiler - Real traf fi c API Warm up - JIT C2 Optimization 정리하기
58. - Java, JIT Compiler - Real traf fi c API Warm up - JIT C2 Optimization 정리하기
59. - Java, JIT Compiler - Real traf fi c API Warm up - JIT C2 Optimization 정리하기
60. 감사합니다
61. 참고 문헌 1) 벤저민 J. 에번스, 제임스 고프, 크리스 뉴랜드, 『Optimizing Java』, 이일웅 옮김, 한빛미디어, 2019 2) Scott Oaks, 『Java Performance』, 2nd Edition, 2020, Ch4, https:/ /www.oreilly.com/library/view/java - performance-2nd/9781492056102/ch04.html 3) Oracle, The Java HotSpot Performance Engine Architecture, https:/ /www.oracle.com/java/technologies/whitepaper.html 4) OpenJDK open source, https:/ /github.com/openjdk/jdk11u