How to use Macrobenchmark

Google I/O extended Android Korea 2024 에서 벤치마크 사용 설명서란 주제로 발표한 내용입니다. https://festa.io/events/5509

Transcript

1. # Performance Macrobenchmark ࢎਊ ࢸݺࢲ ߓ೙઱ | GDG Korea Android
2. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ ਗੋ ౵ঈ ࢿמ ޙઁܳ ೧Ѿೞח ߑߨ ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү 🙆 Intro
3. Intro ௏٘ী ऀয੓ח Unstable Parameters ܳ ݽف ୶੸ೞҊ ࣻ੿ೞ૑ ݃ࣁਃ. 🙅 хਵ۽ ନ૑ ݈ӝ! ೧׼ ஹನ੷࠶੄ ܻஹನ૑࣌ਸ ਬߊೞח গפݫ੉࣌ਸ ࢏ઁೞ૑݃ࣁਃ. ૒хী ٮۄ ੍ӝ য۰਍ ୭੸ചܳ प೯ೞ૑ ݃ࣁਃ.
4. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ ਗੋ ౵ঈ ࢿמ ޙઁܳ ೧Ѿೞח ߑߨ Intro ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү
5. ࢿמ ޙઁܳ ખ ؊ औѱ ଺ӝ ਤೠ ஏ੿ী ؀ೠ ੉ঠӝ ൓݃ߨ੉ ইתפ׮. 🧙 # Performance
6. Intro জ ղࠗ ز੘ ঌ ೙ਃ হ੉ UiAutomatorܳ ࢎ ਊ೧ࢲ ࢿמਸ ஏ੿೤פ׮. Jetpack Macrobenchmark ਬ੷о ࠁח Ѫҗ э਷ ࢿמਸ ഛੋ೤פ׮. Performance ബҗ ஏ੿ਸ ਤೠ ୶ୌೞח ߑߨ ࢿמ ஏ੿ पઁ ࢎਊ ജ҃੄ ࢿמ CIীࢲ प೯ ଵҊ : పझ౟ী ؀ೠ ӝࠄ੸ੋ ੉೧
7. ޖ঺ਸ ஏ੿ ೡ ࣻ ੓աਃ? Intro জ੉ द੘ೡ ٸ৬ ੹୓ ۽٘ ؼ ٸ ঴݃ա Ѧܻח૑ Startup Timing 01 Frame Timing 02 Frameਸ ࢤࢿೞחؘ ঴݃ա Ѧܻח૑ Trace Section 03 ׮নೠ ஏ੿ਸ ೧ࠅ ࣻ ੓ח ׮ਊب API Memory Usage 04 জ੉ ঴݃ա ݆਷ ݫݽܻܳ ࢎਊೞҊ ੓ח૑ Power 05 জ੉ ঴݃ա ݆਷ ߓఠܻܳ ࣗݽೞҊ ੓ח૑ + https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
8. https://developer.android.com/codelabs/jetpack-compose-performance#0 Practical performance problem solving in Jetpack Compose ஏ੿ೞӝ
9. ஏ੿ೞӝ Macrobenchmark ݽٕ Macrobenchmarkח জ ௏٘৬ ܻ࠙ غয জ ஏ੿ పझ౟ प೯ਸ ׸׼ೞח com.android.test ݽٕ੉ ೙ਃ೤פ׮.
10. ޖѢ਍ Composable ୭੸ച ೞӝ ஏ੿ೞӝ
11. ஏ੿ೞӝ ஏ੿ ҅ദ ࣁ਋ӝ • Layout Inspectorܳ ࢎਊ೧ࢲ ҳઑܳ ౵ঈ • п Layout ߹ दрਸ ஏ੿
12. ஏ੿ೞӝ Layout ҳઑ • Column : HeavyItem • PublishedText • PublishDate.format • Layout : ImageDisplay • ImagePlaceHolder • ItemTag
13. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } }
14. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test fun accelerateHeavyScreenCompilation() fun benchmark(compilationMode: CompilationMode) { } } class MacrobenchmarkRule : TestRule JUnit ӏ஗ਸ ࢎਊೞৈ গ೒ܻா੉࣌੄ द੘, झ௼܀݂ ژח গפݫ੉࣌җ э਷ ؀ӏݽ ੘সਸ ߮஖݃ఊೞח ߑߨਸ ઁҕ೤פ׮. https://developer.android.com/reference/kotlin/androidx/benchmark/macro/junit4/MacrobenchmarkRule
15. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } }
16. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) } https://developer.android.com/reference/androidx/benchmark/macro/CompilationMode CompilationMode.Partial জ द੘ ࢿמ ஏ੿ೞҊ ੉೧ೞחؘ ࢎਊ (Default) CompilationMode.Full জ प೯ ઺ runtime ࢿמ ࠺Ү ߂ ѐࢶ JIT ୭੸ച۽ ੋೠ ࢿמ ߸ز ਃੋ ઁѢ CompilationMode.None ࢎ੹ ஹ౵ੌ হ਺. Baseline Pro fi le җ Benchmark ղীࢲ ࠺Ү CompilationMode
17. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } }
18. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( . . . startupMode = StartupMode.WARM, } Sta rt upMode https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics StartupMode.COLD Sta rt up Benchmarkী ࢎਊ setupBlock৬ mesureBlock ࢎ੉ী জ ೐۽ࣁझ ઙܐ StartupMode.WARM Frame timing Benchmarkী ࢎਊ setupBlock റী ೐۽ࣁझо ઙܐغ૑ ঋ਺ ೐۽ࣁझܳ ઙܐೞ૑ ঋҊ ݽٚ प೯ ઺ੋ Activityܳ ײ਺ Activity द੘ ߑधҗ పझ౟ द੘ द੄ ೐۽ࣁझ ࢚క ߸҃ StartupMode.HOT п Activity ߹ நय ݫழפ્ਸ ஏ੿ೞחؘ ࢎਊ ೐۽ࣁझ, ੉੹ী प೯ ઺੉؍ Activityب ੤द੘ غ૑ ঋ਺ Null Macrobenchmark о ইޖѪب ೞ૑ ঋח ݽ٘ killProcess() ۽ ૒੽ ஶ౟܀
19. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } } SetupBlock ਗೞח ஏ੿ ചݶ ੹ Frame ө૑੄ ز੘ (۽Ӓੋ ١)
20. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } } MeasureBlock ஏ੿ਸ ਗೞח ز੘
21. MeasureBlock ஏ੿ਸ ਗೞח ز੘ override fun MacrobenchmarkScope.measureBlock() { pressHome() startTaskActivity("accelerate_heavy") device.wait( Until.hasObject(By.res("list_of_items")), 5_000) val feed = device.findObject(By.res("list_of_items")) feed.setGestureMargin(device.displayWidth / 5) repeat(2) { feed.drag( Point( feed.visibleCenter.x, feed.visibleBounds.top ) ) Thread.sleep(500) } }
22. MeasureBlock TestTagܳ ࠢৈࢲ ਗೞח resource ଺ইоӝ @Composable fun ScreenContent(items: List<HeavyItem>) { LazyVerticalGrid( modifier = Modifier .fillMaxSize() .testTag(“list_of_items"), . . . ) { items(items) { item -> HeavyItem(item) } } } device.wait(Until.hasObject(By.res("list_of_items")), 5_000)
23. MeasureBlock TestTagܳ ࠢৈࢲ ਗೞח resource ଺ইоӝ @Composable fun ScreenContent(items: List<HeavyItem>) { LazyVerticalGrid( modifier = Modifier .fillMaxSize() .testTag(“list_of_items"), . . . ) { items(items) { item -> HeavyItem(item) } } } device.wait(Until.hasObject(By.res("list_of_items")), 5_000)
24. MeasureBlock ஏ੿ਸ ਗೞח ز੘ - झ௼܀ override fun MacrobenchmarkScope.measureBlock() { pressHome() startTaskActivity("accelerate_heavy") device.wait( Until.hasObject(By.res("list_of_items")), 5_000) val feed = device.findObject(By.res("list_of_items")) feed.setGestureMargin(device.displayWidth / 5) repeat(2) { feed.drag( Point( feed.visibleCenter.x, feed.visibleBounds.top ) ) Thread.sleep(500) } }
25. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { @get:Rule val benchmarkRule = MacrobenchmarkRule() @Test fun accelerateHeavyScreenCompilation() = benchmark(CompilationMode.Partial()) fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( packageName = "com.compose.performance", metrics = listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum) ), compilationMode = compilationMode, startupMode = StartupMode.WARM, iterations = 10, setupBlock = { setupBlock() }, measureBlock = { measureBlock() } ) } } ஏ੿ ೦ݾ https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
26. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( . . . metrics = listOf( FrameTimingMetric() . . .), } frameDurationCpuMs P50 11.9, P90 24.0, P95 30.3, P99 90.6 frameOverrunMs P50 3.5, P90 21.2, P95 25.9, P99 252.0 FrameTimingMetric 👉 CPUীࢲ UI झۨ٘৬ RenterThreadীࢲ ೐ۨ੐੉ ࢤࢿغחؘ Ѧܽ दр https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
27. @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( . . . metrics = listOf( FrameTimingMetric() . . .), } frameDurationCpuMs P50 11.9, P90 24.0, P95 30.3, P99 90.6 frameOverrunMs P50 3.5, P90 21.2, P95 25.9, P99 252.0 FrameTimingMetric 👉 CPUীࢲ UI झۨ٘৬ RenterThreadীࢲ ೐ۨ੐੉ ࢤࢿغחؘ Ѧܽ दр 👉 ઱য૓ ೐ۨ੐੉ ӝೠਸ ֈӟ दр +(নࣻ)ੌ ࣻ۾ ߡߢѢܿ, - (਺ࣻ)ח ঴݃ա ࡅܲ૑ Android 12(API 31) https://developer.android.com/topic/performance/benchmarking/macrobenchmark-metrics
28. ஏ੿ೞӝ TraceSection Metric • Column : HeavyItem • PublishedText • PublishDate.format • Layout : ImageDisplay • ImagePlaceHolder • ItemTag Layout ҳઑ ߹۽ ࢸ੿
29. listOf( FrameTimingMetric(), TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum), TraceSectionMetric("PublishedText", TraceSectionMetric.Mode.Sum), TraceSectionMetric("PublishDate.format", TraceSectionMetric.Mode.Sum), TraceSectionMetric("ImageDisplay", TraceSectionMetric.Mode.Sum), TraceSectionMetric("ImagePlaceholder", TraceSectionMetric.Mode.Sum), TraceSectionMetric("ItemTag", TraceSectionMetric.Mode.Sum) ) @RunWith(AndroidJUnit4::class) class HeavyScreenBenchmark { fun benchmark(compilationMode: CompilationMode) { rule.measureRepeated( . . . metrics = listOf(FrameTimingMetric(). . .), } TraceSectionMetric Trace Section ੉ ઁҕػ sectionName җ ੌ஖ೞח പࣻ৬ ࣗਃػ दр ஭୛ ୭ࣗदр, ઺ঔ ч, ୭؀ दр (ms) h tt ps://developer.android.com/topic/pe rf ormance/benchmarking/macrobenchmark-metrics?hl=ko#trace-section
30. TraceSectionMetric.Mode TraceSectionMetric("HeavyItem", TraceSectionMetric.Mode.Sum), https://developer.android.com/reference/androidx/benchmark/macro/TraceSectionMetric.Mode Mode.First SectionName ੌ஖ೞח ୐ߣ૩ ࣂ࣌੄ instance ࣗਃदр ஭୛ Mode.Sum SectionName ੌ஖ೞח ݽٚ ࣂ࣌ instance ࣗਃ दр ೤҅ Mode.Max SectionName ੌ஖ೞח ੹୓ ࣂ࣌ ઺ о੢ ௾ ч ӝ۾ Mode.Min SectionName ੌ஖ೞח ੹୓ ࣂ࣌ ઺ о੢ ੘਷ ч ӝ۾ Mode.Average SectionName ੌ஖ೞח ੹୓ ࣂ࣌੄ ಣӐ Mode.Count SectionName ੌ஖ೞח ࣂ࣌ ѐࣻ 1.3.0 beta+ 1.3.0 beta+ 1.3.0 beta+ 1.3.0 beta+
31. ஏ੿ೞӝ _count : SectionName җ ੌ஖ೞח іࣻ _ms : Section ஏ੿ػ ࣗਃ दр TraceSectionMetric ஏ੿ Ѿҗ
32. ஏ੿ೞӝ HeavyItem PublishedText PublishDate.format ImageDisplay ImagePlaceHolder ItemTag Min Median Max HeavyItem 46.4 50.2 53.0 PublishedText 6.9 7.7 8.6 PublishDate.format 0.7 0.8 0.9 ImageDisplay 27.1 29.1 30.4 ImagePlaceHolder 17.4 18.6 19.3 ItemTag 2.7 3.6 5.4 TraceSectionMetric ஏ੿ Ѿҗ
33. System Trace ࠁ۞оӝ ਗੋ౵ঈ Test Resultsীࢲ Ѿҗ link ࠗ࠙ ௿ܼ
34. Trace ౵ੌ ਤ஖ Measure > build > outputs > connected_android_text_additional_output > benchmarkRelease > connected > ‘connected device’ ਗੋ౵ঈ
35. ਗੋ౵ঈ Pro fi ler System Trace
36. উ٘۽੉٘ झౚ٣য় प೯ ೙ਃ হ਺ ٣߄੉झ੄ ੹୓ Threadܳ ࠅ ࣻ ੓਺ Trace ઺, प೯غח SQL quries ਗੋ౵ঈ Web-based tool Gives you whole picture https://youtu.be/Z96wfbID_Yc?si=pOe8w7hT7IEbsnb-&t=655 Investigate deeper H tt ps://ui.pe rf e tt o.dev/
37. ਗੋ౵ঈ Pe rf e tt o
38. ਗੋ౵ঈ Expected Timeline दझమীࢲ ೐ۨ੐ ࢤࢿೞח ৘࢚ ఋ੐ۄੋ
39. ਗੋ౵ঈ Actual Timeline ৘࢚ࠁ׮ য়ې Ѧ۰ࢲ ࢤࢿػ ೐ۨ੐ पઁ۽ ೐ۨ੐੉ ࢤࢿػ दр
40. ਗੋ౵ঈ System-tracing call cha r ੹୓ दр ইې ഐ୹ػ ೣࣻ ࣗਃ दр੄ ୨೤ Trace Code Untrace Code
41. ਗੋ౵ঈ о੢ য়ې Ѧܻח ੘স ࠙ࢳೞӝ HeavyItem ੹୓ ࣗਃ दр
42. ਗੋ౵ঈ о੢ ݆੉ ର૑ೞח दр ImageDisplay о੢ য়ې Ѧܻח ੘স ࠙ࢳೞӝ
43. ਗੋ౵ঈ Image ۽٘ೞחؘ Placeholder о о੢ য়ې Ѧܻ֎?? о੢ য়ې Ѧܻח ੘স ࠙ࢳೞӝ
44. ਗੋ౵ঈ https://developer.android.com/develop/ui/compose/tooling/tracing ಴दػ ৔৉݅ ୶੸ য়ߡ೻٘о ੸Ҋ জ ࢿמী ௾ ৔ೱਸ ޷ ஖૑ ঋ਺ জ੄ ݽٚ ೣࣻ ഐ୹ਸ ୶੸ জ ࢿמী ௾ ৔ೱਸ ޷஖૑݅ ߊࢤೞ ח ࢚ട, ഐ୹غח ೣࣻ, ഐ୹ ࠼بܳ ੹୓੸ਵ۽ ౵ঈ Composition function ನೣغয ੓૑ ঋ਺ Composition function ನೣ Composition Tracing +
45. // :app module implementation( “androidx.compose.runtime:runtime-tracing:1.0.0-beta01" ) https://developer.android.com/develop/ui/compose/tooling/tracing // :mesure module implementation( “androidx.tracing:tracing-perfetto:1.0.0") implementation( "androidx.tracing:tracing-perfetto-binary:1.0.0") // gradle configuration androidx.benchmark.fullTracing.enable=true
46. ਗੋ౵ঈ Composition function ҙ଴ оמ
47. ImagePlaceHolder ѐࢶ 1. ੉޷૑ܳ ࠺زӝ۽ ۽٘ 2. ؊ ࡈܻ ۽٘غب۾ ੘਷ ੉޷૑ ࢎਊ 3. ೙ਃೠ ௼ӝী ٮۄ ઑ੿غח Vector Drawable ࢎਊ ௏٘ࣻ੿
48. 1600 x 1600 px @Composable fun imagePlaceholder() = trace("ImagePlaceholder") { painterResource(R.drawable.placeholder) } ௏٘ࣻ੿
49. @Composable fun imagePlaceholder() = trace("ImagePlaceholder") { painterResource(R.drawable.placeholder_vector) } placeholder_vector.xml ௏٘ࣻ੿
50. TraceSectionMetric ஏ੿ Ѿҗ ୡӝ ࢿמҗ ࠺Ү
51. ѐࢶ Ѿҗ Before A ft er Max ӝળ 19.3 ীࢲ 4.4 ۽ 77.2% хࣗ ୡӝ ࢿמҗ ࠺Ү
52. Before A ft er ୡӝ ࢿמҗ ࠺Ү
53. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ ਗੋ ౵ঈ ࢿמ ޙઁܳ ೧Ѿೞח ߑߨ ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү 🙆 Intro
54. ஏ੿ਸ ా೧ ୡӝ ࢿמਸ ࢸ੿೤פ׮. ޙઁܳ ੌਵఃח ਗੋਸ ഛੋ೤פ׮. ஏ੿ೞӝ ਗੋ ౵ঈ ഛੋೠ ղਊী ٮۄ ௏٘ܳ ࣻ੿೤פ׮. ௏٘ ࣻ੿ ࢿמਸ ஏ੿ೞҊ ୡӝ ࢿמҗ ࠺Ү೤פ׮. ୡӝ ࢿמҗ ࠺Ү ੿ܻ Macrobenchmark ࢸ੿ೞӝ ஏ੿ чٜ੄ ੄޷ ঌইࠁӝ Ѿҗ ч ࠺Үೞӝ Profiler ژח Perfetto chart ੉೧ೞӝ ޖѢ਍ ੘স оߺѱ ѐࢶೞӝ ࣻ੿ റ ബҗо ੓ח૑ ࠺Үೞӝ
55. хࢎ೤פ׮. ߓ೙઱ veronikapj GDG Korea Android Resources https://developer.android.com/codelabs/jetpack-compose-performance