Getting Control Over Your Unit Tests and Build Memory Footprint

As mobile apps developers we usually care about the app’s memory footprint at runtime but we don’t feel the need to do the same for build and unit tests!
In this talk we will be doing a deep dive into how unit tests use JVM memory and how it may impact their stability and how it's related to the Gradle build... Yes, lots of hows to be answered 😄

Transcript

1. Getting Control Memory Footprint Unit Tests and Build Oussama Hafferssas Senior Android Engineer at Trainline Over Your 12 April 2023
2. Getting Control Memory Footprint Unit Tests and Build Oussama Hafferssas Over Your 12 April 2023
3. Getting Control Unit Tests and Build Oussama Hafferssas 12 April 2023 Memory Footprint
4. Getting Control Oussama Hafferssas 12 April 2023 Understand Unit Tests and Build Memory Footprint
5. Getting Control Oussama Hafferssas 12 April 2023 Understand Unit Tests and Build Memory Footprint
6. Getting Control Oussama Hafferssas 12 April 2023 Understand Unit Tests and Build Memory Footprint Not on device/AVD
7. Getting Control Oussama Hafferssas 12 April 2023 Understand Unit Tests and Build Memory Footprint Computer / CI
8. Getting Control Oussama Hafferssas 12 April 2023 Understand Unit Tests and Build Memory Footprint Dev Environment
9. Getting Control Oussama Hafferssas 12 April 2023 Understand Unit Tests and Build Memory Footprint Dev Environment
10. Getting Control Oussama Hafferssas 12 April 2023 Understand Unit Tests and Build Memory Footprint JVM Dev Environment
11. Oussama Hafferssas 12 April 2023 Understand Dev Environment JVM Getting Control Memory Footprint Unit Tests and Build Over Your how the handles memory on
12. Oussama Hafferssas 12 April 2023 Understand Dev Environment JVM Getting Control Memory Footprint Unit Tests and Build Over Your how the handles memory on … during unit tests and build
13. Oussama Hafferssas 12 April 2023 Understand Dev Environment JVM Getting Control Memory Footprint Unit Tests and Build Over Your how the handles memory on … during unit tests and build Naming things Finding talk titles is hard…
14. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your WHY?
15. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your WHY? Features / Fixes Dev team Unit tests on CI
16. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your WHY? Features / Fixes Dev team Unit tests on CI
17. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your WHY? Features / Fixes Dev team Unit tests on CI
18. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your WHY? Features / Fixes Dev team Unit tests on CI
19. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your WHY? OutOfMemory Exception JVM Crashed because of an OutOfMemory Exception
20. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests
21. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest
22. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest
23. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest
24. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest
25. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest !" gradle.properties file org.gradle.jvmargs=-Xmx6136m
26. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest !" gradle.properties file org.gradle.jvmargs=-Xmx6136m
27. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon
28. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon
29. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble
30. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble
31. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources
32. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources
33. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m
34. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m
35. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4
36. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4
37. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests Demo time! https://github.com/duckduckgo/Android Gradle Workers
38. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4
39. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4
40. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4 Building/compiling sources of unit tests Running unit tests
41. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your OutOfMemory Exception during unit tests ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4 Building/compiling sources of unit tests Running unit tests
42. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4 Building/compiling sources of unit tests Running unit tests OutOfMemory Exception during unit tests
43. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4 Building/compiling sources of unit tests Running unit tests OutOfMemory Exception during unit tests
44. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4 Building/compiling sources of unit tests Running unit tests OutOfMemory Exception during unit tests
45. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4 Building/compiling sources of unit tests Running unit tests OutOfMemory Exception during unit tests build.gradle gradle.properties android { testOptions { unitTests.all { maxHeapSize = "768m" } }
46. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your Demo time! https://github.com/duckduckgo/Android Heap size change
47. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your ./gradlew testDebugUnitTest Daemon :assemble :compileDebugUnitTestSources org.gradle.jvmargs=-Xmx6136m GradleWorker #1 GradleWorker #2 GradleWorker #3 GradleWorker #4 Building/compiling sources of unit tests Running unit tests OutOfMemory Exception during unit tests build.gradle gradle.properties android { testOptions { unitTests.all { maxHeapSize = "768m" } }
48. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Daemon Gradle Worker #1 build.gradle Gradle Worker #4 Gradle Worker #2 Gradle Worker #3 Getting Control android { testOptions { unitTests.all { maxHeapSize = "768m" } }
49. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Daemon Gradle Worker #1 build.gradle Gradle Worker #4 Gradle Worker #2 Gradle Worker #3 Getting Control OOM android { testOptions { unitTests.all { maxHeapSize = "768m" } }
50. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Daemon Gradle Worker #1 build.gradle Gradle Worker #4 Gradle Worker #2 Gradle Worker #3 Getting Control OOM android { testOptions { unitTests.all { maxHeapSize = "768m" jvmArgs '-XX:+HeapDumpOnOutOfMemoryError' } }
51. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Daemon Gradle Worker #1 build.gradle Gradle Worker #4 Gradle Worker #2 Gradle Worker #3 Getting Control OOM android { testOptions { unitTests.all { maxHeapSize = "768m" jvmArgs '-XX:+HeapDumpOnOutOfMemoryError' } } ! Memory consumption evolution " Heap dump inspection
52. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control ! Memory consumption evolution " Heap dump inspection Next:
53. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control ! Memory consumption evolution " Heap dump inspection Next:
54. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control " Heap dump inspection ! Memory consumption evolution Previously:
55. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your Demo time! https://github.com/duckduckgo/Android IntelliJ IDEA profiler
56. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control " Heap dump inspection ! Memory consumption evolution Previously:
57. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control " Heap dump inspection ! Memory consumption evolution Previously:
58. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control " Heap dump inspection ! Memory consumption evolution Previously:
59. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control " Heap dump inspection ! Memory consumption evolution Previously: class ClearInlineMocksRule : TestWatcher() { override fun finished(description: Description) { Mockito.framework().clearInlineMocks() } }
60. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control " Heap dump inspection ! Memory consumption evolution class ClearInlineMocksRule : TestWatcher() { override fun finished(description: Description) { Mockito.framework().clearInlineMocks() } } + ➡ ➡
61. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control OutOfMemory Exception JVM Crashed because of an OutOfMemory Exception
62. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control OutOfMemory Exception JVM Crashed because of an OutOfMemory Exception
63. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Daemon Gradle Worker #1 Gradle Worker #4 Gradle Worker #2 Gradle Worker #3 Getting Control OOM android { testOptions { unitTests.all { maxHeapSize = "768m" jvmArgs '-XX:+HeapDumpOnOutOfMemoryError' } } JVM Crashed because of an OutOfMemory Exception org.gradle.jvmargs=-Xmx6136m
64. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Daemon Gradle Worker #1 Gradle Worker #4 Gradle Worker #2 Gradle Worker #3 Getting Control OOM android { testOptions { unitTests.all { maxHeapSize = "768m" jvmArgs '-XX:+HeapDumpOnOutOfMemoryError' } } JVM Crashed because of an OutOfMemory Exception org.gradle.jvmargs=-Xmx6136m
65. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Daemon Gradle Worker #1 Gradle Worker #4 Gradle Worker #2 Gradle Worker #3 Getting Control OOM android { testOptions { unitTests.all { maxHeapSize = "768m" jvmArgs '-XX:+HeapDumpOnOutOfMemoryError' } } JVM Crashed because of an OutOfMemory Exception org.gradle.jvmargs=-Xmx6136m
66. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control OOM android { testOptions { unitTests.all { maxHeapSize = "768m" jvmArgs '-XX:+HeapDumpOnOutOfMemoryError' } } JVM Crashed because of an OutOfMemory Exception org.gradle.jvmargs=-Xmx6136m
67. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control OOM JVM Crashed because of an OutOfMemory Exception JVM Heap Memory
68. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception JVM Heap Memory NON Heap Memory
69. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception JVM Heap Memory NON Heap Memory Up to 10% of heap!
70. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception JVM Heap Memory NON Heap Memory Up to 10% of heap! 768 MB
71. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception JVM Heap Memory NON Heap Memory Up to 10% of heap! 768 MB 768 MB
72. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception JVM Heap Memory NON Heap Memory Up to 10% of heap! 768 MB 768 MB 4 GB
73. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception 768 MB 768 MB 4 GB $
74. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception 768 MB 768 MB 4 GB $ The solution: JVM tuning The process of adjusting the default parameters to match our application needs
75. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception 768 MB 768 MB 4 GB $ The solution: JVM tuning The process of adjusting the default parameters to match our application needs
76. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception 768 MB 768 MB 4 GB $ The solution: JVM tuning The process of adjusting the default parameters to match our application needs
77. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control JVM Crashed because of an OutOfMemory Exception 768 MB 768 MB 4 GB The solution: JVM tuning The process of adjusting the default parameters to match our application needs
78. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control 768 MB 768 MB 4 GB The solution: JVM tuning The process of adjusting the default parameters to match our application needs How: JPS Java Process Tools JVM Crashed because of an OutOfMemory Exception
79. Oussama Hafferssas 12 April 2023 Getting Control Memory Footprint Unit Tests and Build Over Your Demo time! https://github.com/duckduckgo/Android JPS
80. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your 6 GB 768 MB 768 MB 768 MB 768 MB Getting Control 768 MB 768 MB 4 GB The solution: JVM tuning The process of adjusting the default parameters to match our application needs How: JPS Java Process Tools JVM Crashed because of an OutOfMemory Exception
81. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control Takeaways • Memory issues are hard!
82. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control Takeaways • Memory issues are hard! • Choose your tools
83. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control Takeaways • Memory issues are hard! • Choose your tools • Profiler • Scripts
84. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control Takeaways • Memory issues are hard! • Choose your tools • Profiler • Scripts • Look everywhere (codebase + libraries)
85. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control Takeaways • Memory issues are hard! • Choose your tools • Profiler • Scripts • Look everywhere (codebase + libraries) • Think holistically
86. Oussama Hafferssas 12 April 2023 Memory Footprint Unit Tests and Build Over Your Getting Control Takeaways • Memory issues are hard! • Choose your tools • Profiler • Scripts • Try to be patient • Look everywhere (codebase + libraries) • Think holistically
87. Getting Control And Build Memory Footprint Unit Tests Oussama Hafferssas Over Your Thank you