Accelerating Maven Builds: From 🐌 Snail's Pace to 🚀 Rocket Speed (JCON 2024)

Are you tired of watching Maven builds crawl at a snail’s pace, wasting precious development time? Spending too much time at the coffee machine, or fighting battles with wooden swords under the excuse “my code’s compiling”?

Join us to learn how to supercharge your Maven build! We’ll cover three main mechanisms that you can apply in your project. Learn how each speeds up your build, when they provide the biggest gains, and what pitfalls await.

So take the next step in boosting your developer productivity by learning practical tips to decrease context switching and increase development speed and feedback cycle. Your journey from snail’s pace to rocket speed begins today!

Transcript

1. ACCELERATING MAVEN BUILDS FROM SNAIL'S PACE TO ROCKET SPEED #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
2. , licensed under CC BY-NC 2.5, adopted to match the dark background and not hurt your eyes. "Compiling" by Randall Munroe #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
3. (Hans Dockter, Gradle Inc.) “ We haven’t seen a company yet where investing into more ef cient builds was not leading to a signi cant return on investment. The most successful software teams on the planet are the ones with an ef cient build infrastructure. Quantifying the Costs of Builds #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
4. HOW TO ACCELERATE? ⏱ Measure Ľ Analyse Ƣ Improve #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
5. MEASURE YOUR BUILD MEASURE YOUR BUILD MEASURE YOUR BUILD MEASURE YOUR BUILD MEASURE YOUR BUILD #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
6. THE OPENTELEMETRY MAVEN EXTENSION Time execution recorder for Maven Log time taken by each Mojo in the build lifecycle Comes as a "Plugin on steroids": replaces or hooks into Maven core features Maven extension #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
7. INSTALL THE EXTENSION 1. Create .mvn/extensions.xml in your project 2. Register the extension 3. Start an OpenTelemetry collector (such as Jaeger "all-in-one") → <extensions> <extension> <groupId>io.opentelemetry.contrib groupId> <artifactId>opentelemetry maven extension artifactId> <version>1.33.0-alpha version> extension> extensions> Simpler alternatives #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
8. ACTIVATE THE EXTENSION Run Maven as you did before, but add -Dotel.traces.exporter=otlp When done, open OpenTelemetry #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
9. DEMO | ⌛ OpenTelemetry Screenshot #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
10. PARALLEL TEST EXECUTION PARALLEL TEST EXECUTION PARALLEL TEST EXECUTION PARALLEL TEST EXECUTION PARALLEL TEST EXECUTION #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
11. WHAT IS IT? Parallel execution of tests will run your tests* in parallel. * from a single module #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
12. WHEN TO APPLY If you have idle cores or tests waiting for I/O. 1. Executes large number of tests in parallel 2. Executes long-running tests in parallel #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
13. ⚠ CAVEATS 1. Tests must be parallelizable Non-thread safe tests will make the build will fail miserably. Ƣ Fix the tests Ƣ Add @Execute(SAME_THREAD) (JUnit speci c) 2. Can bottleneck the build Use resources that would've built another Maven module. Ƣ Disable parallel test execution for speci c modules Applies to parallel builds - more on that later. #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
14. HOW TO EXECUTE TESTS IN PARALLEL? 1. Through Sure re 2. Through JUnit #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
15. 1. USING MAVEN'S SUREFIRE PLUGIN Specify what to run in parallel using the parallel parameter. Allowed values: 1. none 2. methods 3. classes 4. classesAndMethods 5. all #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
16. 1. USING MAVEN'S SUREFIRE PLUGIN (CTD.) On the command line: mvn -Dparallel=classesAndMethods verify. Or in your pom.xml les: <plugin> <groupId>org.apache.maven.plugins groupId> <artifactId>maven suref re plugin artifactId> <version>3.2.5 version> <conf guration> <parallel>classesAndMethods parallel> conf guration> plugin> #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
17. PROS AND CONS Advantages: Supports both JUnit 4 and TestNG. Maintained by the Maven core team Disadvantages: All tests need to be thread-safe in a module to use it. Con guration can be a bit verbose across multiple modules. Does not apply to JUnit Jupiter. #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
18. 2. USING JUNIT JUPITER 1. Using src/test/resources/junit-platform.properties 2. Using system properties in the Sure re argLine 3. Using Java annotations #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
19. Example src/test/resources/junit-platform.properties # activation of parallel test execution junit.jupiter.execution.parallel.enabled = true # tests per class (but classes in sequence) junit.jupiter.execution.parallel.mode.default = concurrent # classes in parallel junit.jupiter.execution.parallel.mode.classes.default = concurrent # both options can be combined! #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
20. Example POM <plugin> <groupId>org.apache.maven.plugins groupId> <artifactId>maven suref re plugin artifactId> <version>3.2.5 version> <conf guration> <argLine> -Djunit.jupiter.execution.parallel.enabled=true -Djunit.jupiter.execution.parallel.mode.default=concurrent argLine> conf guration> plugin> #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
21. Example with Java annotations @Execution(CONCURRENT) class MyServiceTest { test methods are now being executed in parallel } #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
22. PROS AND CONS Advantages: Finer granularity of test executions Run single tests selectively in sequence Maintained by the JUnit Jupiter team Great IDE support w/ property les and annotations Disadvantages: Speci c to JUnit Jupiter, not in TestNG, Spock and others Harder to control from the command line #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
23. DEMO #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
24. WITH SEQUENTIAL TESTS #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
25. WITH PARALLEL TESTS #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
26. MAVEN DAEMON MAVEN DAEMON MAVEN DAEMON MAVEN DAEMON MAVEN DAEMON #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
27. WHAT IS IT? Provides faster Maven builds using a daemon process. #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
28. HOW DOES IT HELP? 1. Keeps the JVM and plugins "warm" Save on JVM startup time, POM parsing and plugin loading 2. Runs multi-threaded by default without cluttering output Maven Daemon runs your build in parallel mode by default 3. Improved parallelism compared to "vanilla" Maven Maven Daemon comes with which builds in topological order. Takari Smart builder #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
29. WHEN TO APPLY? 1. Large project Many modules, with large project object models — parsed only once. 2. Many plugins Plugin loading happens only once. 3. "Heavy" plugins Plugins are kept warm once loaded into the JVM. #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
30. ⚠ CAVEATS 1. Multi-threaded by default If you use non-threadsafe plugins, your build may fail miserably. Ƣ Circumvent by specifying -T 1 Ƣ Address the root cause: x the plugin(s) 2. Re-implements some parts of Maven Parsing .mvn/maven.config has some bugs: , . apache/maven- mvnd#913 apache/maven-mvnd#917 #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
31. HOW? 1. Install Maven Daemon 2. Retrain nger gymnastics: mvn → mvnd #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
32. HOW? (CTD.) A few new tricks: Use mvnd --status to inspect details of the daemon process. Add -Dmvnd.buildTime=true for simple timing report, incl. bottleneck analysis. Add -Dsmartbuilder.profiling=true for advanced timing report, incl. detailed bottleneck analysis. #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
33. DEMO Screenshots: | Maven Maven Daemon #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
34. PARALLELIZING YOUR BUILD PARALLELIZING YOUR BUILD PARALLELIZING YOUR BUILD PARALLELIZING YOUR BUILD PARALLELIZING YOUR BUILD Ȃ MAVEN MODULES EVERYWHERE Ȃ #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
35. CASE STUDY: HEXAGONAL ARCHITECTURE #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
36. INITIAL MAVEN MODULES #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
37. REFACTORED MAVEN MODULES #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
38. EVALUATION PROS Faster builds when there are enough modules with few, distinct dependencies (Almost) no need for ArchUnit or alike Easier to exclude transient dependencies via Maven CONS Could hinder navigation between classes and modules Needs awareness of runtime vs compile dependencies May not be suitable for libraries #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
39. TO DO OR NOT TO DO? As always: it depends! Pro ts vs. added complexity For libraries: impact for consumers? For applications: does it impact architecture? #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
40. DEMO This is what happens when you do not have any long-running tests: SHIRO W/ 62 MODULES (2:01 MIN) SHIRO W/ 58 MODULES (1:59 MIN) #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
41. RESUME A FAILED BUILD RESUME A FAILED BUILD RESUME A FAILED BUILD RESUME A FAILED BUILD RESUME A FAILED BUILD #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
42. RESUMING AFTER A BUILD FAILURE #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
43. RESUMING AFTER A BUILD FAILURE? — NOPE! The Maven Reactor 1. trims the project list 2. resolves inter-project dependencies 3. can't nd a module that was dropped in step 1 With Maven 3, the work-around was ( mvn clean install before running the actual build. → Broken Builds and Bad Behaviours - The Maven Story #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
44. WHEN TO APPLY With Maven 4 , 1. ... when troubleshooting the build locally 2. ... when performing TDD You can run ✅ mvn verify - as you'd expect Ƹ #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
45. ⚠ CAVEATS 1. Only useful with interactive use of Maven Does not make sense on a build server 2. Some unof cial Maven plugins might not work Ƣ Please report to the plugins developers #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
46. HOW TO RESUME A FAILED BUILD? RTFM *) F is for FINE, of course #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
47. DEMO #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
48. MAVEN BUILD CACHE EXTENSION MAVEN BUILD CACHE EXTENSION MAVEN BUILD CACHE EXTENSION MAVEN BUILD CACHE EXTENSION MAVEN BUILD CACHE EXTENSION #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
49. HOW DOES THE BUILD CACHE WORK? The Maven Build Cache extension reduces build time by skipping parts of the build which don't need to be executed. It uses a combination of four strategies: 1. Only execute parts of the build which changed 2. Build subtrees of a multi-module project in isolation 3. Normalize the version to avoid rebuilding across forks 4. Project state restoration (e.g. artifacts) #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
50. 1. WHICH PARTS HAVE CHANGED? A. MODIFICATIONS TO THE POM If you add any dependencies or change scopes, modify plugins or executions, the cache will be invalidated. B. MODIFICATION OF SOURCE CODE The cache extension uses source code content ngerprinting. #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
51. WHEN TO USE THE CACHE EXTENSION? 1. You already use Maven 3.9 or Maven 4 2. Your project does not rely on 'quirks' 3. You want to speed up your local developer build 4. You want to speed up your CI builds (PRs) 5. You want to share the cache with co-workers 6. You are willing to invest time to make the cache portable #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
52. INSTALL THE EXTENSION 1. Create .mvn/extensions.xml in your project 2. Register the extension by adding the following content: 3. (Optional) extend the default cache con guration <?xml version="1.0" encoding="UTF-8" ?> <extensions> <extension> <groupId>org.apache.maven.extensions groupId> <artifactId>maven build cache extension artifactId> <version>1.1.0 version> extension> extensions> #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
53. DEMO DEMO DEMO DEMO DEMO #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
54. ⚠ CAVEATS 1. Con gure relevant input les 2. Add plugin parameters for reconciliation 3. Filter non-essential les (IDE, etc) 4. Expect your tests not to run 5. Remote cache not used (OS, pro les...) 6. Cache eviction , #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
55. HONOURABLE MENTIONS HONOURABLE MENTIONS HONOURABLE MENTIONS HONOURABLE MENTIONS HONOURABLE MENTIONS We did not cover: Reduce logging (Logback, Log4J, mvn -q) — try it! Christian Stein's "junit5-looming" (showcase): JUnit 5 test engine with virtual thread support #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
56. WHAT DID WE LEARN? WHAT DID WE LEARN? WHAT DID WE LEARN? WHAT DID WE LEARN? WHAT DID WE LEARN? 1. Measure your builds 2. Parallel test execution per module 3. Using the Maven Daemon 4. Feed mvnd with architecture improvements 5. Maven 4: resume after failure 6. Maven Build Cache Extension #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)
57. REFERENCES bit.ly/snail-pace-rocket-speed-demos blog.bmarwell.de maarten.mulders.it @bmarwell @mthmulders #Maven #SnailToRocket Benjamin Marwell (@bmarwell) & Maarten Mulders (@mthmulders)