From Maven to Gradle: The Story Behind the Switch

A build system is an important tool which provides robust testing and distribution. While unseen by most users, it’s central to any successful software. Late last year, Elasticsearch switched to use Gradle. This talk will explore the history and motivations behind the switch, as well as further improvements now made possible.

Transcript

1. ‹#› Ryan Ernst @ryanjernst From Maven to Gradle: The story behind the switch
2. Build System Timeline 2 Gradle All plugins in ES repo 0.4 0.19 2.0 5.0 Maven Plugins moved to external repos Maven Multi Module Plugins moved back in ES repo Gradle
3. Build System Timeline 3 Gradle All plugins in ES repo 0.4 0.19 2.0 5.0 Maven Plugins moved to external repos Maven Multi Module Plugins moved back in ES repo Gradle
4. Build Needs 4 • Patterns (eg nocommit) • Forbidden apis • Dependency licenses • License headers • Jar hell (up front) • Third party audit Static Checks • Randomized • Enforce good practices Unit Tests • Run against a real ES • Run with plugins • Run with special config • Run with mock services (eg s3, ec2, hdfs) Integration Tests
5. Maven to Gradle 5 Maven Multi Module Static checks Integration tests Transitive dependencies Modules 1 2 3 4 5
7. Gradle multi project • Project substitutions • Maven coordinates in projects • Build substitutes local projects • Configures the entire root project • Does not skip projects not being built! 7 Source: Gray Arial 10pt
8. Maven to Gradle Integration tests Transitive dependencies Modules Maven Multi Module 1 Static checks 2 3 4 5 Maven Multi Module
10. Gradle static checks • Added “precommit" task in gradle • Runs independently of compile, test, and integTest • Just a dummy task which depends on each check • Adding new checks is easier, just part of buildSrc 10 Source: Gray Arial 10pt
11. Maven to Gradle Transitive dependencies Modules Integration tests 3 Static checks 2 Maven Multi Module 1 4 5 Maven Multi Module
13. Gradle integration tests • No more xml! • Rewritten in groovy as gradle plugin • Configuration is easily changed per project • Reuse is much easier, no more copy/pasting • Added “fixtures” • Processes started before elasticsearch in integration test • Can mock services like hdfs, s3, ec2, etc 13 Source: Gray Arial 10pt
14. Maven to Gradle Transitive dependencies 4 Modules Static checks 2 Maven Multi Module 1 Integration tests 3 5 Maven Multi Module
16. Gradle transitive dependencies • Gradle allows disabling transitive deps • On a configuration • On a specific dependency • Added custom logic for disabling per dependency (ignoring ES) • Still need to publish maven pom • Generated pom with non-explicit dependencies excluded 16 Source: Gray Arial 10pt
17. Modules 5 Maven to Gradle Static checks 2 Maven Multi Module 1 Integration tests 3 Transitive dependencies 4 Maven Multi Module
19. Gradle elasticsearch modules • Wanted to simplify core, moving some functionality out to plugins • Isolate for security, removing permissions from core • Decouple to improve plugin api • But still include as part of distribution • Less than a day in grade • automatic for new modules • not published to maven (they are just internal to build) • One week in maven • Extremely manual process to add new ones • Must publish, no concept of “local” maven module 19 Source: Gray Arial 10pt
20. Questions? 20 Source: Gray Arial 10pt Twitter: @ryanjernst Github: rjernst Or AMA booth. Thank you!