Surviving Dependency Hell with Maven

As a developer advocate working with customers, Ray has seen all sorts of issues due to dependency conflicts. Dependency conflicts come in many different forms and have different impacts on your applications. This presentation examines common causes of a dependency conflict, how you can mitigate it as a library developer, and how end users can resolve it. It also covers what Google has been documenting in terms of best practices and what tools it has created to help, based on its learnings.

Transcript

1. Surviving Dependency Hell
   With Maven
   

   View Slide

2. 2
   @aalmiray @saturnism
   Ray Tsang
   Developer Advocate
   @ Google
   Java Champion
   Spring Cloud GCP
   JDeferred
   Contributors to JLBP.dev
   @saturnism | saturnism.me
   

   View Slide

3. 3
   @aalmiray @saturnism
   Robert Scholte
   Java Champion
   Founder, Source Ground
   Apache Maven PMC Chair
   Member Expert Group
   JSR 376 - Jigsaw
   @rfscholte
   

   View Slide

4. 4
   @aalmiray @saturnism
   Andres Almiray
   Seasoned Sourceror
   @ Oracle
   Java Champion
   Hackergarten
   JDeferred & many others!
   @aalmiray | andresalmiray.com
   

   View Slide

5. 5
   @aalmiray @saturnism
   Let's see the code
   

   View Slide

6. 6
   @aalmiray @saturnism
   NoSuchMethodError
   NoSuchFieldError
   NoClassDefFoundError
   

   View Slide

7. 7
   @aalmiray @saturnism
   mvn dependency:tree
   mvn dependency:tree -Dverbose=true
   mvn dependency:tree -Dverbose=true -Dincludes=...
   

   View Slide

8. 8
   @aalmiray @saturnism
   Example 1
   Translate Truth
   Guava 19
   Guava 21
   Which version to use?
   

   View Slide

9. 9
   @aalmiray @saturnism
   Any dependency there can be only ONE version
   Classpath - First class wins
   Maven - Nearest wins
   

   View Slide

10. 10
    @aalmiray @saturnism
    Maven does not understand Semver
    Nor compatibility
    

    View Slide

11. 11
    @aalmiray @saturnism
    Use Maven Enforcer
    Convergence vs Upper bound
    http://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html
    (See Code!)
    

    View Slide

12. 12
    @aalmiray @saturnism
    Ensuring only ONE version
    of the dependency in tree
    Exclusions, or Dependency Management
    (See Code)
    

    View Slide

13. 13
    @aalmiray @saturnism
    Upper bound if
    higher version is backwards compatible
    [JLBP-7] Make breaking transitions easy
    [JLBP-10] Maintain API stability as long as needed for consumers
    (Guava 21 and up are backwards compatible)
    

    View Slide

14. 14
    @aalmiray @saturnism
    If upper version is breaking lower version…
    Or, system classpath has an incompatible version
    (Hadoooooop)
    

    View Slide

15. 15
    @aalmiray @saturnism
    Shade + Relocation :(
    Classloaders
    OSGi
    

    View Slide

16. 16
    @aalmiray @saturnism
    GA libraries don't depend on non-GA APIs
    alpha, beta, RC, 0.xx, @UnstableApi, @Beta, @Internal
    [JLBP-4] Avoid dependencies on unstable libraries and features
    

    View Slide

17. 17
    @aalmiray @saturnism
    Major release, breaking changes
    Use new Group ID or Artifact ID - different coordinate!
    AND Use a new package name
    [JLBP-6] Rename artifacts and packages together
    

    View Slide

18. 18
    @aalmiray @saturnism
    How do we change versions in transitive
    dependencies?
    

    View Slide

19. 19
    @aalmiray @saturnism
    Maven Dependency Management
    

    View Slide

20. 20
    @aalmiray @saturnism
    Let's see Example 3e
    

    View Slide

21. 21
    @aalmiray @saturnism
    Example 1
    PubSub Firebase
    guava-jdk5 17.0
    guava 28.0
    

    View Slide

22. 22
    @aalmiray @saturnism
    Ban Duplicate Classes
    https://www.mojohaus.org/extra-enforcer-rules/banDuplicateClasses.html
    

    View Slide

23. 23
    @aalmiray @saturnism
    2 artifacts should not have overlapping classes
    [JLBP-5] Avoid dependencies that overlap classes with other dependencies
    [JLBP-19] Place each package in only one module
    

    View Slide

24. 24
    @aalmiray @saturnism
    [JLBP-11] Keep dependencies up to date!
    

    View Slide

25. 25
    @aalmiray @saturnism
    Let's fix all of these
    And see the next example 3c
    

    View Slide

26. 26
    @aalmiray @saturnism
    Example
    Pubsub Trace
    grpc-stub 1.10.1
    grpc-all 1.0.1
    grpc-core 1.0.1 grpc-... 1.0.1
    

    View Slide

27. 27
    @aalmiray @saturnism
    Version Misalignment
    

    View Slide

28. 28
    @aalmiray @saturnism
    How do we change versions in transitive
    dependencies?
    

    View Slide

29. 29
    @aalmiray @saturnism
    Example
    Pubsub Trace
    grpc-stub 1.10.1
    grpc-all 1.10.1
    grpc-core 1.10.1 grpc-... 1.10.1
    grpc-bom 1.10.1
    

    View Slide

30. 30
    @aalmiray @saturnism
    Multi-module projects Produce a BOM!
    [JLBP-15] Publish a BOM for multi-module projects
    

    View Slide

31. 31
    @aalmiray @saturnism
    See an example!
    

    View Slide

32. 32
    @aalmiray @saturnism
    Power → Responsibility
    BOM should not overreach!
    ONLY modules within the multi-project project
    

    View Slide

33. 33
    @aalmiray @saturnism
    gRPC also uses another method
    Strict Version Ranges for inter-module dependencies,
    e.g. [1.10.1]
    

    View Slide

34. 34
    @aalmiray @saturnism
    Linkage Checker
    Static Analysis to identify missing links
    https://github.com/GoogleCloudPlatform/cloud-opensource-java/tree/master/enforcer-rules
    

    View Slide

35. 35
    @aalmiray @saturnism
    Bigger issues with
    Dependency Management...
    

    View Slide

36. 36
    @aalmiray @saturnism
    What you build & test with
    Is not what consumers get!
    

    View Slide

37. 37
    @aalmiray @saturnism
    Flatten the POM
    https://github.com/mojohaus/flatten-maven-plugin/pull/93
    

    View Slide

38. 38
    @aalmiray @saturnism
    Bonus
    mvn dependency:analyze - find dependencies you don't need
    Maven 3.6.3 - fixes our exclusion fix in Maven 3.6.2
    

    View Slide

39. 39
    @aalmiray @saturnism
    Enforcer Rules
    Enforcer Version Convergence or Upperbound
    Ban Duplicate Classes
    Linkage Checker
    Manage Transitive Versions
    
    BOM for Multi-Module
    Import BOM
    Maven Flatten Plugin
    

    View Slide

40. 40
    @aalmiray @saturnism
    Visit the best practices at JLBP.dev
    Thanks!
    Andres Almiray @aalmiray
    Ray Tsang @saturnism
    Maven used by 60% to 80% of Java projects
    92 projects, 50+ plugins, lots of libraries - Contribution Appreciated!
    

    View Slide