Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Surviving Dependency Hell with Maven

Ray Tsang
February 21, 2020

Surviving Dependency Hell with Maven

As a developer advocate working with customers, Ray has seen all sorts of issues due to dependency conflicts. Dependency conflicts come in many different forms and have different impacts on your applications. This presentation examines common causes of a dependency conflict, how you can mitigate it as a library developer, and how end users can resolve it. It also covers what Google has been documenting in terms of best practices and what tools it has created to help, based on its learnings.

Ray Tsang

February 21, 2020
Tweet

More Decks by Ray Tsang

Other Decks in Programming

Transcript

  1. Surviving Dependency Hell
    With Maven

    View Slide

  2. 2
    @aalmiray @saturnism
    Ray Tsang
    Developer Advocate
    @ Google
    Java Champion
    Spring Cloud GCP
    JDeferred
    Contributors to JLBP.dev
    @saturnism | saturnism.me

    View Slide

  3. 3
    @aalmiray @saturnism
    Robert Scholte
    Java Champion
    Founder, Source Ground
    Apache Maven PMC Chair
    Member Expert Group
    JSR 376 - Jigsaw
    @rfscholte

    View Slide

  4. 4
    @aalmiray @saturnism
    Andres Almiray
    Seasoned Sourceror
    @ Oracle
    Java Champion
    Hackergarten
    JDeferred & many others!
    @aalmiray | andresalmiray.com

    View Slide

  5. 5
    @aalmiray @saturnism
    Let's see the code

    View Slide

  6. 6
    @aalmiray @saturnism
    NoSuchMethodError
    NoSuchFieldError
    NoClassDefFoundError

    View Slide

  7. 7
    @aalmiray @saturnism
    mvn dependency:tree
    mvn dependency:tree -Dverbose=true
    mvn dependency:tree -Dverbose=true -Dincludes=...

    View Slide

  8. 8
    @aalmiray @saturnism
    Example 1
    Translate Truth
    Guava 19
    Guava 21
    Which version to use?

    View Slide

  9. 9
    @aalmiray @saturnism
    Any dependency there can be only ONE version
    Classpath - First class wins
    Maven - Nearest wins

    View Slide

  10. 10
    @aalmiray @saturnism
    Maven does not understand Semver
    Nor compatibility

    View Slide

  11. 11
    @aalmiray @saturnism
    Use Maven Enforcer
    Convergence vs Upper bound
    http://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html
    (See Code!)

    View Slide

  12. 12
    @aalmiray @saturnism
    Ensuring only ONE version
    of the dependency in tree
    Exclusions, or Dependency Management
    (See Code)

    View Slide

  13. 13
    @aalmiray @saturnism
    Upper bound if
    higher version is backwards compatible
    [JLBP-7] Make breaking transitions easy
    [JLBP-10] Maintain API stability as long as needed for consumers
    (Guava 21 and up are backwards compatible)

    View Slide

  14. 14
    @aalmiray @saturnism
    If upper version is breaking lower version…
    Or, system classpath has an incompatible version
    (Hadoooooop)

    View Slide

  15. 15
    @aalmiray @saturnism
    Shade + Relocation :(
    Classloaders
    OSGi

    View Slide

  16. 16
    @aalmiray @saturnism
    GA libraries don't depend on non-GA APIs
    alpha, beta, RC, 0.xx, @UnstableApi, @Beta, @Internal
    [JLBP-4] Avoid dependencies on unstable libraries and features

    View Slide

  17. 17
    @aalmiray @saturnism
    Major release, breaking changes
    Use new Group ID or Artifact ID - different coordinate!
    AND Use a new package name
    [JLBP-6] Rename artifacts and packages together

    View Slide

  18. 18
    @aalmiray @saturnism
    How do we change versions in transitive
    dependencies?

    View Slide

  19. 19
    @aalmiray @saturnism
    Maven Dependency Management

    View Slide

  20. 20
    @aalmiray @saturnism
    Let's see Example 3e

    View Slide

  21. 21
    @aalmiray @saturnism
    Example 1
    PubSub Firebase
    guava-jdk5 17.0
    guava 28.0

    View Slide

  22. 22
    @aalmiray @saturnism
    Ban Duplicate Classes
    https://www.mojohaus.org/extra-enforcer-rules/banDuplicateClasses.html

    View Slide

  23. 23
    @aalmiray @saturnism
    2 artifacts should not have overlapping classes
    [JLBP-5] Avoid dependencies that overlap classes with other dependencies
    [JLBP-19] Place each package in only one module

    View Slide

  24. 24
    @aalmiray @saturnism
    [JLBP-11] Keep dependencies up to date!

    View Slide

  25. 25
    @aalmiray @saturnism
    Let's fix all of these
    And see the next example 3c

    View Slide

  26. 26
    @aalmiray @saturnism
    Example
    Pubsub Trace
    grpc-stub 1.10.1
    grpc-all 1.0.1
    grpc-core 1.0.1 grpc-... 1.0.1

    View Slide

  27. 27
    @aalmiray @saturnism
    Version Misalignment

    View Slide

  28. 28
    @aalmiray @saturnism
    How do we change versions in transitive
    dependencies?

    View Slide

  29. 29
    @aalmiray @saturnism
    Example
    Pubsub Trace
    grpc-stub 1.10.1
    grpc-all 1.10.1
    grpc-core 1.10.1 grpc-... 1.10.1
    grpc-bom 1.10.1

    View Slide

  30. 30
    @aalmiray @saturnism
    Multi-module projects Produce a BOM!
    [JLBP-15] Publish a BOM for multi-module projects

    View Slide

  31. 31
    @aalmiray @saturnism
    See an example!

    View Slide

  32. 32
    @aalmiray @saturnism
    Power → Responsibility
    BOM should not overreach!
    ONLY modules within the multi-project project

    View Slide

  33. 33
    @aalmiray @saturnism
    gRPC also uses another method
    Strict Version Ranges for inter-module dependencies,
    e.g. [1.10.1]

    View Slide

  34. 34
    @aalmiray @saturnism
    Linkage Checker
    Static Analysis to identify missing links
    https://github.com/GoogleCloudPlatform/cloud-opensource-java/tree/master/enforcer-rules

    View Slide

  35. 35
    @aalmiray @saturnism
    Bigger issues with
    Dependency Management...

    View Slide

  36. 36
    @aalmiray @saturnism
    What you build & test with
    Is not what consumers get!

    View Slide

  37. 37
    @aalmiray @saturnism
    Flatten the POM
    https://github.com/mojohaus/flatten-maven-plugin/pull/93

    View Slide

  38. 38
    @aalmiray @saturnism
    Bonus
    mvn dependency:analyze - find dependencies you don't need
    Maven 3.6.3 - fixes our exclusion fix in Maven 3.6.2

    View Slide

  39. 39
    @aalmiray @saturnism
    Enforcer Rules
    Enforcer Version Convergence or Upperbound
    Ban Duplicate Classes
    Linkage Checker
    Manage Transitive Versions

    BOM for Multi-Module
    Import BOM
    Maven Flatten Plugin

    View Slide

  40. 40
    @aalmiray @saturnism
    Visit the best practices at JLBP.dev
    Thanks!
    Andres Almiray @aalmiray
    Ray Tsang @saturnism
    Maven used by 60% to 80% of Java projects
    92 projects, 50+ plugins, lots of libraries - Contribution Appreciated!

    View Slide