Multi-project management tool 「gradle-multi-project-support」

Transcript

1. Minseok Oh / 2020-10-07 gradle-multi-project-support Multi-project Management Tool

2. About me • September 2017, join LINE auth team as

   a server developer • From June 2020, working for blockchain platform • Interested in development productivity • Common keyboard mania around you

3. Mono-repo • In MSA, multiple source code repositories occur usually.

   • There can be a task required to write your change across two or more repositories. • Known issues in this case • Work several times for just one task. • It needs to check and manage which version is compatible with each related repository. (a.k.a dependency hell)
4. None

5. Mono-repo (Cont'd) • Consider mono-repo as an alternative • Mono-repo's

   advantages • Easy to reuse code • Room to simplify the management of dependencies • Atomic commits available • Large-scale code refactoring

6. Mono-repo (Cont'd) • Changes after mono-repo • Accelerated changes across

   multiple services • Relocation of features to change service structure • Extensive code refactoring to improve code quality

7. Mono-repo (Cont'd) • From multi-repo to mono-repo • There are

   some well-known problems.
 (ex: partial damage to semantic versioning) • However migration to mono-repo is agreed. • We believed that the increase in productivity brought by mono-repo will be significant.

8. Gradle • There was the time when we used two

   build tools when we migrating to mono-repo. • Maven, Gradle

9. Gradle (Cont'd) • As a result, integrated into Gradle. •

   Performance and developer experience were considered mainly • Gradle shows relatively better build performance. • Incrementality : Gradle avoids work by tracking input and output of tasks and only running what is necessary, and only processing files that changed when possible. • Build Cache: Reuses the build outputs of any other Gradle build with the same inputs, including between machines. • For better developer experience, • Gradle provides Kotlin-based DSL. • The programming language used for development to match the required for build process

10. Many duplications • After the service structure change and code

    refactoring, a lot of duplications were still found. • Client settings except for the business interfaces • Common features required by multiple services • Configuration classes, properties, and so on

11. Many duplications (Cont'd) • Extract the duplications into Gradle's sub-projects.

    • Use other sub-projects as dependencies • In this step, • Code duplication had been greatly reduced. • Reuse in mono-repo was increased.

12. Many duplications (Cont'd) • But, new duplications were found. •

    As written more sub-projects, the same build scripts appear multiple times. • Some sub-projects with Spring Boot auto-configuration had similar but slightly different settings. • Some sub-projects were omitted from the build script refactoring by human fault. • New approach was required.
13. None

14. Gradle's configure (Cont'd) • Order of running scripts in Gradle

    1. settings.gradle(.kts) 2. build.gradle(.kts) in project root 3. build.gradle(.kts) in each sub-project

15. Gradle's configure (Cont'd) • Extract the duplications of build script

    using Gradle's configure. • Almost all build scripts were extracted except for some dependency declarations. • Root build script should know all sub-project's names and those characteristics.

16. Gradle's configure (Cont'd) • Introduce 'type' property • type property

    represents each sub-project's characteristics.
 (ex: java-lib, kotlin-lib, kotlin- boot-lib, java-boot-lib) • Sub-projects' names were eliminated in root build script by type property.

17. line/gradle-multi-project-support • Retrospective with members • Significant improvement in productivity

    • All the benefits of mono-repo we want • Easy to create new sub-projects using type property • Low readability of the build script using type property • As the number of sub-projects increases, • Difficult to determine the impact of changes. • Not easy anymore to create a release note.

18. line/gradle-multi-project-support (Cont'd) • Overcome the problems using Gradle's buildSrc. •

    Add functions to improve readability • Write Gradle plugins to print the affected Git commits for each sub-project.

19. line/gradle-multi-project-support (Cont'd) • And we released these codes as OSS(open

    source software). • Share approaches only to avoid creating new copy & paste. • The key idea is to allow users to frame mono-repo in their own style.

20. Wrap-up • Migrate multi-repo into mono-repo • Replace Maven with

    Gradle • Modularize common codes • Extract common build scripts • Improve readability of build script • Write some useful Gradle plugins

21. Give it a try! https://github.com/line/gradle-multi-project-support