Minseok Oh / 2020-10-07 gradle-multi-project-support Multi-project Management Tool

About me • September 2017, join LINE auth team as a server developer • From June 2020, working for blockchain platform • Interested in development productivity • Common keyboard mania around you

Mono-repo • In MSA, multiple source code repositories occur usually. • There can be a task required to write your change across two or more repositories. • Known issues in this case • Work several times for just one task. • It needs to check and manage which version is compatible with each related repository. (a.k.a dependency hell)

No content

Mono-repo (Cont'd) • Consider mono-repo as an alternative • Mono-repo's advantages • Easy to reuse code • Room to simplify the management of dependencies • Atomic commits available • Large-scale code refactoring

Mono-repo (Cont'd) • Changes after mono-repo • Accelerated changes across multiple services • Relocation of features to change service structure • Extensive code refactoring to improve code quality

Mono-repo (Cont'd) • From multi-repo to mono-repo • There are some well-known problems.
 (ex: partial damage to semantic versioning) • However migration to mono-repo is agreed. • We believed that the increase in productivity brought by mono-repo will be significant.

Gradle • There was the time when we used two build tools when we migrating to mono-repo. • Maven, Gradle

Gradle (Cont'd) • As a result, integrated into Gradle. • Performance and developer experience were considered mainly • Gradle shows relatively better build performance. • Incrementality : Gradle avoids work by tracking input and output of tasks and only running what is necessary, and only processing files that changed when possible. • Build Cache: Reuses the build outputs of any other Gradle build with the same inputs, including between machines. • For better developer experience, • Gradle provides Kotlin-based DSL. • The programming language used for development to match the required for build process

Many duplications • After the service structure change and code refactoring, a lot of duplications were still found. • Client settings except for the business interfaces • Common features required by multiple services • Configuration classes, properties, and so on

Many duplications (Cont'd) • Extract the duplications into Gradle's sub-projects. • Use other sub-projects as dependencies • In this step, • Code duplication had been greatly reduced. • Reuse in mono-repo was increased.

Many duplications (Cont'd) • But, new duplications were found. • As written more sub-projects, the same build scripts appear multiple times. • Some sub-projects with Spring Boot auto-configuration had similar but slightly different settings. • Some sub-projects were omitted from the build script refactoring by human fault. • New approach was required.

No content

Gradle's configure (Cont'd) • Order of running scripts in Gradle 1. settings.gradle(.kts) 2. build.gradle(.kts) in project root 3. build.gradle(.kts) in each sub-project

Gradle's configure (Cont'd) • Extract the duplications of build script using Gradle's configure. • Almost all build scripts were extracted except for some dependency declarations. • Root build script should know all sub-project's names and those characteristics.

Gradle's configure (Cont'd) • Introduce 'type' property • type property represents each sub-project's characteristics.
 (ex: java-lib, kotlin-lib, kotlin- boot-lib, java-boot-lib) • Sub-projects' names were eliminated in root build script by type property.

line/gradle-multi-project-support • Retrospective with members • Significant improvement in productivity • All the benefits of mono-repo we want • Easy to create new sub-projects using type property • Low readability of the build script using type property • As the number of sub-projects increases, • Difficult to determine the impact of changes. • Not easy anymore to create a release note.

line/gradle-multi-project-support (Cont'd) • Overcome the problems using Gradle's buildSrc. • Add functions to improve readability • Write Gradle plugins to print the affected Git commits for each sub-project.

line/gradle-multi-project-support (Cont'd) • And we released these codes as OSS(open source software). • Share approaches only to avoid creating new copy & paste. • The key idea is to allow users to frame mono-repo in their own style.

Wrap-up • Migrate multi-repo into mono-repo • Replace Maven with Gradle • Modularize common codes • Extract common build scripts • Improve readability of build script • Write some useful Gradle plugins

Give it a try! https://github.com/line/gradle-multi-project-support