Tweet
Tweet

Slide 1

Slide 1 text

State of the Art Gradle Multi-Module Builds Benjamin Muschko, Gradle Inc.

Slide 2

Slide 2 text

Complex software consists of modules

Slide 3

Slide 3 text

Benefits of modularization Maintainability Reusability Separation  of  concerns

Slide 4

Slide 4 text

Enablers for implementing separation of concerns Minimize  coupling Maximize  cohesion

Slide 5

Slide 5 text

It’s all about your view of the world

Slide 6

Slide 6 text

Your view might involve legacy structures

Slide 7

Slide 7 text

Configurability is key

Slide 8

Slide 8 text

Multi-module or multi-project? A B C  Component  or  module  is  term   originating  from  software  engineering

Slide 9

Slide 9 text

Multi-module or multi-project? A B C  Component  or  module  is  term   originating  from  software  engineering A B C Project  is  a  term  referring  to   the  domain  object  in  Gradle

Slide 10

Slide 10 text

Projects can depend on each other A B C No  project  depends  on  the     output  of  another  project

Slide 11

Slide 11 text

Projects can depend on each other A B C A B C No  project  depends  on  the     output  of  another  project        Projects  define  one  or  many   dependencies  on  other  project(s)

Slide 12

Slide 12 text

Project structure - hierarchical or flat    Hierarchical:  deeply  nested  structure A B C D E

Slide 13

Slide 13 text

Project structure - hierarchical or flat A Flat:  structure  only  one  level  deep    Hierarchical:  deeply  nested  structure B C A B C D E

Slide 14

Slide 14 text

Project vs. binary dependencies A B C Different  VCS  repositories,   dependencies  are  published   artifacts  in  a  binary  repository VCS  repo VCS  repo VCS  repo

Slide 15

Slide 15 text

Project vs. binary dependencies A B C A B C Different  VCS  repositories,   dependencies  are  published   artifacts  in  a  binary  repository      Projects  usually  live  in            same  VCS  repository VCS  repo VCS  repo VCS  repo VCS  repo

Slide 16

Slide 16 text

Declaring project dependencies Method  Project#project(String)   provides  access  to  Project  reference 
 dependencies {
 compile project(':c')
 } build.gradle A C Creates  JAR  of  project  C  and  adds  it  to  the  compile  classpath  of  project  A.

Slide 17

Slide 17 text

Declaring module dependencies A  binary  dependency  is  resolved  by  its   coordinates  referring  to  a  published  artifact 
 dependencies {
 compile 'org.gradle:c:1.0'
 } build.gradle A C Binary  repository

Slide 18

Slide 18 text

Main role of the settings file Declares  projects  that  should  take  part  in  the   build 
 include 'a'
 include 'b:e'
 include 'c' settings.gradle 
 includeFlat 'a'
 includeFlat 'b'
 includeFlat 'c' settings.gradle                      Flat     project  structure        Hierarchical     project  structure

Slide 19

Slide 19 text

How many? Where to put it? Single  settings  file  per  multi-­‐project  build Root  project  name  is  derived  of  dir.  name  containing  settings  file. A B C settings.gradle A B C master settings.gradle

Slide 20

Slide 20 text

Resolving the settings file 2-­‐step  approach   1. Search  for  file  in  directory  named  master with  same  nesting  level  as  current  directory.   2. If  no  file  is  found  search  in  parent   directories  starting  from  current  directory.

Slide 21

Slide 21 text

Evaluation of logic in settings file Initialization            Phase Configuration              Phase Execution        Phase Gradle’s  Build  Lifecycle Executed  in  the  first  phase  of  Gradle’s  build  lifecycle. settings.gradle

Slide 22

Slide 22 text

Configuration injection Define  common  configuration  for  1..n   projects

Slide 23

Slide 23 text

Configuration injection Define  common  configuration  for  1..n   projects 
 subprojects {
 ...
 } All  subprojects  of  the  project     that  defines  this  logic

Slide 24

Slide 24 text

Configuration injection Define  common  configuration  for  1..n   projects 
 subprojects {
 ...
 } 
 allprojects {
 ...
 } All  subprojects  of  the  project     that  defines  this  logic All  subprojects  of  the  project     that  defines  this  logic  and  the   project  itself

Slide 25

Slide 25 text

Using configuration injection A B C D E

Slide 26

Slide 26 text

Using configuration injection A B C D E 
 subprojects {
 apply plugin: 'java'
 } C D E Java Java

Slide 27

Slide 27 text

Using configuration injection A B C D E 
 subprojects {
 apply plugin: 'java'
 } 
 allprojects {
 apply plugin: 'java'
 } C D E C D E Java Java Java Java Java

Slide 28

Slide 28 text

Filtering based on subproject configuration A B C D E 
 def webProjects() {
 subprojects.findAll { subproject -> subproject.plugins.hasPlugin('war') }
 }
 
 gradle.projectsEvaluated {
 configure(webProjects()) {
 ...
 }
 }

Slide 29

Slide 29 text

Filtering based on subproject configuration A B C D E 
 def webProjects() {
 subprojects.findAll { subproject -> subproject.plugins.hasPlugin('war') }
 }
 
 gradle.projectsEvaluated {
 configure(webProjects()) {
 ...
 }
 } Alternative:   evaluationDependsOnChildren()

Slide 30

Slide 30 text

Task execution based on name matching Find  and  execute  task(s)  with  requested  name A B C D E 
 ~/dev/myApp/c $ gradle war :d:compileJava :d:processResources UP-TO-DATE :d:classes :d:jar :e:compileJava :e:processResources UP-TO-DATE :e:classes :e:war BUILD SUCCESSFUL

Slide 31

Slide 31 text

Task execution based on name matching Find  and  execute  task(s)  with  requested  name A B C D E 
 ~/dev/myApp/c $ gradle war :d:compileJava :d:processResources UP-TO-DATE :d:classes :d:jar :e:compileJava :e:processResources UP-TO-DATE :e:classes :e:war BUILD SUCCESSFUL Project  E  depends  on  D

Slide 32

Slide 32 text

Project and task execution path notation Full  path  notation  allows  for  being  specific A B C D E 
 ~/dev/myApp/c $ gradle e:compileJava :d:compileJava :d:processResources UP-TO-DATE :d:classes :d:jar :e:compileJava BUILD SUCCESSFUL Project Task

Slide 33

Slide 33 text

Selecting project(s) A B C D E

Slide 34

Slide 34 text

Selecting project(s) A B C D E 
 project.rootProject A

Slide 35

Slide 35 text

Selecting project(s) A B C D E 
 project.rootProject A 
 project.project(':c').children D E

Slide 36

Slide 36 text

Selecting project(s) A B C D E 
 project.rootProject A 
 project.project(':c').children D E 
 project.project(':')    Top-­‐level     root  project

Slide 37

Slide 37 text

Case-study application • High-­‐level  information  about  available   Gradle  plugins   • Query  Bintray  API  to  retrieve  data   • Render  information  on  page

Slide 38

Slide 38 text

DEMO Showcasing  the  application  functionality

Slide 39

Slide 39 text

Single build file Only  recommended  for  builds  with  little  logic 
 project(':a') {
 ...
 }
 
 project(':b') {
 ...
 }
 
 project(':c') {
 ...
 } 
 include 'a'
 include 'b'
 include 'c' build.gradle settings.gradle Requires  explicit  use  of  project  method.

Slide 40

Slide 40 text

DEMO Using  a  single  build  file  to  structure  logic

Slide 41

Slide 41 text

Multiple build files Best  default  option  for  organizing  build  logic A B C build.gradle A  project  is  not  required  to  have  a  build.gradle  file. 
 include 'a'
 include 'b'
 include 'c' settings.gradle build.gradle build.gradle build.gradle

Slide 42

Slide 42 text

DEMO Using  a  multiple  build  files  to  structure  logic

Slide 43

Slide 43 text

Hybrid approach Avoid  using  this  option  as  it  might  cause   potential  maintenance  issues  down  the  road A B C build.gradle 
 project(':b') {
 ...
 } Root  project  build.gradle 
 include 'a'
 include 'b'
 include 'c' settings.gradle build.gradle build.gradle

Slide 44

Slide 44 text

DEMO Using  a  hybrid  approach  to  structure  logic

Slide 45

Slide 45 text

Partial builds of a subproject “build  this  project  but  none  of  its  project  dependencies”:   --no-rebuild, -a “build  &  test  this  project  and  its  project  dependencies”:   buildNeeded “build  &  test  this  project  and  projects  that  depend  on   current  project”:   buildDependents

Slide 46

Slide 46 text

DEMO Using  partial  build  options

Slide 47

Slide 47 text

Highly customized Settings configuration Typical  scenarios:   • Changing  default  project  names   • Changing  default  project  directories   • Changing  default  build  file  names   • Including  projects  through  external  file   definition

Slide 48

Slide 48 text

Changing project names I  inherited  a  legacy  project  structure.  Directory  names  are   not  descriptive  enough.  How  do  I  change  the  project  names?

Slide 49

Slide 49 text

Changing project names I  inherited  a  legacy  project  structure.  Directory  names  are   not  descriptive  enough.  How  do  I  change  the  project  names? 
 rootProject.name = 'myApp'
 include 'shared', 'web', 'client' 
 rootProject.children.each {
 it.name = 'myApp-' + it.name
 }
 settings.gradle

Slide 50

Slide 50 text

DEMO Assigning  custom  project  names

Slide 51

Slide 51 text

Changing build file names It  gets  pretty  confusing  in  my  IDE  if  I  work  on  multiple   build.gradle  files  in  parallel.  Can  I  give  them  custom  names?

Slide 52

Slide 52 text

Changing build file names It  gets  pretty  confusing  in  my  IDE  if  I  work  on  multiple   build.gradle  files  in  parallel.  Can  I  give  them  custom  names? 
 rootProject.name = 'myApp'
 include 'shared', 'web', 'client' 
 rootProject.children.each {
 it.buildFileName = it.name + '.gradle' - 'myApp' }
 settings.gradle

Slide 53

Slide 53 text

DEMO Assigning  custom  project  file  names

Slide 54

Slide 54 text

Including projects by external file definition My  list  of  projects  and  their  build  file  names  has  become  very     long.  Can  I  externalize  this  definition  into  another  file?

Slide 55

Slide 55 text

Including projects by external file definition My  list  of  projects  and  their  build  file  names  has  become  very     long.  Can  I  externalize  this  definition  into  another  file? 
 {
 "projects": {
 "shared": {
 "name": "myApp-shared",
 "buildFileName": "shared.gradle"
 },
 "web": {
 "name": "myApp-web",
 "buildFileName": "web.gradle"
 }
 }
 }

Slide 56

Slide 56 text

DEMO Deriving  project  inclusions  from  JSON  file

Slide 57

Slide 57 text

Externalizing Settings logic into a plugin As  soon  as  the  logic  becomes  more  complex,   write  a  plugin. 
 package org.gradle.training
 
 import org.gradle.api.Plugin
 import org.gradle.api.initialization.Settings
 
 class DynamicProjectsSettingsPlugin implements Plugin {
 @Override
 void apply(Settings settings) {
 // work on Settings instance
 }
 }

Slide 58

Slide 58 text

DEMO Writing  a  Settings  plugin

Slide 59

Slide 59 text

Working on componentized projects A B C VCS  repo D VCS  repo E VCS  repo F VCS  repo Deliverable  artifact  comprises  of  all  components.

Slide 60

Slide 60 text

Declared module dependencies A B C VCS  repo D VCS  repo E VCS  repo F VCS  repo 
 dependencies {
 compile 'org.gradle:f:1.0'
 } 
 dependencies {
 compile 'org.gradle:d:2.3'
 }

Slide 61

Slide 61 text

Working on multiple components during development A B C VCS  repo D VCS  repo E VCS  repo F VCS  repo Requires  publishing  of  F  and  D.  What  about  E?

Slide 62

Slide 62 text

Replacing module with project dependencies A B C VCS  repo D VCS  repo E VCS  repo F VCS  repo 
 dependencies {
 compile project(':f')
 } 
 dependencies {
 compile project(':d')
 }

Slide 63

Slide 63 text

The problem Dynamically  switching  between  module  and   project  dependencies   • Different  scenarios  (e.g.  development   environment  vs.  CI).   • How  do  I  tell  Gradle  which  one  to  use?   • How  do  I  make  it  work  in  the  IDE?

Slide 64

Slide 64 text

Open Source approaches 
 dynamicDependencies {
 compile group: 'com.example', name: 'example', version: '1.0'
 } 
 dependencies {
 compile elastic('com.example:example:1.0', 'example')
 } Prezi  Pride  (https:/ /github.com/prezi/pride) Elastic  Deps     (https:/ /github.com/pniederw/elastic-­‐deps)

Slide 65

Slide 65 text

DEMO Exploring  a  simple  custom  solution

Slide 66

Slide 66 text

Dependency substitution rules Gradle’s  API  cousin  to  dependency  resolve   rules   • Replace  external  with  project  dependency   • Replace  project  with  external  dependency Gradle  2.5

Slide 67

Slide 67 text

Replace external with project dependency Scenario:  You  are  working  on  a  module  and   want  to  constantly  integrate  the  code  -­‐  “fast   turnaround”. 
 configurations.all {
 resolutionStrategy {
 dependencySubstitution {
 withModule('org.utils:api') {
 useTarget project(':api') 
 }
 }
 }
 }

Slide 68

Slide 68 text

Replace project with external dependency Scenario:  You  know  the  code  of  the  module   hasn’t  changed  -­‐  “build  avoidance”. 
 configurations.all {
 resolutionStrategy {
 dependencySubstitution {
 withProject(':api') {
 useTarget 'org.utils:api:1.3' }
 }
 }
 }

Slide 69

Slide 69 text

More down the road… Current  functionality  is  a  low-­‐level  API   intended  to  build  on  top. Low-­‐level            API DSL              IDE   capabilities

Slide 70

Slide 70 text

DEMO Using  dependency  substitution  rules  in   practice

Slide 71

Slide 71 text

IDE support No  support  for  IntelliJ  at  the  moment Eclipse  STS  available  through  custom           model  used  with  tooling  API Planned  support  for  Buildship

Slide 72

Slide 72 text

DEMO Showcasing  Eclipse  STS  support

Slide 73

Slide 73 text

Managing large multi-project builds Configuration  time  increases  with  #  projects

Slide 74

Slide 74 text

Upgrade to >= Gradle 2.4 Significant  performance  improvements  in   Gradle  internals Speeds  up  build  up  to  40%!

Slide 75

Slide 75 text

Performance-related command line options Building  projects  in  parallel:     --parallel, --parallel-threads Only  configure  relevant  projects:   --configure-on-demand Avoid  rebuilding  project  dependencies:   --no-rebuild, -a

Slide 76

Slide 76 text

Using the new configuration model Caching  and  reusing  the  project  model A B C D E    Cache Work  in  progress

Slide 77

Slide 77 text

Modeling projects as SourceSets Minimize  time  spend  in  configuration  phase   by  not  using  projects/project  dependencies A B C D E SourceSet  A SourceSet  B SourceSet  C SourceSet  D SourceSet  E compileClasspath Requires  a  lot  of  custom  code!

Slide 78

Slide 78 text

Thank  You! Please  ask  questions… https:/ /www.github.com/bmuschko @bmuschko