Case Study: Spring Projects on Gradle

21874091836ab7d3a769c1a89a0702f3?s=47 Chris Beams
September 29, 2011

Case Study: Spring Projects on Gradle

Slides from presentation delivered at SpringOne/2GX 2011 in Washington DC. Note that the PDF export here is lossy with regard to style. Follow the instructions at https://github.com/cbeams-archive/spring-on-gradle#to-view-the-slides to view the slides in the original format.

21874091836ab7d3a769c1a89a0702f3?s=128

Chris Beams

September 29, 2011
Tweet

Transcript

  1. None
  2. None
  3. -- on --

  4. Goal

  5. Goal: Share Why How Pro Con

  6. SpringSource OSS Projects

  7. (partial list)

  8. (just the JVM stuff)

  9. SpringSource JVM Projects spring-framework spring-security spring-ws spring-webflow spring-batch spring-integration spring-dm

    spring-ldap spring-roo
  10. ... grails groovy spring-amqp spring-android spring-data spring-gemfire spring-mobile spring-social

  11. Who's on Gradle?

  12. Spring Security

  13. Spring Integration

  14. Groovy

  15. Grails

  16. Spring Android

  17. Spring Mobile

  18. Spring Social

  19. Who's on Gradle? spring-security spring-integration groovy* grails spring-android spring-mobile spring-social

    spring-amqp* spring-data*
  20. Who's on Gradle? 7 out of 17 projects > 40%

    All moves made within the last year All have released at least one milestone or GA
  21. Why?

  22. Why? To separate concerns

  23. Why? To make concepts explicit

  24. Why? To unify dependency metadata

  25. Why? To make the build concise

  26. Why? To make the build easy to use

  27. Why? To make the build fast

  28. Why? To facilitate reuse

  29. Why? To fully automate the release process

  30. (details on all that coming up...)

  31. But first...

  32. Debate

  33. (We might as well get it out of the way)

  34. Gradle may be nice, but... Maven POMs are critical ∴

    the build must be Maven-based ... right?
  35. Maybe not

  36. apply plugin: 'maven'

  37. apply plugin: 'maven' Poms are: Generated from Gradle metadata Installed

    with jars to local .m2 cache Uploaded to remote Maven repositor(ies)
  38. apply plugin: 'maven' $ gradle install # poms and jars

    to local m2 cache $ gradle uploadArchives # poms and jars to remote repo
  39. Result Single source of dependency metadata Clean, minimal poms Separates

    the concerns of build instructions vs dependency manifest Interoperability with Maven-based builds
  40. Debate

  41. Gradle may be nice, but... Many Spring users know Maven

    ∴ the build must be Maven-based ... right?
  42. Maybe not

  43. Assumes Spring users build from source Relatively uncommon But for

    those that do... What is actually required?
  44. Building from source needs to: be simple facilitate IDE import

    install Maven poms to local .m2 cache
  45. What's the status quo? Spring 3 Ant Build Six steps

    - not easy IDE import is complicated We can definitely do better
  46. Maven makes it easy $ git clone $GIT_URL # Import

    poms into m2eclipse / IDEA $ mvn install ...
  47. So does Gradle $ git clone $GIT_URL $ cd spring-project

    $ ./gradlew eclipse # or 'idea' # Import projects into IDE $ ./gradlew install ...
  48. Assertion As long as building from source is simple, works

    with their IDE and interoperates with Maven... ... most users will be happy
  49. Ultimately build system == implementation decision must meet basic requirements

    (like Maven interop) project lead decides what's best
  50. </debate>?

  51. Hardly.

  52. Every organization has different needs So the debate differs

  53. Needs at SpringSource Common deployment infrastructure and procedures Distributed team

    No dedicated CM team or buildmaster Many projects Many users Many Maven repositories Special versioning needs ...
  54. Case Study

  55. Spring Integration

  56. (brief intro)

  57. None
  58. None
  59. It's all about

  60. Messages

  61. Channels

  62. Endpoints

  63. And the patterns they form

  64. None
  65. None
  66. None
  67. None
  68. Spring Integration 22 sub-projects 2.0 GA released on Nov 22nd

    2010
  69. Separate Concerns

  70. Build User Roles Build Consumer Build Operator Build Master

  71. Build Consumer Depends on Maven artifacts Downloads distribution zip Reads

    documentation Probably never thinks about the build system
  72. Build Operator Builds from source (compile, test, etc) May update

    dependency metadata May add new subprojects Needs IDE integration
  73. Build Master Everything an operator does Releases the project Maintains

    the project build Configures CI Writes custom build logic for special needs
  74. Let's design a build with these users in mind Start

    by separating concerns
  75. Build Consumer Publish poms and jars to maven repos Upload

    docs and dist zips Pretty simple
  76. Build Operator Make checkout/build/IDE-import dead simple Optimize common use cases

    like compile/test/docs Separate declarative from imperative in build scripts Make the release process as easy as possible Document anything that's not obvious
  77. Build Master Treat imperative build sources like any other project

    Make it editable in the IDE Write tests for the build Keep common build code DRY
  78. Make Concepts Explicit

  79. Licensing

  80. Spring Versioning

  81. Unify Metadata

  82. Hand-maintained Metadata under Spring (Ant) Build Ivy (ivy.xml) - 'canonical'

    Eclipse (.classpath, .project) IDEA (.iml, .ipr) OSGi (Bundlor template.mf) Maven (pom.xml)
  83. Concise Scripts

  84. Concise Output

  85. $ mvn clean compile [INFO] Scanning for projects... [INFO] -------------------------------------------------------------

    [INFO] Building Spring Integration Core [INFO] task-segment: [clean, compile] [INFO] ------------------------------------------------------------- [INFO] [clean:clean {execution: default-clean}] [INFO] Deleting directory /Users/cbeams/Work/pax/spring-integration/ [INFO] [resources:resources {execution: default-resources}] [INFO] Using 'UTF8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] Copying 6 resources [INFO] [compiler:compile {execution: default-compile}] [INFO] Compiling 297 source files to /Users/cbeams/Work/pax/spring-i [INFO] ------------------------------------------------------------- [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------- [INFO] Total time: 6 seconds [INFO] Finished at: Wed Nov 24 15:42:19 MST 2010 [INFO] Final Memory: 21M/81M [INFO] -------------------------------------------------------------
  86. $ gradle clean compileJava :spring-integration-core:clean :spring-integration-core:compileJava BUILD SUCCESSFUL Total time:

    5.621 secs
  87. $ mvn clean compile [INFO] Scanning for projects... [INFO] -------------------------------------------------------------

    [INFO] Building Spring Integration Core [INFO] task-segment: [clean, compile] [INFO] ------------------------------------------------------------- [INFO] [clean:clean {execution: default-clean}] [INFO] Deleting directory /Users/cbeams/Work/pax/spring-integration/ [INFO] [resources:resources {execution: default-resources}] [INFO] Using 'UTF8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] Copying 6 resources [INFO] [compiler:compile {execution: default-compile}] [INFO] Compiling 297 source files to /Users/cbeams/Work/pax/spring-i [INFO] ------------------------------------------------------------- [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------- [INFO] Total time: 6 seconds [INFO] Finished at: Wed Nov 24 15:42:19 MST 2010 [INFO] Final Memory: 21M/81M [INFO] -------------------------------------------------------------
  88. $ gradle clean compileJava :spring-integration-core:clean :spring-integration-core:compileJava BUILD SUCCESSFUL Total time:

    5.621 secs
  89. None
  90. Broken Windows To be fixed... ... they must first be

    seen
  91. $ mvn clean compile [INFO] Scanning for projects... [INFO] -------------------------------------------------------------

    [INFO] Building Spring Integration FTP Support [INFO] task-segment: [clean, compile] [INFO] ------------------------------------------------------------- [INFO] [clean:clean {execution: default-clean}] [INFO] Deleting directory /Users/cbeams/Work/pax/spring-integration/ [INFO] [resources:resources {execution: default-resources}] [INFO] Using 'UTF8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] Copying 3 resources [INFO] [compiler:compile {execution: default-compile}] [INFO] Compiling 10 source files to /Users/cbeams/Work/pax/spring-in [WARNING] /Users/cbeams/Work/pax/spring-integration/spring-integrati found : org.apache.commons.net.ftp.FTPFile[] required: F[] [INFO] ------------------------------------------------------------- [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------- [INFO] Total time: 5 seconds [INFO] Finished at: Wed Nov 24 17:10:18 MST 2010 [INFO] Final Memory: 21M/81M [INFO] -------------------------------------------------------------
  92. $ gradle clean compileJava :spring-integration-ftp:compileJava /Users/cbeams/Work/pax/spring-integration/spring-integration-ftp/src found : org.apache.commons.net.ftp.FTPFile[] required:

    F[] public FTPFile[] ls(String path) { ^ 1 warning BUILD SUCCESSFUL Total time: 4.874 secs
  93. $ mvn clean compile | wc 351 1626 22858 $

    gradle clean compileJava | wc 113 257 6378
  94. $ mvn clean test | wc 550432 2046401 74486400 $

    gradle clean test | wc 22748 84781 3057486
  95. 25x less output not bad but 22,748 lines is still

    way too much
  96. test { logging.captureStandardOutput(LogLevel.INFO) }

  97. $ mvn clean test | wc 550432 2046401 74486400 $

    gradle clean test | wc 227 317 8792
  98. 8,472x less output that's better

  99. Ease of Use

  100. Ease of Use Gradle Wrapper Partial builds Task discoverability Support

    source poms gradle help AMQP test exclusions Multi-repository deployment
  101. Get running with Gradle $ git clone --recursive $GIT_URL Cloning

    into spring-integration... remote: Counting objects: 19483, done. remote: Compressing objects: 100% (10387/10387), done. ... $ cd spring-integration && ./gradlew eclipse :spring-integration-core:eclipseClasspath :spring-integration-core:eclipseJdt :spring-integration-core:eclipseProject :spring-integration-core:eclipse :spring-integration-event:eclipseClasspath :spring-integration-event:eclipseJdt :spring-integration-event:eclipseProject :spring-integration-event:eclipse ... BUILD SUCCESSFUL
  102. Wrapper == self-contained builds $ git clone --recursive $GIT_URL $

    cd spring-integration $ ./gradlew build Downloading http://dist.codehaus.org/gradle/gradle-0.9-rc-3-bin.zip ................................................................... ................................................................... Unzipping /Users/cbeams/.gradle/wrapper/dists/gradle-0.9-rc-3-bin.zi Set executable permissions for: /Users/cbeams/.gradle/wrapper/dists/ > Loading ... BUILD SUCCESSFUL
  103. What about CI?

  104. Wrapper makes this easy

  105. Make it Fast

  106. Make it Fast Fast at startup Fast clean build Fast

    average build
  107. Fast at startup Gradle daemon

  108. Fast clean build Head-to-head with Maven

  109. Fast average build Incremental build

  110. Automate the release process

  111. Release One-page release instructions Literally "pushbutton" releases not necessarily desirable

  112. Getting Help

  113. Getting Help User Guide JavaDoc Mailing list

  114. Look at other projects All the Spring projects Gradle itself

    Hibernate Not yet clear what 'idomatic' Gradle looks like
  115. What Gradle needs

  116. DSL documentation

  117. The most important move Ant ever made

  118. None
  119. None
  120. DSL doc vs User Guide Gradle's got a great User

    Guide Explains concepts Walks through certain scenarios Very helpful when getting started
  121. DSL doc vs API doc Gradle's got JavaDoc Often useful

    But it's not enough DSL often relies on Groovy metaprogramming JavaDoc can't cover that
  122. e.g. subprojects { // what properties are available? }

  123. DSL doc Would be, well, DSL-centric Every day reference for

    build masters
  124. Good News

  125. Plugin Portal

  126. None
  127. None
  128. Test-Driven Builds

  129. Test-Driven Builds For the same reasons we test other code

    Special challenges
  130. Tooling

  131. Tooling Gradle UI is there IDEA integrates it Eclipse tooling

    is a must Bash-completion / Gradle shell Project quick-start (archetypes)
  132. Key Plugins

  133. Key Plugins Docbook AspectJ ...

  134. Rich Deployment/Rollback

  135. Rich Deployment/Rollback Building blocks are there Whole subdomain that needs

    modeling Look to Capistrano for inspiration
  136. Patterns

  137. None
  138. Patterns of Enterprise Application Architecture

  139. Patterns of Enterprise Application Architecture

  140. Patterns of Enterprise Project Automation

  141. Patterns of Enterprise Project Automation Level the playing field Provide

    common concepts Good for competition
  142. Somebody please write this! (Hans, I'm looking at you)

  143. Summary

  144. Benefits we've gained Clearly modeled and separated concerns Flexibility Concise

    scripts & output Ease of use Fast execution Reusable build sources
  145. Challenges remaining Reuse model not ideal Maintaining our own docbook

    plugin Eclipse tooling is much needed
  146. Lessons learned Document the build Treat the build like code

    Design the build for user roles and their workflows Total build homogeny == false goal
  147. Thanks! Slides: https://github.com/cbeams/spring-on-gradle Source: https://github.com/springsource/spring-build- gradle