Case Study: Spring Projects on Gradle

Slides from presentation delivered at SpringOne/2GX 2011 in Washington DC. Note that the PDF export here is lossy with regard to style. Follow the instructions at https://github.com/cbeams-archive/spring-on-gradle#to-view-the-slides to view the slides in the original format.

Transcript

3. -- on --
4. Goal
5. Goal: Share Why How Pro Con
6. SpringSource OSS Projects
7. (partial list)
8. (just the JVM stuff)
9. SpringSource JVM Projects spring-framework spring-security spring-ws spring-webflow spring-batch spring-integration spring-dm spring-ldap spring-roo
10. ... grails groovy spring-amqp spring-android spring-data spring-gemfire spring-mobile spring-social
11. Who's on Gradle?
12. Spring Security
13. Spring Integration
14. Groovy
15. Grails
16. Spring Android
17. Spring Mobile
18. Spring Social
19. Who's on Gradle? spring-security spring-integration groovy* grails spring-android spring-mobile spring-social spring-amqp* spring-data*
20. Who's on Gradle? 7 out of 17 projects > 40% All moves made within the last year All have released at least one milestone or GA
21. Why?
22. Why? To separate concerns
23. Why? To make concepts explicit
24. Why? To unify dependency metadata
25. Why? To make the build concise
26. Why? To make the build easy to use
27. Why? To make the build fast
28. Why? To facilitate reuse
29. Why? To fully automate the release process
30. (details on all that coming up...)
31. But first...
32. Debate
33. (We might as well get it out of the way)
34. Gradle may be nice, but... Maven POMs are critical ∴ the build must be Maven-based ... right?
35. Maybe not
36. apply plugin: 'maven'
37. apply plugin: 'maven' Poms are: Generated from Gradle metadata Installed with jars to local .m2 cache Uploaded to remote Maven repositor(ies)
38. apply plugin: 'maven' $ gradle install # poms and jars to local m2 cache $ gradle uploadArchives # poms and jars to remote repo
39. Result Single source of dependency metadata Clean, minimal poms Separates the concerns of build instructions vs dependency manifest Interoperability with Maven-based builds
40. Debate
41. Gradle may be nice, but... Many Spring users know Maven ∴ the build must be Maven-based ... right?
42. Maybe not
43. Assumes Spring users build from source Relatively uncommon But for those that do... What is actually required?
44. Building from source needs to: be simple facilitate IDE import install Maven poms to local .m2 cache
45. What's the status quo? Spring 3 Ant Build Six steps - not easy IDE import is complicated We can definitely do better
46. Maven makes it easy $ git clone $GIT_URL # Import poms into m2eclipse / IDEA $ mvn install ...
47. So does Gradle $ git clone $GIT_URL $ cd spring-project $ ./gradlew eclipse # or 'idea' # Import projects into IDE $ ./gradlew install ...
48. Assertion As long as building from source is simple, works with their IDE and interoperates with Maven... ... most users will be happy
49. Ultimately build system == implementation decision must meet basic requirements (like Maven interop) project lead decides what's best
50. </debate>?
51. Hardly.
52. Every organization has different needs So the debate differs
53. Needs at SpringSource Common deployment infrastructure and procedures Distributed team No dedicated CM team or buildmaster Many projects Many users Many Maven repositories Special versioning needs ...
54. Case Study
55. Spring Integration
56. (brief intro)
59. It's all about
60. Messages
61. Channels
62. Endpoints
63. And the patterns they form
68. Spring Integration 22 sub-projects 2.0 GA released on Nov 22nd 2010
69. Separate Concerns
70. Build User Roles Build Consumer Build Operator Build Master
71. Build Consumer Depends on Maven artifacts Downloads distribution zip Reads documentation Probably never thinks about the build system
72. Build Operator Builds from source (compile, test, etc) May update dependency metadata May add new subprojects Needs IDE integration
73. Build Master Everything an operator does Releases the project Maintains the project build Configures CI Writes custom build logic for special needs
74. Let's design a build with these users in mind Start by separating concerns
75. Build Consumer Publish poms and jars to maven repos Upload docs and dist zips Pretty simple
76. Build Operator Make checkout/build/IDE-import dead simple Optimize common use cases like compile/test/docs Separate declarative from imperative in build scripts Make the release process as easy as possible Document anything that's not obvious
77. Build Master Treat imperative build sources like any other project Make it editable in the IDE Write tests for the build Keep common build code DRY
78. Make Concepts Explicit
79. Licensing
80. Spring Versioning
81. Unify Metadata
82. Hand-maintained Metadata under Spring (Ant) Build Ivy (ivy.xml) - 'canonical' Eclipse (.classpath, .project) IDEA (.iml, .ipr) OSGi (Bundlor template.mf) Maven (pom.xml)
83. Concise Scripts
84. Concise Output
85. $ mvn clean compile [INFO] Scanning for projects... [INFO] ------------------------------------------------------------- [INFO] Building Spring Integration Core [INFO] task-segment: [clean, compile] [INFO] ------------------------------------------------------------- [INFO] [clean:clean {execution: default-clean}] [INFO] Deleting directory /Users/cbeams/Work/pax/spring-integration/ [INFO] [resources:resources {execution: default-resources}] [INFO] Using 'UTF8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] Copying 6 resources [INFO] [compiler:compile {execution: default-compile}] [INFO] Compiling 297 source files to /Users/cbeams/Work/pax/spring-i [INFO] ------------------------------------------------------------- [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------- [INFO] Total time: 6 seconds [INFO] Finished at: Wed Nov 24 15:42:19 MST 2010 [INFO] Final Memory: 21M/81M [INFO] -------------------------------------------------------------
86. $ gradle clean compileJava :spring-integration-core:clean :spring-integration-core:compileJava BUILD SUCCESSFUL Total time: 5.621 secs
87. $ mvn clean compile [INFO] Scanning for projects... [INFO] ------------------------------------------------------------- [INFO] Building Spring Integration Core [INFO] task-segment: [clean, compile] [INFO] ------------------------------------------------------------- [INFO] [clean:clean {execution: default-clean}] [INFO] Deleting directory /Users/cbeams/Work/pax/spring-integration/ [INFO] [resources:resources {execution: default-resources}] [INFO] Using 'UTF8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] Copying 6 resources [INFO] [compiler:compile {execution: default-compile}] [INFO] Compiling 297 source files to /Users/cbeams/Work/pax/spring-i [INFO] ------------------------------------------------------------- [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------- [INFO] Total time: 6 seconds [INFO] Finished at: Wed Nov 24 15:42:19 MST 2010 [INFO] Final Memory: 21M/81M [INFO] -------------------------------------------------------------
88. $ gradle clean compileJava :spring-integration-core:clean :spring-integration-core:compileJava BUILD SUCCESSFUL Total time: 5.621 secs
90. Broken Windows To be fixed... ... they must first be seen
91. $ mvn clean compile [INFO] Scanning for projects... [INFO] ------------------------------------------------------------- [INFO] Building Spring Integration FTP Support [INFO] task-segment: [clean, compile] [INFO] ------------------------------------------------------------- [INFO] [clean:clean {execution: default-clean}] [INFO] Deleting directory /Users/cbeams/Work/pax/spring-integration/ [INFO] [resources:resources {execution: default-resources}] [INFO] Using 'UTF8' encoding to copy filtered resources. [INFO] Copying 0 resource [INFO] Copying 3 resources [INFO] [compiler:compile {execution: default-compile}] [INFO] Compiling 10 source files to /Users/cbeams/Work/pax/spring-in [WARNING] /Users/cbeams/Work/pax/spring-integration/spring-integrati found : org.apache.commons.net.ftp.FTPFile[] required: F[] [INFO] ------------------------------------------------------------- [INFO] BUILD SUCCESSFUL [INFO] ------------------------------------------------------------- [INFO] Total time: 5 seconds [INFO] Finished at: Wed Nov 24 17:10:18 MST 2010 [INFO] Final Memory: 21M/81M [INFO] -------------------------------------------------------------
92. $ gradle clean compileJava :spring-integration-ftp:compileJava /Users/cbeams/Work/pax/spring-integration/spring-integration-ftp/src found : org.apache.commons.net.ftp.FTPFile[] required: F[] public FTPFile[] ls(String path) { ^ 1 warning BUILD SUCCESSFUL Total time: 4.874 secs
93. $ mvn clean compile | wc 351 1626 22858 $ gradle clean compileJava | wc 113 257 6378
94. $ mvn clean test | wc 550432 2046401 74486400 $ gradle clean test | wc 22748 84781 3057486
95. 25x less output not bad but 22,748 lines is still way too much
96. test { logging.captureStandardOutput(LogLevel.INFO) }
97. $ mvn clean test | wc 550432 2046401 74486400 $ gradle clean test | wc 227 317 8792
98. 8,472x less output that's better
99. Ease of Use
100. Ease of Use Gradle Wrapper Partial builds Task discoverability Support source poms gradle help AMQP test exclusions Multi-repository deployment
101. Get running with Gradle $ git clone --recursive $GIT_URL Cloning into spring-integration... remote: Counting objects: 19483, done. remote: Compressing objects: 100% (10387/10387), done. ... $ cd spring-integration && ./gradlew eclipse :spring-integration-core:eclipseClasspath :spring-integration-core:eclipseJdt :spring-integration-core:eclipseProject :spring-integration-core:eclipse :spring-integration-event:eclipseClasspath :spring-integration-event:eclipseJdt :spring-integration-event:eclipseProject :spring-integration-event:eclipse ... BUILD SUCCESSFUL
102. Wrapper == self-contained builds $ git clone --recursive $GIT_URL $ cd spring-integration $ ./gradlew build Downloading http://dist.codehaus.org/gradle/gradle-0.9-rc-3-bin.zip ................................................................... ................................................................... Unzipping /Users/cbeams/.gradle/wrapper/dists/gradle-0.9-rc-3-bin.zi Set executable permissions for: /Users/cbeams/.gradle/wrapper/dists/ > Loading ... BUILD SUCCESSFUL
103. What about CI?
104. Wrapper makes this easy
105. Make it Fast
106. Make it Fast Fast at startup Fast clean build Fast average build
107. Fast at startup Gradle daemon
108. Fast clean build Head-to-head with Maven
109. Fast average build Incremental build
110. Automate the release process
111. Release One-page release instructions Literally "pushbutton" releases not necessarily desirable
112. Getting Help
113. Getting Help User Guide JavaDoc Mailing list
114. Look at other projects All the Spring projects Gradle itself Hibernate Not yet clear what 'idomatic' Gradle looks like
115. What Gradle needs
116. DSL documentation
117. The most important move Ant ever made
120. DSL doc vs User Guide Gradle's got a great User Guide Explains concepts Walks through certain scenarios Very helpful when getting started
121. DSL doc vs API doc Gradle's got JavaDoc Often useful But it's not enough DSL often relies on Groovy metaprogramming JavaDoc can't cover that
122. e.g. subprojects { // what properties are available? }
123. DSL doc Would be, well, DSL-centric Every day reference for build masters
124. Good News
125. Plugin Portal
128. Test-Driven Builds
129. Test-Driven Builds For the same reasons we test other code Special challenges
130. Tooling
131. Tooling Gradle UI is there IDEA integrates it Eclipse tooling is a must Bash-completion / Gradle shell Project quick-start (archetypes)
132. Key Plugins
133. Key Plugins Docbook AspectJ ...
134. Rich Deployment/Rollback
135. Rich Deployment/Rollback Building blocks are there Whole subdomain that needs modeling Look to Capistrano for inspiration
136. Patterns
138. Patterns of Enterprise Application Architecture
139. Patterns of Enterprise Application Architecture
140. Patterns of Enterprise Project Automation
141. Patterns of Enterprise Project Automation Level the playing field Provide common concepts Good for competition
142. Somebody please write this! (Hans, I'm looking at you)
143. Summary
144. Benefits we've gained Clearly modeled and separated concerns Flexibility Concise scripts & output Ease of use Fast execution Reusable build sources
145. Challenges remaining Reuse model not ideal Maintaining our own docbook plugin Eclipse tooling is much needed
146. Lessons learned Document the build Treat the build like code Design the build for user roles and their workflows Total build homogeny == false goal
147. Thanks! Slides: https://github.com/cbeams/spring-on-gradle Source: https://github.com/springsource/spring-build- gradle