One Build To Rule Them All Building a Full Application Stack With Gradle John Engelman Object Partners Inc @johnrengelman github.com/johnrengelman

What is a full application stack? For the purposes of this talk, a "full application stack" is a multi-project build that consists of multiple technologies, languages, frameworks, and/or external integrations that share components and participate in a single build to produce a set of linked output artifacts.

Origin Single Repository Multi-Project Build ~20 Developers 77 Projects (lots of small libs) 4 Grails Applications w/ AngularJS Grunt/Gulp & Bower to build JS code 9 Dropwizard Applications 2 Gradle Plugins Avg. Build Time: ~ 40 mins

Topics 1. Multiproject Best Practices 2. Building Grails Applications & Plugins 3. Building Javascript projects 4. Building Application Distributions

Gradle Multi-Project Best Practices

Project Layout Flat vs Nested

Flat Layout m y p r o j e c t / | + - - f o o - l i b / | + - - b a r - s e r v i c e / | + - - b a z - s e r v i c e /

Nested Layout m y p r o j e c t / | + - - l i b s / | | | + - - f o o - l i b / | + - - s e r v i c e s / | + - - b a r - s e r v i c e / | + - - b a z - s e r v i c e /

Auto-Detecting Sub-Projects Don't need to explicitly include each project Automatically find subprojects / / s e t t i n g s . g r a d l e d e f s k i p p e d D i r s = [ ' b u i l d S r c ' , ' g r a d l e ' ] d e f p a t h = [ ] a s L i n k e d L i s t r o o t D i r . t r a v e r s e ( t y p e : F i l e T y p e . F I L E S , n a m e F i l t e r : ~ / . * \ . g r a d l e / , / / f i n d . g r a d l e f i l e s m a x D e p t h : 1 , / / l i m i t s e a r c h d e p t h p r e D i r : { p a t h < < i t . n a m e / / b u i l d u p t h e d i r e c t o r y s t r u c t u r e i f ( s k i p p e d D i r s . c o n t a i n s ( i t . n a m e ) ) { / / i g n o r e s k i p p e d d i r e c t o r i e s r e t u r n F i l e V i s i t R e s u l t . S K I P _ S U B T R E E } } , p o s t D i r : { p a t h . r e m o v e L a s t ( ) } / / d r o p t h e . g r a d l e f i l e o f f t h e p a t h ) { i f ( p a t h ) { i n c l u d e p a t h . j o i n ( ' : ' ) / / r e g i s t e r t h e p r o j e c t p a t h u s i n g : n o t a t i o n } }

Name Gradle files after Project names Quicker access to project's build file / / s e t t i n g s . g r a d l e r o o t P r o j e c t . c h i l d r e n . e a c h { p r o j e c t - > s e t S u b p r o j e c t B u i l d F i l e ( p r o j e c t ) } v o i d s e t S u b p r o j e c t B u i l d F i l e ( d e f p r o j e c t ) { S t r i n g f i l e B a s e N a m e = p r o j e c t . n a m e . r e p l a c e A l l ( " \ \ p { U p p e r } " ) { " - $ { i t . t o L o w e r C a s e ( ) } " } p r o j e c t . b u i l d F i l e N a m e = " $ { f i l e B a s e N a m e } . g r a d l e " a s s e r t p r o j e c t . b u i l d F i l e . i s F i l e ( ) p r o j e c t . c h i l d r e n . e a c h { s u b p r o j e c t - > s e t S u b p r o j e c t B u i l d F i l e ( s u b p r o j e c t ) } }

m y p r o j e c t / | + - - f o o L i b / | | | + - - f o o - l i b . g r a d l e | + - - b a r S e r v i c e / | | | + - - b a r - s e r v i c e . g r a d l e | + - - b u i l d . g r a d l e | + - - s e t t i n g s . g r a d l e

Demo

Building Grails Projects

Quick Start b u i l d s c r i p t { r e p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' o r g . g r a i l s : g r a i l s - g r a d l e - p l u g i n : 2 . 1 . 0 ' } } a p p l y p l u g i n : ' g r a i l s ' g r a i l s { g r a i l s V e r s i o n ' 2 . 3 . 8 ' g r o o v y V e r s i o n ' 2 . 1 . 9 ' } r e p o s i t o r i e s { g r a i l s . c e n t r a l ( ) } d e p e n d e n c i e s { b o o t s t r a p ' o r g . g r a i l s . p l u g i n s : t o m c a t : 7 . 0 . 5 0 . 1 ' }

Configuration Options g r a i l s { g r a i l s V e r s i o n = ' 2 . 3 . 8 ' g r o o v y V e r s i o n = ' 2 . 1 . 9 ' s p r i n g L o a d e d V e r s i o n = ' 1 . 1 . 3 ' }

Resolving Grails Plugins r e p o s i t o r i e s { g r a i l s . c e n t r a l ( ) / / A d d t h e G r a i l s C e n t r a l M a v e n r e p o } d e p e n d e n c i e s { b o o t s t r a p ' o r g . g r a i l s . p l u g i n s : t o m c a t : 7 . 0 . 5 0 . 1 ' / / b o o t s t r a p , c o m p i l e , r u n t i m e , t e s t , p r o v i d e d / / M U S T a d d ' o r g . g r a i l s . p l u g i n s ' }

Useful Tasks Task Action init creates a new Grails application project init-plugin creates a new Grails Plugin project run $ grails run-app test $ grails test-app war $ grails war packagePlugin $ grails package-plugin

Build Pipeline Participation Task Dependent Tasks clean delete buildPlugins/ dir assemble war, packagePlugin check no default build assemble, check

Run Any Grails Script $ g r a d l e g r a i l s - < s c r i p t - n a m e >

Configure Grails tasks r u n { e n v = ' p r o d ' a r g s + = ' - - s t a c k t r a c e ' }

Question: How Does it work? Answer: Grails Launcher

Gradle is NOT building Grails project

What Gradle Does: 1. Resolve dependencies 2. Create GrailsLaunchContext 3. Serialize to file 4. Fork GrailsLauncher.Main in new JVM

What GrailsLauncher Does: 1. Deserialize GrailsLaunchContext from file 2. Create ClassLoader with bootstrap classpath from context 3. Load GrailsScriptRunner class 4. Call GrailsScriptRunner.executeCommand(scriptName, [args], env)

Plugin Gotchyas

Do NOT apply Java/Groovy plugin Do NOT apply plugins that apply the Java/Groovy plugin Support for plugin publishing lacks Grails Release Plugin generates POM from BuildConfig Issues with generating plugin.xml, pom.xml, and packagePlugin UP-TO-DATE

Demo

Building Java Script Projects

Lots of Tools Available Grunt Gulp Bower Tomorrow's next cool thing

Integrating with Bower & Grunt

Step 1 - Get Node b u i l d s c r i p t { r e p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' c o m . m o o w o r k . g r a d l e : g r a d l e - n o d e - p l u g i n : 0 . 5 ' } } a p p l y p l u g i n : ' n o d e '

Step 2 - Configure Node a l l p r o j e c t s { p l u g i n s . w i t h T y p e ( N o d e P l u g i n ) { n o d e { v e r s i o n = ' 0 . 1 0 . 2 6 ' d o w n l o a d = t r u e w o r k D i r = r o o t P r o j e c t . f i l e ( " $ { r o o t P r o j e c t . b u i l d D i r } / n o d e j s " ) } } } p r o j e c t . a f t e r E v a l u a t e { n o d e S e t u p { i n p u t s . p r o p e r t y ' v e r s i o n ' , n o d e . v e r s i o n o n l y I f { ! ( i t . v a r i a n t . n o d e D i r ) . e x i s t s ( ) } } }

Step 3 - Configure Project Node tasks t a s k n p m C l e a n ( t y p e : D e l e t e ) { d e l e t e ' n o d e _ m o d u l e s ' } p r o j e c t . a f t e r E v a l u a t e { / / u s e t h e 1 N o d e i n s t a l l f r o m r o o t n o d e S e t u p . e n a b l e d = f a l s e n p m I n s t a l l . d e p e n d s O n r o o t P r o j e c t . t a s k s . n o d e S e t u p }

Step 4 - Get Grunt b u i l d s c r i p t { r e p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' c o m . m o o w o r k . g r a d l e : g r a d l e - g r u n t - p l u g i n : 0 . 5 ' } } a p p l y p l u g i n : ' g r u n t ' / / M a k e s u r e N P M i s i n s t a l l e d b e f o r e r u n n i n g G r u n t p r o j e c t . a f t e r E v a l u a t e { p r o j e c t . t a s k s . w i t h T y p e ( G r u n t T a s k ) { t a s k - > t a s k . d e p e n d s O n n p m I n s t a l l } }

Step 5 - Configure Grunt Tasks t a s k c o m p i l e J s ( t y p e : G r u n t T a s k ) { a r g s = [ ' c o m p i l e ' ] } a s s e m b l e . d e p e n d s O n c o m p i l e J s

Step 6 - Get Bower c l a s s B o w e r T a s k e x t e n d s N o d e T a s k { p r i v a t e S t r i n g b o w e r S c r i p t P a t h B o w e r T a s k ( ) { g r o u p = ' B o w e r ' } S t r i n g g e t B o w e r S c r i p t ( ) { b o w e r S c r i p t P a t h ? : ' n o d e _ m o d u l e s / b o w e r / b i n / b o w e r ' } @ O v e r r i d e v o i d e x e c ( ) { d e f l o c a l B o w e r = p r o j e c t . f i l e ( b o w e r S c r i p t ) i f ( ! l o c a l B o w e r . f i l e ) { t h r o w n e w G r a d l e E x c e p t i o n ( ' B o w e r n o t i n s t a l l e d ' ) } s e t S c r i p t ( l o c a l B o w e r ) s u p e r . e x e c ( ) } } p r o j e c t . a f t e r E v a l u a t e { p r o j e c t . t a s k s . w i t h T y p e ( B o w e r T a s k ) { t a s k - > t a s k . d e p e n d s O n n p m I n s t a l l } }

Step 7 - Configure Bower Tasks t a s k b o w e r I n s t a l l ( t y p e : B o w e r T a s k ) { a r g s = [ ' i n s t a l l ' ] i n p u t s . f i l e ' b o w e r . j s o n ' i n p u t s . f i l e ' . b o w e r r c ' } a s s e m b l e . d e p e n d s O n b o w e r I n s t a l l

Step 8 - Task Ordering Example: Grails WAR contains Grunt/Bower output p r o j e c t . t a s k s . ' g r a i l s - w a r ' . m u s t R u n A f t e r c o m p i l e J s , b o w e r I n s t a l l

Demo

Building Application Distributions

Many options 1. Zip/Jar tasks 2. War task 3. Applications plugin 4. Deploy with Gradle 5. Fat (Uber) Jars

Zip/Jar tasks Pros Compiles project classes/resources into a single file Part of normal build cycle Built in to Gradle Cons No dependency mgmt after build

War task Pros Bundles application code with dependencies into a single file Easy to deploy to container Cons Only works for things that understand WAR (Tomcat)

Application Plugin Pros Creates single Zip file for application Bundles all dependencies Creates shell scripts for executing program Cons Zip is just a package for the app. Requires unzipping for deploy

Deploying with Gradle Pros Relies on Jar files and POMs Benefit from Gradle's dependency resolution Gradle dependency caching Gradle Wrapper (or provisioning Gradle instance) Light-weight artifact (just need a build.gradle file) Cons Complex to initially setup Resolution time ...

Fat (Uber) Jars Pros Single jar file with all dependencies Launch as "java -jar file.jar" Runs anywhere a JRE is available (of the right version) Cons Longer build time Bigger artifacts (= longer downloads) Still need to manager JAVA_OPTS externally

Ways to FatJar

Like Node integration, lots of ways Use Gradle's zipTree and Copy Gradle FatJar Plugin Gradle OneJar Plugin Gradle Shadow Plugin

Using zipTree & Copy I/O expensive. Has to write out all the jars to disk before creating a new jar But, has all the goodies that Gradle provides for copying (filtering, mapping, etc.) t a s k f a t J a t ( t y p e : J a r ) { f r o m s o u r c e S e t s . m a i n . o u t p u t f r o m { m a i n S o u r c e S e t . r u n t i m e C l a s s p a t h . c o l l e c t { i t ( i t . n a m e . e n d s W i t h ( ' . z i p ' ) | | i t . n a m e . e n d s W i t h ( ' . j a r ' ) ) { p r o j e c t . z i p T r e e ( i t ) } e l s e { p r o j e c t . f i l e s ( i t ) } } } }

Gradle Shadow Plugin

Based on Maven Shade Uber-jarring Resource transformation File filtering Class relocating Based on Gradle's Jar task (and all it's inherent abilities) Uses JarInputStream & JarOutputStream to write file (fast!)

Quick Start b u i l d s c r i p t { r e s p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' c o m . g i t h u b . j e n g e l m a n . g r a d l e . p l u g i n s : s h a d o w : 1 . 0 . 2 ' } } a p p l y p l u g i n : ' c o m . g i t h u b . j o h n r e n g e l m a n . s h a d o w ' j a r { m a n i f e s t { a t t r i b u t e s ' M a i n - C l a s s ' : ' m y a p p . M a i n ' } } s h a d o w J a r { m e r g e S e r v i c e F i l e s ( ) } $ g r a d l e s h a d o w J a r

Plugin Defaults Includes all dependencies in 'runtime' Excludes any 'META-INF/INDEX.LIST', 'META-INF/*.DSA', and 'META-INF/*.RSA' files Uses same 'MANIFEST.MF' as 'jar' task Classifier is 'all' Creates 'shadow' configuration & component and assigns output as an artifact

Speed Comparison Example Ratpack Gradle App Total files in resulting Jar: ~4074 Plugin Time zipTree (RatPack plugin) 1167 ms oneJar 452 ms fatJar 2325 ms shadow 62.25 ms

No content

Integration with Application plugin a p p l y p l u g i n : ' a p p l i c a t i o n ' a p p l y p l u g i n : ' c o m . g i t h u b . j o h n r e n g e l m a n . s h a d o w ' m a i n C l a s s N a m e = ' m y a p p . M a i n ' $ g r a d l e i n s t a l l S h a d o w $ g r a d l e r u n S h a d o w $ g r a d l e d i s t S h a d o w Z i p $ g r a d l e d i s t S h a d o w T a r

Demo

OPI Java, Groovy, Javascript, Mobile, Open Source ~ 100 Senior Consultants Minneapolis, MN & Omaha, NE Chicago, IL & Denver, CO Average tenure > 5 years Founded in 1996 objectpartners.com @objectpartners objectpartners.com/blog