Slide 1

Slide 1 text

One Build To Rule Them All Building a Full Application Stack With Gradle John Engelman Object Partners Inc @johnrengelman github.com/johnrengelman

Slide 2

Slide 2 text

What is a full application stack? For the purposes of this talk, a "full application stack" is a multi-project build that consists of multiple technologies, languages, frameworks, and/or external integrations that share components and participate in a single build to produce a set of linked output artifacts.

Slide 3

Slide 3 text

Origin Single Repository Multi-Project Build ~20 Developers 77 Projects (lots of small libs) 4 Grails Applications w/ AngularJS Grunt/Gulp & Bower to build JS code 9 Dropwizard Applications 2 Gradle Plugins Avg. Build Time: ~ 40 mins

Slide 4

Slide 4 text

Topics 1. Multiproject Best Practices 2. Building Grails Applications & Plugins 3. Building Javascript projects 4. Building Application Distributions

Slide 5

Slide 5 text

Gradle Multi-Project Best Practices

Slide 6

Slide 6 text

Project Layout Flat vs Nested

Slide 7

Slide 7 text

Flat Layout m y p r o j e c t / | + - - f o o - l i b / | + - - b a r - s e r v i c e / | + - - b a z - s e r v i c e /

Slide 8

Slide 8 text

Nested Layout m y p r o j e c t / | + - - l i b s / | | | + - - f o o - l i b / | + - - s e r v i c e s / | + - - b a r - s e r v i c e / | + - - b a z - s e r v i c e /

Slide 9

Slide 9 text

Auto-Detecting Sub-Projects Don't need to explicitly include each project Automatically find subprojects / / s e t t i n g s . g r a d l e d e f s k i p p e d D i r s = [ ' b u i l d S r c ' , ' g r a d l e ' ] d e f p a t h = [ ] a s L i n k e d L i s t r o o t D i r . t r a v e r s e ( t y p e : F i l e T y p e . F I L E S , n a m e F i l t e r : ~ / . * \ . g r a d l e / , / / f i n d . g r a d l e f i l e s m a x D e p t h : 1 , / / l i m i t s e a r c h d e p t h p r e D i r : { p a t h < < i t . n a m e / / b u i l d u p t h e d i r e c t o r y s t r u c t u r e i f ( s k i p p e d D i r s . c o n t a i n s ( i t . n a m e ) ) { / / i g n o r e s k i p p e d d i r e c t o r i e s r e t u r n F i l e V i s i t R e s u l t . S K I P _ S U B T R E E } } , p o s t D i r : { p a t h . r e m o v e L a s t ( ) } / / d r o p t h e . g r a d l e f i l e o f f t h e p a t h ) { i f ( p a t h ) { i n c l u d e p a t h . j o i n ( ' : ' ) / / r e g i s t e r t h e p r o j e c t p a t h u s i n g : n o t a t i o n } }

Slide 10

Slide 10 text

Name Gradle files after Project names Quicker access to project's build file / / s e t t i n g s . g r a d l e r o o t P r o j e c t . c h i l d r e n . e a c h { p r o j e c t - > s e t S u b p r o j e c t B u i l d F i l e ( p r o j e c t ) } v o i d s e t S u b p r o j e c t B u i l d F i l e ( d e f p r o j e c t ) { S t r i n g f i l e B a s e N a m e = p r o j e c t . n a m e . r e p l a c e A l l ( " \ \ p { U p p e r } " ) { " - $ { i t . t o L o w e r C a s e ( ) } " } p r o j e c t . b u i l d F i l e N a m e = " $ { f i l e B a s e N a m e } . g r a d l e " a s s e r t p r o j e c t . b u i l d F i l e . i s F i l e ( ) p r o j e c t . c h i l d r e n . e a c h { s u b p r o j e c t - > s e t S u b p r o j e c t B u i l d F i l e ( s u b p r o j e c t ) } }

Slide 11

Slide 11 text

m y p r o j e c t / | + - - f o o L i b / | | | + - - f o o - l i b . g r a d l e | + - - b a r S e r v i c e / | | | + - - b a r - s e r v i c e . g r a d l e | + - - b u i l d . g r a d l e | + - - s e t t i n g s . g r a d l e

Slide 12

Slide 12 text

Demo

Slide 13

Slide 13 text

Building Grails Projects

Slide 14

Slide 14 text

Quick Start b u i l d s c r i p t { r e p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' o r g . g r a i l s : g r a i l s - g r a d l e - p l u g i n : 2 . 1 . 0 ' } } a p p l y p l u g i n : ' g r a i l s ' g r a i l s { g r a i l s V e r s i o n ' 2 . 3 . 8 ' g r o o v y V e r s i o n ' 2 . 1 . 9 ' } r e p o s i t o r i e s { g r a i l s . c e n t r a l ( ) } d e p e n d e n c i e s { b o o t s t r a p ' o r g . g r a i l s . p l u g i n s : t o m c a t : 7 . 0 . 5 0 . 1 ' }

Slide 15

Slide 15 text

Configuration Options g r a i l s { g r a i l s V e r s i o n = ' 2 . 3 . 8 ' g r o o v y V e r s i o n = ' 2 . 1 . 9 ' s p r i n g L o a d e d V e r s i o n = ' 1 . 1 . 3 ' }

Slide 16

Slide 16 text

Resolving Grails Plugins r e p o s i t o r i e s { g r a i l s . c e n t r a l ( ) / / A d d t h e G r a i l s C e n t r a l M a v e n r e p o } d e p e n d e n c i e s { b o o t s t r a p ' o r g . g r a i l s . p l u g i n s : t o m c a t : 7 . 0 . 5 0 . 1 ' / / b o o t s t r a p , c o m p i l e , r u n t i m e , t e s t , p r o v i d e d / / M U S T a d d ' o r g . g r a i l s . p l u g i n s ' }

Slide 17

Slide 17 text

Useful Tasks Task Action init creates a new Grails application project init-plugin creates a new Grails Plugin project run $ grails run-app test $ grails test-app war $ grails war packagePlugin $ grails package-plugin

Slide 18

Slide 18 text

Build Pipeline Participation Task Dependent Tasks clean delete buildPlugins/ dir assemble war, packagePlugin check no default build assemble, check

Slide 19

Slide 19 text

Run Any Grails Script $ g r a d l e g r a i l s - < s c r i p t - n a m e >

Slide 20

Slide 20 text

Configure Grails tasks r u n { e n v = ' p r o d ' a r g s + = ' - - s t a c k t r a c e ' }

Slide 21

Slide 21 text

Question: How Does it work? Answer: Grails Launcher

Slide 22

Slide 22 text

Gradle is NOT building Grails project

Slide 23

Slide 23 text

What Gradle Does: 1. Resolve dependencies 2. Create GrailsLaunchContext 3. Serialize to file 4. Fork GrailsLauncher.Main in new JVM

Slide 24

Slide 24 text

What GrailsLauncher Does: 1. Deserialize GrailsLaunchContext from file 2. Create ClassLoader with bootstrap classpath from context 3. Load GrailsScriptRunner class 4. Call GrailsScriptRunner.executeCommand(scriptName, [args], env)

Slide 25

Slide 25 text

Plugin Gotchyas

Slide 26

Slide 26 text

Do NOT apply Java/Groovy plugin Do NOT apply plugins that apply the Java/Groovy plugin Support for plugin publishing lacks Grails Release Plugin generates POM from BuildConfig Issues with generating plugin.xml, pom.xml, and packagePlugin UP-TO-DATE

Slide 27

Slide 27 text

Demo

Slide 28

Slide 28 text

Building Java Script Projects

Slide 29

Slide 29 text

Lots of Tools Available Grunt Gulp Bower Tomorrow's next cool thing

Slide 30

Slide 30 text

Integrating with Bower & Grunt

Slide 31

Slide 31 text

Step 1 - Get Node b u i l d s c r i p t { r e p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' c o m . m o o w o r k . g r a d l e : g r a d l e - n o d e - p l u g i n : 0 . 5 ' } } a p p l y p l u g i n : ' n o d e '

Slide 32

Slide 32 text

Step 2 - Configure Node a l l p r o j e c t s { p l u g i n s . w i t h T y p e ( N o d e P l u g i n ) { n o d e { v e r s i o n = ' 0 . 1 0 . 2 6 ' d o w n l o a d = t r u e w o r k D i r = r o o t P r o j e c t . f i l e ( " $ { r o o t P r o j e c t . b u i l d D i r } / n o d e j s " ) } } } p r o j e c t . a f t e r E v a l u a t e { n o d e S e t u p { i n p u t s . p r o p e r t y ' v e r s i o n ' , n o d e . v e r s i o n o n l y I f { ! ( i t . v a r i a n t . n o d e D i r ) . e x i s t s ( ) } } }

Slide 33

Slide 33 text

Step 3 - Configure Project Node tasks t a s k n p m C l e a n ( t y p e : D e l e t e ) { d e l e t e ' n o d e _ m o d u l e s ' } p r o j e c t . a f t e r E v a l u a t e { / / u s e t h e 1 N o d e i n s t a l l f r o m r o o t n o d e S e t u p . e n a b l e d = f a l s e n p m I n s t a l l . d e p e n d s O n r o o t P r o j e c t . t a s k s . n o d e S e t u p }

Slide 34

Slide 34 text

Step 4 - Get Grunt b u i l d s c r i p t { r e p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' c o m . m o o w o r k . g r a d l e : g r a d l e - g r u n t - p l u g i n : 0 . 5 ' } } a p p l y p l u g i n : ' g r u n t ' / / M a k e s u r e N P M i s i n s t a l l e d b e f o r e r u n n i n g G r u n t p r o j e c t . a f t e r E v a l u a t e { p r o j e c t . t a s k s . w i t h T y p e ( G r u n t T a s k ) { t a s k - > t a s k . d e p e n d s O n n p m I n s t a l l } }

Slide 35

Slide 35 text

Step 5 - Configure Grunt Tasks t a s k c o m p i l e J s ( t y p e : G r u n t T a s k ) { a r g s = [ ' c o m p i l e ' ] } a s s e m b l e . d e p e n d s O n c o m p i l e J s

Slide 36

Slide 36 text

Step 6 - Get Bower c l a s s B o w e r T a s k e x t e n d s N o d e T a s k { p r i v a t e S t r i n g b o w e r S c r i p t P a t h B o w e r T a s k ( ) { g r o u p = ' B o w e r ' } S t r i n g g e t B o w e r S c r i p t ( ) { b o w e r S c r i p t P a t h ? : ' n o d e _ m o d u l e s / b o w e r / b i n / b o w e r ' } @ O v e r r i d e v o i d e x e c ( ) { d e f l o c a l B o w e r = p r o j e c t . f i l e ( b o w e r S c r i p t ) i f ( ! l o c a l B o w e r . f i l e ) { t h r o w n e w G r a d l e E x c e p t i o n ( ' B o w e r n o t i n s t a l l e d ' ) } s e t S c r i p t ( l o c a l B o w e r ) s u p e r . e x e c ( ) } } p r o j e c t . a f t e r E v a l u a t e { p r o j e c t . t a s k s . w i t h T y p e ( B o w e r T a s k ) { t a s k - > t a s k . d e p e n d s O n n p m I n s t a l l } }

Slide 37

Slide 37 text

Step 7 - Configure Bower Tasks t a s k b o w e r I n s t a l l ( t y p e : B o w e r T a s k ) { a r g s = [ ' i n s t a l l ' ] i n p u t s . f i l e ' b o w e r . j s o n ' i n p u t s . f i l e ' . b o w e r r c ' } a s s e m b l e . d e p e n d s O n b o w e r I n s t a l l

Slide 38

Slide 38 text

Step 8 - Task Ordering Example: Grails WAR contains Grunt/Bower output p r o j e c t . t a s k s . ' g r a i l s - w a r ' . m u s t R u n A f t e r c o m p i l e J s , b o w e r I n s t a l l

Slide 39

Slide 39 text

Demo

Slide 40

Slide 40 text

Building Application Distributions

Slide 41

Slide 41 text

Many options 1. Zip/Jar tasks 2. War task 3. Applications plugin 4. Deploy with Gradle 5. Fat (Uber) Jars

Slide 42

Slide 42 text

Zip/Jar tasks Pros Compiles project classes/resources into a single file Part of normal build cycle Built in to Gradle Cons No dependency mgmt after build

Slide 43

Slide 43 text

War task Pros Bundles application code with dependencies into a single file Easy to deploy to container Cons Only works for things that understand WAR (Tomcat)

Slide 44

Slide 44 text

Application Plugin Pros Creates single Zip file for application Bundles all dependencies Creates shell scripts for executing program Cons Zip is just a package for the app. Requires unzipping for deploy

Slide 45

Slide 45 text

Deploying with Gradle Pros Relies on Jar files and POMs Benefit from Gradle's dependency resolution Gradle dependency caching Gradle Wrapper (or provisioning Gradle instance) Light-weight artifact (just need a build.gradle file) Cons Complex to initially setup Resolution time ...

Slide 46

Slide 46 text

Fat (Uber) Jars Pros Single jar file with all dependencies Launch as "java -jar file.jar" Runs anywhere a JRE is available (of the right version) Cons Longer build time Bigger artifacts (= longer downloads) Still need to manager JAVA_OPTS externally

Slide 47

Slide 47 text

Ways to FatJar

Slide 48

Slide 48 text

Like Node integration, lots of ways Use Gradle's zipTree and Copy Gradle FatJar Plugin Gradle OneJar Plugin Gradle Shadow Plugin

Slide 49

Slide 49 text

Using zipTree & Copy I/O expensive. Has to write out all the jars to disk before creating a new jar But, has all the goodies that Gradle provides for copying (filtering, mapping, etc.) t a s k f a t J a t ( t y p e : J a r ) { f r o m s o u r c e S e t s . m a i n . o u t p u t f r o m { m a i n S o u r c e S e t . r u n t i m e C l a s s p a t h . c o l l e c t { i t ( i t . n a m e . e n d s W i t h ( ' . z i p ' ) | | i t . n a m e . e n d s W i t h ( ' . j a r ' ) ) { p r o j e c t . z i p T r e e ( i t ) } e l s e { p r o j e c t . f i l e s ( i t ) } } } }

Slide 50

Slide 50 text

Gradle Shadow Plugin

Slide 51

Slide 51 text

Based on Maven Shade Uber-jarring Resource transformation File filtering Class relocating Based on Gradle's Jar task (and all it's inherent abilities) Uses JarInputStream & JarOutputStream to write file (fast!)

Slide 52

Slide 52 text

Quick Start b u i l d s c r i p t { r e s p o s i t o r i e s { j c e n t e r ( ) } d e p e n d e n c i e s { c l a s s p a t h ' c o m . g i t h u b . j e n g e l m a n . g r a d l e . p l u g i n s : s h a d o w : 1 . 0 . 2 ' } } a p p l y p l u g i n : ' c o m . g i t h u b . j o h n r e n g e l m a n . s h a d o w ' j a r { m a n i f e s t { a t t r i b u t e s ' M a i n - C l a s s ' : ' m y a p p . M a i n ' } } s h a d o w J a r { m e r g e S e r v i c e F i l e s ( ) } $ g r a d l e s h a d o w J a r

Slide 53

Slide 53 text

Plugin Defaults Includes all dependencies in 'runtime' Excludes any 'META-INF/INDEX.LIST', 'META-INF/*.DSA', and 'META-INF/*.RSA' files Uses same 'MANIFEST.MF' as 'jar' task Classifier is 'all' Creates 'shadow' configuration & component and assigns output as an artifact

Slide 54

Slide 54 text

Speed Comparison Example Ratpack Gradle App Total files in resulting Jar: ~4074 Plugin Time zipTree (RatPack plugin) 1167 ms oneJar 452 ms fatJar 2325 ms shadow 62.25 ms

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

Integration with Application plugin a p p l y p l u g i n : ' a p p l i c a t i o n ' a p p l y p l u g i n : ' c o m . g i t h u b . j o h n r e n g e l m a n . s h a d o w ' m a i n C l a s s N a m e = ' m y a p p . M a i n ' $ g r a d l e i n s t a l l S h a d o w $ g r a d l e r u n S h a d o w $ g r a d l e d i s t S h a d o w Z i p $ g r a d l e d i s t S h a d o w T a r

Slide 57

Slide 57 text

Demo

Slide 58

Slide 58 text

OPI Java, Groovy, Javascript, Mobile, Open Source ~ 100 Senior Consultants Minneapolis, MN & Omaha, NE Chicago, IL & Denver, CO Average tenure > 5 years Founded in 1996 objectpartners.com @objectpartners objectpartners.com/blog