Gradle plugin best practices by example Benjamin Muschko, Gradle Inc.

The dark past of build logic Unstructured  spaghetti  code Copy  &  paste  of  code  snippets The  build  tool  tells  you  how  to  structure  code Build  can  only  be  understood  by  build  guru™ Testing  through  manual  execution

Build logic “-ilities” Typical  non-­‐functional  software  requirements   also  apply  to  build  code…   • Reusability   • Testability   • Maintainability   • Extensibility   • Configurability

Reusability Avoid  copy/paste!  Allow  code  to  be  shared   among  independent  projects.

Testability Build  code  is  no  different  from  application   code.  Test  it  on  multiple  levels!

Maintainability Avoid  the  big  ball  of  mud!  Cohesion  and   separation  of  concerns  are  important.

Extensibility Extend  Gradle's  build  language  by  your  own   declarative  &  expressive  language  constructs.

Configurability Don't  box  in  your  users!  Implement   convention  over  configuration  with  ease.

The end goal Maintainable,  reusable  &  tested  code Consumers  only  configure  the  code Complex  implement.  details  are  hidden

How to get there? Concepts  that  help  implement  these   requirements…   • Good  software  engineering  practices   • Custom  tasks   • Plugins   • Gradle's  extension  mechanisms   • Testing  capabilities

Techniques covered in talk • Declarative  vs.  imperative  logic   • Convention  over  configuration   • Capabilities  vs.  conventions   • Gradle's  extension  mechanisms   • Testing  plugin  logic   • Publishing  the  plugin  artifacts   • Writing  and  generating  documentation   • Setting  up  Continuous  Integration/Delivery

Case study: Gradle Docker plugin                      Serves  as  showcase  plugin  project.   • Plugin  for  managing  Docker  images  and   containers  through  Docker  remote  API   • API  communication  via  Docker  Java  library   • Written  in  Groovy   • Source  code  available  on  GitHub  (https:/ / github.com/bmuschko/gradle-­‐docker-­‐plugin)

Always check in the Wrapper Project  becomes  instantly  buildable  for  every   developer  and  on  the  CI  machine. Check  in  Wrapper  files   into  VCS  repository.

Hiding complex, imperative logic Concept  applies  to  tasks  and  plugins.   • Reusable  and  configurable   • Easy  to  structure,  refactor  and  test   • Avoid  global  properties  and  methods

Task type > ad-hoc task Prefer  implementing  task  types  to   implementing  ad-­‐hoc  tasks. 
 task createDockerfile(type: Dockerfile) {
 destFile = file("$buildDir/mydockerfile/Dockerfile")
 from 'ubuntu:12.04'
 maintainer 'Benjamin Muschko "[email protected]"'
 }
 
 task buildImage(type: DockerBuildImage) {
 dependsOn createDockerfile
 inputDir = createDockerfile.destFile.parentFile
 tag = 'bmuschko/myimage'
 }

Binary plugin > script plugin Use  script  plugins  to  organize  build  logic   based  on  functional  boundaries  in  project. 
 apply from: "$rootDir/gradle/dependencies.gradle"
 apply from: "$rootDir/gradle/test-setup.gradle"
 apply from: "$rootDir/gradle/integration-test.gradle"
 apply from: "$rootDir/gradle/functional-test.gradle"
 apply from: "$rootDir/gradle/additional-artifacts.gradle"
 apply from: "$rootDir/gradle/publishing.gradle"
 apply from: "$rootDir/gradle/documentation.gradle"
 apply from: "$rootDir/gradle/release.gradle"

Convention over configuration Provide  sensible  defaults  and  standards.  Expose   a  way  to  re-­‐configure  them  to  user’s  needs.   Examples:   • src/main/java  for  Java-­‐based  applications   • Project  name  derived  from  directory  name   • Output  directory  is  build

Convention Plugin  consumers  will  be  most  comfortable   with  using  convention  plugins.  Pick  sensible   defaults  for  the  user.   Good  indicator:   The  less  a  consumer  has  to  re-­‐configure   defaults  the  better.

Configuration Real-­‐world  project  might  need  to  re-­‐ configure  the  defaults.  Make  it  convenient  to   change  defaults.   Reason:   Their  view  of  the  world  might  look  different   than  yours.

Users only declare the “what” Expose  a  custom  DSL  from  your  binary  plugin  to   configure  runtime  behavior.   The  “how”  is  the  responsibility  of  the  plugin  impl. 
 apply plugin: 'com.bmuschko.docker-remote-api'
 
 docker {
 url = 'https://192.168.59.103:2376'
 certPath = new File(System.properties['user.home'], '.boot2docker/certs/boot2docker-vm')
 }

Example: Providing default dependencies Plugins  often  rely  on  external  libraries.   Automatically  resolve  default  version,  but   make  it  configurable.

Customizing Docker Java library Introduce  custom  configura]on  in  plugin. 
 project.configurations.create('docker')
 .setVisible(false)
 .setTransitive(true)
 .setDescription('The Docker Java libraries to be used.') Setting  dependency  from  consuming  build. 
 dependencies {
 docker 'com.github.docker-java:docker-java:2.0.0'
 docker 'org.slf4j:slf4j-simple:1.7.5'
 }

Providing default (but customizable) dependency Only  use  defaults,  if  no  dependencies  are   assigned  to  custom  configuration. 
 Configuration config = project.configurations.create('docker')
 
 project.tasks.withType(AbstractDockerRemoteApiTask) {
 config.defaultDependencies { deps ->
 deps.add(project.dependencies .create('com.github.docker-java:docker-java:2.1.0'))
 deps.add(project.dependencies .create('org.slf4j:slf4j-simple:1.7.5'))
 }
 
 conventionMapping.with {
 classpath = { config } 
 }
 } Gradle  2.5

Capabilities vs. conventions Separating  general-­‐purpose  functionality   from  pre-­‐configured,  opinionated   functionality.   Finding  the  right  balance  between  both   aspects  is  key.

Capabilities • Un-­‐opinionated  functionality   • Provide  general-­‐purpose  concepts   Examples:   • Custom  task  types  without  creating  an  actual   task  instance   • Add  new  concepts  without  configuring  them   (e.g.  source  sets,  build  types  and  flavors)

Conventions • Opinionated  functionality   • Instantiate  and/or  pre-­‐configure  concepts   Examples:   • Standardized  directory  layouts   • Enhanced  tasks  created  by  a  plugin   • Adding  default  values  to  extensions   • Declaring  task  dependencies  to  form  a  lifecycle

Plugin composition Plugins  can  build  upon  other  plugins.  This  is  a   common  pattern.   A  base  plugin  provides     generic  capabilities   Another  plugin  builds  on  the  base,     adding  opinionated  conventions

Example: Docker plugins Allows  users  to  pick  functionality  they  need   in  their  projects.   Plugin  that  adds  task  types  for     interacting  with  Docker  remote  API   Plugin  for  creating  &  pushing     Docker  image  for  Java  applications

Example: Docker Java application plugin Applying  plugin  by  identifier  or  type.   
 import org.gradle.api.Plugin
 import org.gradle.api.Project
 
 class DockerJavaApplicationPlugin implements Plugin {
 @Override
 void apply(Project project) {
 project.apply(plugin: DockerRemoteApiPlugin)
 }
 }

Build for further extension • Anticipate  more  specific  extensions  to  your   plugin   • Implemented  through  plugin  composition   • Base  plugin  act  as  enablers  for  custom   convention  plugins   • Enterprise  build  convention  plugins  re-­‐ configure  your  plugins

Put yourself in the shoes of consumer Do  not  automatically  apply  plugins  in  your   plugin!   Reason:     Makes  your  plugin  highly  opinionated.

Reacting to plugins React  to  plugins  that  might  be  applied  in   consuming  build  script.  Only  then  logic  is   executed.   The  same  concept  applies  to  tasks.

Using configuration rules Reacting  to  plugins 
 project.plugins.withType(IdeaPlugin) {
 // configure Idea plugin in the context of your plugin
 }
 Reacting  to  task  types 
 project.tasks.withType(War) {
 // configure all War tasks in the context of your plugin
 }

Testing build logic Build  logic  needs  to  be  testable  on  multiple   levels.

Implementing test types with Gradle Unit  testing:     No  Gradle  tooling  needed   Integration  testing:     Use  ProjectBuilder  to  create  pseudo  Project   instance   Functional  testing:   Nebula  Test,  Gradle  TestKit

Example: Writing unit test with Spock 
 package com.bmuschko.gradle.docker.tasks.image
 
 import spock.lang.Specification
 
 import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.*
 
 class DockerfileTest extends Specification {
 def "Instruction String representation is built correctly"() {
 expect:
 instructionInstance.keyword == keyword
 instructionInstance.build() == builtInstruction 
 where:
 instructionInstance | keyword | builtInstruction
 new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04'
 new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04'
 ...
 }
 }

Separating test type source code    Default  unit  test  directory Custom  integration  test  directory Custom  functional  test  directory

Example: Integration test source set and task 
 sourceSets {
 integrationTest {
 groovy.srcDir file('src/integTest/groovy')
 resources.srcDir file('src/integTest/resources')
 compileClasspath += sourceSets.main.output + configurations.testRuntime
 runtimeClasspath += output + compileClasspath
 }
 }
 
 task integrationTest(type: Test) {
 description = 'Runs the integration tests.'
 group = 'verification'
 testClassesDir = sourceSets.integrationTest.output.classesDir
 classpath = sourceSets.integrationTest.runtimeClasspath
 mustRunAfter test
 }
 
 check.dependsOn integrationTest

Example: Writing integration test with Spock 
 import org.gradle.api.Project
 import org.gradle.testfixtures.ProjectBuilder class DockerJavaApplicationPluginIntegrationTest extends Specification {
 @Rule TemporaryFolder temporaryFolder = new TemporaryFolder()
 Project project
 
 def setup() {
 project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build()
 }
 
 def "Creates tasks out-of-the-box when application plugin is applied"() {
 when:
 project.apply(plugin: DockerJavaApplicationPlugin)
 project.apply(plugin: 'application')
 
 then: project.tasks.findByName(DockerJavaApplicationPlugin .COPY_DIST_RESOURCES_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.BUILD_IMAGE_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.PUSH_IMAGE_TASK_NAME)
 }
 }

Functional testing Testing  the  build  logic  from  the  end  user’s   perspective.   • Exercising  build  logic  as  part  of  a   programmatically  executed  build.   • Make  assertions  about  build  outcome.  

Using the Gradle TestKit Functional  testing  support  in  Gradle  core.   • Uses  Tooling  API  as  test  execution  harness.   • Agnostic  of  test  framework.   • Assertions  made  based  on  build  output,   build  logging  or  test  of  tasks  +  their  result.   Gradle  2.6

TestKit usage Declaring  TestKit  dependency 
 dependencies {
 testCompile gradleTestKit()
 }
 Declaring  the  Spock  dependency 
 dependencies {
 testCompile 'org.spockframework:spock-core:1.0-groovy-2.3'
 }

Example: Writing functional tests with TestKit 
 def "can successfully create Dockerfile"() {
 given:
 buildFile << """
 import com.bmuschko.gradle.docker.tasks.image.Dockerfile
 
 task dockerfile(type: Dockerfile) {
 from 'ubuntu:14.04'
 maintainer 'Benjamin Muschko "[email protected]"'
 } """
 
 when:
 def result = GradleRunner.create()
 .withProjectDir(testProjectDir.root)
 .withArguments('dockerfile')
 .build()
 
 then:
 result.task(":dockerfile").outcome == SUCCESS testProjectDir.file('Dockerfile').exists()
 }

DEMO Functional  testing  with  TestKit

Roadmap for TestKit Gradle  2.6   Mechanics  for  executing  tests   Functional  tests  can  query  build  result   Gradle  2.7   Isolation  of  test  environment,  dedicated  daemons   Gradle  2.8   Convenient  injection  of  code  under  test

Cross-version compatibility tests Forward  and  backward  compatibility   independent  of  Gradle  version  used  to  build   plugin  artifact. 2.4 2.5 2.6 2.3 2.2  Version  used   to  build  plugin

Implementing compatibility tests No  built-­‐in  support  in  Gradle  yet.  On  the   roadmap  for  TestKit.   Intermediate  options:   • Custom  implementation  using  Tooling  API.   • Community  plugins  like  https:/ /github.com/ ysb33r/gradleTest

Importance of documentation Plugins  are  not  self-­‐documenting.   Thorough  documentation  is  crucial  for  plugin   consumers.  

Answer questions for consumers How  can  I  use  the  plugin  and  configure  it?   What  tasks  &  extensions  are  provided?   What  impact  does  the  plugin  have  on  my   project?

Impact on consumers De-­‐mystify  added  functionality.   Give  consumers  the  feeling  that  they  are   under  control.

What should be documented? • Purpose  of  plugin,  the  repository  that  hosts  the  plugin   • Plugins:  Identifier,  type  and  description  bundled  in  a  JAR   • Enhanced  tasks:  their  name,  type  and  dependencies   • Custom  task  types:  Javadocs,  Groovydocs   • Conventions,  custom  extensions  (e.g.  DSLs)   • Extension  properties/methods   • Usage  examples  in  textual  form   • A  good  set  of  functional  tests  that  demonstrate  the  use   of  the  plugin

Example: Plugin description & usage

Example: Extension usage and properties

Example: Textual usage & functional tests

Example: Linking custom tasks to API docs

Example: Publishing API docs to GitHub pages Javadocs/Groovydocs  are  essential  to  allow  users  discover   API  classes  exposed  by  plugin.   • Create  new  branch  gh-pages   • Remove  all  files  from  working  tree  and  index   • Push  new  branch   Available  under   http(s)://.github.io/

Example: Publishing Javadocs to GitHub pages Use  gradle-­‐git  plugin  to  push  to  automatically   publish  Javadocs.   
 apply plugin: 'org.ajoberstar.github-pages'
 
 githubPages {
 repoUri = '[email protected]:bmuschko/gradle-docker-plugin.git'
 
 pages {
 from(javadoc.outputs.files) {
 into 'docs/javadoc'
 }
 }
 }

DEMO Publishing  changed  API  documentation

Publishing the plugin binaries Make  artifact(s)  available  to  consumers  by   uploading  them  to  one  or  many  binary   repositories.

Provide appropriate metadata 
 apply plugin: 'maven-publish'
 
 publishing {
 publications {
 mavenJava(MavenPublication) {
 pom.withXml {
 def root = asNode()
 root.appendNode('name', 'Gradle Docker plugin')
 root.appendNode('description', 'Gradle plugin for managing Docker images and containers.')
 root.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin')
 root.appendNode('inceptionYear', '2014')
 
 def scm = root.appendNode('scm')
 scm.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin')
 scm.appendNode('connection', 'scm:https://[email protected]/bmuschko/gradle-docker-plugin.git')
 scm.appendNode('developerConnection', 'scm:git://github.com/bmuschko/gradle-docker-plugin.git')
 
 def license = root.appendNode('licenses').appendNode('license')
 license.appendNode('name', 'The Apache Software License, Version 2.0')
 license.appendNode('url', 'http://www.apache.org/licenses/LICENSE-2.0.txt')
 license.appendNode('distribution', 'repo')
 
 def developers = root.appendNode('developers')
 def developer = developers.appendNode('developer')
 developer.appendNode('id', 'bmuschko')
 developer.appendNode('name', 'Benjamin Muschko')
 developer.appendNode('email', '[email protected]')
 }
 }
 }
 }

Publishing doesn’t stop with plugin JAR 
 task sourcesJar(type: Jar) {
 classifier 'sources'
 from sourceSets.main.allSource
 }
 
 task groovydocJar(type: Jar, dependsOn: groovydoc) {
 classifier 'groovydoc'
 from groovydoc.destinationDir
 }
 
 task javadocJar(type: Jar, dependsOn: javadoc) {
 classifier 'javadoc'
 from javadoc.destinationDir
 }
 
 artifacts {
 archives sourcesJar
 archives groovydocJar
 archives javadocJar
 }

Share Open Source plugins on the portal https:/ /plugins.gradle.org/

Publishing to the Gradle plugin portal Requires  use  of  the  “Plugin  Publishing  Plugin” 
 buildscript {
 repositories {
 maven {
 url 'https://plugins.gradle.org/m2/'
 }
 } 
 dependencies {
 classpath 'com.gradle.publish:plugin-publish-plugin:0.9.1'
 }
 }
 
 apply plugin: 'com.gradle.plugin-publish'

Declaring required portal plugin metadata 
 pluginBundle {
 website = 'https://github.com/bmuschko/gradle-docker-plugin'
 vcsUrl = 'https://github.com/bmuschko/gradle-docker-plugin.git'
 description = 'Gradle plugin for managing Docker images and containers.'
 tags = ['gradle', 'docker', 'container', 'image', 'lightweight', 'vm', 'linux']
 
 plugins {
 dockerRemoteApi {
 id = 'com.bmuschko.docker-remote-api'
 displayName = 'Provides custom tasks for interacting with Docker via its remote API.'
 }
 
 dockerJavaApplication {
 id = 'com.bmuschko.docker-java-application'
 displayName = 'Creates and pushes a Docker image for a Java application.'
 }
 }
 }

Continuous Integration A  plugin  might  start  small  but  will  grow  in   complexity  over  time.  Avoid  integration  hell!   • Fast  feedback  with  every  code  integration   • Always  execute  (an  assorted  set  of)  tests   • Use  CI  cloud-­‐based  product  or  host  in-­‐ house

Adding a CI badge Give  consumers  and  contributor  a  chance  to   see  the  build  status.

Working towards Continuous Delivery Make  sure  your  plugin  is  always  production   ready.   • End  goal:  avoid  manual  steps   • Build  and  visualize  suitable  delivery  pipeline   • Go  the  extra  mile  -­‐  model  push-­‐button   release  capabilities

Example: Potential release steps  Publi Assemble   artifact(s)    Tag  VCS   repository    Publish   artifact(s) Plugin  JAR   Sources  JAR   Docs  JAR Create  tag   Push  tag Create  metadata   Upload  artifact(s)   Update  API  docs

DEMO Implementing  a  plugin  release  process

Example: Build pipeline with Snap CI                      Build,  test  and  deploy  in  the  cloud.   • Free  of  charge  for  public  Github  repos   • Model  build  pipelines  with  automatic  and/ or  manual  execution  steps   • Execute  build  for  pull  requests

Example: Snap CI pipeline for Docker plugin https:/ /snap-­‐ci.com/bmuschko/gradle-­‐docker-­‐plugin/branch/master Compilation      Unit  tests Integration            tests Functional            tests Release

DEMO Showcasing  build  pipeline  on  Snap  CI

Other aspects • Be  mindful  of  the  performance  impact  your   plugin  might  have  on  consuming  build   • Avoid  using  internal  Gradle  APIs  as  much  as   possible

Thank  You! Please  ask  questions… https:/ /www.github.com/bmuschko @bmuschko