Gradle plugin best practices by example

Gradle plugin best practices by example

Gradle is a general purpose, multi-platform build tool. Plugins allow for extending Gradle's core capabilities with reusable and targeted functionality. In this session, we'll discuss techniques and best practices for developing your own Gradle plugins by dissecting the structure, code, documentation and supporting infrastructure of an existing Gradle plugin.

Gradle plugins are extremly simple to write. But there's far more to learn. Once you get beyond the basics, you will want to know about guidelines and best practices. In this session, we'll identify concepts and techniques that work well in practice by having a look at the Gradle Docker plugin.

We'll mainly focus on the following topics:

* Brief introduction to Gradle plugin concepts
* Applying proven software engineering practices
* Separating capabilities from conventions
* Implementing convention over configuration
* Writing flexible custom tasks
* Depending on external libraries
* Exposing a custom DSL to plugin consumers
* Using the Gradle Wrapper
* Writing different types of tests for plugin code
* Publishing the plugin binaries
* Creating functional documentation
* Registering the plugin with Gradle's plugin portal
* Setting up Continuous Integration

8f2248c6bfcc6df39a2cd8edf4267cb5?s=128

Benjamin Muschko

September 16, 2015
Tweet

Transcript

  1. 2.

    The dark past of build logic Unstructured  spaghetti  code Copy

     &  paste  of  code  snippets The  build  tool  tells  you  how  to  structure  code Build  can  only  be  understood  by  build  guru™ Testing  through  manual  execution
  2. 3.

    Build logic “-ilities” Typical  non-­‐functional  software  requirements   also  apply

     to  build  code…   • Reusability   • Testability   • Maintainability   • Extensibility   • Configurability
  3. 6.

    Maintainability Avoid  the  big  ball  of  mud!  Cohesion  and  

    separation  of  concerns  are  important.
  4. 9.

    The end goal Maintainable,  reusable  &  tested  code Consumers  only

     configure  the  code Complex  implement.  details  are  hidden
  5. 10.

    How to get there? Concepts  that  help  implement  these  

    requirements…   • Good  software  engineering  practices   • Custom  tasks   • Plugins   • Gradle's  extension  mechanisms   • Testing  capabilities
  6. 11.

    Techniques covered in talk • Declarative  vs.  imperative  logic  

    • Convention  over  configuration   • Capabilities  vs.  conventions   • Gradle's  extension  mechanisms   • Testing  plugin  logic   • Publishing  the  plugin  artifacts   • Writing  and  generating  documentation   • Setting  up  Continuous  Integration/Delivery
  7. 12.

    Case study: Gradle Docker plugin          

               Serves  as  showcase  plugin  project.   • Plugin  for  managing  Docker  images  and   containers  through  Docker  remote  API   • API  communication  via  Docker  Java  library   • Written  in  Groovy   • Source  code  available  on  GitHub  (https:/ / github.com/bmuschko/gradle-­‐docker-­‐plugin)
  8. 13.

    Always check in the Wrapper Project  becomes  instantly  buildable  for

     every   developer  and  on  the  CI  machine. Check  in  Wrapper  files   into  VCS  repository.
  9. 14.

    Hiding complex, imperative logic Concept  applies  to  tasks  and  plugins.

      • Reusable  and  configurable   • Easy  to  structure,  refactor  and  test   • Avoid  global  properties  and  methods
  10. 15.

    Task type > ad-hoc task Prefer  implementing  task  types  to

      implementing  ad-­‐hoc  tasks. 
 task createDockerfile(type: Dockerfile) {
 destFile = file("$buildDir/mydockerfile/Dockerfile")
 from 'ubuntu:12.04'
 maintainer 'Benjamin Muschko "benjamin.muschko@gmail.com"'
 }
 
 task buildImage(type: DockerBuildImage) {
 dependsOn createDockerfile
 inputDir = createDockerfile.destFile.parentFile
 tag = 'bmuschko/myimage'
 }
  11. 16.

    Binary plugin > script plugin Use  script  plugins  to  organize

     build  logic   based  on  functional  boundaries  in  project. 
 apply from: "$rootDir/gradle/dependencies.gradle"
 apply from: "$rootDir/gradle/test-setup.gradle"
 apply from: "$rootDir/gradle/integration-test.gradle"
 apply from: "$rootDir/gradle/functional-test.gradle"
 apply from: "$rootDir/gradle/additional-artifacts.gradle"
 apply from: "$rootDir/gradle/publishing.gradle"
 apply from: "$rootDir/gradle/documentation.gradle"
 apply from: "$rootDir/gradle/release.gradle"
  12. 17.

    Convention over configuration Provide  sensible  defaults  and  standards.  Expose  

    a  way  to  re-­‐configure  them  to  user’s  needs.   Examples:   • src/main/java  for  Java-­‐based  applications   • Project  name  derived  from  directory  name   • Output  directory  is  build
  13. 18.

    Convention Plugin  consumers  will  be  most  comfortable   with  using

     convention  plugins.  Pick  sensible   defaults  for  the  user.   Good  indicator:   The  less  a  consumer  has  to  re-­‐configure   defaults  the  better.
  14. 19.

    Configuration Real-­‐world  project  might  need  to  re-­‐ configure  the  defaults.

     Make  it  convenient  to   change  defaults.   Reason:   Their  view  of  the  world  might  look  different   than  yours.
  15. 20.

    Users only declare the “what” Expose  a  custom  DSL  from

     your  binary  plugin  to   configure  runtime  behavior.   The  “how”  is  the  responsibility  of  the  plugin  impl. 
 apply plugin: 'com.bmuschko.docker-remote-api'
 
 docker {
 url = 'https://192.168.59.103:2376'
 certPath = new File(System.properties['user.home'], '.boot2docker/certs/boot2docker-vm')
 }
  16. 21.

    Example: Providing default dependencies Plugins  often  rely  on  external  libraries.

      Automatically  resolve  default  version,  but   make  it  configurable.
  17. 22.

    Customizing Docker Java library Introduce  custom  configura]on  in  plugin. 


    project.configurations.create('docker')
 .setVisible(false)
 .setTransitive(true)
 .setDescription('The Docker Java libraries to be used.') Setting  dependency  from  consuming  build. 
 dependencies {
 docker 'com.github.docker-java:docker-java:2.0.0'
 docker 'org.slf4j:slf4j-simple:1.7.5'
 }
  18. 23.

    Providing default (but customizable) dependency Only  use  defaults,  if  no

     dependencies  are   assigned  to  custom  configuration. 
 Configuration config = project.configurations.create('docker')
 
 project.tasks.withType(AbstractDockerRemoteApiTask) {
 config.defaultDependencies { deps ->
 deps.add(project.dependencies .create('com.github.docker-java:docker-java:2.1.0'))
 deps.add(project.dependencies .create('org.slf4j:slf4j-simple:1.7.5'))
 }
 
 conventionMapping.with {
 classpath = { config } 
 }
 } Gradle  2.5
  19. 24.

    Capabilities vs. conventions Separating  general-­‐purpose  functionality   from  pre-­‐configured,  opinionated

      functionality.   Finding  the  right  balance  between  both   aspects  is  key.
  20. 25.

    Capabilities • Un-­‐opinionated  functionality   • Provide  general-­‐purpose  concepts  

    Examples:   • Custom  task  types  without  creating  an  actual   task  instance   • Add  new  concepts  without  configuring  them   (e.g.  source  sets,  build  types  and  flavors)
  21. 26.

    Conventions • Opinionated  functionality   • Instantiate  and/or  pre-­‐configure  concepts

      Examples:   • Standardized  directory  layouts   • Enhanced  tasks  created  by  a  plugin   • Adding  default  values  to  extensions   • Declaring  task  dependencies  to  form  a  lifecycle
  22. 27.

    Plugin composition Plugins  can  build  upon  other  plugins.  This  is

     a   common  pattern.   A  base  plugin  provides     generic  capabilities   Another  plugin  builds  on  the  base,     adding  opinionated  conventions
  23. 28.

    Example: Docker plugins Allows  users  to  pick  functionality  they  need

      in  their  projects.   Plugin  that  adds  task  types  for     interacting  with  Docker  remote  API   Plugin  for  creating  &  pushing     Docker  image  for  Java  applications
  24. 29.

    Example: Docker Java application plugin Applying  plugin  by  identifier  or

     type.   
 import org.gradle.api.Plugin
 import org.gradle.api.Project
 
 class DockerJavaApplicationPlugin implements Plugin<Project> {
 @Override
 void apply(Project project) {
 project.apply(plugin: DockerRemoteApiPlugin)
 }
 }
  25. 30.

    Build for further extension • Anticipate  more  specific  extensions  to

     your   plugin   • Implemented  through  plugin  composition   • Base  plugin  act  as  enablers  for  custom   convention  plugins   • Enterprise  build  convention  plugins  re-­‐ configure  your  plugins
  26. 31.

    Put yourself in the shoes of consumer Do  not  automatically

     apply  plugins  in  your   plugin!   Reason:     Makes  your  plugin  highly  opinionated.
  27. 32.

    Reacting to plugins React  to  plugins  that  might  be  applied

     in   consuming  build  script.  Only  then  logic  is   executed.   The  same  concept  applies  to  tasks.
  28. 33.

    Using configuration rules Reacting  to  plugins 
 project.plugins.withType(IdeaPlugin) {
 //

    configure Idea plugin in the context of your plugin
 }
 Reacting  to  task  types 
 project.tasks.withType(War) {
 // configure all War tasks in the context of your plugin
 }
  29. 35.

    Implementing test types with Gradle Unit  testing:     No

     Gradle  tooling  needed   Integration  testing:     Use  ProjectBuilder  to  create  pseudo  Project   instance   Functional  testing:   Nebula  Test,  Gradle  TestKit
  30. 36.

    Example: Writing unit test with Spock 
 package com.bmuschko.gradle.docker.tasks.image
 


    import spock.lang.Specification
 
 import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.*
 
 class DockerfileTest extends Specification {
 def "Instruction String representation is built correctly"() {
 expect:
 instructionInstance.keyword == keyword
 instructionInstance.build() == builtInstruction 
 where:
 instructionInstance | keyword | builtInstruction
 new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04'
 new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04'
 ...
 }
 }
  31. 37.

    Separating test type source code    Default  unit  test  directory

    Custom  integration  test  directory Custom  functional  test  directory
  32. 38.

    Example: Integration test source set and task 
 sourceSets {


    integrationTest {
 groovy.srcDir file('src/integTest/groovy')
 resources.srcDir file('src/integTest/resources')
 compileClasspath += sourceSets.main.output + configurations.testRuntime
 runtimeClasspath += output + compileClasspath
 }
 }
 
 task integrationTest(type: Test) {
 description = 'Runs the integration tests.'
 group = 'verification'
 testClassesDir = sourceSets.integrationTest.output.classesDir
 classpath = sourceSets.integrationTest.runtimeClasspath
 mustRunAfter test
 }
 
 check.dependsOn integrationTest
  33. 39.

    Example: Writing integration test with Spock 
 import org.gradle.api.Project
 import

    org.gradle.testfixtures.ProjectBuilder class DockerJavaApplicationPluginIntegrationTest extends Specification {
 @Rule TemporaryFolder temporaryFolder = new TemporaryFolder()
 Project project
 
 def setup() {
 project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build()
 }
 
 def "Creates tasks out-of-the-box when application plugin is applied"() {
 when:
 project.apply(plugin: DockerJavaApplicationPlugin)
 project.apply(plugin: 'application')
 
 then: project.tasks.findByName(DockerJavaApplicationPlugin .COPY_DIST_RESOURCES_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.BUILD_IMAGE_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.PUSH_IMAGE_TASK_NAME)
 }
 }
  34. 40.

    Functional testing Testing  the  build  logic  from  the  end  user’s

      perspective.   • Exercising  build  logic  as  part  of  a   programmatically  executed  build.   • Make  assertions  about  build  outcome.  
  35. 41.

    Using the Gradle TestKit Functional  testing  support  in  Gradle  core.

      • Uses  Tooling  API  as  test  execution  harness.   • Agnostic  of  test  framework.   • Assertions  made  based  on  build  output,   build  logging  or  test  of  tasks  +  their  result.   Gradle  2.6
  36. 42.

    TestKit usage Declaring  TestKit  dependency 
 dependencies {
 testCompile gradleTestKit()


    }
 Declaring  the  Spock  dependency 
 dependencies {
 testCompile 'org.spockframework:spock-core:1.0-groovy-2.3'
 }
  37. 43.

    Example: Writing functional tests with TestKit 
 def "can successfully

    create Dockerfile"() {
 given:
 buildFile << """
 import com.bmuschko.gradle.docker.tasks.image.Dockerfile
 
 task dockerfile(type: Dockerfile) {
 from 'ubuntu:14.04'
 maintainer 'Benjamin Muschko "benjamin.muschko@gmail.com"'
 } """
 
 when:
 def result = GradleRunner.create()
 .withProjectDir(testProjectDir.root)
 .withArguments('dockerfile')
 .build()
 
 then:
 result.task(":dockerfile").outcome == SUCCESS testProjectDir.file('Dockerfile').exists()
 }
  38. 45.

    Roadmap for TestKit Gradle  2.6   Mechanics  for  executing  tests

      Functional  tests  can  query  build  result   Gradle  2.7   Isolation  of  test  environment,  dedicated  daemons   Gradle  2.8   Convenient  injection  of  code  under  test
  39. 46.

    Cross-version compatibility tests Forward  and  backward  compatibility   independent  of

     Gradle  version  used  to  build   plugin  artifact. 2.4 2.5 2.6 2.3 2.2  Version  used   to  build  plugin
  40. 47.

    Implementing compatibility tests No  built-­‐in  support  in  Gradle  yet.  On

     the   roadmap  for  TestKit.   Intermediate  options:   • Custom  implementation  using  Tooling  API.   • Community  plugins  like  https:/ /github.com/ ysb33r/gradleTest
  41. 49.

    Answer questions for consumers How  can  I  use  the  plugin

     and  configure  it?   What  tasks  &  extensions  are  provided?   What  impact  does  the  plugin  have  on  my   project?
  42. 51.

    What should be documented? • Purpose  of  plugin,  the  repository

     that  hosts  the  plugin   • Plugins:  Identifier,  type  and  description  bundled  in  a  JAR   • Enhanced  tasks:  their  name,  type  and  dependencies   • Custom  task  types:  Javadocs,  Groovydocs   • Conventions,  custom  extensions  (e.g.  DSLs)   • Extension  properties/methods   • Usage  examples  in  textual  form   • A  good  set  of  functional  tests  that  demonstrate  the  use   of  the  plugin
  43. 56.

    Example: Publishing API docs to GitHub pages Javadocs/Groovydocs  are  essential

     to  allow  users  discover   API  classes  exposed  by  plugin.   • Create  new  branch  gh-pages   • Remove  all  files  from  working  tree  and  index   • Push  new  branch   Available  under   http(s)://<username>.github.io/<projectname>
  44. 57.

    Example: Publishing Javadocs to GitHub pages Use  gradle-­‐git  plugin  to

     push  to  automatically   publish  Javadocs.   
 apply plugin: 'org.ajoberstar.github-pages'
 
 githubPages {
 repoUri = 'git@github.com:bmuschko/gradle-docker-plugin.git'
 
 pages {
 from(javadoc.outputs.files) {
 into 'docs/javadoc'
 }
 }
 }
  45. 59.

    Publishing the plugin binaries Make  artifact(s)  available  to  consumers  by

      uploading  them  to  one  or  many  binary   repositories.
  46. 60.

    Provide appropriate metadata 
 apply plugin: 'maven-publish'
 
 publishing {


    publications {
 mavenJava(MavenPublication) {
 pom.withXml {
 def root = asNode()
 root.appendNode('name', 'Gradle Docker plugin')
 root.appendNode('description', 'Gradle plugin for managing Docker images and containers.')
 root.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin')
 root.appendNode('inceptionYear', '2014')
 
 def scm = root.appendNode('scm')
 scm.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin')
 scm.appendNode('connection', 'scm:https://bmuschko@github.com/bmuschko/gradle-docker-plugin.git')
 scm.appendNode('developerConnection', 'scm:git://github.com/bmuschko/gradle-docker-plugin.git')
 
 def license = root.appendNode('licenses').appendNode('license')
 license.appendNode('name', 'The Apache Software License, Version 2.0')
 license.appendNode('url', 'http://www.apache.org/licenses/LICENSE-2.0.txt')
 license.appendNode('distribution', 'repo')
 
 def developers = root.appendNode('developers')
 def developer = developers.appendNode('developer')
 developer.appendNode('id', 'bmuschko')
 developer.appendNode('name', 'Benjamin Muschko')
 developer.appendNode('email', 'benjamin.muschko@gmail.com')
 }
 }
 }
 }
  47. 61.

    Publishing doesn’t stop with plugin JAR 
 task sourcesJar(type: Jar)

    {
 classifier 'sources'
 from sourceSets.main.allSource
 }
 
 task groovydocJar(type: Jar, dependsOn: groovydoc) {
 classifier 'groovydoc'
 from groovydoc.destinationDir
 }
 
 task javadocJar(type: Jar, dependsOn: javadoc) {
 classifier 'javadoc'
 from javadoc.destinationDir
 }
 
 artifacts {
 archives sourcesJar
 archives groovydocJar
 archives javadocJar
 }
  48. 63.

    Publishing to the Gradle plugin portal Requires  use  of  the

     “Plugin  Publishing  Plugin” 
 buildscript {
 repositories {
 maven {
 url 'https://plugins.gradle.org/m2/'
 }
 } 
 dependencies {
 classpath 'com.gradle.publish:plugin-publish-plugin:0.9.1'
 }
 }
 
 apply plugin: 'com.gradle.plugin-publish'
  49. 64.

    Declaring required portal plugin metadata 
 pluginBundle {
 website =

    'https://github.com/bmuschko/gradle-docker-plugin'
 vcsUrl = 'https://github.com/bmuschko/gradle-docker-plugin.git'
 description = 'Gradle plugin for managing Docker images and containers.'
 tags = ['gradle', 'docker', 'container', 'image', 'lightweight', 'vm', 'linux']
 
 plugins {
 dockerRemoteApi {
 id = 'com.bmuschko.docker-remote-api'
 displayName = 'Provides custom tasks for interacting with Docker via its remote API.'
 }
 
 dockerJavaApplication {
 id = 'com.bmuschko.docker-java-application'
 displayName = 'Creates and pushes a Docker image for a Java application.'
 }
 }
 }
  50. 65.

    Continuous Integration A  plugin  might  start  small  but  will  grow

     in   complexity  over  time.  Avoid  integration  hell!   • Fast  feedback  with  every  code  integration   • Always  execute  (an  assorted  set  of)  tests   • Use  CI  cloud-­‐based  product  or  host  in-­‐ house
  51. 67.

    Working towards Continuous Delivery Make  sure  your  plugin  is  always

     production   ready.   • End  goal:  avoid  manual  steps   • Build  and  visualize  suitable  delivery  pipeline   • Go  the  extra  mile  -­‐  model  push-­‐button   release  capabilities
  52. 68.

    Example: Potential release steps  Publi Assemble   artifact(s)    Tag

     VCS   repository    Publish   artifact(s) Plugin  JAR   Sources  JAR   Docs  JAR Create  tag   Push  tag Create  metadata   Upload  artifact(s)   Update  API  docs
  53. 70.

    Example: Build pipeline with Snap CI        

                 Build,  test  and  deploy  in  the  cloud.   • Free  of  charge  for  public  Github  repos   • Model  build  pipelines  with  automatic  and/ or  manual  execution  steps   • Execute  build  for  pull  requests
  54. 71.

    Example: Snap CI pipeline for Docker plugin https:/ /snap-­‐ci.com/bmuschko/gradle-­‐docker-­‐plugin/branch/master Compilation

         Unit  tests Integration            tests Functional            tests Release
  55. 73.

    Other aspects • Be  mindful  of  the  performance  impact  your

      plugin  might  have  on  consuming  build   • Avoid  using  internal  Gradle  APIs  as  much  as   possible