Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gradle plugin best practices by example

Gradle plugin best practices by example

Gradle is a general purpose, multi-platform build tool. Plugins allow for extending Gradle's core capabilities with reusable and targeted functionality. In this session, we'll discuss techniques and best practices for developing your own Gradle plugins by dissecting the structure, code, documentation and supporting infrastructure of an existing Gradle plugin.

Gradle plugins are extremly simple to write. But there's far more to learn. Once you get beyond the basics, you will want to know about guidelines and best practices. In this session, we'll identify concepts and techniques that work well in practice by having a look at the Gradle Docker plugin.

We'll mainly focus on the following topics:

* Brief introduction to Gradle plugin concepts
* Applying proven software engineering practices
* Separating capabilities from conventions
* Implementing convention over configuration
* Writing flexible custom tasks
* Depending on external libraries
* Exposing a custom DSL to plugin consumers
* Using the Gradle Wrapper
* Writing different types of tests for plugin code
* Publishing the plugin binaries
* Creating functional documentation
* Registering the plugin with Gradle's plugin portal
* Setting up Continuous Integration

Benjamin Muschko

September 16, 2015
Tweet

More Decks by Benjamin Muschko

Other Decks in Programming

Transcript

  1. Gradle plugin best practices
    by example
    Benjamin Muschko, Gradle Inc.

    View Slide

  2. The dark past of build logic
    Unstructured  spaghetti  code
    Copy  &  paste  of  code  snippets
    The  build  tool  tells  you  how  to  structure  code
    Build  can  only  be  understood  by  build  guru™
    Testing  through  manual  execution

    View Slide

  3. Build logic “-ilities”
    Typical  non-­‐functional  software  requirements  
    also  apply  to  build  code…  
    • Reusability  
    • Testability  
    • Maintainability  
    • Extensibility  
    • Configurability

    View Slide

  4. Reusability
    Avoid  copy/paste!  Allow  code  to  be  shared  
    among  independent  projects.

    View Slide

  5. Testability
    Build  code  is  no  different  from  application  
    code.  Test  it  on  multiple  levels!

    View Slide

  6. Maintainability
    Avoid  the  big  ball  of  mud!  Cohesion  and  
    separation  of  concerns  are  important.

    View Slide

  7. Extensibility
    Extend  Gradle's  build  language  by  your  own  
    declarative  &  expressive  language  constructs.

    View Slide

  8. Configurability
    Don't  box  in  your  users!  Implement  
    convention  over  configuration  with  ease.

    View Slide

  9. The end goal
    Maintainable,  reusable  &  tested  code
    Consumers  only  configure  the  code
    Complex  implement.  details  are  hidden

    View Slide

  10. How to get there?
    Concepts  that  help  implement  these  
    requirements…  
    • Good  software  engineering  practices  
    • Custom  tasks  
    • Plugins  
    • Gradle's  extension  mechanisms  
    • Testing  capabilities

    View Slide

  11. Techniques covered in talk
    • Declarative  vs.  imperative  logic  
    • Convention  over  configuration  
    • Capabilities  vs.  conventions  
    • Gradle's  extension  mechanisms  
    • Testing  plugin  logic  
    • Publishing  the  plugin  artifacts  
    • Writing  and  generating  documentation  
    • Setting  up  Continuous  Integration/Delivery

    View Slide

  12. Case study: Gradle Docker plugin
                         Serves  as  showcase  plugin  project.  
    • Plugin  for  managing  Docker  images  and  
    containers  through  Docker  remote  API  
    • API  communication  via  Docker  Java  library  
    • Written  in  Groovy  
    • Source  code  available  on  GitHub  (https:/
    /
    github.com/bmuschko/gradle-­‐docker-­‐plugin)

    View Slide

  13. Always check in the Wrapper
    Project  becomes  instantly  buildable  for  every  
    developer  and  on  the  CI  machine.
    Check  in  Wrapper  files  
    into  VCS  repository.

    View Slide

  14. Hiding complex, imperative logic
    Concept  applies  to  tasks  and  plugins.  
    • Reusable  and  configurable  
    • Easy  to  structure,  refactor  and  test  
    • Avoid  global  properties  and  methods

    View Slide

  15. Task type > ad-hoc task
    Prefer  implementing  task  types  to  
    implementing  ad-­‐hoc  tasks.

    task createDockerfile(type: Dockerfile) {

    destFile = file("$buildDir/mydockerfile/Dockerfile")

    from 'ubuntu:12.04'

    maintainer 'Benjamin Muschko "[email protected]"'

    }


    task buildImage(type: DockerBuildImage) {

    dependsOn createDockerfile

    inputDir = createDockerfile.destFile.parentFile

    tag = 'bmuschko/myimage'

    }

    View Slide

  16. Binary plugin > script plugin
    Use  script  plugins  to  organize  build  logic  
    based  on  functional  boundaries  in  project.

    apply from: "$rootDir/gradle/dependencies.gradle"

    apply from: "$rootDir/gradle/test-setup.gradle"

    apply from: "$rootDir/gradle/integration-test.gradle"

    apply from: "$rootDir/gradle/functional-test.gradle"

    apply from: "$rootDir/gradle/additional-artifacts.gradle"

    apply from: "$rootDir/gradle/publishing.gradle"

    apply from: "$rootDir/gradle/documentation.gradle"

    apply from: "$rootDir/gradle/release.gradle"

    View Slide

  17. Convention over configuration
    Provide  sensible  defaults  and  standards.  Expose  
    a  way  to  re-­‐configure  them  to  user’s  needs.  
    Examples:  
    • src/main/java  for  Java-­‐based  applications  
    • Project  name  derived  from  directory  name  
    • Output  directory  is  build

    View Slide

  18. Convention
    Plugin  consumers  will  be  most  comfortable  
    with  using  convention  plugins.  Pick  sensible  
    defaults  for  the  user.  
    Good  indicator:  
    The  less  a  consumer  has  to  re-­‐configure  
    defaults  the  better.

    View Slide

  19. Configuration
    Real-­‐world  project  might  need  to  re-­‐
    configure  the  defaults.  Make  it  convenient  to  
    change  defaults.  
    Reason:  
    Their  view  of  the  world  might  look  different  
    than  yours.

    View Slide

  20. Users only declare the “what”
    Expose  a  custom  DSL  from  your  binary  plugin  to  
    configure  runtime  behavior.  
    The  “how”  is  the  responsibility  of  the  plugin  impl.

    apply plugin: 'com.bmuschko.docker-remote-api'


    docker {

    url = 'https://192.168.59.103:2376'

    certPath = new File(System.properties['user.home'],
    '.boot2docker/certs/boot2docker-vm')

    }

    View Slide

  21. Example: Providing default dependencies
    Plugins  often  rely  on  external  libraries.  
    Automatically  resolve  default  version,  but  
    make  it  configurable.

    View Slide

  22. Customizing Docker Java library
    Introduce  custom  configura]on  in  plugin.

    project.configurations.create('docker')

    .setVisible(false)

    .setTransitive(true)

    .setDescription('The Docker Java libraries to be used.')
    Setting  dependency  from  consuming  build.

    dependencies {

    docker 'com.github.docker-java:docker-java:2.0.0'

    docker 'org.slf4j:slf4j-simple:1.7.5'

    }

    View Slide

  23. Providing default (but customizable) dependency
    Only  use  defaults,  if  no  dependencies  are  
    assigned  to  custom  configuration.

    Configuration config = project.configurations.create('docker')


    project.tasks.withType(AbstractDockerRemoteApiTask) {

    config.defaultDependencies { deps ->

    deps.add(project.dependencies
    .create('com.github.docker-java:docker-java:2.1.0'))

    deps.add(project.dependencies
    .create('org.slf4j:slf4j-simple:1.7.5'))

    }


    conventionMapping.with {

    classpath = { config } 

    }

    }
    Gradle  2.5

    View Slide

  24. Capabilities vs. conventions
    Separating  general-­‐purpose  functionality  
    from  pre-­‐configured,  opinionated  
    functionality.  
    Finding  the  right  balance  between  both  
    aspects  is  key.

    View Slide

  25. Capabilities
    • Un-­‐opinionated  functionality  
    • Provide  general-­‐purpose  concepts  
    Examples:  
    • Custom  task  types  without  creating  an  actual  
    task  instance  
    • Add  new  concepts  without  configuring  them  
    (e.g.  source  sets,  build  types  and  flavors)

    View Slide

  26. Conventions
    • Opinionated  functionality  
    • Instantiate  and/or  pre-­‐configure  concepts  
    Examples:  
    • Standardized  directory  layouts  
    • Enhanced  tasks  created  by  a  plugin  
    • Adding  default  values  to  extensions  
    • Declaring  task  dependencies  to  form  a  lifecycle

    View Slide

  27. Plugin composition
    Plugins  can  build  upon  other  plugins.  This  is  a  
    common  pattern.  
    A  base  plugin  provides    
    generic  capabilities  
    Another  plugin  builds  on  the  base,    
    adding  opinionated  conventions

    View Slide

  28. Example: Docker plugins
    Allows  users  to  pick  functionality  they  need  
    in  their  projects.  
    Plugin  that  adds  task  types  for    
    interacting  with  Docker  remote  API  
    Plugin  for  creating  &  pushing    
    Docker  image  for  Java  applications

    View Slide

  29. Example: Docker Java application plugin
    Applying  plugin  by  identifier  or  type.  

    import org.gradle.api.Plugin

    import org.gradle.api.Project


    class DockerJavaApplicationPlugin implements Plugin {

    @Override

    void apply(Project project) {

    project.apply(plugin: DockerRemoteApiPlugin)

    }

    }

    View Slide

  30. Build for further extension
    • Anticipate  more  specific  extensions  to  your  
    plugin  
    • Implemented  through  plugin  composition  
    • Base  plugin  act  as  enablers  for  custom  
    convention  plugins  
    • Enterprise  build  convention  plugins  re-­‐
    configure  your  plugins

    View Slide

  31. Put yourself in the shoes of consumer
    Do  not  automatically  apply  plugins  in  your  
    plugin!  
    Reason:    
    Makes  your  plugin  highly  opinionated.

    View Slide

  32. Reacting to plugins
    React  to  plugins  that  might  be  applied  in  
    consuming  build  script.  Only  then  logic  is  
    executed.  
    The  same  concept  applies  to  tasks.

    View Slide

  33. Using configuration rules
    Reacting  to  plugins

    project.plugins.withType(IdeaPlugin) {

    // configure Idea plugin in the context of your plugin

    }

    Reacting  to  task  types

    project.tasks.withType(War) {

    // configure all War tasks in the context of your plugin

    }

    View Slide

  34. Testing build logic
    Build  logic  needs  to  be  testable  on  multiple  
    levels.

    View Slide

  35. Implementing test types with Gradle
    Unit  testing:    
    No  Gradle  tooling  needed  
    Integration  testing:    
    Use  ProjectBuilder  to  create  pseudo  Project  
    instance  
    Functional  testing:  
    Nebula  Test,  Gradle  TestKit

    View Slide

  36. Example: Writing unit test with Spock

    package com.bmuschko.gradle.docker.tasks.image


    import spock.lang.Specification


    import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.*


    class DockerfileTest extends Specification {

    def "Instruction String representation is built correctly"() {

    expect:

    instructionInstance.keyword == keyword

    instructionInstance.build() == builtInstruction

    where:

    instructionInstance | keyword | builtInstruction

    new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04'

    new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04'

    ...

    }

    }

    View Slide

  37. Separating test type source code
       Default  unit  test  directory
    Custom  integration  test  directory
    Custom  functional  test  directory

    View Slide

  38. Example: Integration test source set and task

    sourceSets {

    integrationTest {

    groovy.srcDir file('src/integTest/groovy')

    resources.srcDir file('src/integTest/resources')

    compileClasspath += sourceSets.main.output + configurations.testRuntime

    runtimeClasspath += output + compileClasspath

    }

    }


    task integrationTest(type: Test) {

    description = 'Runs the integration tests.'

    group = 'verification'

    testClassesDir = sourceSets.integrationTest.output.classesDir

    classpath = sourceSets.integrationTest.runtimeClasspath

    mustRunAfter test

    }


    check.dependsOn integrationTest

    View Slide

  39. Example: Writing integration test with Spock

    import org.gradle.api.Project

    import org.gradle.testfixtures.ProjectBuilder
    class DockerJavaApplicationPluginIntegrationTest extends Specification {

    @Rule TemporaryFolder temporaryFolder = new TemporaryFolder()

    Project project


    def setup() {

    project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build()

    }


    def "Creates tasks out-of-the-box when application plugin is applied"() {

    when:

    project.apply(plugin: DockerJavaApplicationPlugin)

    project.apply(plugin: 'application')


    then:
    project.tasks.findByName(DockerJavaApplicationPlugin
    .COPY_DIST_RESOURCES_TASK_NAME)

    project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME)

    project.tasks.findByName(DockerJavaApplicationPlugin.BUILD_IMAGE_TASK_NAME)

    project.tasks.findByName(DockerJavaApplicationPlugin.PUSH_IMAGE_TASK_NAME)

    }

    }

    View Slide

  40. Functional testing
    Testing  the  build  logic  from  the  end  user’s  
    perspective.  
    • Exercising  build  logic  as  part  of  a  
    programmatically  executed  build.  
    • Make  assertions  about  build  outcome.  

    View Slide

  41. Using the Gradle TestKit
    Functional  testing  support  in  Gradle  core.  
    • Uses  Tooling  API  as  test  execution  harness.  
    • Agnostic  of  test  framework.  
    • Assertions  made  based  on  build  output,  
    build  logging  or  test  of  tasks  +  their  result.  
    Gradle  2.6

    View Slide

  42. TestKit usage
    Declaring  TestKit  dependency

    dependencies {

    testCompile gradleTestKit()

    }

    Declaring  the  Spock  dependency

    dependencies {

    testCompile 'org.spockframework:spock-core:1.0-groovy-2.3'

    }

    View Slide

  43. Example: Writing functional tests with TestKit

    def "can successfully create Dockerfile"() {

    given:

    buildFile << """

    import com.bmuschko.gradle.docker.tasks.image.Dockerfile


    task dockerfile(type: Dockerfile) {

    from 'ubuntu:14.04'

    maintainer 'Benjamin Muschko "[email protected]"'

    }
    """


    when:

    def result = GradleRunner.create()

    .withProjectDir(testProjectDir.root)

    .withArguments('dockerfile')

    .build()


    then:

    result.task(":dockerfile").outcome == SUCCESS
    testProjectDir.file('Dockerfile').exists()

    }

    View Slide

  44. DEMO
    Functional  testing  with  TestKit

    View Slide

  45. Roadmap for TestKit
    Gradle  2.6  
    Mechanics  for  executing  tests  
    Functional  tests  can  query  build  result  
    Gradle  2.7  
    Isolation  of  test  environment,  dedicated  daemons  
    Gradle  2.8  
    Convenient  injection  of  code  under  test

    View Slide

  46. Cross-version compatibility tests
    Forward  and  backward  compatibility  
    independent  of  Gradle  version  used  to  build  
    plugin  artifact.
    2.4 2.5 2.6
    2.3
    2.2
     Version  used  
    to  build  plugin

    View Slide

  47. Implementing compatibility tests
    No  built-­‐in  support  in  Gradle  yet.  On  the  
    roadmap  for  TestKit.  
    Intermediate  options:  
    • Custom  implementation  using  Tooling  API.  
    • Community  plugins  like  https:/
    /github.com/
    ysb33r/gradleTest

    View Slide

  48. Importance of documentation
    Plugins  are  not  self-­‐documenting.  
    Thorough  documentation  is  crucial  for  plugin  
    consumers.  

    View Slide

  49. Answer questions for consumers
    How  can  I  use  the  plugin  and  configure  it?  
    What  tasks  &  extensions  are  provided?  
    What  impact  does  the  plugin  have  on  my  
    project?

    View Slide

  50. Impact on consumers
    De-­‐mystify  added  functionality.  
    Give  consumers  the  feeling  that  they  are  
    under  control.

    View Slide

  51. What should be documented?
    • Purpose  of  plugin,  the  repository  that  hosts  the  plugin  
    • Plugins:  Identifier,  type  and  description  bundled  in  a  JAR  
    • Enhanced  tasks:  their  name,  type  and  dependencies  
    • Custom  task  types:  Javadocs,  Groovydocs  
    • Conventions,  custom  extensions  (e.g.  DSLs)  
    • Extension  properties/methods  
    • Usage  examples  in  textual  form  
    • A  good  set  of  functional  tests  that  demonstrate  the  use  
    of  the  plugin

    View Slide

  52. Example: Plugin description & usage

    View Slide

  53. Example: Extension usage and properties

    View Slide

  54. Example: Textual usage & functional tests

    View Slide

  55. Example: Linking custom tasks to API docs

    View Slide

  56. Example: Publishing API docs to GitHub pages
    Javadocs/Groovydocs  are  essential  to  allow  users  discover  
    API  classes  exposed  by  plugin.  
    • Create  new  branch  gh-pages  
    • Remove  all  files  from  working  tree  and  index  
    • Push  new  branch  
    Available  under  
    http(s)://.github.io/

    View Slide

  57. Example: Publishing Javadocs to GitHub pages
    Use  gradle-­‐git  plugin  to  push  to  automatically  
    publish  Javadocs.  

    apply plugin: 'org.ajoberstar.github-pages'


    githubPages {

    repoUri = '[email protected]:bmuschko/gradle-docker-plugin.git'


    pages {

    from(javadoc.outputs.files) {

    into 'docs/javadoc'

    }

    }

    }

    View Slide

  58. DEMO
    Publishing  changed  API  documentation

    View Slide

  59. Publishing the plugin binaries
    Make  artifact(s)  available  to  consumers  by  
    uploading  them  to  one  or  many  binary  
    repositories.

    View Slide

  60. Provide appropriate metadata

    apply plugin: 'maven-publish'


    publishing {

    publications {

    mavenJava(MavenPublication) {

    pom.withXml {

    def root = asNode()

    root.appendNode('name', 'Gradle Docker plugin')

    root.appendNode('description', 'Gradle plugin for managing Docker images and containers.')

    root.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin')

    root.appendNode('inceptionYear', '2014')


    def scm = root.appendNode('scm')

    scm.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin')

    scm.appendNode('connection', 'scm:https://[email protected]/bmuschko/gradle-docker-plugin.git')

    scm.appendNode('developerConnection', 'scm:git://github.com/bmuschko/gradle-docker-plugin.git')


    def license = root.appendNode('licenses').appendNode('license')

    license.appendNode('name', 'The Apache Software License, Version 2.0')

    license.appendNode('url', 'http://www.apache.org/licenses/LICENSE-2.0.txt')

    license.appendNode('distribution', 'repo')


    def developers = root.appendNode('developers')

    def developer = developers.appendNode('developer')

    developer.appendNode('id', 'bmuschko')

    developer.appendNode('name', 'Benjamin Muschko')

    developer.appendNode('email', '[email protected]')

    }

    }

    }

    }

    View Slide

  61. Publishing doesn’t stop with plugin JAR

    task sourcesJar(type: Jar) {

    classifier 'sources'

    from sourceSets.main.allSource

    }


    task groovydocJar(type: Jar, dependsOn: groovydoc) {

    classifier 'groovydoc'

    from groovydoc.destinationDir

    }


    task javadocJar(type: Jar, dependsOn: javadoc) {

    classifier 'javadoc'

    from javadoc.destinationDir

    }


    artifacts {

    archives sourcesJar

    archives groovydocJar

    archives javadocJar

    }

    View Slide

  62. Share Open Source plugins on the portal
    https:/
    /plugins.gradle.org/

    View Slide

  63. Publishing to the Gradle plugin portal
    Requires  use  of  the  “Plugin  Publishing  Plugin”

    buildscript {

    repositories {

    maven {

    url 'https://plugins.gradle.org/m2/'

    }

    }

    dependencies {

    classpath 'com.gradle.publish:plugin-publish-plugin:0.9.1'

    }

    }


    apply plugin: 'com.gradle.plugin-publish'

    View Slide

  64. Declaring required portal plugin metadata

    pluginBundle {

    website = 'https://github.com/bmuschko/gradle-docker-plugin'

    vcsUrl = 'https://github.com/bmuschko/gradle-docker-plugin.git'

    description = 'Gradle plugin for managing Docker images and containers.'

    tags = ['gradle', 'docker', 'container', 'image', 'lightweight',
    'vm', 'linux']


    plugins {

    dockerRemoteApi {

    id = 'com.bmuschko.docker-remote-api'

    displayName = 'Provides custom tasks for interacting with
    Docker via its remote API.'

    }


    dockerJavaApplication {

    id = 'com.bmuschko.docker-java-application'

    displayName = 'Creates and pushes a Docker image for a Java
    application.'

    }

    }

    }

    View Slide

  65. Continuous Integration
    A  plugin  might  start  small  but  will  grow  in  
    complexity  over  time.  Avoid  integration  hell!  
    • Fast  feedback  with  every  code  integration  
    • Always  execute  (an  assorted  set  of)  tests  
    • Use  CI  cloud-­‐based  product  or  host  in-­‐
    house

    View Slide

  66. Adding a CI badge
    Give  consumers  and  contributor  a  chance  to  
    see  the  build  status.

    View Slide

  67. Working towards Continuous Delivery
    Make  sure  your  plugin  is  always  production  
    ready.  
    • End  goal:  avoid  manual  steps  
    • Build  and  visualize  suitable  delivery  pipeline  
    • Go  the  extra  mile  -­‐  model  push-­‐button  
    release  capabilities

    View Slide

  68. Example: Potential release steps
     Publi
    Assemble  
    artifact(s)
       Tag  VCS  
    repository
       Publish  
    artifact(s)
    Plugin  JAR  
    Sources  JAR  
    Docs  JAR
    Create  tag  
    Push  tag
    Create  metadata  
    Upload  artifact(s)  
    Update  API  docs

    View Slide

  69. DEMO
    Implementing  a  plugin  release  process

    View Slide

  70. Example: Build pipeline with Snap CI
                         Build,  test  and  deploy  in  the  cloud.  
    • Free  of  charge  for  public  Github  repos  
    • Model  build  pipelines  with  automatic  and/
    or  manual  execution  steps  
    • Execute  build  for  pull  requests

    View Slide

  71. Example: Snap CI pipeline for Docker plugin
    https:/
    /snap-­‐ci.com/bmuschko/gradle-­‐docker-­‐plugin/branch/master
    Compilation  
       Unit  tests
    Integration  
             tests
    Functional  
             tests
    Release

    View Slide

  72. DEMO
    Showcasing  build  pipeline  on  Snap  CI

    View Slide

  73. Other aspects
    • Be  mindful  of  the  performance  impact  your  
    plugin  might  have  on  consuming  build  
    • Avoid  using  internal  Gradle  APIs  as  much  as  
    possible

    View Slide

  74. Thank  You!
    Please  ask  questions…
    https:/
    /www.github.com/bmuschko
    @bmuschko

    View Slide