Share
Gradle plugin best practices by example
Gradle is a general purpose, multi-platform build tool. Plugins allow for extending Gradle's core capabilities with reusable and targeted functionality. In this session, we'll discuss techniques and best practices for developing your own Gradle plugins by dissecting the structure, code, documentation and supporting infrastructure of an existing Gradle plugin.
Gradle plugins are extremly simple to write. But there's far more to learn. Once you get beyond the basics, you will want to know about guidelines and best practices. In this session, we'll identify concepts and techniques that work well in practice by having a look at the Gradle Docker plugin.
We'll mainly focus on the following topics:
* Brief introduction to Gradle plugin concepts
* Applying proven software engineering practices
* Separating capabilities from conventions
* Implementing convention over configuration
* Writing flexible custom tasks
* Depending on external libraries
* Exposing a custom DSL to plugin consumers
* Using the Gradle Wrapper
* Writing different types of tests for plugin code
* Publishing the plugin binaries
* Creating functional documentation
* Registering the plugin with Gradle's plugin portal
* Setting up Continuous Integration
Benjamin Muschko
Transcript
74 slides
-
The dark past of build logic Unstructured spaghetti code Copy
& paste of code snippets The build tool tells you how to structure code Build can only be understood by build guru™ Testing through manual execution -
Build logic “-ilities” Typical non-‐functional software requirements also apply
to build code… • Reusability • Testability • Maintainability • Extensibility • Configurability -
Reusability Avoid copy/paste! Allow code to be shared among
independent projects. -
Testability Build code is no different from application code.
Test it on multiple levels! -
Maintainability Avoid the big ball of mud! Cohesion and
separation of concerns are important. -
Extensibility Extend Gradle's build language by your own declarative
& expressive language constructs. -
Configurability Don't box in your users! Implement convention over
configuration with ease. -
The end goal Maintainable, reusable & tested code Consumers only
configure the code Complex implement. details are hidden -
How to get there? Concepts that help implement these
requirements… • Good software engineering practices • Custom tasks • Plugins • Gradle's extension mechanisms • Testing capabilities -
Techniques covered in talk • Declarative vs. imperative logic
• Convention over configuration • Capabilities vs. conventions • Gradle's extension mechanisms • Testing plugin logic • Publishing the plugin artifacts • Writing and generating documentation • Setting up Continuous Integration/Delivery -
Case study: Gradle Docker plugin
Serves as showcase plugin project. • Plugin for managing Docker images and containers through Docker remote API • API communication via Docker Java library • Written in Groovy • Source code available on GitHub (https:/ / github.com/bmuschko/gradle-‐docker-‐plugin) -
Always check in the Wrapper Project becomes instantly buildable for
every developer and on the CI machine. Check in Wrapper files into VCS repository. -
Hiding complex, imperative logic Concept applies to tasks and plugins.
• Reusable and configurable • Easy to structure, refactor and test • Avoid global properties and methods -
Task type > ad-hoc task Prefer implementing task types to
implementing ad-‐hoc tasks. task createDockerfile(type: Dockerfile) { destFile = file("$buildDir/mydockerfile/Dockerfile") from 'ubuntu:12.04' maintainer 'Benjamin Muschko "[email protected]"' } task buildImage(type: DockerBuildImage) { dependsOn createDockerfile inputDir = createDockerfile.destFile.parentFile tag = 'bmuschko/myimage' } -
Binary plugin > script plugin Use script plugins to organize
build logic based on functional boundaries in project. apply from: "$rootDir/gradle/dependencies.gradle" apply from: "$rootDir/gradle/test-setup.gradle" apply from: "$rootDir/gradle/integration-test.gradle" apply from: "$rootDir/gradle/functional-test.gradle" apply from: "$rootDir/gradle/additional-artifacts.gradle" apply from: "$rootDir/gradle/publishing.gradle" apply from: "$rootDir/gradle/documentation.gradle" apply from: "$rootDir/gradle/release.gradle" -
Convention over configuration Provide sensible defaults and standards. Expose
a way to re-‐configure them to user’s needs. Examples: • src/main/java for Java-‐based applications • Project name derived from directory name • Output directory is build -
Convention Plugin consumers will be most comfortable with using
convention plugins. Pick sensible defaults for the user. Good indicator: The less a consumer has to re-‐configure defaults the better. -
Configuration Real-‐world project might need to re-‐ configure the defaults.
Make it convenient to change defaults. Reason: Their view of the world might look different than yours. -
Users only declare the “what” Expose a custom DSL from
your binary plugin to configure runtime behavior. The “how” is the responsibility of the plugin impl. apply plugin: 'com.bmuschko.docker-remote-api' docker { url = 'https://192.168.59.103:2376' certPath = new File(System.properties['user.home'], '.boot2docker/certs/boot2docker-vm') } -
Example: Providing default dependencies Plugins often rely on external libraries.
Automatically resolve default version, but make it configurable. -
Customizing Docker Java library Introduce custom configura]on in plugin.
project.configurations.create('docker') .setVisible(false) .setTransitive(true) .setDescription('The Docker Java libraries to be used.') Setting dependency from consuming build. dependencies { docker 'com.github.docker-java:docker-java:2.0.0' docker 'org.slf4j:slf4j-simple:1.7.5' } -
Providing default (but customizable) dependency Only use defaults, if no
dependencies are assigned to custom configuration. Configuration config = project.configurations.create('docker') project.tasks.withType(AbstractDockerRemoteApiTask) { config.defaultDependencies { deps -> deps.add(project.dependencies .create('com.github.docker-java:docker-java:2.1.0')) deps.add(project.dependencies .create('org.slf4j:slf4j-simple:1.7.5')) } conventionMapping.with { classpath = { config } } } Gradle 2.5 -
Capabilities vs. conventions Separating general-‐purpose functionality from pre-‐configured, opinionated
functionality. Finding the right balance between both aspects is key. -
Capabilities • Un-‐opinionated functionality • Provide general-‐purpose concepts
Examples: • Custom task types without creating an actual task instance • Add new concepts without configuring them (e.g. source sets, build types and flavors) -
Conventions • Opinionated functionality • Instantiate and/or pre-‐configure concepts
Examples: • Standardized directory layouts • Enhanced tasks created by a plugin • Adding default values to extensions • Declaring task dependencies to form a lifecycle -
Plugin composition Plugins can build upon other plugins. This is
a common pattern. A base plugin provides generic capabilities Another plugin builds on the base, adding opinionated conventions -
Example: Docker plugins Allows users to pick functionality they need
in their projects. Plugin that adds task types for interacting with Docker remote API Plugin for creating & pushing Docker image for Java applications -
Example: Docker Java application plugin Applying plugin by identifier or
type. import org.gradle.api.Plugin import org.gradle.api.Project class DockerJavaApplicationPlugin implements Plugin<Project> { @Override void apply(Project project) { project.apply(plugin: DockerRemoteApiPlugin) } } -
Build for further extension • Anticipate more specific extensions to
your plugin • Implemented through plugin composition • Base plugin act as enablers for custom convention plugins • Enterprise build convention plugins re-‐ configure your plugins -
Put yourself in the shoes of consumer Do not automatically
apply plugins in your plugin! Reason: Makes your plugin highly opinionated. -
Reacting to plugins React to plugins that might be applied
in consuming build script. Only then logic is executed. The same concept applies to tasks. -
Using configuration rules Reacting to plugins project.plugins.withType(IdeaPlugin) { //
configure Idea plugin in the context of your plugin } Reacting to task types project.tasks.withType(War) { // configure all War tasks in the context of your plugin } -
Testing build logic Build logic needs to be testable on
multiple levels. -
Implementing test types with Gradle Unit testing: No
Gradle tooling needed Integration testing: Use ProjectBuilder to create pseudo Project instance Functional testing: Nebula Test, Gradle TestKit -
Example: Writing unit test with Spock package com.bmuschko.gradle.docker.tasks.image
import spock.lang.Specification import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.* class DockerfileTest extends Specification { def "Instruction String representation is built correctly"() { expect: instructionInstance.keyword == keyword instructionInstance.build() == builtInstruction where: instructionInstance | keyword | builtInstruction new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04' new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04' ... } } -
Separating test type source code Default unit test directory
Custom integration test directory Custom functional test directory -
Example: Integration test source set and task sourceSets {
integrationTest { groovy.srcDir file('src/integTest/groovy') resources.srcDir file('src/integTest/resources') compileClasspath += sourceSets.main.output + configurations.testRuntime runtimeClasspath += output + compileClasspath } } task integrationTest(type: Test) { description = 'Runs the integration tests.' group = 'verification' testClassesDir = sourceSets.integrationTest.output.classesDir classpath = sourceSets.integrationTest.runtimeClasspath mustRunAfter test } check.dependsOn integrationTest -
Example: Writing integration test with Spock import org.gradle.api.Project import
org.gradle.testfixtures.ProjectBuilder class DockerJavaApplicationPluginIntegrationTest extends Specification { @Rule TemporaryFolder temporaryFolder = new TemporaryFolder() Project project def setup() { project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build() } def "Creates tasks out-of-the-box when application plugin is applied"() { when: project.apply(plugin: DockerJavaApplicationPlugin) project.apply(plugin: 'application') then: project.tasks.findByName(DockerJavaApplicationPlugin .COPY_DIST_RESOURCES_TASK_NAME) project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME) project.tasks.findByName(DockerJavaApplicationPlugin.BUILD_IMAGE_TASK_NAME) project.tasks.findByName(DockerJavaApplicationPlugin.PUSH_IMAGE_TASK_NAME) } } -
Functional testing Testing the build logic from the end user’s
perspective. • Exercising build logic as part of a programmatically executed build. • Make assertions about build outcome. -
Using the Gradle TestKit Functional testing support in Gradle core.
• Uses Tooling API as test execution harness. • Agnostic of test framework. • Assertions made based on build output, build logging or test of tasks + their result. Gradle 2.6 -
TestKit usage Declaring TestKit dependency dependencies { testCompile gradleTestKit()
} Declaring the Spock dependency dependencies { testCompile 'org.spockframework:spock-core:1.0-groovy-2.3' } -
Example: Writing functional tests with TestKit def "can successfully
create Dockerfile"() { given: buildFile << """ import com.bmuschko.gradle.docker.tasks.image.Dockerfile task dockerfile(type: Dockerfile) { from 'ubuntu:14.04' maintainer 'Benjamin Muschko "[email protected]"' } """ when: def result = GradleRunner.create() .withProjectDir(testProjectDir.root) .withArguments('dockerfile') .build() then: result.task(":dockerfile").outcome == SUCCESS testProjectDir.file('Dockerfile').exists() } -
Roadmap for TestKit Gradle 2.6 Mechanics for executing tests
Functional tests can query build result Gradle 2.7 Isolation of test environment, dedicated daemons Gradle 2.8 Convenient injection of code under test -
Cross-version compatibility tests Forward and backward compatibility independent of
Gradle version used to build plugin artifact. 2.4 2.5 2.6 2.3 2.2 Version used to build plugin -
Implementing compatibility tests No built-‐in support in Gradle yet. On
the roadmap for TestKit. Intermediate options: • Custom implementation using Tooling API. • Community plugins like https:/ /github.com/ ysb33r/gradleTest -
Importance of documentation Plugins are not self-‐documenting. Thorough documentation
is crucial for plugin consumers. -
Answer questions for consumers How can I use the plugin
and configure it? What tasks & extensions are provided? What impact does the plugin have on my project? -
Impact on consumers De-‐mystify added functionality. Give consumers the
feeling that they are under control. -
What should be documented? • Purpose of plugin, the repository
that hosts the plugin • Plugins: Identifier, type and description bundled in a JAR • Enhanced tasks: their name, type and dependencies • Custom task types: Javadocs, Groovydocs • Conventions, custom extensions (e.g. DSLs) • Extension properties/methods • Usage examples in textual form • A good set of functional tests that demonstrate the use of the plugin -
Example: Publishing API docs to GitHub pages Javadocs/Groovydocs are essential
to allow users discover API classes exposed by plugin. • Create new branch gh-pages • Remove all files from working tree and index • Push new branch Available under http(s)://<username>.github.io/<projectname> -
Example: Publishing Javadocs to GitHub pages Use gradle-‐git plugin to
push to automatically publish Javadocs. apply plugin: 'org.ajoberstar.github-pages' githubPages { repoUri = '[email protected]:bmuschko/gradle-docker-plugin.git' pages { from(javadoc.outputs.files) { into 'docs/javadoc' } } } -
Publishing the plugin binaries Make artifact(s) available to consumers by
uploading them to one or many binary repositories. -
Provide appropriate metadata apply plugin: 'maven-publish' publishing {
publications { mavenJava(MavenPublication) { pom.withXml { def root = asNode() root.appendNode('name', 'Gradle Docker plugin') root.appendNode('description', 'Gradle plugin for managing Docker images and containers.') root.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin') root.appendNode('inceptionYear', '2014') def scm = root.appendNode('scm') scm.appendNode('url', 'https://github.com/bmuschko/gradle-docker-plugin') scm.appendNode('connection', 'scm:https://[email protected]/bmuschko/gradle-docker-plugin.git') scm.appendNode('developerConnection', 'scm:git://github.com/bmuschko/gradle-docker-plugin.git') def license = root.appendNode('licenses').appendNode('license') license.appendNode('name', 'The Apache Software License, Version 2.0') license.appendNode('url', 'http://www.apache.org/licenses/LICENSE-2.0.txt') license.appendNode('distribution', 'repo') def developers = root.appendNode('developers') def developer = developers.appendNode('developer') developer.appendNode('id', 'bmuschko') developer.appendNode('name', 'Benjamin Muschko') developer.appendNode('email', '[email protected]') } } } } -
Publishing doesn’t stop with plugin JAR task sourcesJar(type: Jar)
{ classifier 'sources' from sourceSets.main.allSource } task groovydocJar(type: Jar, dependsOn: groovydoc) { classifier 'groovydoc' from groovydoc.destinationDir } task javadocJar(type: Jar, dependsOn: javadoc) { classifier 'javadoc' from javadoc.destinationDir } artifacts { archives sourcesJar archives groovydocJar archives javadocJar } -
Publishing to the Gradle plugin portal Requires use of the
“Plugin Publishing Plugin” buildscript { repositories { maven { url 'https://plugins.gradle.org/m2/' } } dependencies { classpath 'com.gradle.publish:plugin-publish-plugin:0.9.1' } } apply plugin: 'com.gradle.plugin-publish' -
Declaring required portal plugin metadata pluginBundle { website =
'https://github.com/bmuschko/gradle-docker-plugin' vcsUrl = 'https://github.com/bmuschko/gradle-docker-plugin.git' description = 'Gradle plugin for managing Docker images and containers.' tags = ['gradle', 'docker', 'container', 'image', 'lightweight', 'vm', 'linux'] plugins { dockerRemoteApi { id = 'com.bmuschko.docker-remote-api' displayName = 'Provides custom tasks for interacting with Docker via its remote API.' } dockerJavaApplication { id = 'com.bmuschko.docker-java-application' displayName = 'Creates and pushes a Docker image for a Java application.' } } } -
Continuous Integration A plugin might start small but will grow
in complexity over time. Avoid integration hell! • Fast feedback with every code integration • Always execute (an assorted set of) tests • Use CI cloud-‐based product or host in-‐ house -
Adding a CI badge Give consumers and contributor a chance
to see the build status. -
Working towards Continuous Delivery Make sure your plugin is always
production ready. • End goal: avoid manual steps • Build and visualize suitable delivery pipeline • Go the extra mile -‐ model push-‐button release capabilities -
Example: Potential release steps Publi Assemble artifact(s) Tag
VCS repository Publish artifact(s) Plugin JAR Sources JAR Docs JAR Create tag Push tag Create metadata Upload artifact(s) Update API docs -
Example: Build pipeline with Snap CI
Build, test and deploy in the cloud. • Free of charge for public Github repos • Model build pipelines with automatic and/ or manual execution steps • Execute build for pull requests -
Example: Snap CI pipeline for Docker plugin https:/ /snap-‐ci.com/bmuschko/gradle-‐docker-‐plugin/branch/master Compilation
Unit tests Integration tests Functional tests Release -
Other aspects • Be mindful of the performance impact your
plugin might have on consuming build • Avoid using internal Gradle APIs as much as possible