Pro Yearly is on sale from $80 to $50! »

Testing the build with TestKit

Testing the build with TestKit

Build code contributes to the success of a company to a large extent. It enables an organization to deliver their software to the end user fast, frequently and reliably. With Continuous Delivery becoming a standard practice throughout the industry and build logic increasing in complexity, a need arises to verify the logic powering these processes. Gradle provides powerful support for testing build code out-of-the-box. In this demo-driven session, we will discuss the ins and outs of TestKit, a Gradle core library that helps with functional testing of build logic.

In the course of this talk, we’ll touch on aspects such as:

– Setting up TestKit for a build
– Executing a build with a specific Gradle distribution
– Asserting expected build outcome and output
– Verifying proper runtime behavior of custom plugins and extensions
– Cross-version testing of builds
– Debugging test execution in the IDE

8f2248c6bfcc6df39a2cd8edf4267cb5?s=128

Benjamin Muschko

June 25, 2016
Tweet

Transcript

  1. Gradle Summit Testing the build with TestKit
 Benjamin Muschko 2016

  2. About to ship code to production…

  3. …but the build logic contained bugs

  4. Why not test build logic?

  5. Testing build logic Build logic needs to be testable on

    multiple levels.
  6. Writing unit tests in Gradle Class under test does not

    use Gradle API Test logic in isolation Testing a single class
  7. Example: Writing a unit test with Spock 
 package com.bmuschko.gradle.docker.tasks.image


    
 import spock.lang.Specification
 
 import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.*
 
 class DockerfileTest extends Specification {
 def "Instruction String representation is built correctly"() {
 expect:
 instructionInstance.keyword == keyword
 instructionInstance.build() == builtInstruction 
 where:
 instructionInstance | keyword | builtInstruction
 new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04'
 new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04'
 ...
 }
 }
  8. Writing integration tests in Gradle Class(es) under test use Gradle

    API Usually interacts with Project instance Does not execute a full build script
  9. Example: Writing an integration test using ProjectBuilder 
 import org.gradle.api.Project


    import org.gradle.testfixtures.ProjectBuilder class DockerJavaApplicationPluginIntegrationTest extends Specification {
 @Rule TemporaryFolder temporaryFolder = new TemporaryFolder()
 Project project
 
 def setup() {
 project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build()
 }
 
 def "Creates tasks out-of-the-box when application plugin is applied"() {
 when:
 project.apply(plugin: DockerJavaApplicationPlugin)
 project.apply(plugin: 'application')
 
 then: project.tasks.findByName(DockerJavaApplicationPlugin .COPY_DIST_RESOURCES_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.BUILD_IMAGE_TASK_NAME)
 project.tasks.findByName(DockerJavaApplicationPlugin.PUSH_IMAGE_TASK_NAME)
 }
 }
  10. Writing functional tests in Gradle Executes build script similar to

    end user Examines build outcome and output Isolated test environment
  11. Example: Writing a functional test with TestKit 
 def "can

    successfully create Dockerfile"() {
 given:
 buildFile << """
 import com.bmuschko.gradle.docker.tasks.image.Dockerfile
 
 task dockerfile(type: Dockerfile) {
 from 'ubuntu:14.04'
 maintainer 'Benjamin Muschko "benjamin.muschko@gmail.com"'
 } """
 
 when:
 def result = GradleRunner.create()
 .withProjectDir(testProjectDir.root)
 .withArguments('dockerfile')
 .build()
 
 then:
 result.task(":dockerfile").outcome == SUCCESS testProjectDir.file('Dockerfile').exists()
 }
  12. What’s the Gradle TestKit? - Uses Tooling API as test

    execution harness - Agnostic of test framework - Assertions made based on build output, build logging or executed tasks + their result Functional testing support in Gradle core. Gradle 2.6
  13. TestKit usage Declaring TestKit dependency 
 dependencies {
 testCompile gradleTestKit()


    }
 Declaring dependency on test framework 
 dependencies {
 testCompile 'org.spockframework:spock-core:1.0-groovy-2.4'
 }
  14. DEMO Functional testing of a build script

  15. Cross-version compatibility tests Forward and backward compatibility independent of Gradle

    version used to build logic. 2.4 2.5 2.6 2.3 2.2 Version used for writing build logic
  16. Retrieving Gradle distributions Enterprise Server Local or Shared Disk Build

    Machine Public Server
  17. Specifying a Gradle version Using a Gradle version available on

    server Using a Gradle installation on disk Using a Gradle distribution via URL GradleRunner.withGradleVersion(String) GradleRunner.withGradleInstallation(File) GradleRunner.withGradleDistribution(URI)
  18. DEMO Cross-version compatibility testing

  19. Testing a plugin - Tooling API runs in separate process

    - Does not share classpath and classloaders as test process - Requires injection of code under test Requires additional work!
  20. DEMO Manually injecting the plugin classpath

  21. That’s some complex machinery!

  22. Simplifying setup with plugin development plugin - Declaration of gradleApi()

    and gradleTestKit() - Generates plugin classpath manifest file - Plugin classpath injection needs to be declared for GradleRunner instance explicitly Apply sensible default and conventions…
  23. Automatic injection of plugin code Applying the Java Gradle plugin

    dev. plugin 
 apply plugin: 'java-gradle-plugin' Injecting classpath into GradleRunner 
 GradleRunner.create()
 .withPluginClasspath()
 .build() Gradle 2.13
  24. DEMO Testing a plugin with TestKit

  25. Configurable conventions Source set containing code under test Source set

    used for injecting the plugin classpath Reconfigurable with the help of the class GradlePluginDevelopmentExtension sourceSets.main sourceSets.test
  26. DEMO Configuring a dedicated test source set

  27. Debugging test execution Setting system property for ad-hoc testing Enabling

    debugging programmatically -Dorg.gradle.testkit.debug=true GradleRunner.withDebug(true)
  28. DEMO Debugging tests from the IDE

  29. Feature limitations Support for features determined by version of Tooling

    API used to execute test. More info: https:/ /docs.gradle.org/current/userguide/test_kit.html
  30. Future enhancements for TestKit - Convenience test fixtures - Hooking

    into IDE plugins - Integration with JaCoCo plugin More info: https://github.com/gradle/gradle/blob/master/ design-docs/testing-toolkit.md
  31. Thank You! Please ask questions… https:/ /www.github.com/bmuschko @bmuschko

  32. Introducing Gradle Cloud Services - Insights into your build -

    View and share via URL - Debug, optimize and refine - Analyze all of your builds Stop by the Gradle Lounge and create a Build Scan! The first service, Gradle Build Scans, is now available.
  33. Who Are We Motto: Build Happiness Mission: To revolutionize the

    way software is built and shipped. We exist to end once-and-for-all the worst things about big software and restore the reason you got into coding in the first place. We’re Hiring: Gradle is hiring front-end, back-end, and core software engineers. Visit gradle.org/jobs to apply
  34. Learn more at www.gradle.org