Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Testing the build with TestKit

Testing the build with TestKit

Build code contributes to the success of a company to a large extent. It enables an organization to deliver their software to the end user fast, frequently and reliably. With Continuous Delivery becoming a standard practice throughout the industry and build logic increasing in complexity, a need arises to verify the logic powering these processes. Gradle provides powerful support for testing build code out-of-the-box. In this demo-driven session, we will discuss the ins and outs of TestKit, a Gradle core library that helps with functional testing of build logic.

In the course of this talk, we’ll touch on aspects such as:

– Setting up TestKit for a build
– Executing a build with a specific Gradle distribution
– Asserting expected build outcome and output
– Verifying proper runtime behavior of custom plugins and extensions
– Cross-version testing of builds
– Debugging test execution in the IDE

Benjamin Muschko

June 25, 2016
Tweet

More Decks by Benjamin Muschko

Other Decks in Programming

Transcript

  1. Gradle Summit
    Testing the build with TestKit

    Benjamin Muschko
    2016

    View Slide

  2. About to ship code to production…

    View Slide

  3. …but the build logic contained bugs

    View Slide

  4. Why not test build logic?

    View Slide

  5. Testing build logic
    Build logic needs to be testable on multiple
    levels.

    View Slide

  6. Writing unit tests in Gradle
    Class under test does
    not use Gradle API
    Test logic in isolation
    Testing a single class

    View Slide

  7. Example: Writing a unit test with Spock

    package com.bmuschko.gradle.docker.tasks.image


    import spock.lang.Specification


    import static com.bmuschko.gradle.docker.tasks.image.Dockerfile.*


    class DockerfileTest extends Specification {

    def "Instruction String representation is built correctly"() {

    expect:

    instructionInstance.keyword == keyword

    instructionInstance.build() == builtInstruction

    where:

    instructionInstance | keyword | builtInstruction

    new FromInstruction('ubuntu:14.04') | 'FROM' | 'FROM ubuntu:14.04'

    new FromInstruction({ 'ubuntu:14.04' }) | 'FROM' | 'FROM ubuntu:14.04'

    ...

    }

    }

    View Slide

  8. Writing integration tests in Gradle
    Class(es) under test
    use Gradle API
    Usually interacts with
    Project instance
    Does not execute
    a full build script

    View Slide

  9. Example: Writing an integration test using ProjectBuilder

    import org.gradle.api.Project

    import org.gradle.testfixtures.ProjectBuilder
    class DockerJavaApplicationPluginIntegrationTest extends Specification {

    @Rule TemporaryFolder temporaryFolder = new TemporaryFolder()

    Project project


    def setup() {

    project = ProjectBuilder.builder().withProjectDir(temporaryFolder.root).build()

    }


    def "Creates tasks out-of-the-box when application plugin is applied"() {

    when:

    project.apply(plugin: DockerJavaApplicationPlugin)

    project.apply(plugin: 'application')


    then:
    project.tasks.findByName(DockerJavaApplicationPlugin
    .COPY_DIST_RESOURCES_TASK_NAME)

    project.tasks.findByName(DockerJavaApplicationPlugin.DOCKERFILE_TASK_NAME)

    project.tasks.findByName(DockerJavaApplicationPlugin.BUILD_IMAGE_TASK_NAME)

    project.tasks.findByName(DockerJavaApplicationPlugin.PUSH_IMAGE_TASK_NAME)

    }

    }

    View Slide

  10. Writing functional tests in Gradle
    Executes build script
    similar to end user
    Examines build
    outcome and output
    Isolated test
    environment

    View Slide

  11. Example: Writing a functional test with TestKit

    def "can successfully create Dockerfile"() {

    given:

    buildFile << """

    import com.bmuschko.gradle.docker.tasks.image.Dockerfile


    task dockerfile(type: Dockerfile) {

    from 'ubuntu:14.04'

    maintainer 'Benjamin Muschko "[email protected]"'

    }
    """


    when:

    def result = GradleRunner.create()

    .withProjectDir(testProjectDir.root)

    .withArguments('dockerfile')

    .build()


    then:

    result.task(":dockerfile").outcome == SUCCESS
    testProjectDir.file('Dockerfile').exists()

    }

    View Slide

  12. What’s the Gradle TestKit?
    - Uses Tooling API as test execution harness
    - Agnostic of test framework
    - Assertions made based on build output, build
    logging or executed tasks + their result
    Functional testing support in Gradle core.
    Gradle 2.6

    View Slide

  13. TestKit usage
    Declaring TestKit dependency

    dependencies {

    testCompile gradleTestKit()

    }

    Declaring dependency on test framework

    dependencies {

    testCompile 'org.spockframework:spock-core:1.0-groovy-2.4'

    }

    View Slide

  14. DEMO
    Functional testing of a build script

    View Slide

  15. Cross-version compatibility tests
    Forward and backward compatibility independent
    of Gradle version used to build logic.
    2.4 2.5 2.6
    2.3
    2.2
    Version used for
    writing build logic

    View Slide

  16. Retrieving Gradle distributions
    Enterprise
    Server
    Local or
    Shared Disk
    Build Machine
    Public Server

    View Slide

  17. Specifying a Gradle version
    Using a Gradle version available on server
    Using a Gradle installation on disk
    Using a Gradle distribution via URL
    GradleRunner.withGradleVersion(String)
    GradleRunner.withGradleInstallation(File)
    GradleRunner.withGradleDistribution(URI)

    View Slide

  18. DEMO
    Cross-version compatibility testing

    View Slide

  19. Testing a plugin
    - Tooling API runs in separate process
    - Does not share classpath and classloaders as
    test process
    - Requires injection of code under test
    Requires additional work!

    View Slide

  20. DEMO
    Manually injecting the plugin classpath

    View Slide

  21. That’s some complex machinery!

    View Slide

  22. Simplifying setup with plugin development plugin
    - Declaration of gradleApi() and
    gradleTestKit()
    - Generates plugin classpath manifest file
    - Plugin classpath injection needs to be declared
    for GradleRunner instance explicitly
    Apply sensible default and conventions…

    View Slide

  23. Automatic injection of plugin code
    Applying the Java Gradle plugin dev. plugin

    apply plugin: 'java-gradle-plugin'
    Injecting classpath into GradleRunner

    GradleRunner.create()

    .withPluginClasspath()

    .build()
    Gradle 2.13

    View Slide

  24. DEMO
    Testing a plugin with TestKit

    View Slide

  25. Configurable conventions
    Source set containing code under test
    Source set used for injecting the plugin classpath
    Reconfigurable with the help of the class
    GradlePluginDevelopmentExtension
    sourceSets.main
    sourceSets.test

    View Slide

  26. DEMO
    Configuring a dedicated test source set

    View Slide

  27. Debugging test execution
    Setting system property for ad-hoc testing
    Enabling debugging programmatically
    -Dorg.gradle.testkit.debug=true
    GradleRunner.withDebug(true)

    View Slide

  28. DEMO
    Debugging tests from the IDE

    View Slide

  29. Feature limitations
    Support for features determined by version of
    Tooling API used to execute test.
    More info:
    https:/
    /docs.gradle.org/current/userguide/test_kit.html

    View Slide

  30. Future enhancements for TestKit
    - Convenience test fixtures
    - Hooking into IDE plugins
    - Integration with JaCoCo plugin
    More info:
    https://github.com/gradle/gradle/blob/master/
    design-docs/testing-toolkit.md

    View Slide

  31. Thank You!
    Please ask questions…
    https:/
    /www.github.com/bmuschko
    @bmuschko

    View Slide

  32. Introducing Gradle Cloud Services
    - Insights into your build
    - View and share via URL
    - Debug, optimize and refine
    - Analyze all of your builds
    Stop by the Gradle Lounge and create a Build Scan!
    The first service, Gradle Build Scans, is now available.

    View Slide

  33. Who Are We
    Motto: Build Happiness
    Mission: To revolutionize the way software is built and
    shipped. We exist to end once-and-for-all the worst things
    about big software and restore the reason you got into coding
    in the first place.
    We’re Hiring: Gradle is hiring front-end, back-end, and core
    software engineers. Visit gradle.org/jobs to apply

    View Slide

  34. Learn more at www.gradle.org

    View Slide