$30 off During Our Annual Pro Sale. View Details »

Getting the most out of your next generation JVM testing framework

Getting the most out of your next generation JVM testing framework

With the rise of newer JVM languages like Groovy and Kotlin, feature-rich test frameworks emerged. The test framework Spock became a welcome alternative for many projects fearless to adopt Groovy as part of their polyglot software stack.

With the first GA release of JUnit 5 in September 2017, the JUnit team brought real innovation to the established space of testing JVM code. Not only is the release packed with new features comparable to the ones provided by Spock, JUnit 5 also serves as a platform for launching other test frameworks on the JVM.

In this demo-driven talk, we will compare typical testing usage patterns and features available for JUnit 5 and Spock. You'll leave with a firm understanding of the benefits and tradeoffs delivered by both test frameworks.

Benjamin Muschko

January 10, 2019
Tweet

More Decks by Benjamin Muschko

Other Decks in Programming

Transcript

  1. Ge#ng the most out of your
    next genera2on JVM tes2ng framework
    JUnit 5 Spock
    Benjamin Muschko

    View Slide

  2. AUTOMATED
    ASCENT
    bmuschko
    bmuschko
    bmuschko.com
    About the speaker
    automatedascent.com

    View Slide

  3. Which test
    framework do you
    use?

    View Slide

  4. JUnit 5 in a nutshell
    Complete rewrite of JUnit
    Modular & extensible
    Forward & backward compa2ble

    View Slide

  5. Far more than a version upgrade

    View Slide

  6. JUnit Jupiter
    The programming
    and extension
    model

    View Slide

  7. Spock Framework
    BDD tes2ng
    framework

    View Slide

  8. Spock, short and sweet
    Tests are wriLen in Groovy
    Highly expressive, extensible
    JUnit 4-compa2ble with test runner

    View Slide

  9. given
    when
    then
    defines the key ac2on
    known state or data
    verifies expected outcome
    Specifica2on of a test

    View Slide

  10. Tests need to
    be expressive

    View Slide

  11. import org.junit.jupiter.api.DisplayName;

    import org.junit.jupiter.api.Test;


    import static org.junit.jupiter.api.Assertions.assertEquals;


    public class DescriptiveTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @Test

    @DisplayName("can add two numbers")

    void canAdd() {

    assertEquals(3, arithmeticOperation.add(1, 2));

    }

    }
    Descrip2ve tests

    View Slide

  12. Execu2ng tests in IntelliJ

    View Slide

  13. import spock.lang.Specification

    import spock.lang.Subject


    class DescriptiveTest extends Specification {

    @Subject def arithmeticOperation =↵
    new ArithmeticOperation()


    def "can add two numbers"() {

    when:

    def result = arithmeticOperation.add(1, 2)
    then:
    result == 3 

    }

    }
    Descrip2ve tests

    View Slide

  14. Execu2ng tests in IntelliJ

    View Slide

  15. Tests
    may require
    preparation

    View Slide

  16. import org.junit.jupiter.api.AfterEach;

    import org.junit.jupiter.api.BeforeEach;


    public class FixtureSetupCleanup {

    private Path testFile;


    @BeforeEach

    void setup() throws IOException {

    testFile = Files.createTempFile("junit5", ".tmp");

    }


    @AfterEach

    void cleanup() {

    testFile.toFile().delete();

    }

    }
    Fixture set up and tear down

    View Slide

  17. import spock.lang.Specification

    import spock.lang.Subject


    class FixtureSetupCleanup extends Specification {

    def testFile


    def setup() {

    testFile = Files.createTempFile("junit5", ".tmp")

    }


    def cleanup() {

    testFile.toFile().delete()

    }

    }
    Fixture set up and tear down

    View Slide

  18. Putting tests
    to sleep

    View Slide

  19. import org.junit.jupiter.api.Disabled;


    public class IgnoredTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @Test

    @Disabled("for demonstration purposes")

    void canAdd() {

    assertEquals(3, arithmeticOperation.add(1, 2));

    }

    }
    Disabling tests

    View Slide

  20. import spock.lang.Ignore


    class IgnoredTest extends Specification {

    @Subject def arithmeticOperation =↵
    new ArithmeticOperation()


    @Ignore("for demonstration purposes")

    def canAdd() {

    expect:

    arithmeticOperation.add(1, 2) == 3

    }

    }
    Disabling tests

    View Slide

  21. import org.junit.jupiter.api.condition.EnabledIfSystemProperty;


    public class ConditionalExecutionTest {

    private final static String SYS_PROP_KEY = "junit5.test.enabled";

    private final static String SYS_PROP_TRUE_VALUE = "true";

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @EnabledIfSystemProperty(named = SYS_PROP_KEY,↵
    matches = SYS_PROP_TRUE_VALUE)

    @Test

    void testOnlyOnSystemSystemProperty() {

    assertEquals(3, arithmeticOperation.add(1, 2));

    }

    }
    Built-in condi2onals

    View Slide

  22. import static org.junit.jupiter.api.Assumptions.assumeTrue;


    public class ConditionalExecutionTest {

    private final static String SYS_PROP_KEY = "junit5.test.enabled";

    private final static String SYS_PROP_TRUE_VALUE = "true";

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @Test

    void testOnlyOnSystemPropertySet() {

    assumeTrue(SYS_PROP_TRUE_VALUE↵
    .equals(System.getProperty(SYS_PROP_KEY)));

    assertEquals(3, arithmeticOperation.add(1, 2));

    }

    }
    Alterna2ve condi2onal syntax

    View Slide

  23. import org.junit.jupiter.api.extension.*;


    import static org.junit.jupiter.api.extension.ConditionEvaluationResult.*;


    public static class SystemPropertyConditionalExtension implements ExecutionCondition {

    @Override

    public ConditionEvaluationResult evaluateExecutionCondition(ExtensionContext↵

    context) {

    String sysPropValue = System.getProperty(SYS_PROP_KEY);

    boolean enabled = SYS_PROP_TRUE_VALUE.equals(sysPropValue);


    if (enabled) {

    return enabled(String.format("System property '%s' evaluates to true",↵

    SYS_PROP_KEY));

    }


    return disabled(String.format("System property '%s' evaluates to false”,↵

    SYS_PROP_KEY));

    }

    }
    Custom condi2onals

    View Slide

  24. import org.junit.jupiter.api.extension.ExtendWith;


    public class ConditionalExecutionTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @ExtendWith(SystemPropertyConditionalExtension.class)

    @Test

    void testOnlyOnSystemPropertySetByExtension() {

    assertEquals(3, arithmeticOperation.add(1, 2));

    }

    }
    Using custom condi2onals

    View Slide

  25. import spock.lang.Requires


    class ConditionalExecutionTest extends Specification {

    private final static String SYS_PROP_KEY = "spock.test.enabled"

    private final static String SYS_PROP_TRUE_VALUE = "true"

    @Subject def arithmeticOperation = new ArithmeticOperation()


    @Requires({ SYS_PROP_TRUE_VALUE == sys[SYS_PROP_KEY] })

    def "can add"() {

    expect:

    arithmeticOperation.add(1, 2) == 3

    }

    }
    Condi2onal test execu2on

    View Slide

  26. Expect
    things to
    misbehave

    View Slide

  27. import static org.junit.jupiter.api.Assertions.assertThrows;


    public class ExpectedExceptionTest {

    private final FileReader fileReader =↵
    new DefaultFileReader();


    @Test

    void cannotReadNonExistentFile() {

    assertThrows(IOException.class, () -> {

    fileReader.readContent(Paths.get("hello.text"));

    });

    }

    }
    Expec2ng thrown excep2ons

    View Slide

  28. import spock.lang.FailsWith


    class ExpectedExceptionTest extends Specification {

    @Subject def fileReader = new DefaultFileReader()


    @FailsWith(IOException)

    def "throws exception if file contents cannot be read"() {

    expect:

    fileReader.readContent(Paths.get('hello.text'))

    }

    }
    Expec2ng thrown excep2ons

    View Slide

  29. class ExpectedExceptionTest extends Specification {

    @Subject def fileReader = new DefaultFileReader()


    def "throws exception if file contents cannot be read↵
    and assert message"() {

    when:

    fileReader.readContent(Paths.get('hello.text'))


    then:

    def t = thrown(IOException)

    t.message == 'File does not exist'

    }
    }
    Expec2ng thrown excep2ons

    View Slide

  30. import org.junit.jupiter.api.RepeatedTest;


    public class RepetitionTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @RepeatedTest(value = 5, name = "Iteration {currentRepetition}↵
    of {totalRepetitions}")

    void canSubstract() {

    assertEquals(1, arithmeticOperation.substract(2, 1));

    }

    }
    Repea2ng test execu2on

    View Slide

  31. import spock.lang.Unroll
    class RepetitionTest extends Specification {

    @Subject def arithmeticOperation =↵
    new ArithmeticOperation()

    @Unroll

    def "Iteration #i of 5"() {

    expect:

    arithmeticOperation.substract(2, 1) == 1


    where:

    i << (1..5)

    }

    }
    Repea2ng test execu2on

    View Slide

  32. import static java.time.Duration.ofSeconds;

    import static org.junit.jupiter.api.Assertions.assertTimeout;


    public class TimeoutTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @Test

    void canAdd() {

    assertTimeout(ofSeconds(2), () -> {

    assertEquals(3, arithmeticOperation.add(1, 2));

    });

    }

    }
    Declaring test execu2on 2meouts

    View Slide

  33. import spock.lang.Timeout


    class TimeoutTest extends Specification {

    @Subject def arithmeticOperation =↵
    new ArithmeticOperation()


    @Timeout(2)

    def canAdd() {

    expect:

    arithmeticOperation.add(1, 2) == 3

    }

    }
    Declaring test execu2on 2meouts

    View Slide

  34. Feeding
    data to a
    scenario

    View Slide

  35. import org.junit.jupiter.params.*;

    import java.util.stream.Stream;


    public class DataDrivenTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @ParameterizedTest(name = "can add {0} to {1} and receive {2}")

    @MethodSource("additionProvider")

    void canAddAndAssertExactResult(int a, int b, int result) {

    assertEquals(result, arithmeticOperation.add(a, b));

    }


    static Stream additionProvider() {

    return Stream.of(

    Arguments.of(1, 3, 4),

    Arguments.of(3, 4, 7)

    );

    }

    }
    Data-driven tests

    View Slide

  36. Data-driven tests in IntelliJ

    View Slide

  37. import spock.lang.Unroll


    class DataDrivenTest extends Specification {

    @Subject def arithmeticOperation = new ArithmeticOperation()


    @Unroll

    def "can add #a to #b and receive #result"() {

    expect:

    arithmeticOperation.add(a, b) == result


    where:

    a | b | result

    1 | 3 | 4

    3 | 4 | 7

    }

    }
    Data-driven tests

    View Slide

  38. Data-driven tests in IntelliJ

    View Slide

  39. Testing
    in isolation

    View Slide

  40. import static org.mockito.Mockito.mock;

    import static org.mockito.Mockito.when;


    public class MockTest {

    ...


    @Test

    void canMockFileReadOperation() throws IOException {

    String text = "hello";

    FileReader fileReader = mock(FileReader.class);

    when(fileReader.readContent(testFile)).thenReturn(text);

    FileManager fileManager = new DefaultFileManager(fileReader);

    Files.write(testFile, text.getBytes());

    assertEquals(text, fileManager.readContent(testFile));

    }

    }
    Mocking dependencies

    View Slide

  41. class MockTest extends Specification {

    def fileReader = Mock(FileReader)


    def "can mock file read operation"() {

    given:

    def text = "hello"

    Files.write(testFile, text.getBytes())


    when:

    def content = fileManager.readContent(testFile)


    then:

    1 * fileReader.readContent(testFile) >> text

    content == text

    }

    }
    Mocking dependencies

    View Slide

  42. Categorized
    test execution

    View Slide

  43. import org.junit.jupiter.api.Tag;


    public class TaggedTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @Tag("slow")

    @Test

    void runsSlowly() {

    assertEquals(3, arithmeticOperation.add(1, 2));

    }


    @Tag("fast")

    @Test

    void runsFast() {

    assertEquals(1, arithmeticOperation.substract(2, 1));

    }

    }
    Labeling and filtering

    View Slide

  44. import org.junit.platform.runner.JUnitPlatform;

    import org.junit.platform.suite.api.IncludeTags;

    import org.junit.platform.suite.api.SelectPackages;

    import org.junit.runner.RunWith;


    @RunWith(JUnitPlatform.class)

    @SelectPackages("com.bmuschko.test.comparison.junit5.tagged")

    @IncludeTags("fast")

    public class FilteredFastTest {

    }
    Labeling and filtering

    View Slide

  45. class TaggedTest extends Specification {

    @Subject def arithmeticOperation = new ArithmeticOperation()


    @Slow

    def "runs slowly"() {

    expect:

    arithmeticOperation.add(1, 2) == 3

    }


    @Fast

    def "runs fast"() {

    expect:

    arithmeticOperation.substract(2, 1) == 1

    }

    }
    Labeling and filtering

    View Slide

  46. @Retention(RetentionPolicy.RUNTIME)

    @Target([ElementType.TYPE, ElementType.METHOD])

    @interface Slow {

    }


    @Retention(RetentionPolicy.RUNTIME)

    @Target([ElementType.TYPE, ElementType.METHOD])

    @interface Fast {

    }

    Labeling and filtering

    View Slide

  47. import com.bmuschko.test.comparison.spock.tagged.Fast


    runner {

    include Fast

    }
    Labeling and filtering
    SpockConfig.groovy

    View Slide

  48. Extending the test framework

    View Slide

  49. Third-party extensions
    JUnit Pioneer
    junit-pioneer.org
    Test engines, Docker, Mockito, WireMock and more…
    github.com/junit-team/junit5/wiki/Third-party-Extensions

    View Slide

  50. Consuming an extension
    import org.junit.jupiter.api.extension.ExtendWith;


    @ExtendWith(BeforeAfterLoggingExtension.class)

    public class LoggingTest {

    private final ArithmeticOperation arithmeticOperation =↵
    new ArithmeticOperation();


    @Test

    void canAdd() {

    assertEquals(3, arithmeticOperation.add(1, 2));

    }

    }

    View Slide

  51. Wri2ng a simple extension
    import org.junit.jupiter.api.extension.*;


    public class BeforeAfterLoggingExtension implements↵
    BeforeTestExecutionCallback, AfterTestExecutionCallback {

    @Override

    public void beforeTestExecution(ExtensionContext context) {

    Method testMethod = context.getRequiredTestMethod();

    System.out.println(String.format("Starting test method %s.%s”,↵
    testMethod.getDeclaringClass(), testMethod.getName()));

    }


    @Override

    public void afterTestExecution(ExtensionContext context) {

    Method testMethod = context.getRequiredTestMethod();

    System.out.println(String.format("Finishing test method %s.%s”,↵
    testMethod.getDeclaringClass(), testMethod.getName()));

    }

    }

    View Slide

  52. Third-party extensions
    very liLle out there…

    View Slide

  53. Consuming an extension
    @BeforeAfterLogging

    class LoggingTest extends Specification {

    @Subject def arithmeticOperation = new ArithmeticOperation()


    def "can add"() {

    expect:

    arithmeticOperation.add(1, 2) == 3

    }

    }

    View Slide

  54. Wri2ng a simple extension
    import org.spockframework.runtime.extension.AbstractAnnotationDrivenExtension

    import org.spockframework.runtime.model.SpecInfo


    class BeforeAfterLoggingExtension extends↵
    AbstractAnnotationDrivenExtension {

    @Override

    void visitSpecAnnotation(BeforeAfterLogging annotation, SpecInfo spec) {

    spec.addListener(new BeforeAfterEventListener())

    }

    }
    import org.spockframework.runtime.extension.ExtensionAnnotation


    @Retention(RetentionPolicy.RUNTIME)

    @Target(ElementType.TYPE)

    @ExtensionAnnotation(BeforeAfterLoggingExtension.class)

    @interface BeforeAfterLogging {

    }

    View Slide

  55. “Which framework is beLer?”

    View Slide

  56. Accommodate your team’s skill set

    View Slide

  57. Tests as readable user stories

    View Slide

  58. Which features are crucial to you?

    View Slide

  59. Resources
    junit.org/junit5
    spockframework.org
    bmuschko.com/blog/junit5-vs-spock-showdown
    github.com/bmuschko/junit5-vs-spock-feature-comparison

    View Slide

  60. Thanks!
    Please ask ques2ons.

    View Slide