Pro Yearly is on sale from $80 to $50! »

Getting the most out of your next generation JVM testing framework

Getting the most out of your next generation JVM testing framework

With the rise of newer JVM languages like Groovy and Kotlin, feature-rich test frameworks emerged. The test framework Spock became a welcome alternative for many projects fearless to adopt Groovy as part of their polyglot software stack.

With the first GA release of JUnit 5 in September 2017, the JUnit team brought real innovation to the established space of testing JVM code. Not only is the release packed with new features comparable to the ones provided by Spock, JUnit 5 also serves as a platform for launching other test frameworks on the JVM.

In this demo-driven talk, we will compare typical testing usage patterns and features available for JUnit 5 and Spock. You'll leave with a firm understanding of the benefits and tradeoffs delivered by both test frameworks.

8f2248c6bfcc6df39a2cd8edf4267cb5?s=128

Benjamin Muschko

January 10, 2019
Tweet

Transcript

  1. Ge#ng the most out of your next genera2on JVM tes2ng

    framework JUnit 5 Spock Benjamin Muschko
  2. AUTOMATED ASCENT bmuschko bmuschko bmuschko.com About the speaker automatedascent.com

  3. Which test framework do you use?

  4. JUnit 5 in a nutshell Complete rewrite of JUnit Modular

    & extensible Forward & backward compa2ble
  5. Far more than a version upgrade

  6. JUnit Jupiter The programming and extension model

  7. Spock Framework BDD tes2ng framework

  8. Spock, short and sweet Tests are wriLen in Groovy Highly

    expressive, extensible JUnit 4-compa2ble with test runner
  9. given when then defines the key ac2on known state or

    data verifies expected outcome Specifica2on of a test
  10. Tests need to be expressive

  11. import org.junit.jupiter.api.DisplayName;
 import org.junit.jupiter.api.Test;
 
 import static org.junit.jupiter.api.Assertions.assertEquals;
 
 public

    class DescriptiveTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 @DisplayName("can add two numbers")
 void canAdd() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Descrip2ve tests
  12. Execu2ng tests in IntelliJ

  13. import spock.lang.Specification
 import spock.lang.Subject
 
 class DescriptiveTest extends Specification {


    @Subject def arithmeticOperation =↵ new ArithmeticOperation()
 
 def "can add two numbers"() {
 when:
 def result = arithmeticOperation.add(1, 2) then: result == 3 
 }
 } Descrip2ve tests
  14. Execu2ng tests in IntelliJ

  15. Tests may require preparation

  16. import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 
 public class FixtureSetupCleanup {
 private

    Path testFile;
 
 @BeforeEach
 void setup() throws IOException {
 testFile = Files.createTempFile("junit5", ".tmp");
 }
 
 @AfterEach
 void cleanup() {
 testFile.toFile().delete();
 }
 } Fixture set up and tear down
  17. import spock.lang.Specification
 import spock.lang.Subject
 
 class FixtureSetupCleanup extends Specification {


    def testFile
 
 def setup() {
 testFile = Files.createTempFile("junit5", ".tmp")
 }
 
 def cleanup() {
 testFile.toFile().delete()
 }
 } Fixture set up and tear down
  18. Putting tests to sleep

  19. import org.junit.jupiter.api.Disabled;
 
 public class IgnoredTest {
 private final ArithmeticOperation

    arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 @Disabled("for demonstration purposes")
 void canAdd() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Disabling tests
  20. import spock.lang.Ignore
 
 class IgnoredTest extends Specification {
 @Subject def

    arithmeticOperation =↵ new ArithmeticOperation()
 
 @Ignore("for demonstration purposes")
 def canAdd() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 } Disabling tests
  21. import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
 
 public class ConditionalExecutionTest {
 private final static

    String SYS_PROP_KEY = "junit5.test.enabled";
 private final static String SYS_PROP_TRUE_VALUE = "true";
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @EnabledIfSystemProperty(named = SYS_PROP_KEY,↵ matches = SYS_PROP_TRUE_VALUE)
 @Test
 void testOnlyOnSystemSystemProperty() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Built-in condi2onals
  22. import static org.junit.jupiter.api.Assumptions.assumeTrue;
 
 public class ConditionalExecutionTest {
 private final

    static String SYS_PROP_KEY = "junit5.test.enabled";
 private final static String SYS_PROP_TRUE_VALUE = "true";
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 void testOnlyOnSystemPropertySet() {
 assumeTrue(SYS_PROP_TRUE_VALUE↵ .equals(System.getProperty(SYS_PROP_KEY)));
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Alterna2ve condi2onal syntax
  23. import org.junit.jupiter.api.extension.*;
 
 import static org.junit.jupiter.api.extension.ConditionEvaluationResult.*;
 
 public static class

    SystemPropertyConditionalExtension implements ExecutionCondition {
 @Override
 public ConditionEvaluationResult evaluateExecutionCondition(ExtensionContext↵
 context) {
 String sysPropValue = System.getProperty(SYS_PROP_KEY);
 boolean enabled = SYS_PROP_TRUE_VALUE.equals(sysPropValue);
 
 if (enabled) {
 return enabled(String.format("System property '%s' evaluates to true",↵
 SYS_PROP_KEY));
 }
 
 return disabled(String.format("System property '%s' evaluates to false”,↵
 SYS_PROP_KEY));
 }
 } Custom condi2onals
  24. import org.junit.jupiter.api.extension.ExtendWith;
 
 public class ConditionalExecutionTest {
 private final ArithmeticOperation

    arithmeticOperation =↵ new ArithmeticOperation();
 
 @ExtendWith(SystemPropertyConditionalExtension.class)
 @Test
 void testOnlyOnSystemPropertySetByExtension() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Using custom condi2onals
  25. import spock.lang.Requires
 
 class ConditionalExecutionTest extends Specification {
 private final

    static String SYS_PROP_KEY = "spock.test.enabled"
 private final static String SYS_PROP_TRUE_VALUE = "true"
 @Subject def arithmeticOperation = new ArithmeticOperation()
 
 @Requires({ SYS_PROP_TRUE_VALUE == sys[SYS_PROP_KEY] })
 def "can add"() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 } Condi2onal test execu2on
  26. Expect things to misbehave

  27. import static org.junit.jupiter.api.Assertions.assertThrows;
 
 public class ExpectedExceptionTest {
 private final

    FileReader fileReader =↵ new DefaultFileReader();
 
 @Test
 void cannotReadNonExistentFile() {
 assertThrows(IOException.class, () -> {
 fileReader.readContent(Paths.get("hello.text"));
 });
 }
 } Expec2ng thrown excep2ons
  28. import spock.lang.FailsWith
 
 class ExpectedExceptionTest extends Specification {
 @Subject def

    fileReader = new DefaultFileReader()
 
 @FailsWith(IOException)
 def "throws exception if file contents cannot be read"() {
 expect:
 fileReader.readContent(Paths.get('hello.text'))
 }
 } Expec2ng thrown excep2ons
  29. class ExpectedExceptionTest extends Specification {
 @Subject def fileReader = new

    DefaultFileReader()
 
 def "throws exception if file contents cannot be read↵ and assert message"() {
 when:
 fileReader.readContent(Paths.get('hello.text'))
 
 then:
 def t = thrown(IOException)
 t.message == 'File does not exist'
 } } Expec2ng thrown excep2ons
  30. import org.junit.jupiter.api.RepeatedTest;
 
 public class RepetitionTest {
 private final ArithmeticOperation

    arithmeticOperation =↵ new ArithmeticOperation();
 
 @RepeatedTest(value = 5, name = "Iteration {currentRepetition}↵ of {totalRepetitions}")
 void canSubstract() {
 assertEquals(1, arithmeticOperation.substract(2, 1));
 }
 } Repea2ng test execu2on
  31. import spock.lang.Unroll class RepetitionTest extends Specification {
 @Subject def arithmeticOperation

    =↵ new ArithmeticOperation()
 @Unroll
 def "Iteration #i of 5"() {
 expect:
 arithmeticOperation.substract(2, 1) == 1
 
 where:
 i << (1..5)
 }
 } Repea2ng test execu2on
  32. import static java.time.Duration.ofSeconds;
 import static org.junit.jupiter.api.Assertions.assertTimeout;
 
 public class TimeoutTest

    {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 void canAdd() {
 assertTimeout(ofSeconds(2), () -> {
 assertEquals(3, arithmeticOperation.add(1, 2));
 });
 }
 } Declaring test execu2on 2meouts
  33. import spock.lang.Timeout
 
 class TimeoutTest extends Specification {
 @Subject def

    arithmeticOperation =↵ new ArithmeticOperation()
 
 @Timeout(2)
 def canAdd() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 } Declaring test execu2on 2meouts
  34. Feeding data to a scenario

  35. import org.junit.jupiter.params.*;
 import java.util.stream.Stream;
 
 public class DataDrivenTest {
 private

    final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @ParameterizedTest(name = "can add {0} to {1} and receive {2}")
 @MethodSource("additionProvider")
 void canAddAndAssertExactResult(int a, int b, int result) {
 assertEquals(result, arithmeticOperation.add(a, b));
 }
 
 static Stream<Arguments> additionProvider() {
 return Stream.of(
 Arguments.of(1, 3, 4),
 Arguments.of(3, 4, 7)
 );
 }
 } Data-driven tests
  36. Data-driven tests in IntelliJ

  37. import spock.lang.Unroll
 
 class DataDrivenTest extends Specification {
 @Subject def

    arithmeticOperation = new ArithmeticOperation()
 
 @Unroll
 def "can add #a to #b and receive #result"() {
 expect:
 arithmeticOperation.add(a, b) == result
 
 where:
 a | b | result
 1 | 3 | 4
 3 | 4 | 7
 }
 } Data-driven tests
  38. Data-driven tests in IntelliJ

  39. Testing in isolation

  40. import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.when;
 
 public class MockTest

    {
 ...
 
 @Test
 void canMockFileReadOperation() throws IOException {
 String text = "hello";
 FileReader fileReader = mock(FileReader.class);
 when(fileReader.readContent(testFile)).thenReturn(text);
 FileManager fileManager = new DefaultFileManager(fileReader);
 Files.write(testFile, text.getBytes());
 assertEquals(text, fileManager.readContent(testFile));
 }
 } Mocking dependencies
  41. class MockTest extends Specification {
 def fileReader = Mock(FileReader)
 


    def "can mock file read operation"() {
 given:
 def text = "hello"
 Files.write(testFile, text.getBytes())
 
 when:
 def content = fileManager.readContent(testFile)
 
 then:
 1 * fileReader.readContent(testFile) >> text
 content == text
 }
 } Mocking dependencies
  42. Categorized test execution

  43. import org.junit.jupiter.api.Tag;
 
 public class TaggedTest {
 private final ArithmeticOperation

    arithmeticOperation =↵ new ArithmeticOperation();
 
 @Tag("slow")
 @Test
 void runsSlowly() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 
 @Tag("fast")
 @Test
 void runsFast() {
 assertEquals(1, arithmeticOperation.substract(2, 1));
 }
 } Labeling and filtering
  44. import org.junit.platform.runner.JUnitPlatform;
 import org.junit.platform.suite.api.IncludeTags;
 import org.junit.platform.suite.api.SelectPackages;
 import org.junit.runner.RunWith;
 
 @RunWith(JUnitPlatform.class)


    @SelectPackages("com.bmuschko.test.comparison.junit5.tagged")
 @IncludeTags("fast")
 public class FilteredFastTest {
 } Labeling and filtering
  45. class TaggedTest extends Specification {
 @Subject def arithmeticOperation = new

    ArithmeticOperation()
 
 @Slow
 def "runs slowly"() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 
 @Fast
 def "runs fast"() {
 expect:
 arithmeticOperation.substract(2, 1) == 1
 }
 } Labeling and filtering
  46. @Retention(RetentionPolicy.RUNTIME)
 @Target([ElementType.TYPE, ElementType.METHOD])
 @interface Slow {
 }
 
 @Retention(RetentionPolicy.RUNTIME)
 @Target([ElementType.TYPE,

    ElementType.METHOD])
 @interface Fast {
 }
 Labeling and filtering
  47. import com.bmuschko.test.comparison.spock.tagged.Fast
 
 runner {
 include Fast
 } Labeling and

    filtering SpockConfig.groovy
  48. Extending the test framework

  49. Third-party extensions JUnit Pioneer junit-pioneer.org Test engines, Docker, Mockito, WireMock

    and more… github.com/junit-team/junit5/wiki/Third-party-Extensions
  50. Consuming an extension import org.junit.jupiter.api.extension.ExtendWith;
 
 @ExtendWith(BeforeAfterLoggingExtension.class)
 public class LoggingTest

    {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 void canAdd() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 }
  51. Wri2ng a simple extension import org.junit.jupiter.api.extension.*;
 
 public class BeforeAfterLoggingExtension

    implements↵ BeforeTestExecutionCallback, AfterTestExecutionCallback {
 @Override
 public void beforeTestExecution(ExtensionContext context) {
 Method testMethod = context.getRequiredTestMethod();
 System.out.println(String.format("Starting test method %s.%s”,↵ testMethod.getDeclaringClass(), testMethod.getName()));
 }
 
 @Override
 public void afterTestExecution(ExtensionContext context) {
 Method testMethod = context.getRequiredTestMethod();
 System.out.println(String.format("Finishing test method %s.%s”,↵ testMethod.getDeclaringClass(), testMethod.getName()));
 }
 }
  52. Third-party extensions very liLle out there…

  53. Consuming an extension @BeforeAfterLogging
 class LoggingTest extends Specification {
 @Subject

    def arithmeticOperation = new ArithmeticOperation()
 
 def "can add"() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 }
  54. Wri2ng a simple extension import org.spockframework.runtime.extension.AbstractAnnotationDrivenExtension
 import org.spockframework.runtime.model.SpecInfo
 
 class

    BeforeAfterLoggingExtension extends↵ AbstractAnnotationDrivenExtension<BeforeAfterLogging> {
 @Override
 void visitSpecAnnotation(BeforeAfterLogging annotation, SpecInfo spec) {
 spec.addListener(new BeforeAfterEventListener())
 }
 } import org.spockframework.runtime.extension.ExtensionAnnotation
 
 @Retention(RetentionPolicy.RUNTIME)
 @Target(ElementType.TYPE)
 @ExtensionAnnotation(BeforeAfterLoggingExtension.class)
 @interface BeforeAfterLogging {
 }
  55. “Which framework is beLer?”

  56. Accommodate your team’s skill set

  57. Tests as readable user stories

  58. Which features are crucial to you?

  59. Resources junit.org/junit5 spockframework.org bmuschko.com/blog/junit5-vs-spock-showdown github.com/bmuschko/junit5-vs-spock-feature-comparison

  60. Thanks! Please ask ques2ons.