Getting the most out of your next generation JVM testing framework

With the rise of newer JVM languages like Groovy and Kotlin, feature-rich test frameworks emerged. The test framework Spock became a welcome alternative for many projects fearless to adopt Groovy as part of their polyglot software stack.

With the first GA release of JUnit 5 in September 2017, the JUnit team brought real innovation to the established space of testing JVM code. Not only is the release packed with new features comparable to the ones provided by Spock, JUnit 5 also serves as a platform for launching other test frameworks on the JVM.

In this demo-driven talk, we will compare typical testing usage patterns and features available for JUnit 5 and Spock. You'll leave with a firm understanding of the benefits and tradeoffs delivered by both test frameworks.

Transcript

1. Ge#ng the most out of your next genera2on JVM tes2ng framework JUnit 5 Spock Benjamin Muschko
2. AUTOMATED ASCENT bmuschko bmuschko bmuschko.com About the speaker automatedascent.com
3. Which test framework do you use?
4. JUnit 5 in a nutshell Complete rewrite of JUnit Modular & extensible Forward & backward compa2ble
5. Far more than a version upgrade
6. JUnit Jupiter The programming and extension model
7. Spock Framework BDD tes2ng framework
8. Spock, short and sweet Tests are wriLen in Groovy Highly expressive, extensible JUnit 4-compa2ble with test runner
9. given when then defines the key ac2on known state or data verifies expected outcome Specifica2on of a test
10. Tests need to be expressive
11. import org.junit.jupiter.api.DisplayName;
 import org.junit.jupiter.api.Test;
 
 import static org.junit.jupiter.api.Assertions.assertEquals;
 
 public class DescriptiveTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 @DisplayName("can add two numbers")
 void canAdd() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Descrip2ve tests
12. Execu2ng tests in IntelliJ
13. import spock.lang.Specification
 import spock.lang.Subject
 
 class DescriptiveTest extends Specification {
 @Subject def arithmeticOperation =↵ new ArithmeticOperation()
 
 def "can add two numbers"() {
 when:
 def result = arithmeticOperation.add(1, 2) then: result == 3 
 }
 } Descrip2ve tests
14. Execu2ng tests in IntelliJ
15. Tests may require preparation
16. import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeEach;
 
 public class FixtureSetupCleanup {
 private Path testFile;
 
 @BeforeEach
 void setup() throws IOException {
 testFile = Files.createTempFile("junit5", ".tmp");
 }
 
 @AfterEach
 void cleanup() {
 testFile.toFile().delete();
 }
 } Fixture set up and tear down
17. import spock.lang.Specification
 import spock.lang.Subject
 
 class FixtureSetupCleanup extends Specification {
 def testFile
 
 def setup() {
 testFile = Files.createTempFile("junit5", ".tmp")
 }
 
 def cleanup() {
 testFile.toFile().delete()
 }
 } Fixture set up and tear down
18. Putting tests to sleep
19. import org.junit.jupiter.api.Disabled;
 
 public class IgnoredTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 @Disabled("for demonstration purposes")
 void canAdd() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Disabling tests
20. import spock.lang.Ignore
 
 class IgnoredTest extends Specification {
 @Subject def arithmeticOperation =↵ new ArithmeticOperation()
 
 @Ignore("for demonstration purposes")
 def canAdd() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 } Disabling tests
21. import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
 
 public class ConditionalExecutionTest {
 private final static String SYS_PROP_KEY = "junit5.test.enabled";
 private final static String SYS_PROP_TRUE_VALUE = "true";
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @EnabledIfSystemProperty(named = SYS_PROP_KEY,↵ matches = SYS_PROP_TRUE_VALUE)
 @Test
 void testOnlyOnSystemSystemProperty() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Built-in condi2onals
22. import static org.junit.jupiter.api.Assumptions.assumeTrue;
 
 public class ConditionalExecutionTest {
 private final static String SYS_PROP_KEY = "junit5.test.enabled";
 private final static String SYS_PROP_TRUE_VALUE = "true";
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 void testOnlyOnSystemPropertySet() {
 assumeTrue(SYS_PROP_TRUE_VALUE↵ .equals(System.getProperty(SYS_PROP_KEY)));
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Alterna2ve condi2onal syntax
23. import org.junit.jupiter.api.extension.*;
 
 import static org.junit.jupiter.api.extension.ConditionEvaluationResult.*;
 
 public static class SystemPropertyConditionalExtension implements ExecutionCondition {
 @Override
 public ConditionEvaluationResult evaluateExecutionCondition(ExtensionContext↵
 context) {
 String sysPropValue = System.getProperty(SYS_PROP_KEY);
 boolean enabled = SYS_PROP_TRUE_VALUE.equals(sysPropValue);
 
 if (enabled) {
 return enabled(String.format("System property '%s' evaluates to true",↵
 SYS_PROP_KEY));
 }
 
 return disabled(String.format("System property '%s' evaluates to false”,↵
 SYS_PROP_KEY));
 }
 } Custom condi2onals
24. import org.junit.jupiter.api.extension.ExtendWith;
 
 public class ConditionalExecutionTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @ExtendWith(SystemPropertyConditionalExtension.class)
 @Test
 void testOnlyOnSystemPropertySetByExtension() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 } Using custom condi2onals
25. import spock.lang.Requires
 
 class ConditionalExecutionTest extends Specification {
 private final static String SYS_PROP_KEY = "spock.test.enabled"
 private final static String SYS_PROP_TRUE_VALUE = "true"
 @Subject def arithmeticOperation = new ArithmeticOperation()
 
 @Requires({ SYS_PROP_TRUE_VALUE == sys[SYS_PROP_KEY] })
 def "can add"() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 } Condi2onal test execu2on
26. Expect things to misbehave
27. import static org.junit.jupiter.api.Assertions.assertThrows;
 
 public class ExpectedExceptionTest {
 private final FileReader fileReader =↵ new DefaultFileReader();
 
 @Test
 void cannotReadNonExistentFile() {
 assertThrows(IOException.class, () -> {
 fileReader.readContent(Paths.get("hello.text"));
 });
 }
 } Expec2ng thrown excep2ons
28. import spock.lang.FailsWith
 
 class ExpectedExceptionTest extends Specification {
 @Subject def fileReader = new DefaultFileReader()
 
 @FailsWith(IOException)
 def "throws exception if file contents cannot be read"() {
 expect:
 fileReader.readContent(Paths.get('hello.text'))
 }
 } Expec2ng thrown excep2ons
29. class ExpectedExceptionTest extends Specification {
 @Subject def fileReader = new DefaultFileReader()
 
 def "throws exception if file contents cannot be read↵ and assert message"() {
 when:
 fileReader.readContent(Paths.get('hello.text'))
 
 then:
 def t = thrown(IOException)
 t.message == 'File does not exist'
 } } Expec2ng thrown excep2ons
30. import org.junit.jupiter.api.RepeatedTest;
 
 public class RepetitionTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @RepeatedTest(value = 5, name = "Iteration {currentRepetition}↵ of {totalRepetitions}")
 void canSubstract() {
 assertEquals(1, arithmeticOperation.substract(2, 1));
 }
 } Repea2ng test execu2on
31. import spock.lang.Unroll class RepetitionTest extends Specification {
 @Subject def arithmeticOperation =↵ new ArithmeticOperation()
 @Unroll
 def "Iteration #i of 5"() {
 expect:
 arithmeticOperation.substract(2, 1) == 1
 
 where:
 i << (1..5)
 }
 } Repea2ng test execu2on
32. import static java.time.Duration.ofSeconds;
 import static org.junit.jupiter.api.Assertions.assertTimeout;
 
 public class TimeoutTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 void canAdd() {
 assertTimeout(ofSeconds(2), () -> {
 assertEquals(3, arithmeticOperation.add(1, 2));
 });
 }
 } Declaring test execu2on 2meouts
33. import spock.lang.Timeout
 
 class TimeoutTest extends Specification {
 @Subject def arithmeticOperation =↵ new ArithmeticOperation()
 
 @Timeout(2)
 def canAdd() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 } Declaring test execu2on 2meouts
34. Feeding data to a scenario
35. import org.junit.jupiter.params.*;
 import java.util.stream.Stream;
 
 public class DataDrivenTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @ParameterizedTest(name = "can add {0} to {1} and receive {2}")
 @MethodSource("additionProvider")
 void canAddAndAssertExactResult(int a, int b, int result) {
 assertEquals(result, arithmeticOperation.add(a, b));
 }
 
 static Stream<Arguments> additionProvider() {
 return Stream.of(
 Arguments.of(1, 3, 4),
 Arguments.of(3, 4, 7)
 );
 }
 } Data-driven tests
36. Data-driven tests in IntelliJ
37. import spock.lang.Unroll
 
 class DataDrivenTest extends Specification {
 @Subject def arithmeticOperation = new ArithmeticOperation()
 
 @Unroll
 def "can add #a to #b and receive #result"() {
 expect:
 arithmeticOperation.add(a, b) == result
 
 where:
 a | b | result
 1 | 3 | 4
 3 | 4 | 7
 }
 } Data-driven tests
38. Data-driven tests in IntelliJ
39. Testing in isolation
40. import static org.mockito.Mockito.mock;
 import static org.mockito.Mockito.when;
 
 public class MockTest {
 ...
 
 @Test
 void canMockFileReadOperation() throws IOException {
 String text = "hello";
 FileReader fileReader = mock(FileReader.class);
 when(fileReader.readContent(testFile)).thenReturn(text);
 FileManager fileManager = new DefaultFileManager(fileReader);
 Files.write(testFile, text.getBytes());
 assertEquals(text, fileManager.readContent(testFile));
 }
 } Mocking dependencies
41. class MockTest extends Specification {
 def fileReader = Mock(FileReader)
 
 def "can mock file read operation"() {
 given:
 def text = "hello"
 Files.write(testFile, text.getBytes())
 
 when:
 def content = fileManager.readContent(testFile)
 
 then:
 1 * fileReader.readContent(testFile) >> text
 content == text
 }
 } Mocking dependencies
42. Categorized test execution
43. import org.junit.jupiter.api.Tag;
 
 public class TaggedTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Tag("slow")
 @Test
 void runsSlowly() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 
 @Tag("fast")
 @Test
 void runsFast() {
 assertEquals(1, arithmeticOperation.substract(2, 1));
 }
 } Labeling and filtering
44. import org.junit.platform.runner.JUnitPlatform;
 import org.junit.platform.suite.api.IncludeTags;
 import org.junit.platform.suite.api.SelectPackages;
 import org.junit.runner.RunWith;
 
 @RunWith(JUnitPlatform.class)
 @SelectPackages("com.bmuschko.test.comparison.junit5.tagged")
 @IncludeTags("fast")
 public class FilteredFastTest {
 } Labeling and filtering
45. class TaggedTest extends Specification {
 @Subject def arithmeticOperation = new ArithmeticOperation()
 
 @Slow
 def "runs slowly"() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 
 @Fast
 def "runs fast"() {
 expect:
 arithmeticOperation.substract(2, 1) == 1
 }
 } Labeling and filtering
46. @Retention(RetentionPolicy.RUNTIME)
 @Target([ElementType.TYPE, ElementType.METHOD])
 @interface Slow {
 }
 
 @Retention(RetentionPolicy.RUNTIME)
 @Target([ElementType.TYPE, ElementType.METHOD])
 @interface Fast {
 }
 Labeling and filtering
47. import com.bmuschko.test.comparison.spock.tagged.Fast
 
 runner {
 include Fast
 } Labeling and filtering SpockConfig.groovy
48. Extending the test framework
49. Third-party extensions JUnit Pioneer junit-pioneer.org Test engines, Docker, Mockito, WireMock and more… github.com/junit-team/junit5/wiki/Third-party-Extensions
50. Consuming an extension import org.junit.jupiter.api.extension.ExtendWith;
 
 @ExtendWith(BeforeAfterLoggingExtension.class)
 public class LoggingTest {
 private final ArithmeticOperation arithmeticOperation =↵ new ArithmeticOperation();
 
 @Test
 void canAdd() {
 assertEquals(3, arithmeticOperation.add(1, 2));
 }
 }
51. Wri2ng a simple extension import org.junit.jupiter.api.extension.*;
 
 public class BeforeAfterLoggingExtension implements↵ BeforeTestExecutionCallback, AfterTestExecutionCallback {
 @Override
 public void beforeTestExecution(ExtensionContext context) {
 Method testMethod = context.getRequiredTestMethod();
 System.out.println(String.format("Starting test method %s.%s”,↵ testMethod.getDeclaringClass(), testMethod.getName()));
 }
 
 @Override
 public void afterTestExecution(ExtensionContext context) {
 Method testMethod = context.getRequiredTestMethod();
 System.out.println(String.format("Finishing test method %s.%s”,↵ testMethod.getDeclaringClass(), testMethod.getName()));
 }
 }
52. Third-party extensions very liLle out there…
53. Consuming an extension @BeforeAfterLogging
 class LoggingTest extends Specification {
 @Subject def arithmeticOperation = new ArithmeticOperation()
 
 def "can add"() {
 expect:
 arithmeticOperation.add(1, 2) == 3
 }
 }
54. Wri2ng a simple extension import org.spockframework.runtime.extension.AbstractAnnotationDrivenExtension
 import org.spockframework.runtime.model.SpecInfo
 
 class BeforeAfterLoggingExtension extends↵ AbstractAnnotationDrivenExtension<BeforeAfterLogging> {
 @Override
 void visitSpecAnnotation(BeforeAfterLogging annotation, SpecInfo spec) {
 spec.addListener(new BeforeAfterEventListener())
 }
 } import org.spockframework.runtime.extension.ExtensionAnnotation
 
 @Retention(RetentionPolicy.RUNTIME)
 @Target(ElementType.TYPE)
 @ExtensionAnnotation(BeforeAfterLoggingExtension.class)
 @interface BeforeAfterLogging {
 }
55. “Which framework is beLer?”
56. Accommodate your team’s skill set
57. Tests as readable user stories
58. Which features are crucial to you?
59. Resources junit.org/junit5 spockframework.org bmuschko.com/blog/junit5-vs-spock-showdown github.com/bmuschko/junit5-vs-spock-feature-comparison
60. Thanks! Please ask ques2ons.