Spock vs JUnit 5 - Clash of the Titans

Slide 1

Slide 1 text

Spock vs JUnit 5 - Clash of the Titans Spock vs JUnit 5 - Clash of the Titans Marcin Zajączkowski Madrid, 29 March 2019 @SolidSoftBlog https://blog.solidsoft.info/

Slide 2

Slide 2 text

About me Areas of expertise Automatic Testing / TDD Software Craftsmanship / Code Quality Deployment Automation / Continuous Delivery Concurrency / Parallel Computing / Reactive Systems . FOSS projects author and contributor blogger trainer Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 3

Slide 3 text

Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 4

Slide 4 text

Shameless personal plug - Quo vadis? Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 5

Slide 5 text

Shameless personal plug - Quo vadis? Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 6

Slide 6 text

Personal plug - I'm looking I'm looking for interesting paths to follow game changing tools creation a product/solution for developers - FOSS projects are welcome :-) internally within an organization for other teams challenging & trailblazing projects where my experience and technical skills could be adequately leveraged Preferred areas: automatic testing and code quality reactive systems / concurrency / high performance Continuous Delivery & deployment automation something innovative & challenging Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Presentation goal Presentation goal Help you decide what is the best testing framework for Java code Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 9

Slide 9 text

Presentation plan historical outline selected JUnit 5 & Spock features comparison summary Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 10

Slide 10 text

Version statement Version statement This comparison is valid for JUnit 5.4.1 and Spock 1.3 Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 11

Slide 11 text

Historical outline Historical outline Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 12

Slide 12 text

Unit testing in Java - historical outline JUnit - ﬁrst xUnit for Java - 2000 Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 13

Slide 13 text

Unit testing in Java - historical outline JUnit - ﬁrst xUnit for Java - 2000 TestNG - Java 5 leveraged in tests - 2004 with some unique (at the time) features Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 14

Slide 14 text

Unit testing in Java - historical outline JUnit - ﬁrst xUnit for Java - 2000 TestNG - Java 5 leveraged in tests - 2004 with some unique (at the time) features JUnit 4 - Java 5 support - 2006 catching up TestNG features over years de facto standard, steady evolution rather than revolution Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 15

Slide 15 text

Unit testing in Java - historical outline JUnit - ﬁrst xUnit for Java - 2000 TestNG - Java 5 leveraged in tests - 2004 with some unique (at the time) features JUnit 4 - Java 5 support - 2006 catching up TestNG features over years de facto standard, steady evolution rather than revolution Spock - revamped test creation - 2009 power of Groovy under the hood Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 16

Slide 16 text

Unit testing in Java - historical outline JUnit - ﬁrst xUnit for Java - 2000 TestNG - Java 5 leveraged in tests - 2004 with some unique (at the time) features JUnit 4 - Java 5 support - 2006 catching up TestNG features over years de facto standard, steady evolution rather than revolution Spock - revamped test creation - 2009 power of Groovy under the hood JUnit 5 - redesigned and rewritten from scratch - 2017 new king of the hill? Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 17

Slide 17 text

Development Development Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 18

Slide 18 text

. inception year: number of GitHub stars: number of commits: development activity: number of active committers: number of contributors (ever): Development Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 19

Slide 19 text

. inception year: number of GitHub stars: number of commits: development activity: number of active committers: number of contributors (ever): JUnit 5 (as of 5.3) 2015 ~2,5K ~4,9K high (young project) 3 ~90 Development Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 20

Slide 20 text

. inception year: number of GitHub stars: number of commits: development activity: number of active committers: number of contributors (ever): JUnit 5 (as of 5.3) 2015 ~2,5K ~4,9K high (young project) 3 ~90 Spock (as of 1.2) 2009 ~2K ~2,4K medium (mature project) 2 ~70 Development Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 21

Slide 21 text

Development Development partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 22

Slide 22 text

Development Development partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ JUnit 5 JUnit 5 JUnit 5 wins wins wins

Slide 23

Slide 23 text

Tool support Tool support Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 24

Slide 24 text

. Java 11: IntelliJ IDEA: Eclipse: Netbeans: Maven: Gradle: SonarQube: PIT - mutation testing: Tool support Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 25

Slide 25 text

. Java 11: IntelliJ IDEA: Eclipse: Netbeans: Maven: Gradle: SonarQube: PIT - mutation testing: JUnit 5 very good built-in built-in 10.0+ (Maven only) plugin (Surefire) built-in built-in plugin (official) Spock good (with Groovy 2.5.3+) built-in (some Groovy limitations) built-in (some Groovy limitations) unknown plugin (GMavenPlus for Groovy) built-in plugin (official for Groovy) built-in Tool support Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 26

Slide 26 text

Tool support Tool support partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 27

Slide 27 text

Tool support Tool support partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ JUnit 5 JUnit 5 JUnit 5 wins wins wins

Slide 28

Slide 28 text

Test structure Test structure Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Test structure - BDD-like specification - in general clear separation in 3 blocks with predefined responsibility given - creation, initialization and stubbing when - operation to test then - assertion and interaction verification unified (procedures for) test creation improved readability easier to get started with writing (good) test especially for less experienced people Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 31

Slide 31 text

Test structure - JUnit 5 class SimpleCalculatorTest { @Test void shouldAddTwoNumbers() { //given Calculator calculator = new Calculator(); //when int result = calculator.add(1, 2); //then assertEquals(3, result); } } just plain code comments easy to forgot if template is not used can be easily lost/misplaced in refactoring Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 32

Slide 32 text

Test structure - Spock class SimpleCalculatorSpec extends Specification { def "should add two numbers"() { given: Calculator calculator = new Calculator() when: int result = calculator.add(1, 2) then: result == 3 } } [given]/when/then (or expect) required to compile code especially handy with predeﬁned test template Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 33

Slide 33 text

Test structure Test structure partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 34

Slide 34 text

Test structure Test structure partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ Spock Spock Spock wins wins wins

Slide 35

Slide 35 text

Exception testing Exception testing Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 36

Slide 36 text

Exception testing - in general verification if proper exception was thrown often in particular line in test usually accompanied by verification of proper error message extra field(s) set in exception instance Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 37

Slide 37 text

Exception testing - JUnit 5 @Test(expected = NullPointerException.class) for one-liners Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 38

Slide 38 text

Exception testing - JUnit 5 @Test(expected = NullPointerException.class) for one-liners assertThrows() for ﬁne grained assertions similar to catchThrowable() from AssertJ @Test void shouldThrowBusinessExceptionOnCommunicationProblem() { //when Executable e = () -> client.sendPing(TEST_REQUEST_ID) //then CommunicationException thrown = assertThrows(CommunicationException.class, e); assertEquals("Communication problem when sending request with id: " + TEST_REQUEST_ID, thrown.getMessage()); assertEquals(TEST_REQUEST_ID, thrown.getRequestId()); } compact syntax (thanks to lambda expression) further assertions possible on returned instance Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 39

Slide 39 text

Exception testing - Spock @FailsWith(NullPointerException) for one-liners Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 40

Slide 40 text

Exception testing - Spock @FailsWith(NullPointerException) for one-liners thrown() method to catch exceptions in when block def "should capture exception"() { when: client.sendPing(TEST_REQUEST_ID) then: CommunicationException e = thrown() e.message == "Communication problem when sending request with id: $TEST_REQUEST_ID" e.requestId == TEST_REQUEST_ID } further assertions possible on returned instance very compact syntax (thanks to Groovy AST transformations) smart type inference Exceptions utility class to deal with cause chain Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 41

Slide 41 text

Exception testing Exception testing partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 42

Slide 42 text

Exception testing Exception testing partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ Spock Spock Spock wins wins wins

Slide 43

Slide 43 text

Conditional test execution Conditional test execution Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 44

Slide 44 text

Conditional test execution - in general test executed only if given condition is (not) met . Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 45

Slide 45 text

Conditional test execution - in general test executed only if given condition is (not) met . common cases particular Java version reflection hack to use newer version features compatibility testing for lower version specific operating system notifications about changed files on Mac are much delayed testing symlinks on Windows makes no sense tests executed only on CI server, stage environment, ... Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 46

Slide 46 text

Conditional test execution - JUnit 5 annotation-based @Enabled*/@Disabled* conditions predeﬁned set of conditions JVM version, operating system system property, environment variable @Test @DisabledOnOs(OS.WINDOWS) void shouldTestSymlinksBasedLogic() { ... } @Test @EnabledIfSystemProperty(named = "os.arch", matches = ".*32.*") void shouldBeRunOn32BitSystems() { ... } Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 47

Slide 47 text

Conditional test execution - JUnit 5 - cont'd good code completion (enums) easily composable as meta-annotations Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 48

Slide 48 text

Conditional test execution - JUnit 5 - cont'd good code completion (enums) easily composable as meta-annotations experimental support for custom logic in script-based conditions @Test @DisabledIf("'Travis' == systemEnvironment.get('CI_SERVER')") void shouldBeDisabledOnTravis() { ... } Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Conditional test execution - Spock @Requires/@IgnoreIf built-in extensions predeﬁned set of conditions JVM version, operating system system property, environment variable @IgnoreIf({ !jvm.java8Compatible }) def "should return empty Optional by default for unstubbed methods with Java 8+"() { ... } no code completion by default (possible with small trick) custom logic in Groovy closure @Requires({ isStrongCryptographyEnabled() }) //custom static method def "should test strong cryptography-based features"() { ... } Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 51

Slide 51 text

Conditional test execution Conditional test execution partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 52

Slide 52 text

Conditional test execution Conditional test execution partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ Spock Spock Spock wins wins wins

Slide 53

Slide 53 text

Mocking Mocking Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 54

Slide 54 text

Mocking - in general testing with mocks (stubs) instead real collaborators stubbing call executions verifying interactions Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 55

Slide 55 text

Mocking - JUnit 5 no built-in mocking framework Mockito - ﬁrst port of call @Test public void should_not_call_remote_service_if_found_in_cache() { //given given(cacheMock.getCachedOperator(CACHED_MOBILE_NUMBER)).willReturn(Optional.of(PLUS)); //when service.checkOperator(CACHED_MOBILE_NUMBER); //then then(wsMock).should(never()).checkOperator(CACHED_MOBILE_NUMBER); // verify(wsMock, never()).checkOperator(CACHED_MOBILE_NUMBER); //alternative syntax } Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 56

Slide 56 text

Mocking - JUnit 5 no built-in mocking framework Mockito - ﬁrst port of call @Test public void should_not_call_remote_service_if_found_in_cache() { //given given(cacheMock.getCachedOperator(CACHED_MOBILE_NUMBER)).willReturn(Optional.of(PLUS)); //when service.checkOperator(CACHED_MOBILE_NUMBER); //then then(wsMock).should(never()).checkOperator(CACHED_MOBILE_NUMBER); // verify(wsMock, never()).checkOperator(CACHED_MOBILE_NUMBER); //alternative syntax } industrial standard rich set of features ﬁrst-class support for integration testing with Spring Boot (@MockBean) Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 57

Slide 57 text

Slide 58

Slide 58 text

Mocking - Spock built-in mocking subsystem def "should not hit remote service if found in cache"() { given: cacheMock.getCachedOperator(CACHED_MOBILE_NUMBER) >> Optional.of(PLUS) when: service.checkOperator(CACHED_MOBILE_NUMBER) then: 0 * wsMock.checkOperator(CACHED_MOBILE_NUMBER) } extra short and more meaningful syntax thanks to Groovy operator overloading & AST transformations unbeatable by anything written in pure Java * Spring (Boot) support starting with Spock 1.2 its own limitations and quirks Mockito can be used selectively if preferred/needed Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 59

Slide 59 text

Mocking Mocking partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 60

Slide 60 text

Mocking Mocking partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ Draw Draw Draw

Slide 61

Slide 61 text

Mocking Mocking partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ Draw Draw Draw Spock Spock Spock wins wins wins

Slide 62

Slide 62 text

Parameterized tests Parameterized tests Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 63

Slide 63 text

Parameterized tests - in general one test for various input data reduce code duplication can hide speciﬁc business use cases Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 64

Slide 64 text

Parameterized tests - JUnit 5 in JUnit 4 very late added and poorly designed @ParameterizedTest @CsvSource({"1, 2, 3", "-2, 3, 1", "-1, -2, -3"}) void shouldSumTwoIntegers(int x, int y, int expectedResult) { //when int result = calculator.add(x, y); //expect assertEquals(expectedResult, result); } nice implicit argument conversion from String for various types (date, ﬁle/path, currency, UUID, etc.) Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 65

Slide 65 text

Parameterized tests - JUnit 5 input parameters rendered in report with @ParameterizedTest complex declaration of custom data provider (method source) @ParameterizedTest(name = "summing {0} and {1} should give {2}") @MethodSource("integersProvider") void shouldSumTwoIntegers(int x, int y, int expectedResult) { //when int result = calculator.add(x, y); //expect assertEquals(expectedResult, result); } private static Stream integersProvider() { return Stream.of( Arguments.of(1, 2, 3), Arguments.of(-2, 3, 1), Arguments.of(-1, -2, -3) ); } Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 66

Slide 66 text

Slide 67

Slide 67 text

Parameterized tests - Spock @Unroll def "should sum two integers (#x + #y = #expectedResult)"() { when: int result = calculator.add(x, y) then: result == expectedResult where: x | y || expectedResult 1 | 2 || 3 -2 | 3 || 1 -1 | -2 || -3 } state of the art where keyword with table-like formatting very readable and natural to read variables added implicitly (with proper type inferred by IDE) input parameters possible to use in test name directly (with #var syntax) Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 68

Slide 68 text

Parameterized tests - Spock data pipes and data providers for advanced use cases @Unroll("#pesel is valid (#dbId)") def "should validate PESEL correctness (CSV)"() { expect: sut.validate(pesel) where: [dbId, _, _, pesel] << readValidPeopleFromCSVFile() .readLines().collect { it.split(',') } //ugly way to read CSV - don't do this } Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 69

Slide 69 text

Parameterized tests Parameterized tests partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 70

Slide 70 text

Parameterized tests Parameterized tests partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ Spock Spock Spock wins wins wins

Slide 71

Slide 71 text

Migration Migration Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 72

Slide 72 text

Migration to JUnit 5 (from JUnit 4) can coexist with JUnit 4 tests dedicated submodule - junit5-vintage similar test structure with new keywords, annotations, assertions, features new base class package Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 73

Slide 73 text

Migration to Spock (from JUnit 4) can coexist with JUnit 4 tests in fact Spock is a (sophisticated) JUnit runner unoﬃcial tool for automatic test migration completely new test structure Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 74

Slide 74 text

Migration Migration partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 75

Slide 75 text

Migration Migration partial verdict Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/ JUnit 5 JUnit 5 JUnit 5 wins wins wins

Slide 76

Slide 76 text

Summary Summary Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 77

Slide 77 text

Summary JUnit 5 - great progress over JUnit 4 Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 78

Slide 78 text

Summary JUnit 5 - great progress over JUnit 4 many features (previously) unique to Spock now available in JUnit 5 Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 79

Slide 79 text

Summary JUnit 5 - great progress over JUnit 4 many features (previously) unique to Spock now available in JUnit 5 Spock still excels in some areas Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 80

Slide 80 text

Summary JUnit 5 - great progress over JUnit 4 many features (previously) unique to Spock now available in JUnit 5 Spock still excels in some areas best choice is no longer obvious Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 81

Slide 81 text

Summary JUnit 5 - great progress over JUnit 4 many features (previously) unique to Spock now available in JUnit 5 Spock still excels in some areas best choice is no longer obvious it depends on individual preferences Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 82

Slide 82 text

. development: learning curve: tool support: test structure: exception testing: conditional execution: mocking: parameterized tests: migration from JUnit 4: JUnit 5 very good very good (similar to 4) very good (trending up) good good good (limited custom logic) good (Mockito) good very good Spock good (new developers recently) good (Groovy to grasp) good (weaker compile time checks) very good (BDD by default) very good very good (custom logic) good (very compact, some quirks) very good (exceptional!) good Summary - features comparison Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 83

Slide 83 text

Summary - the choice is yours if you prefer old good Java as the only language stability and being mainstream stronger compile time error checking - choose JUnit 5 . Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 84

Slide 84 text

Summary - the choice is yours if you prefer old good Java as the only language stability and being mainstream stronger compile time error checking - choose JUnit 5 . if you favor simplicity and readability power of Groovy under the hood beautiful parameterized tests and exception testing - choose Spock . Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 85

Slide 85 text

Summary - the choice is yours if you prefer old good Java as the only language stability and being mainstream stronger compile time error checking - choose JUnit 5 . if you favor simplicity and readability power of Groovy under the hood beautiful parameterized tests and exception testing - choose Spock . would be great to have only that kind of dilemma :-) Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 86

Slide 86 text

Thank you! (and remember about the feedback - https://greach.contestia.es/) Marcin Zajączkowski https://blog.solidsoft.info/ @SolidSoftBlog [email protected] OpenPGP: 0x48A84C5100F47FB6 06FA 6793 8DD1 7603 B007 5522 48A8 4C51 00F4 7FB6 Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/

Slide 87

Slide 87 text

Questions? (and remember about the feedback - https://greach.contestia.es/) Marcin Zajączkowski https://blog.solidsoft.info/ @SolidSoftBlog [email protected] OpenPGP: 0x48A84C5100F47FB6 06FA 6793 8DD1 7603 B007 5522 48A8 4C51 00F4 7FB6 Marcin Zajączkowski @SolidSoftBlog https://blog.solidsoft.info/