JUnit 5: Motivation, Architecture, Programming and Extension Model

Slides for a 1,5 h talk/lecture I gave at TU Delft in April 2017.

Code examples: https://github.com/marcphilipp/junit5-demo/tree/20170404-delft

Transcript

1. JUnit 5
2. Marc Philipp • Senior Software Engineer @ in Germany • JUnit Maintainer since 2012 • Twitter: @marcphilipp • Web: http://www.marcphilipp.de
4. SaaS Micro Services DevOps Spring Boot AWS AWS Lambda ~ 3000 Employees ~ 200 in KA Erlang Node.js
5. The Story Behind JUnit Lambda
7. Renaming a private field should not break anything, right? 4.11 4.12-beta-1
8. What if we actually had time to work on JUnit?
9. JUnit Lambda
10. Sponsors & Crowdfunding • Three companies had pledged to sponsor a developer each for 6 weeks (prior to crowdfunding). • Additional developers were funded through a crowdfunding campaign on Indiegogo.
12. Thank you!
13. Limitations of JUnit 4
14. Runner • Very powerful: Almost every aspect of test execution can be changed • But: You can only have one Runner per test class! • You can’t combine Runners, e.g.
 SpringJUnit4ClassRunner + Parameterized
15. Rules • Extension mechanism introduced in JUnit 4.7 • Wraps execution of a test (@Rule) or a test class (@ClassRule) • Designed to be combined — great for simple use cases • But: a single rule cannot be used for method-level and class-level callbacks, no support for instance-level callbacks
16. Existing Architecture Everyone uses the junit.jar.
17. http://blog.takipi.com/the-top-100-java-libraries-in-2016-after-analyzing-47251-dependencies/
18. –Johannes Link, https://jaxenter.com/crowdfunding-for-junit-lambda-is- underway-119546.html „The success of JUnit as a platform prevents the development of JUnit as a tool.“
19. Vision
20. Vision • Decouple test execution and reporting from test definition to ease future development • Rethink JUnit’s extensibility story • Making use of Java 8 features (e.g. Lambdas, Streams, Interface default methods) for better assertions, generating test cases, etc.
21. Stakeholders? • Developers who write tests • Extension authors • Tools: IDEs and build tools • Maintainers • …
22. Challenges • Changing the way JUnit tests are being run from outside requires support from IDE vendors and build tool providers. • Compatibility with older releases has to be preserved: Old tests still need to work
23. Kickoff IDE and build tool vendors, sponsors, the team
24. Design Goals • Modern programming model for writing tests (Java 8!) • Powerful extension model with a focus on composability • API Segregation: Decouple test execution/reporting from test definition • Compatibility with older releases + migration path • Modularization + no external dependencies
25. Roadmap Kickoff: 2015-09-19 to 2015-09-23 ✔ Prototype: 2015-12-02 ✔ Alpha: 2016-02-01 ✔ Milestones: M1, M2, M3, M4, M5 Release Candidates GA Release
26. –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „In this industry, where the common opinion is that complete rewrites generally result in failure, what made the JUnit 5 team decide to go for a complete rewrite?“
27. How we cheated the Big Redesign Fallacy • We didn’t have to rely on JUnit 4 for the requirements • Very little ongoing, parallel development on JUnit 4 • No race between old and new
28. Programming Model
29. DEMO
30. Meta Annotations Annotations can be combined to enable re-use:
 
 @Target(ElementType.METHOD)
 @Retention(RetentionPolicy.RUNTIME)
 @Tag("fast")
 @Test
 public @interface FastTest {} Usage:
 
 @FastTest
 void test() {}
 Equivalent:
 
 @Tag("fast")
 @Test
 void test() {}
31. @Nested Tests @DisplayName("A stack") class TestingAStackDemo { @Test @DisplayName("is instantiated with new Stack()") void isInstantiatedWithNew() {/* ... */} @Nested @DisplayName("when new") class WhenNew { @BeforeEach void createNewStack() {/* ... */} @Test @DisplayName("is empty") void isEmpty() {/* ... */} // ... @Nested @DisplayName("after pushing an element") class AfterPushing { @BeforeEach void pushAnElement() {/* ... */} @Test @DisplayName("it is no longer empty") void isNotEmpty() {/* ... */} // ... } } }
32. Dynamic Tests @TestFactory Stream<DynamicTest> dynamicTestsFromStream() { return IntStream.iterate(0, n -> n + 2).limit(100) .mapToObj(n -> dynamicTest("test" + n, () -> { assertTrue(n % 2 == 0); })); }
33. Modularization
34. Separation of Concerns 1. An API to write tests (Jupiter API) 2. Extensible mechanisms to discover and execute tests (Test Engine SPI) 3. An API for test execution by tools (Launcher API)
35. P L AT F O R M J U P I T E R V I N TA G E P A R T Y T H I R D
36. P L AT F O R M J U P I T E R V I N TA G E P A R T Y T H I R D
37. JUnitPlatform Runner for a single class import org.junit.jupiter.api.Test; @RunWith(JUnitPlatform.class) public class JupiterTest { @Test void someTest() { // test something } }
38. JUnitPlatform Runner in suite mode @RunWith(JUnitPlatform.class) @SelectPackages("com.acme") @IncludeEngines({"junit-jupiter", "junit-vintage"}) public class JUnit4SuiteDemo { // this class can be run using JUnit 4 }
39. Test Execution • IDEs: • IntelliJ supports JUnit 5 ≥ M2 since 2016.2 • Eclipse support is available on a branch (see Instructions).
 Official release slated for Oxygen.1. • Interim solution for other IDEs: JUnitPlatform Runner • Gradle/Maven: Plugin/Provider available • see https://github.com/junit-team/junit5-samples • Manually: ConsoleLauncher
40. Compatibility • Backward compatibility (junit-vintage-engine) enables gradual migration of tests to Jupiter API • Forward compatibility (JUnitPlatform Runner) allows test execution with “old” tools
41. Extensions
42. –Kevin Cooney, https://github.com/junit-team/junit5/wiki/Core- Principles „Prefer extension points over features: It's better to enable new functionality by creating or augmenting an extension point rather than adding the functionality as a core feature.“
43. Extension Points over Features • Once we create an API, we often cannot easily modify it. • Third-party libraries can make mistakes and fix them because they have fewer users. • An extension point can potentially be used for different features, or for implementing the same feature in a more flexible way.
44. –David Saff, https://github.com/junit-team/junit5/wiki/Core-Principles „Complementarily, an extension point should be good at one thing.“
45. Good at one thing • Simpler extension points allow for easier composition. • Don't be afraid to introduce new extension points to handle weak points in existing ones.
46. Registration via @ExtendWith • Annotate your test classes or methods to register extensions • Supports an arbitrary number of extensions at the same time • May be used as a meta-annotation
47. DEMO
48. Extension Points • Conditional Test Execution • ContainerExecutionCondition • TestExecutionCondition • General Purpose • TestInstancePostProcessor • TestExecutionExceptionHandler • ParameterResolver • TestTemplateInvocationContextProvider • Test Lifecycle Callbacks • BeforeAllCallback • BeforeEachCallback • BeforeTestExecutionCallback • AfterTestExecutionCallback • AfterEachCallback • AfterAllCallback
49. Development Tools and Guidelines
50. Gradle • very flexible • lots of excellent plugins • has proven be have been a very good choice
51. Spotless • Used to enforce source code formatting in the Gradle build • No more reviewing of formatting changes like in JUnit 4 pull requests • very annoying at times, but also very helpful in keeping consistency
52. Checkstyle & Doclint • Check Javadoc for consistency, broken links etc. • Check imports are used consistently
53. –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „The JUnit 5 team currently eliminates cyclic dependencies through the DeGraph Gradle plugin. Cyclic dependencies are not ideal, but Java compiles them correctly, so why is the JUnit 5 team so insistent on removing them?“
54. Degraph • Check each module for package cycles • JUnit 4 package structure has deteriorated over time • Keep dependencies between packages clear in the presence of pull requests etc. • Keep door open for moving a package into a separate module etc.
55. Clover • Measure code coverage based on instrumenting source code • Good support for exceptions like in Assertions
56. –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „What is the reason why JUnit 5 is against wild card imports? Is this because of performance and potential name collisions, or are there perhaps different reasons?“
57. Imports • No wildcard imports mainly for clarity when reading code • Order of imports is also mandatory • Unused imports are forbidden
58. CI • Fast feedback on pull requests: • Travis (Linux), multiple JDKs • AppVeyor (Windows) • Jenkins hosted on CloudBees for publishing snapshot artifacts and documentation
59. Asciidoctor • User Guide (snapshot version) is published with every build to GitHub Pages • Example source code is compiled, tests are executed, and then included into User Guide
60. Definition of Done • Particularly important in a distributed team • Part of the pull request template • Includes items that cannot be checked by the build
61. Roadmap
62. M4: @TestTemplate class MyTest { @TestTemplate @ExtendWith(MyTestTemplateInvocationContextProvider.class) void testTemplate() { // test something } } class MyTestTemplateInvocationContextProvider implements TestTemplateInvocationContextProvider { @Override public Stream<TestTemplateInvocationContext> provide(ContainerExtensionContext context) { return Stream.of(new MyTestTemplateInvocationContext(), …); } } class MyTestTemplateInvocationContext implements TestTemplateInvocationContext { @Override public String getDisplayName(int invocationIndex) { return "[" + invocationIndex + "]"; } @Override public List<Extension> getAdditionalExtensions() { return asList(new MyParameterResolver(), new MyTestInstancePostProcessor()); } }
63. M4: @ParameterizedTest class ParameterizedTests { @ParameterizedTest @CsvSource({ "foo, 1", "bar, 2" }) void testWithParametersFromAnnotation(String parameter, int i) { // test something } @ParameterizedTest @MethodSource(names = "providerMethod") void testWithParametersFromMethods(String parameter) { } static Iterable<String> providerMethod() { return asList("foo", "bar"); } @ParameterizedTest @CsvFileSource(resources = { "foo.csv", "/bar.csv" }) void testWithParametersFromFile(String parameter) { } }
64. M4: @RepeatedTest @RepeatedTest(10) void repeatedTest() { // ... }
65. Roadmap to GA • 5.0.0-M4 (April 2017): Parameterized/RepeatedTests ✔ • 5.0.0-M5 (May 2017): Java 9 compatibility • 5.0.0-RC1 (June 2017): Last fixes before GA • 5.0.0 (July 2017): GA
66. Getting Started User Guide:
 http://junit.org/junit5/docs/current/user-guide/ Sample projects for Gradle and Maven:
 https://github.com/junit-team/junit5-samples Javadoc:
 http://junit.org/junit5/docs/current/api/
67. Wanted: Feedback! Website:
 http://junit.org/junit5/ Code & Issues:
 https://github.com/junit-team/junit5/ Twitter:
 https://twitter.com/junitteam