JUnit 5: Motivation, Architecture, Programming and Extension Model

JUnit 5: Motivation, Architecture, Programming and Extension Model

Slides for a 1,5 h talk/lecture I gave at TU Delft in April 2017.

Code examples: https://github.com/marcphilipp/junit5-demo/tree/20170404-delft

956c7d246841e8507a1e1b96842994db?s=128

Marc Philipp

April 04, 2017
Tweet

Transcript

  1. 1.
  2. 2.

    Marc Philipp • Senior Software Engineer @ in Germany •

    JUnit Maintainer since 2012 • Twitter: @marcphilipp • Web: http://www.marcphilipp.de
  3. 3.
  4. 4.

    SaaS Micro Services DevOps Spring Boot AWS AWS Lambda ~

    3000 Employees ~ 200 in KA Erlang Node.js
  5. 6.
  6. 10.

    Sponsors & Crowdfunding • Three companies had pledged to sponsor

    a developer each for 6 weeks (prior to crowdfunding). • Additional developers were funded through a crowdfunding campaign on Indiegogo.
  7. 11.
  8. 14.

    Runner • Very powerful: Almost every aspect of test execution

    can be changed • But: You can only have one Runner per test class! • You can’t combine Runners, e.g.
 SpringJUnit4ClassRunner + Parameterized
  9. 15.

    Rules • Extension mechanism introduced in JUnit 4.7 • Wraps

    execution of a test (@Rule) or a test class (@ClassRule) • Designed to be combined — great for simple use cases • But: a single rule cannot be used for method-level and class-level callbacks, no support for instance-level callbacks
  10. 19.
  11. 20.

    Vision • Decouple test execution and reporting from test definition

    to ease future development • Rethink JUnit’s extensibility story • Making use of Java 8 features (e.g. Lambdas, Streams, Interface default methods) for better assertions, generating test cases, etc.
  12. 21.

    Stakeholders? • Developers who write tests • Extension authors •

    Tools: IDEs and build tools • Maintainers • …
  13. 22.

    Challenges • Changing the way JUnit tests are being run

    from outside requires support from IDE vendors and build tool providers. • Compatibility with older releases has to be preserved: Old tests still need to work
  14. 24.

    Design Goals • Modern programming model for writing tests (Java

    8!) • Powerful extension model with a focus on composability • API Segregation: Decouple test execution/reporting from test definition • Compatibility with older releases + migration path • Modularization + no external dependencies
  15. 25.

    Roadmap Kickoff: 2015-09-19 to 2015-09-23 ✔ Prototype: 2015-12-02 ✔ Alpha:

    2016-02-01 ✔ Milestones: M1, M2, M3, M4, M5 Release Candidates GA Release
  16. 26.

    –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „In

    this industry, where the common opinion is that complete rewrites generally result in failure, what made the JUnit 5 team decide to go for a complete rewrite?“
  17. 27.

    How we cheated the Big Redesign Fallacy • We didn’t

    have to rely on JUnit 4 for the requirements • Very little ongoing, parallel development on JUnit 4 • No race between old and new
  18. 29.
  19. 30.

    Meta Annotations Annotations can be combined to enable re-use:
 


    @Target(ElementType.METHOD)
 @Retention(RetentionPolicy.RUNTIME)
 @Tag("fast")
 @Test
 public @interface FastTest {} Usage:
 
 @FastTest
 void test() {}
 Equivalent:
 
 @Tag("fast")
 @Test
 void test() {}
  20. 31.

    @Nested Tests @DisplayName("A stack") class TestingAStackDemo { @Test @DisplayName("is instantiated

    with new Stack()") void isInstantiatedWithNew() {/* ... */} @Nested @DisplayName("when new") class WhenNew { @BeforeEach void createNewStack() {/* ... */} @Test @DisplayName("is empty") void isEmpty() {/* ... */} // ... @Nested @DisplayName("after pushing an element") class AfterPushing { @BeforeEach void pushAnElement() {/* ... */} @Test @DisplayName("it is no longer empty") void isNotEmpty() {/* ... */} // ... } } }
  21. 32.

    Dynamic Tests @TestFactory Stream<DynamicTest> dynamicTestsFromStream() { return IntStream.iterate(0, n ->

    n + 2).limit(100) .mapToObj(n -> dynamicTest("test" + n, () -> { assertTrue(n % 2 == 0); })); }
  22. 34.

    Separation of Concerns 1. An API to write tests (Jupiter

    API) 2. Extensible mechanisms to discover and execute tests (Test Engine SPI) 3. An API for test execution by tools (Launcher API)
  23. 35.

    P L AT F O R M J U P

    I T E R V I N TA G E P A R T Y T H I R D
  24. 36.

    P L AT F O R M J U P

    I T E R V I N TA G E P A R T Y T H I R D
  25. 39.

    Test Execution • IDEs: • IntelliJ supports JUnit 5 ≥

    M2 since 2016.2 • Eclipse support is available on a branch (see Instructions).
 Official release slated for Oxygen.1. • Interim solution for other IDEs: JUnitPlatform Runner • Gradle/Maven: Plugin/Provider available • see https://github.com/junit-team/junit5-samples • Manually: ConsoleLauncher
  26. 40.

    Compatibility • Backward compatibility (junit-vintage-engine) enables gradual migration of tests

    to Jupiter API • Forward compatibility (JUnitPlatform Runner) allows test execution with “old” tools
  27. 42.

    –Kevin Cooney, https://github.com/junit-team/junit5/wiki/Core- Principles „Prefer extension points over features: It's

    better to enable new functionality by creating or augmenting an extension point rather than adding the functionality as a core feature.“
  28. 43.

    Extension Points over Features • Once we create an API,

    we often cannot easily modify it. • Third-party libraries can make mistakes and fix them because they have fewer users. • An extension point can potentially be used for different features, or for implementing the same feature in a more flexible way.
  29. 45.

    Good at one thing • Simpler extension points allow for

    easier composition. • Don't be afraid to introduce new extension points to handle weak points in existing ones.
  30. 46.

    Registration via @ExtendWith • Annotate your test classes or methods

    to register extensions • Supports an arbitrary number of extensions at the same time • May be used as a meta-annotation
  31. 47.
  32. 48.

    Extension Points • Conditional Test Execution • ContainerExecutionCondition • TestExecutionCondition

    • General Purpose • TestInstancePostProcessor • TestExecutionExceptionHandler • ParameterResolver • TestTemplateInvocationContextProvider • Test Lifecycle Callbacks • BeforeAllCallback • BeforeEachCallback • BeforeTestExecutionCallback • AfterTestExecutionCallback • AfterEachCallback • AfterAllCallback
  33. 50.

    Gradle • very flexible • lots of excellent plugins •

    has proven be have been a very good choice
  34. 51.

    Spotless • Used to enforce source code formatting in the

    Gradle build • No more reviewing of formatting changes like in JUnit 4 pull requests • very annoying at times, but also very helpful in keeping consistency
  35. 52.

    Checkstyle & Doclint • Check Javadoc for consistency, broken links

    etc. • Check imports are used consistently
  36. 53.

    –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „The

    JUnit 5 team currently eliminates cyclic dependencies through the DeGraph Gradle plugin. Cyclic dependencies are not ideal, but Java compiles them correctly, so why is the JUnit 5 team so insistent on removing them?“
  37. 54.

    Degraph • Check each module for package cycles • JUnit

    4 package structure has deteriorated over time • Keep dependencies between packages clear in the presence of pull requests etc. • Keep door open for moving a package into a separate module etc.
  38. 55.

    Clover • Measure code coverage based on instrumenting source code

    • Good support for exceptions like in Assertions
  39. 56.

    –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „What

    is the reason why JUnit 5 is against wild card imports? Is this because of performance and potential name collisions, or are there perhaps different reasons?“
  40. 57.

    Imports • No wildcard imports mainly for clarity when reading

    code • Order of imports is also mandatory • Unused imports are forbidden
  41. 58.

    CI • Fast feedback on pull requests: • Travis (Linux),

    multiple JDKs • AppVeyor (Windows) • Jenkins hosted on CloudBees for publishing snapshot artifacts and documentation
  42. 59.

    Asciidoctor • User Guide (snapshot version) is published with every

    build to GitHub Pages • Example source code is compiled, tests are executed, and then included into User Guide
  43. 60.

    Definition of Done • Particularly important in a distributed team

    • Part of the pull request template • Includes items that cannot be checked by the build
  44. 61.
  45. 62.

    M4: @TestTemplate class MyTest { @TestTemplate @ExtendWith(MyTestTemplateInvocationContextProvider.class) void testTemplate() {

    // test something } } class MyTestTemplateInvocationContextProvider implements TestTemplateInvocationContextProvider { @Override public Stream<TestTemplateInvocationContext> provide(ContainerExtensionContext context) { return Stream.of(new MyTestTemplateInvocationContext(), …); } } class MyTestTemplateInvocationContext implements TestTemplateInvocationContext { @Override public String getDisplayName(int invocationIndex) { return "[" + invocationIndex + "]"; } @Override public List<Extension> getAdditionalExtensions() { return asList(new MyParameterResolver(), new MyTestInstancePostProcessor()); } }
  46. 63.

    M4: @ParameterizedTest class ParameterizedTests { @ParameterizedTest @CsvSource({ "foo, 1", "bar,

    2" }) void testWithParametersFromAnnotation(String parameter, int i) { // test something } @ParameterizedTest @MethodSource(names = "providerMethod") void testWithParametersFromMethods(String parameter) { } static Iterable<String> providerMethod() { return asList("foo", "bar"); } @ParameterizedTest @CsvFileSource(resources = { "foo.csv", "/bar.csv" }) void testWithParametersFromFile(String parameter) { } }
  47. 65.

    Roadmap to GA • 5.0.0-M4 (April 2017): Parameterized/RepeatedTests ✔ •

    5.0.0-M5 (May 2017): Java 9 compatibility • 5.0.0-RC1 (June 2017): Last fixes before GA • 5.0.0 (July 2017): GA
  48. 66.

    Getting Started User Guide:
 http://junit.org/junit5/docs/current/user-guide/ Sample projects for Gradle and

    Maven:
 https://github.com/junit-team/junit5-samples Javadoc:
 http://junit.org/junit5/docs/current/api/