JUnit 5: Motivation, Architecture, Programming and Extension Model

JUnit 5: Motivation, Architecture, Programming and Extension Model

Slides for a 1,5 h talk/lecture I gave at TU Delft in April 2017.

Code examples: https://github.com/marcphilipp/junit5-demo/tree/20170404-delft


Marc Philipp

April 04, 2017


  1. JUnit 5

  2. Marc Philipp • Senior Software Engineer @ in Germany •

    JUnit Maintainer since 2012 • Twitter: @marcphilipp • Web: http://www.marcphilipp.de
  3. None
  4. SaaS Micro Services DevOps Spring Boot AWS AWS Lambda ~

    3000 Employees ~ 200 in KA Erlang Node.js
  5. The Story Behind JUnit Lambda

  6. None
  7. Renaming a private field should not break anything, right? 4.11

  8. What if we actually had time to work on JUnit?

  9. JUnit Lambda

  10. Sponsors & Crowdfunding • Three companies had pledged to sponsor

    a developer each for 6 weeks (prior to crowdfunding). • Additional developers were funded through a crowdfunding campaign on Indiegogo.
  11. None
  12. Thank you!

  13. Limitations of JUnit 4

  14. Runner • Very powerful: Almost every aspect of test execution

    can be changed • But: You can only have one Runner per test class! • You can’t combine Runners, e.g.
 SpringJUnit4ClassRunner + Parameterized
  15. Rules • Extension mechanism introduced in JUnit 4.7 • Wraps

    execution of a test (@Rule) or a test class (@ClassRule) • Designed to be combined — great for simple use cases • But: a single rule cannot be used for method-level and class-level callbacks, no support for instance-level callbacks
  16. Existing Architecture Everyone uses the junit.jar.

  17. http://blog.takipi.com/the-top-100-java-libraries-in-2016-after-analyzing-47251-dependencies/

  18. –Johannes Link, https://jaxenter.com/crowdfunding-for-junit-lambda-is- underway-119546.html „The success of JUnit as a

    platform prevents the development of JUnit as a tool.“
  19. Vision

  20. Vision • Decouple test execution and reporting from test definition

    to ease future development • Rethink JUnit’s extensibility story • Making use of Java 8 features (e.g. Lambdas, Streams, Interface default methods) for better assertions, generating test cases, etc.
  21. Stakeholders? • Developers who write tests • Extension authors •

    Tools: IDEs and build tools • Maintainers • …
  22. Challenges • Changing the way JUnit tests are being run

    from outside requires support from IDE vendors and build tool providers. • Compatibility with older releases has to be preserved: Old tests still need to work
  23. Kickoff IDE and build tool vendors, sponsors, the team

  24. Design Goals • Modern programming model for writing tests (Java

    8!) • Powerful extension model with a focus on composability • API Segregation: Decouple test execution/reporting from test definition • Compatibility with older releases + migration path • Modularization + no external dependencies
  25. Roadmap Kickoff: 2015-09-19 to 2015-09-23 ✔ Prototype: 2015-12-02 ✔ Alpha:

    2016-02-01 ✔ Milestones: M1, M2, M3, M4, M5 Release Candidates GA Release
  26. –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „In

    this industry, where the common opinion is that complete rewrites generally result in failure, what made the JUnit 5 team decide to go for a complete rewrite?“
  27. How we cheated the Big Redesign Fallacy • We didn’t

    have to rely on JUnit 4 for the requirements • Very little ongoing, parallel development on JUnit 4 • No race between old and new
  28. Programming Model

  29. DEMO

  30. Meta Annotations Annotations can be combined to enable re-use:

 public @interface FastTest {} Usage:
 void test() {}
 void test() {}
  31. @Nested Tests @DisplayName("A stack") class TestingAStackDemo { @Test @DisplayName("is instantiated

    with new Stack()") void isInstantiatedWithNew() {/* ... */} @Nested @DisplayName("when new") class WhenNew { @BeforeEach void createNewStack() {/* ... */} @Test @DisplayName("is empty") void isEmpty() {/* ... */} // ... @Nested @DisplayName("after pushing an element") class AfterPushing { @BeforeEach void pushAnElement() {/* ... */} @Test @DisplayName("it is no longer empty") void isNotEmpty() {/* ... */} // ... } } }
  32. Dynamic Tests @TestFactory Stream<DynamicTest> dynamicTestsFromStream() { return IntStream.iterate(0, n ->

    n + 2).limit(100) .mapToObj(n -> dynamicTest("test" + n, () -> { assertTrue(n % 2 == 0); })); }
  33. Modularization

  34. Separation of Concerns 1. An API to write tests (Jupiter

    API) 2. Extensible mechanisms to discover and execute tests (Test Engine SPI) 3. An API for test execution by tools (Launcher API)
  35. P L AT F O R M J U P

    I T E R V I N TA G E P A R T Y T H I R D
  36. P L AT F O R M J U P

    I T E R V I N TA G E P A R T Y T H I R D
  37. JUnitPlatform Runner for a single class import org.junit.jupiter.api.Test; @RunWith(JUnitPlatform.class) public

    class JupiterTest { @Test void someTest() { // test something } }
  38. JUnitPlatform Runner in suite mode @RunWith(JUnitPlatform.class) @SelectPackages("com.acme") @IncludeEngines({"junit-jupiter", "junit-vintage"}) public

    class JUnit4SuiteDemo { // this class can be run using JUnit 4 }
  39. Test Execution • IDEs: • IntelliJ supports JUnit 5 ≥

    M2 since 2016.2 • Eclipse support is available on a branch (see Instructions).
 Official release slated for Oxygen.1. • Interim solution for other IDEs: JUnitPlatform Runner • Gradle/Maven: Plugin/Provider available • see https://github.com/junit-team/junit5-samples • Manually: ConsoleLauncher
  40. Compatibility • Backward compatibility (junit-vintage-engine) enables gradual migration of tests

    to Jupiter API • Forward compatibility (JUnitPlatform Runner) allows test execution with “old” tools
  41. Extensions

  42. –Kevin Cooney, https://github.com/junit-team/junit5/wiki/Core- Principles „Prefer extension points over features: It's

    better to enable new functionality by creating or augmenting an extension point rather than adding the functionality as a core feature.“
  43. Extension Points over Features • Once we create an API,

    we often cannot easily modify it. • Third-party libraries can make mistakes and fix them because they have fewer users. • An extension point can potentially be used for different features, or for implementing the same feature in a more flexible way.
  44. –David Saff, https://github.com/junit-team/junit5/wiki/Core-Principles „Complementarily, an extension point should be good

    at one thing.“
  45. Good at one thing • Simpler extension points allow for

    easier composition. • Don't be afraid to introduce new extension points to handle weak points in existing ones.
  46. Registration via @ExtendWith • Annotate your test classes or methods

    to register extensions • Supports an arbitrary number of extensions at the same time • May be used as a meta-annotation
  47. DEMO

  48. Extension Points • Conditional Test Execution • ContainerExecutionCondition • TestExecutionCondition

    • General Purpose • TestInstancePostProcessor • TestExecutionExceptionHandler • ParameterResolver • TestTemplateInvocationContextProvider • Test Lifecycle Callbacks • BeforeAllCallback • BeforeEachCallback • BeforeTestExecutionCallback • AfterTestExecutionCallback • AfterEachCallback • AfterAllCallback
  49. Development Tools and Guidelines

  50. Gradle • very flexible • lots of excellent plugins •

    has proven be have been a very good choice
  51. Spotless • Used to enforce source code formatting in the

    Gradle build • No more reviewing of formatting changes like in JUnit 4 pull requests • very annoying at times, but also very helpful in keeping consistency
  52. Checkstyle & Doclint • Check Javadoc for consistency, broken links

    etc. • Check imports are used consistently
  53. –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „The

    JUnit 5 team currently eliminates cyclic dependencies through the DeGraph Gradle plugin. Cyclic dependencies are not ideal, but Java compiles them correctly, so why is the JUnit 5 team so insistent on removing them?“
  54. Degraph • Check each module for package cycles • JUnit

    4 package structure has deteriorated over time • Keep dependencies between packages clear in the presence of pull requests etc. • Keep door open for moving a package into a separate module etc.
  55. Clover • Measure code coverage based on instrumenting source code

    • Good support for exceptions like in Assertions
  56. –Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „What

    is the reason why JUnit 5 is against wild card imports? Is this because of performance and potential name collisions, or are there perhaps different reasons?“
  57. Imports • No wildcard imports mainly for clarity when reading

    code • Order of imports is also mandatory • Unused imports are forbidden
  58. CI • Fast feedback on pull requests: • Travis (Linux),

    multiple JDKs • AppVeyor (Windows) • Jenkins hosted on CloudBees for publishing snapshot artifacts and documentation
  59. Asciidoctor • User Guide (snapshot version) is published with every

    build to GitHub Pages • Example source code is compiled, tests are executed, and then included into User Guide
  60. Definition of Done • Particularly important in a distributed team

    • Part of the pull request template • Includes items that cannot be checked by the build
  61. Roadmap

  62. M4: @TestTemplate class MyTest { @TestTemplate @ExtendWith(MyTestTemplateInvocationContextProvider.class) void testTemplate() {

    // test something } } class MyTestTemplateInvocationContextProvider implements TestTemplateInvocationContextProvider { @Override public Stream<TestTemplateInvocationContext> provide(ContainerExtensionContext context) { return Stream.of(new MyTestTemplateInvocationContext(), …); } } class MyTestTemplateInvocationContext implements TestTemplateInvocationContext { @Override public String getDisplayName(int invocationIndex) { return "[" + invocationIndex + "]"; } @Override public List<Extension> getAdditionalExtensions() { return asList(new MyParameterResolver(), new MyTestInstancePostProcessor()); } }
  63. M4: @ParameterizedTest class ParameterizedTests { @ParameterizedTest @CsvSource({ "foo, 1", "bar,

    2" }) void testWithParametersFromAnnotation(String parameter, int i) { // test something } @ParameterizedTest @MethodSource(names = "providerMethod") void testWithParametersFromMethods(String parameter) { } static Iterable<String> providerMethod() { return asList("foo", "bar"); } @ParameterizedTest @CsvFileSource(resources = { "foo.csv", "/bar.csv" }) void testWithParametersFromFile(String parameter) { } }
  64. M4: @RepeatedTest @RepeatedTest(10) void repeatedTest() { // ... }

  65. Roadmap to GA • 5.0.0-M4 (April 2017): Parameterized/RepeatedTests ✔ •

    5.0.0-M5 (May 2017): Java 9 compatibility • 5.0.0-RC1 (June 2017): Last fixes before GA • 5.0.0 (July 2017): GA
  66. Getting Started User Guide:
 http://junit.org/junit5/docs/current/user-guide/ Sample projects for Gradle and

 https://github.com/junit-team/junit5-samples Javadoc:
  67. Wanted: Feedback! Website:
 http://junit.org/junit5/ Code & Issues:
 https://github.com/junit-team/junit5/ Twitter: