JUnit 5: Motivation, Architecture, Programming and Extension Model

Slide 1

Slide 1 text

JUnit 5

Slide 2

Slide 2 text

Marc Philipp • Senior Software Engineer @ in Germany • JUnit Maintainer since 2012 • Twitter: @marcphilipp • Web: http://www.marcphilipp.de

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

SaaS Micro Services DevOps Spring Boot AWS AWS Lambda ~ 3000 Employees ~ 200 in KA Erlang Node.js

Slide 5

Slide 5 text

The Story Behind JUnit Lambda

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Renaming a private ﬁeld should not break anything, right? 4.11 4.12-beta-1

Slide 8

Slide 8 text

What if we actually had time to work on JUnit?

Slide 9

Slide 9 text

JUnit Lambda

Slide 10

Slide 10 text

Sponsors & Crowdfunding • Three companies had pledged to sponsor a developer each for 6 weeks (prior to crowdfunding). • Additional developers were funded through a crowdfunding campaign on Indiegogo.

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Thank you!

Slide 13

Slide 13 text

Limitations of JUnit 4

Slide 14

Slide 14 text

Runner • Very powerful: Almost every aspect of test execution can be changed • But: You can only have one Runner per test class! • You can’t combine Runners, e.g.  SpringJUnit4ClassRunner + Parameterized

Slide 15

Slide 15 text

Rules • Extension mechanism introduced in JUnit 4.7 • Wraps execution of a test (@Rule) or a test class (@ClassRule) • Designed to be combined — great for simple use cases • But: a single rule cannot be used for method-level and class-level callbacks, no support for instance-level callbacks

Slide 16

Slide 16 text

Existing Architecture Everyone uses the junit.jar.

Slide 17

Slide 17 text

http://blog.takipi.com/the-top-100-java-libraries-in-2016-after-analyzing-47251-dependencies/

Slide 18

Slide 18 text

–Johannes Link, https://jaxenter.com/crowdfunding-for-junit-lambda-is- underway-119546.html „The success of JUnit as a platform prevents the development of JUnit as a tool.“

Slide 19

Slide 19 text

Vision

Slide 20

Slide 20 text

Vision • Decouple test execution and reporting from test deﬁnition to ease future development • Rethink JUnit’s extensibility story • Making use of Java 8 features (e.g. Lambdas, Streams, Interface default methods) for better assertions, generating test cases, etc.

Slide 21

Slide 21 text

Stakeholders? • Developers who write tests • Extension authors • Tools: IDEs and build tools • Maintainers • …

Slide 22

Slide 22 text

Challenges • Changing the way JUnit tests are being run from outside requires support from IDE vendors and build tool providers. • Compatibility with older releases has to be preserved: Old tests still need to work

Slide 23

Slide 23 text

Kickoﬀ IDE and build tool vendors, sponsors, the team

Slide 24

Slide 24 text

Design Goals • Modern programming model for writing tests (Java 8!) • Powerful extension model with a focus on composability • API Segregation: Decouple test execution/reporting from test deﬁnition • Compatibility with older releases + migration path • Modularization + no external dependencies

Slide 25

Slide 25 text

Roadmap Kickoﬀ: 2015-09-19 to 2015-09-23 ✔ Prototype: 2015-12-02 ✔ Alpha: 2016-02-01 ✔ Milestones: M1, M2, M3, M4, M5 Release Candidates GA Release

Slide 26

Slide 26 text

–Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „In this industry, where the common opinion is that complete rewrites generally result in failure, what made the JUnit 5 team decide to go for a complete rewrite?“

Slide 27

Slide 27 text

How we cheated the Big Redesign Fallacy • We didn’t have to rely on JUnit 4 for the requirements • Very little ongoing, parallel development on JUnit 4 • No race between old and new

Slide 28

Slide 28 text

Programming Model

Slide 29

Slide 29 text

DEMO

Slide 30

Slide 30 text

Meta Annotations Annotations can be combined to enable re-use:    @Target(ElementType.METHOD)  @Retention(RetentionPolicy.RUNTIME)  @Tag("fast")  @Test  public @interface FastTest {} Usage:    @FastTest  void test() {}  Equivalent:    @Tag("fast")  @Test  void test() {}

Slide 31

Slide 31 text

@Nested Tests @DisplayName("A stack") class TestingAStackDemo { @Test @DisplayName("is instantiated with new Stack()") void isInstantiatedWithNew() {/* ... */} @Nested @DisplayName("when new") class WhenNew { @BeforeEach void createNewStack() {/* ... */} @Test @DisplayName("is empty") void isEmpty() {/* ... */} // ... @Nested @DisplayName("after pushing an element") class AfterPushing { @BeforeEach void pushAnElement() {/* ... */} @Test @DisplayName("it is no longer empty") void isNotEmpty() {/* ... */} // ... } } }

Slide 32

Slide 32 text

Dynamic Tests @TestFactory Stream dynamicTestsFromStream() { return IntStream.iterate(0, n -> n + 2).limit(100) .mapToObj(n -> dynamicTest("test" + n, () -> { assertTrue(n % 2 == 0); })); }

Slide 33

Slide 33 text

Modularization

Slide 34

Slide 34 text

Separation of Concerns 1. An API to write tests (Jupiter API) 2. Extensible mechanisms to discover and execute tests (Test Engine SPI) 3. An API for test execution by tools (Launcher API)

Slide 35

Slide 35 text

P L AT F O R M J U P I T E R V I N TA G E P A R T Y T H I R D

Slide 36

Slide 36 text

P L AT F O R M J U P I T E R V I N TA G E P A R T Y T H I R D

Slide 37

Slide 37 text

JUnitPlatform Runner for a single class import org.junit.jupiter.api.Test; @RunWith(JUnitPlatform.class) public class JupiterTest { @Test void someTest() { // test something } }

Slide 38

Slide 38 text

JUnitPlatform Runner in suite mode @RunWith(JUnitPlatform.class) @SelectPackages("com.acme") @IncludeEngines({"junit-jupiter", "junit-vintage"}) public class JUnit4SuiteDemo { // this class can be run using JUnit 4 }

Slide 39

Slide 39 text

Test Execution • IDEs: • IntelliJ supports JUnit 5 ≥ M2 since 2016.2 • Eclipse support is available on a branch (see Instructions).  Oﬃcial release slated for Oxygen.1. • Interim solution for other IDEs: JUnitPlatform Runner • Gradle/Maven: Plugin/Provider available • see https://github.com/junit-team/junit5-samples • Manually: ConsoleLauncher

Slide 40

Slide 40 text

Compatibility • Backward compatibility (junit-vintage-engine) enables gradual migration of tests to Jupiter API • Forward compatibility (JUnitPlatform Runner) allows test execution with “old” tools

Slide 41

Slide 41 text

Extensions

Slide 42

Slide 42 text

–Kevin Cooney, https://github.com/junit-team/junit5/wiki/Core- Principles „Prefer extension points over features: It's better to enable new functionality by creating or augmenting an extension point rather than adding the functionality as a core feature.“

Slide 43

Slide 43 text

Extension Points over Features • Once we create an API, we often cannot easily modify it. • Third-party libraries can make mistakes and fix them because they have fewer users. • An extension point can potentially be used for different features, or for implementing the same feature in a more flexible way.

Slide 44

Slide 44 text

–David Saﬀ, https://github.com/junit-team/junit5/wiki/Core-Principles „Complementarily, an extension point should be good at one thing.“

Slide 45

Slide 45 text

Good at one thing • Simpler extension points allow for easier composition. • Don't be afraid to introduce new extension points to handle weak points in existing ones.

Slide 46

Slide 46 text

Registration via @ExtendWith • Annotate your test classes or methods to register extensions • Supports an arbitrary number of extensions at the same time • May be used as a meta-annotation

Slide 47

Slide 47 text

DEMO

Slide 48

Slide 48 text

Extension Points • Conditional Test Execution • ContainerExecutionCondition • TestExecutionCondition • General Purpose • TestInstancePostProcessor • TestExecutionExceptionHandler • ParameterResolver • TestTemplateInvocationContextProvider • Test Lifecycle Callbacks • BeforeAllCallback • BeforeEachCallback • BeforeTestExecutionCallback • AfterTestExecutionCallback • AfterEachCallback • AfterAllCallback

Slide 49

Slide 49 text

Development Tools and Guidelines

Slide 50

Slide 50 text

Gradle • very ﬂexible • lots of excellent plugins • has proven be have been a very good choice

Slide 51

Slide 51 text

Spotless • Used to enforce source code formatting in the Gradle build • No more reviewing of formatting changes like in JUnit 4 pull requests • very annoying at times, but also very helpful in keeping consistency

Slide 52

Slide 52 text

Checkstyle & Doclint • Check Javadoc for consistency, broken links etc. • Check imports are used consistently

Slide 53

Slide 53 text

–Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „The JUnit 5 team currently eliminates cyclic dependencies through the DeGraph Gradle plugin. Cyclic dependencies are not ideal, but Java compiles them correctly, so why is the JUnit 5 team so insistent on removing them?“

Slide 54

Slide 54 text

Degraph • Check each module for package cycles • JUnit 4 package structure has deteriorated over time • Keep dependencies between packages clear in the presence of pull requests etc. • Keep door open for moving a package into a separate module etc.

Slide 55

Slide 55 text

Clover • Measure code coverage based on instrumenting source code • Good support for exceptions like in Assertions

Slide 56

Slide 56 text

–Liam Clark, Jean de Leeuw, Benjamin Los, Thomas Overklift „What is the reason why JUnit 5 is against wild card imports? Is this because of performance and potential name collisions, or are there perhaps diﬀerent reasons?“

Slide 57

Slide 57 text

Imports • No wildcard imports mainly for clarity when reading code • Order of imports is also mandatory • Unused imports are forbidden

Slide 58

Slide 58 text

CI • Fast feedback on pull requests: • Travis (Linux), multiple JDKs • AppVeyor (Windows) • Jenkins hosted on CloudBees for publishing snapshot artifacts and documentation

Slide 59

Slide 59 text

Asciidoctor • User Guide (snapshot version) is published with every build to GitHub Pages • Example source code is compiled, tests are executed, and then included into User Guide

Slide 60

Slide 60 text

Deﬁnition of Done • Particularly important in a distributed team • Part of the pull request template • Includes items that cannot be checked by the build

Slide 61

Slide 61 text

Roadmap

Slide 62

Slide 62 text

M4: @TestTemplate class MyTest { @TestTemplate @ExtendWith(MyTestTemplateInvocationContextProvider.class) void testTemplate() { // test something } } class MyTestTemplateInvocationContextProvider implements TestTemplateInvocationContextProvider { @Override public Stream provide(ContainerExtensionContext context) { return Stream.of(new MyTestTemplateInvocationContext(), …); } } class MyTestTemplateInvocationContext implements TestTemplateInvocationContext { @Override public String getDisplayName(int invocationIndex) { return "[" + invocationIndex + "]"; } @Override public List getAdditionalExtensions() { return asList(new MyParameterResolver(), new MyTestInstancePostProcessor()); } }

Slide 63

Slide 63 text

M4: @ParameterizedTest class ParameterizedTests { @ParameterizedTest @CsvSource({ "foo, 1", "bar, 2" }) void testWithParametersFromAnnotation(String parameter, int i) { // test something } @ParameterizedTest @MethodSource(names = "providerMethod") void testWithParametersFromMethods(String parameter) { } static Iterable providerMethod() { return asList("foo", "bar"); } @ParameterizedTest @CsvFileSource(resources = { "foo.csv", "/bar.csv" }) void testWithParametersFromFile(String parameter) { } }

Slide 64

Slide 64 text

M4: @RepeatedTest @RepeatedTest(10) void repeatedTest() { // ... }

Slide 65

Slide 65 text

Roadmap to GA • 5.0.0-M4 (April 2017): Parameterized/RepeatedTests ✔ • 5.0.0-M5 (May 2017): Java 9 compatibility • 5.0.0-RC1 (June 2017): Last ﬁxes before GA • 5.0.0 (July 2017): GA

Slide 66

Slide 66 text

Getting Started User Guide:  http://junit.org/junit5/docs/current/user-guide/ Sample projects for Gradle and Maven:  https://github.com/junit-team/junit5-samples Javadoc:  http://junit.org/junit5/docs/current/api/

Slide 67

Slide 67 text

Wanted: Feedback! Website:  http://junit.org/junit5/ Code & Issues:  https://github.com/junit-team/junit5/ Twitter:  https://twitter.com/junitteam