Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Clean Tests

Yoan
October 18, 2021
1.3k

Clean Tests

Yoan

October 18, 2021
Tweet

Transcript

  1. @yot88
    Clean tests

    View full-size slide

  2. Mythbusters
    In small groups, categorize each sentence about Unit tests in :
    • Myth
    • Truth
    Sentences
    • It makes changes more difficult to make
    • Using them, I don't have a piece of code that I'm afraid to touch
    • Unit tests are only written by testers
    • My code is so simple, so I do not need to write a single test on it
    • Unit tested code is of higher quality than not tested code
    • You only need unit testing when there are many developers
    • It reduces the cost of development
    • It takes too much time to write

    View full-size slide

  3. Mythbusters
    • Makes changes easier to make
    • Let developers refactor without fear (again, again, and again)
    It makes changes more difficult to make

    View full-size slide

  4. Mythbusters
    When you refactor / add new features it acts as a safety
    net and increase your confidence
    Using them, I don't have a piece of code that
    I'm afraid to touch

    View full-size slide

  5. Mythbusters
    • Usually, they don’t…
    • Developers write unit tests
    • Ideally run them every time they make any change on the
    system
    Unit tests are only written by testers

    View full-size slide

  6. Mythbusters
    Simple code requires simple tests, so there are no excuses.
    My code is so simple,
    I do not need to write a single test on it

    View full-size slide

  7. Mythbusters
    • It identifies every defect that may have come up before code is sent further
    for integration testing
    • Writing tests makes you think harder about the problem
    • It exposes the edge cases and makes you write better code
    Unit tested code is of higher quality
    than not tested one

    View full-size slide

  8. Mythbusters
    • Unit testing can help a one-person team just as much as a 50-person team
    • Even more risky to let a single person hold all the cards
    You only need unit testing
    when there are many developers

    View full-size slide

  9. Mythbusters
    • Since the bugs are found early, unit testing helps reduce the cost of bug fixes
    • Bugs detected earlier are easier to fix
    It reduces the cost of development

    View full-size slide

  10. Mythbusters
    • It takes a little while to get used to, but overall will save you time and cut down on wasted time
    • Regression testing will keep things moving forward without having to worry too much
    How do you test your development if you do not write Unit test ?
    Our responsibility to reduce the cost of quality
    It takes too much time to write

    View full-size slide

  11. Testing Principles, Practices and Patterns

    View full-size slide

  12. Goal of Unit Testing
    Enable sustainable growth of the software project
    Project without tests
    Quickly slows down to the point that it’s
    hard to make any progress
    Software Entropy
    Fight entropy
    • Constant cleaning and refactoring
    • Tests act as a safety net
    A tool that provides insurance against a vast
    majority of regressions

    View full-size slide

  13. Good vs Bad Tests
    Badly written tests result in the same picture
    Not all tests are created equal
    • Bad tests : raise false alarms
    • Unit tests are also vulnerable to bugs and require maintenance
    Tests are code too
    View them as part of your code base that aims at solving a
    particular problem: ensuring the application’s correctness

    View full-size slide

  14. What makes a successful test suite?
    • Integrated into the development cycle
    Ideally : execute them on every change
    • Targets the most important parts of your code base
    Domain model – contains the business logic
    Testing this gives the best ROI
    • Provides maximum value with minimum maintenance costs
    Recognizing a valuable test (and, by extension, a test of low value)
    Writing a valuable test

    View full-size slide

  15. What is a unit test?

    View full-size slide

  16. What is a unit test?
    A unit test is an automated test that :
    • Verifies a small piece of code (also known as a unit)
    • Does it quickly
    • And does it in an isolated manner.

    View full-size slide

  17. How would you test this ?
    https://github.com/ythirion/clean-tests
    public class CustomerService {
    public static Try purchase(
    Store store,
    ProductType product,
    Integer quantity
    ) {
    return (!store.hasEnoughInventory(product, quantity)) ?
    Failure(new IllegalArgumentException("Not enough inventory")) :
    Success(store.removeInventory(product, quantity));
    }
    }

    View full-size slide

  18. 2 schools
    London school Classical / Detroit school
    A unit is A class
    Uses test
    doubles for
    A class or a set of classes
    All but immutable dependencies Shared dependencies

    View full-size slide

  19. Test example
    London school Classical / Detroit school
    A unit is A class
    Uses test
    doubles for
    A class or a set of classes / behavior
    All but immutable dependencies Shared dependencies
    class ClassicalCustomerTests {
    private final Store store = new Store(HashMap.empty())
    .addInventory(ProductType.Book, 10);
    @Test
    void it_should_purchase_successfully_when_enough_inventory() {
    final var updatedStore = CustomerService.purchase(store, ProductType.Book, 6);
    assertThat(updatedStore.isSuccess()).isTrue();
    assertThat(updatedStore.get().getInventoryFor(ProductType.Book)).isEqualTo(4);
    }
    @Test
    void it_should_fail_when_not_enough_inventory() {
    final var updatedStore = CustomerService.purchase(store, ProductType.Book, 11);
    assertThat(updatedStore.isFailure()).isTrue();
    assertThat(updatedStore.getCause())
    .isInstanceOf(IllegalArgumentException.class)
    .hasMessageContaining("Not enough inventory");
    }
    }
    @ExtendWith(MockitoExtension.class)
    class LondonCustomerTests {
    private final int QUANTITY = 6;
    @Mock
    private Store storeMock;
    @Test
    void it_should_purchase_successfully_when_enough_inventory() {
    when(storeMock.hasEnoughInventory(ProductType.Book, QUANTITY)).thenReturn(true);
    final var updatedStore = CustomerService.purchase(storeMock, ProductType.Book, 6);
    assertThat(updatedStore.isSuccess()).isTrue();
    verify(storeMock, times(1)).removeInventory(ProductType.Book, QUANTITY);
    }
    @Test
    void it_should_fail_when_not_enough_inventory() {
    final var updatedStore = CustomerService.purchase(storeMock, ProductType.Book, 11);
    assertThat(updatedStore.isFailure()).isTrue();
    assertThat(updatedStore.getCause())
    .isInstanceOf(IllegalArgumentException.class)
    .hasMessageContaining("Not enough inventory");
    verify(storeMock, never()).removeInventory(ProductType.Book, QUANTITY);
    }
    }

    View full-size slide

  20. 3 Pillars / 1 foundation
    • Protection against regressions
    A regression = a software bug
    The larger the code base, the more exposure it has to potential bugs
    Tests should reveal those regressions
    • Resistance to refactoring
    The degree to which a test can sustain a refactoring of the underlying
    application code without turning red (failing)
    • Fast feedback
    The faster the tests, the more of them you can have in the suite and
    the more often you can run them -> shorten the feedback loop
    Maintainability (Maintenance costs)
    How hard it is to understand the test
    How hard it is to run the test
    Good Unit
    Tests
    Protection
    Resistance
    Fast feedback
    Maintainability

    View full-size slide

  21. Properties of Good Unit Tests – F.I.R.S.T
    Fast
    Tests should be fast enough that you won't be discouraged from using them
    Isolates
    Tests should not depend on the state of another test
    Repeatable
    Tests should be repeatable in any environment without varying results
    Self validating
    Each test will have a single boolean output of pass or fail
    Thorough
    The tests we write should
    • cover all happy paths
    • cover edge/corner/boundary cases

    View full-size slide

  22. class CalculatorTests {
    @Test
    void calculator_should_sum_2_numbers() {
    // Arrange | Given
    final var first = 10;
    final var second = 20;
    // Act | When
    final var result = Calculator.sum(first, second);
    // Arrange | Then
    assertThat(result).isEqualTo(30);
    }
    }
    Anatomy of a Unit Test
    Class-container for a
    cohesive set of tests
    Arrange / Given : bring the system under test
    (SUT) and its dependencies to a desired state
    Act / When : Invoke the behavior
    • call method / function on the SUT
    • pass the prepared dependencies
    Assert / Then : verify the outcome
    3 ways to do it :
    • the return value
    • the final state of the SUT and its collaborators
    • or the methods the SUT called on those collaborators
    Name of the test
    Junit Glue
    import org.junit.jupiter.api.Test;

    View full-size slide

  23. Naming Unit Test
    It’s important to give expressive names to your tests
    Proper naming helps you understand what the test verifies and how the underlying system behaves.
    “delivery is invalid”
    “delivery should be invalid”
    “delivery with past date should be invalid”
    Why ?
    Must contain
    Scenario
    Function Under Test
    ExpectedResult
    Why ?

    View full-size slide

  24. Naming Unit Test
    • Don’t follow a rigid naming policy
    You simply can’t fit a high-level description of a complex behavior into the narrow box of such a policy
    Allow freedom of expression
    • Name the test as if you were describing the scenario to a non-programmer who is familiar with the
    problem domain
    A domain expert or a business analyst should understand it
    • Use sentences
    Doing so helps improve readability, especially in long names

    View full-size slide

  25. Test doubles
    Test double : term that describes all kinds of non-production-ready, fake dependencies in tests
    • spies : written manually
    • mocks : created with the help of a mocking framework
    Difference is in how intelligent they are
    • dummy : simple, hardcoded value
    • stub : fully fledged dependency that you configure to return
    different values for different scenarios
    • fake : a fake is the same as a stub for most purposes.
    • Usually implemented to replace a dependency that doesn’t yet exist

    View full-size slide

  26. Stub / Mock
    Help to emulate and examine outcoming interactions
    Help to emulate incoming interactions

    View full-size slide

  27. Stub / Mock in mockito
    Help to emulate and examine outcoming interactions
    Help to emulate incoming interactions
    when(storeMock.hasEnoughInventory(ProductType.Book, QUANTITY)).thenReturn(true);
    when() / then()
    Or given() in BDDMockito
    verify()
    verify(storeMock, times(1)).removeInventory(ProductType.Book, QUANTITY);

    View full-size slide

  28. 3 styles of Unit Tests
    State-based Communication-based
    • Feed an input to the system under test
    (SUT)
    • Check the output it produces
    • Tests substitute the SUT’s collaborators with mocks
    • Verify that the SUT calls those collaborators
    correctly
    Output-based
    Assumes there are no side effects and the only
    result of the SUT’s work is the value it returns
    to the caller
    Also known as functional (side-effect-free code)
    • Verify the state of the system after an
    operation is complete
    • “State” can refer to the state of
    • The SUT itself
    • One of its collaborators
    • Or an out-of-process dependency - the database
    or the filesystem
    Verify the final state of the
    system after an operation is complete

    View full-size slide

  29. 3 styles of Unit Tests - examples
    State-based
    Communication-based
    Output-based
    Product("Kaamelott")
    Product(”Free Guy")
    @Test
    void discount_of_2_products_should_be_2_percent() {
    val product1 = new Product("Kaamelott");
    val product2 = new Product("Free Guy");
    // Call on the SUT (here PriceEngine)
    // No side effects -> Pure function
    val discount = PriceEngine.calculateDiscount(product1, product2);
    assertThat(discount).isEqualTo(0.02);
    }
    @Test
    void greet_a_user_should_send_an_email_to_it() {
    final var email = "[email protected]";
    final var emailGatewayMock = mock(EmailGateway.class);
    // Substitute collaborators with Test Double
    final var sut = new Controller(emailGatewayMock);
    sut.greetUser(email);
    // Verify that the SUT calls those collaborators correctly
    verify(emailGatewayMock, times(1)).sendGreetingsEmail(email);
    }
    @Test
    void it_should_add_given_product_to_the_order() {
    val product = new Product("Free Guy");
    val sut = new Order();
    sut.add(product);
    // Verify the state
    assertThat(sut.getProducts())
    .hasSize(1)
    .allMatch(item -> item.equals(product));
    }

    View full-size slide

  30. 3 styles compared
    Refactorings are harder with this approach
    If you change the interaction you need to change the tests as well

    View full-size slide

  31. 4 types of code
    All production code can be categorized along two dimensions:
    • Complexity or domain significance
    • The number of collaborators
    • Code complexity : defined by the number of decision-
    making (cyclomatic complexity for example)
    • Domain significance : how significant the code is for the
    problem domain of your project
    Number of collaborators : number of collaborators a class or
    a method has code with many collaborators is expensive to
    test

    View full-size slide

  32. 4 types of code
    • Domain model and algorithms : Complex code is often part of the domain model
    • Trivial code : Parameter less constructors / one-line properties: they have few (if any) collaborators and exhibit little complexity or domain
    significance
    • Controllers : This code doesn’t do complex or business critical work by itself but coordinates the work of other components like domain
    classes and external applications
    • Overcomplicated code : Such code scores highly on both metrics: it has a lot of collaborators, and it’s also complex or important.
    • Ex : fat controllers that don’t delegate complex work anywhere and do everything themselves
    Unit testing this gives the best
    return for your efforts
    Refactor overcomplicated
    code by splitting it into algorithms and controllers

    View full-size slide

  33. Unit vs Integration tests

    View full-size slide

  34. Integration tests

    View full-size slide

  35. Let’s practice – Anti-Patterns
    Open the clean-tests project :
    • Identify anti-patterns / code smells in the 3 test classes in the anti-patterns package – 10’
    • Collective debriefing – 10’
    • Let’s refactor – 15’
    • Collective debriefing – 10’
    https://github.com/ythirion/clean-tests

    View full-size slide

  36. @UtilityClass
    public class PriceEngine {
    public static Double calculateDiscount(Product... products) {
    val discount = products.length * 0.01;
    return min(discount, 0.2);
    }
    }
    class PriceEngineTests {
    @Test
    void discount_of_2_products_should_be_2_percent() {
    val products = List.of(new Product("P1") , new Product("P2"), new Product("P3"));
    val discount = PriceEngine.calculateDiscount(products.toArray(new Product[0]));
    assertThat(discount).isEqualTo(products.size() * 0.01);
    }
    }
    Anti-Patterns : Leaking algorithm implementation in tests
    Production code

    View full-size slide

  37. Anti-Patterns : Leaking algorithm implementation in tests
    Fine to use "hardcoded" values in test
    It is the expected result from our test case
    class PriceEngineRefactoredTests {
    @Test
    void discount_of_2_products_should_be_3_percent() {
    val products = List.of(new Product("P1"), new Product("P2"), new Product("P3"));
    val discount = PriceEngine.calculateDiscount(products.toArray(new Product[0]));
    assertThat(discount).isEqualTo(0.03);
    }
    }

    View full-size slide

  38. class TodoTests {
    @Test
    void it_should_call_search_on_repository_with_the_given_text() {
    val todoRepositoryMock = mock(TodoRepository.class);
    val todoService = new TodoService(todoRepositoryMock);
    val searchResults = List.of(
    new Todo("Create miro", "add code samples in the board"),
    new Todo("Add myths in miro", "add mythbusters from ppt in
    the board")
    );
    val searchedText = "miro";
    when(todoRepositoryMock.search(searchedText))
    .thenReturn(searchResults);
    val result = todoRepositoryMock.search(searchedText);
    assertThat(result).isEqualTo(searchResults);
    verify(todoRepositoryMock, times(1)).search(searchedText);
    }
    }
    Anti-Patterns : Test external libs in our tests
    The SUT is a mock…
    • Assert that the call on our mock returns what we setup
    • We test Mockito here...

    View full-size slide

  39. class TodoRefactoredTests {
    @Test
    void it_should_call_search_on_repository_with_the_given_text() {
    val todoRepositoryMock = mock(TodoRepository.class);
    val todoService = new TodoService(todoRepositoryMock);
    val searchResults = List.of(
    new Todo("Create miro", "add code samples in the board"),
    new Todo("Add myths in miro", "add mythbusters from ppt in the board")
    );
    val searchedText = "miro";
    when(todoRepositoryMock.search(searchedText))
    .thenReturn(searchResults);
    val result = todoService.search(searchedText);
    assertThat(result).isEqualTo(searchResults);
    verify(todoRepositoryMock, times(1)).search(searchedText);
    }
    }
    Anti-Patterns : Test external libs in our tests
    Never use a Mock as SUT
    If not familiar : use AAA in comments can help
    Call the true SUT :
    • In this simple version will only delegates call to Repository
    • Would have more responsibility in real life (authorization, quotas, filtering, ...)

    View full-size slide

  40. @Test
    void it_should_return_a_Right_for_valid_comment() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result = article.addComment("Amazing article !!!", "Pablo Escobar");
    assertThat(result.isRight()).isTrue();
    }
    @Test
    void it_should_add_a_comment_with_the_given_text() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val text = "Amazing article !!!";
    val result = article.addComment(text, "Pablo Escobar");
    assertThat(result.get().getComments())
    .hasSize(1)
    .anyMatch(comment -> comment.getText().equals(text));
    }
    @Test
    void it_should_add_a_comment_with_the_given_author() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val author = "Pablo Escobar";
    val result = article.addComment("Amazing article !!!", author);
    assertThat(result.get().getComments())
    .hasSize(1)
    .anyMatch(comment -> comment.getAuthor().equals(author));
    }
    @Test
    void it_should_add_a_comment_with_the_date_of_the_day() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result = article.addComment("Amazing article !!!", "Pablo Escobar");
    }
    Anti-Patterns : Only 1 assert / test
    4 tests to maintain but here we check a single behavior :
    Add a comment in an article
    Tests should be behavior oriented not data oriented

    View full-size slide

  41. @Test
    void it_should_return_a_Right_for_valid_comment() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result = article.addComment("Amazing article !!!", "Pablo Escobar");
    assertThat(result.isRight()).isTrue();
    }
    Anti-Patterns : Technical concepts in tests name
    @Test
    void it_should_return_a_Left_when_adding_existing_comment() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result =
    article.addComment("Amazing article !!!", "Pablo Escobar")
    .map(a -> a.addComment("Amazing article !!!", "Pablo Escobar"))
    .flatMap(r -> r);
    assertThat(result.isLeft()).isTrue();
    }
    What is a Left ?
    What is a Right ?

    View full-size slide

  42. @Test
    void it_should_return_a_Right_for_valid_comment() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result = article.addComment("Amazing article !!!", "Pablo Escobar");
    assertThat(result.isRight()).isTrue();
    }
    Anti-Patterns : ambiguity
    What is a valid comment ?

    View full-size slide

  43. Anti-Patterns : Missing assertions
    Tests without assertions do not provide any value
    Seeing a test failing is as important as
    seeing it passing
    @Test
    void it_should_add_a_comment_with_the_date_of_the_day() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result = article.addComment("Amazing article !!!", "Pablo Escobar");
    }

    View full-size slide

  44. @Test
    void it_should_add_a_comment_with_the_date_of_the_day() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result = article.addComment("Amazing article !!!", "Pablo Escobar");
    }
    @Test
    void it_should_return_a_Left_when_adding_existing_comment() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result =
    article.addComment("Amazing article !!!", "Pablo Escobar")
    .map(a -> a.addComment("Amazing article !!!", "Pablo Escobar"))
    .flatMap(r -> r);
    assertThat(result.isLeft()).isTrue();
    }
    class BlogTests {
    @Test
    void it_should_return_a_Right_for_valid_comment() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val result = article.addComment("Amazing article !!!", "Pablo Escobar");
    assertThat(result.isRight()).isTrue();
    }
    @Test
    void it_should_add_a_comment_with_the_given_text() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val text = "Amazing article !!!";
    val result = article.addComment(text, "Pablo Escobar");
    assertThat(result.get().getComments())
    .hasSize(1)
    .anyMatch(comment -> comment.getText().equals(text));
    }
    @Test
    void it_should_add_a_comment_with_the_given_author() {
    val article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    val author = "Pablo Escobar";
    val result = article.addComment("Amazing article !!!", author);
    assertThat(result.get().getComments())
    .hasSize(1)
    .anyMatch(comment -> comment.getAuthor().equals(author));
    }
    Anti-Patterns : Duplication everywhere
    Tests are code too proceed with the same care

    View full-size slide

  45. Anti-Patterns : Refactored BlogTests
    Removed duplication
    • If you need to instantiate a lot of object, centralize it in TestDataBuilders
    • If you change your models, it will be easier to maintain
    Step by step guide
    Check 1 behavior per test
    • Multiple assertions in test
    • Easier to read / understand / maintain
    • Help identify missing test cases : "add a new comment in an Article containing existing comments"
    Be careful with date
    • Non-deterministic data
    • Ideally handle it by passing a function / clock retrieving Date and Time
    Rename test to be more business oriented
    Avoid generic names like result, context, …
    class Blog_should {
    private final String text = "Amazing article !!!";
    private final String author = "Pablo Escobar";
    private Article article;
    private static void assertComment(Comment comment,
    String expectedText,
    String expectedAuthor) {
    assertThat(comment.getText()).isEqualTo(expectedText);
    assertThat(comment.getAuthor()).isEqualTo(expectedAuthor);
    assertThat(comment.getCreationDate()).isBeforeOrEqualTo(LocalDate.now());
    }
    @BeforeEach
    void init() {
    article = new Article(
    "Lorem Ipsum",
    "consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore"
    );
    }
    @Nested
    class add_a_new_comment {
    @Test
    void in_the_article_including_given_text_and_author() {
    val updatedArticle = article.addComment(text, author);
    assertThat(updatedArticle.isRight()).isTrue();
    assertComment(updatedArticle.get().getComments().head(), text, author);
    }
    @Test
    void in_an_article_containing_existing_ones() {
    val newText = "Finibus Bonorum et Malorum";
    val newAuthor = "Al Capone";
    val updatedArticle = article.addComment(text, author)
    .map(a -> a.addComment(newText, newAuthor))
    .flatMap(r -> r);
    assertThat(updatedArticle.isRight()).isTrue();
    assertThat(updatedArticle.get().getComments()).hasSize(2);
    assertComment(updatedArticle.get().getComments().last(), newText, newAuthor);
    }
    }
    @Nested
    class return_an_error {
    @Test
    void when_adding_an_existing_comment() {
    val updatedArticle = article.addComment(text, author)
    .map(a -> a.addComment(text, author))
    .flatMap(r -> r);
    assertThat(updatedArticle.isLeft()).isTrue();
    assertThat(updatedArticle.getLeft())
    .hasSize(1)
    .allMatch(error -> error.getDescription().equals("Comment already in the article"));
    }
    }
    }

    View full-size slide

  46. Anti-Patterns : Comment out failing tests
    A test that fails is a feedback loop for us
    Always apply these rules:
    q Has a regression been found by the test?
    Fix the code!
    q Is one of the assumptions of the test no longer valid?
    Delete it!!
    q Has the application really changed the functionality under test for a valid reason?
    Update the test!!
    Never comment out failing tests

    View full-size slide

  47. Anti-Patterns : The hunt to 100% code coverage

    View full-size slide

  48. Code Coverage
    A coverage metric shows how much source code a test suite executes, from none to 100%
    Can’t be used to effectively measure the
    quality of a test suite
    • Too little coverage in your code base -> 10% :
    • Demonstrate you are not testing enough
    • The reverse isn’t true:
    • Even 100% coverage isn’t a guarantee that you have a good-quality test suite
    Coverage metrics are a good negative indicator but a bad positive one
    𝐶𝑜𝑑𝑒 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 =
    Lines of code executed
    𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑙𝑖𝑛𝑒𝑠

    View full-size slide

  49. Code Coverage - demo
    𝐶𝑜𝑑𝑒 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 =
    Lines of code executed − 2
    𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑙𝑖𝑛𝑒𝑠 − 3
    public static boolean isLong(String input) {
    if(input.length() > 5) {
    return true;
    }
    return false;
    }
    class DemoTests {
    @Test
    void should_return_false_for_abc() {
    assertThat(Demo.isLong("abc")).isFalse();
    }
    }

    View full-size slide

  50. Code Coverage – refactor the code
    𝐶𝑜𝑑𝑒 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 =
    Lines of code executed − 3
    𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑙𝑖𝑛𝑒𝑠 − 3
    Test still verifies the same number of possible outcomes…
    But we are now at 100% coverage
    class DemoTests {
    @Test
    void should_return_false_for_abc() {
    assertThat(Demo.isLong("abc")).isFalse();
    }
    }
    public static boolean isLong(String input) {
    return input.length() > 5;
    }

    View full-size slide

  51. Any alternative ?

    View full-size slide

  52. Branch Coverage
    Focuses on control structures : if, match statements
    Shows how many of such control structures are traversed by at least one test in the suite
    𝐵𝑟𝑎𝑛𝑐ℎ 𝑐𝑜𝑣𝑒𝑟𝑎𝑔𝑒 =
    𝐵𝑟𝑎𝑛𝑐ℎ𝑒𝑠 𝑡𝑟𝑎𝑣𝑒𝑟𝑠𝑒𝑑
    𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑏𝑟𝑎𝑛𝑐ℎ𝑒𝑠
    2 paths
    length > 5 length <= 5
    50% coverage here
    class DemoTests {
    @Test
    void should_return_false_for_abc() {
    assertThat(Demo.isLong("abc")).isFalse();
    }
    }
    public static boolean isLong(String input) {
    return input.length() > 5;
    }

    View full-size slide

  53. Branch Coverage in IntelliJ
    • Sampling : collecting line coverage with
    negligible slowdown
    • Tracing : enables the accurate collection of
    the branch coverage

    View full-size slide

  54. Enough to check our test quality ?

    View full-size slide

  55. class DemoTests {
    @Test
    void should_return_false_for_abc() {
    assertThat(Demo.isLong("abc"));
    }
    }
    Problems with coverage
    We can’t guarantee that the test verifies all the possible outcomes of the system under test
    No coverage metric can consider code paths in external libraries
    Unit tests must have appropriate assertions
    Assertion-free testing

    View full-size slide

  56. Mutation Testing
    Test our tests by introducing MUTANTS (fault) into our production code during the test execution :
    To check that the test is failing
    If the test pass, there is an issue

    View full-size slide

  57. Best practices ?
    • Tests should test one thing only (one behavior)
    • Create more specific tests to drive more generic solution (triangulate)
    • Give your tests meaningful names (behavior/goal-oriented names) that reflect your
    business domain
    • Always see the test failes for the right reason
    Ensure you have meaningful feedback from failing tests
    • Keep your tests and production code separate
    Organize your tests to reflect your production code (similar project structure)

    View full-size slide

  58. Other testing topics
    • Test Driven Development : Test first development approach
    • Approval Testing : Less assertions / better quality
    • Mutation Testing : Measure the quality of your teste
    • Property-Based Testing : Write less tests for a better coverage
    • Consumer-Driven Contract Testing
    • Behavior-Driven Development
    • Acceptance Test Driven Development
    • …

    View full-size slide

  59. Conclusion
    • What is the most valuable stuff you have learned today ?
    • What do you need to apply those concepts in your day to day ?
    • Think about the latest tests you wrote, what could be improved in it ?
    • On the last problem you have encountered, how Unit Tests could have helped you ?

    View full-size slide

  60. To Go Further
    Unit Testing Principles, Practices and Patterns - Vladimir Khorikov

    View full-size slide