Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Your Unit Tests are Trying to tell you something @ DevoxxUK23

Your Unit Tests are Trying to tell you something @ DevoxxUK23

If tests are hard to write, the production design is crappy – goes an old saying. Indeed, writing unit tests gives you one of the most comprehensive, yet brutal, feedback about the design of your production code, but if it comes too late, many developers can’t stand it anymore and they will either stop testing or test more superficially. At the other extreme, others struggle to write contrived, fragile tests full of mocks that end up frustrating more than helping them. This talk reviews the main hints that unit tests provide you, from the most obvious improvements to some of the most subtle design principles.

Victor Rentea

May 10, 2023
Tweet

More Decks by Victor Rentea

Other Decks in Technology

Transcript

  1. Your tests are trying
    to tell you something ...
    10 design hints you were missing

    Read the article:
    https://victorrentea.ro/blog/design-insights-from-unit-testing/
    About the speaker:
    https://victorrentea.ro

    View Slide

  2. 👉 victorrentea.ro/training-offer
    Hi, I'm Victor Rentea 🇷🇴 PhD(CS)
    Java Champion, 17 years of code, code, code, code, code....
    Consultant & Trainer: 5000 developers of 100+ companies in EU:
    ❤ Clean Code, Architecture, Unit Tes3ng
    🛠 Frameworks: Spring, Hibernate, WebFlux
    ⚡ Java Performance, Secure Coding 🔐
    Conference Speaker – dozens of recorded talks on YouTube
    Founder of European SoMware CraMers (5500+ members)
    🔥 Free monthly webinars, 1 hour a@er work 👉 victorrentea.ro/community
    Past events on youtube.com/vrentea
    Father of 👧👦, servant of a 🐈, weekend gardener 🌼
    h"ps://VictorRentea.ro

    View Slide

  3. 3 VictorRentea.ro
    From the Agile Ideology ...
    Emergent Design
    while we keep shipping shit working so/ware fast,
    the design of the system will naturally evolve by itself
    (as opposed to large up-front design that caused overengineering in waterfall)

    View Slide

  4. 4 VictorRentea.ro
    Wri$ng Tests
    tells you
    when to improve the design
    Emergent Design
    that never emerged
    😩
    We need triggers!
    When should I refine the design?

    View Slide

  5. 5 VictorRentea.ro
    Kent Beck
    Creator of Extreme Programming (XP) in 1999
    the most technical style of Agile
    Inventor of TDD
    Author of JUnit
    Father of Unit Tes3ng

    View Slide

  6. 6 VictorRentea.ro
    1. Passes all Tests 💪
    2. Expresses Intent = SRP, Domain Names
    2. No Duplica;on of Logic = DRY🌵
    3. Keep it Simple = KISS💋
    Rules of Simple Design
    by Kent Beck
    https://martinfowler.com/bliki/BeckDesignRules.html
    è Design Feedback 💎

    View Slide

  7. 7 VictorRentea.ro

    View Slide

  8. 8 VictorRentea.ro
    History of Tes,ng:
    Ice-Cream Cone
    What's this?

    View Slide

  9. 9 VictorRentea.ro
    Testing Anti-Pattern
    30 manual testers
    🧔🧔🧔🧔 🧔🧔🧔
    🧔🧔🧔🧔🧔 ...
    20% end-to-end coverage
    using GUI robots, Selenium,..
    absent (old frameworks)
    scarce unit-tests, 10% coverage
    Ice-Cream Cone
    History of Tes,ng:
    Reality in many
    Successful Monoliths today:

    View Slide

  10. 10 VictorRentea.ro
    https://martinfowler.com/articles/practical-test-pyramid.html
    E2E
    Tes1ng Pyramid
    Test a thin slice of behavior
    MOCKS
    Test a group of modules
    Test everything as whole
    ⭐ Deep Edge Case
    ⭐ Cri,cal Business Flow (eg. checkout)
    overlapping is expected
    surface is propor9onal to quan9ty

    View Slide

  11. 11 VictorRentea.ro
    MOCKS
    💖 / 🤬

    View Slide

  12. 12 VictorRentea.ro
    Why we 💖 Mocks
    Isolated Tests
    from external systems
    Fast 👑
    no framework, DB, external API
    Test Less Logic
    when tes$ng high complexity 😵💫 è
    Alterna$ves:
    - In-mem DB
    - Testcontainers 🐳
    - WireMock, ..
    🤨 Chea%ng↓
    * James Coplien in h/ps://rbcs-us.com/documents/Why-Most-Unit-Tes?ng-is-Waste.pdf

    View Slide

  13. 13 VictorRentea.ro
    public computePrices(!!...) {
    !// A
    for (Product product : products) { +1
    !// B
    if (price !== null) { +1
    !// C
    }
    for (Coupon coupon : customer.getCoupons()) { +1
    if (coupon.autoApply() +1
    !&& coupon.isApplicableFor(product, price) +1
    !&& !usedCoupons.contains(coupon)) { +1
    !// D
    }
    }
    }
    return !!...;
    }
    Code Complexity - Cycloma(c
    - Cogni(ve (Sonar)

    View Slide

  14. 14 VictorRentea.ro
    Logic
    under test
    Tes5ng Complex Code
    Many execu$on paths through code => lots of tests
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    CC=5 CC=6
    f(a) g(b)
    calls
    f() calling g() together have a CC of max ...
    Too Many
    (to cover all branches)
    Too Heavy
    (setup and input data)
    Integra%on Tests for f+g are...
    30
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test

    View Slide

  15. 15 VictorRentea.ro
    MOCK

    View Slide

  16. 16 VictorRentea.ro
    Why we 🤬 Mocks
    Uncaught Bugs 😱
    despite 1000s of GREEN✅ tests: lock, tap, doors, umbrella 🤣
    Fragile Tests 💔
    that break at any refactoring
    Unreadable Tests 😵💫
    eg. a test using 5+ mocks è

    View Slide

  17. 17 VictorRentea.ro
    WHAT AM I TESTING HERE ? syndrome

    View Slide

  18. 18 VictorRentea.ro
    Test has 20 lines full of mocks
    😵💫
    😡
    BURN THE TEST!
    bad cost/benefit ra,o
    HONEYCOMB TESTS
    Integra,on test this!
    WHAT AM I TESTING HERE ?
    syndrome
    Tested prod code has 4 lines
    😩
    f(..) {
    a = api.fetchB(repo1.find(..).getBId());
    d = service.createD(a,b,repo2.find(..));
    repo3.save(d);
    mq.send(d.id);
    }
    g(dto) {
    repo2.save(mapper.fromDto(dto, repo1.find(..)));
    }
    SIMPLIFY PRODUCTION
    Collapse Middle-Man (useless method)

    View Slide

  19. 19 VictorRentea.ro
    https://martinfowler.com/articles/practical-test-pyramid.html
    End-to-end
    The Tes7ng Pyramid
    For Monoliths

    View Slide

  20. 20 VictorRentea.ro
    §The Legacy Monolith ("Big ball of mud") has
    - Terrible complexity behind few entry points
    è Many Unit Tests are needed for the deeper corners of logic è
    §Tes(ng Microservices is different
    - Huge complexity in a single microservice = bad practice è Break it !
    è More APIs, hiding a decent amount of complexity (🙏)
    è Possible to test more at the API level = Honeycomb Tes0ng Strategy

    View Slide

  21. 21 VictorRentea.ro
    Integra4on
    test one microservice endpoint end-to-end
    Integrated
    test the en0re ecosystem
    Implementa4on
    Detail
    Start up all microservices (in dockers / staging)
    Expensive, slow, flaky tests
    Required for business-criJcal flows (eg. checkout)
    Start up one microservice
    Use for: Every flow of the system
    Keep tests isolated without mocks
    Instan0ate several classes.
    Use for: naturally isolated parts of code with high complexity.
    Mocks are tolerated (but sJll avoided):
    Test components with clear roles using social unit tests
    Honeycomb Tes7ng Strategy
    Testcontainers 🐳
    WireMock Contract
    Tests
    (Pact, SCC)
    DB ES Kafka ...
    API
    many tests on
    one entry point
    h,ps://engineering.atspo8fy.com/2018/01/tes8ng-of-microservices/
    complexity
    decouple and
    test in isola/on

    View Slide

  22. 22 VictorRentea.ro
    Test manageable complexity without mocks
    #0 Integra.on
    test one flow end-to-end

    View Slide

  23. 23 VictorRentea.ro
    Give you most Design Feedback 💎
    More Complexity => BeDer Design
    Implementa%on
    Detail Tests
    👍
    MOCKS

    View Slide

  24. 24 VictorRentea.ro
    The dawn of Mocks (2004)

    View Slide

  25. 25 VictorRentea.ro
    Mock Roles, not Objects
    h"p://jmock.org/oopsla2004.pdf
    The dawn of Mocks (2004)

    View Slide

  26. 26 VictorRentea.ro
    The dawn of Mocks (2004)

    View Slide

  27. 27 VictorRentea.ro
    BAD HABIT
    Mock Roles, not Objects
    h1p://jmock.org/oopsla2004.pdf
    You implement a new feature
    > ((click in UI/postman)) > It works > 🎉
    Oups!! I forgot about unit-tests 😱
    ...then you write unit tests
    mocking all the dependencies
    of the prod code you wrote

    Few years later:
    "My tests are fragile and impede refactoring!"
    Contract-Driven Design
    Before mocking a dependency,
    clarify its responsibility
    =
    Changing an API you mocked is painful
    😭

    View Slide

  28. 28 VictorRentea.ro
    write
    Social Unit Tests
    for "components" (groups of objects) with clear responsibili8es
    A B
    ✅ Internal refactoring won't break these tests
    Instead of solitary fine-grained unit tests mocking every dependency,

    View Slide

  29. 29 VictorRentea.ro
    "Unit Tes=ng means mocking all dependencies of a class"
    - common belief
    WRONG!
    "It's perfectly fine for unit tests to talk to databases and filesystems!"- Ian Cooper in his Talk
    Unit Tes$ng
    = ?
    Robust Unit Tes$ng requires
    iden$fying responsibili$es

    View Slide

  30. 30 VictorRentea.ro
    Unit Tes>ng tells you to simplify
    methods and data structures
    #1

    View Slide

  31. 31 VictorRentea.ro
    var bigObj = new BigObj();
    bigObj.setA(a);
    bugObj.setB(b);
    prod.method(bigObj);
    Tests must create bigObj just to pass two inputs🫤
    method(bigObj)
    MUTABLE
    DATA 😱
    in 2023?
    using only 2 of the 15 fields in bigObj
    method(a, b)
    Precise Signatures prod.method(a, b);
    Also, simpler tests:
    Pass only necessary data to func$ons ✅
    when(bigObj.getA()).thenReturn(a);
    ⛔ Don't Mock Ge_ers ⛔
    ✅ Mock behavior, not data
    prod.method(new ABC(a, b, c));
    method(abc)
    Parameter Object
    When tes$ng highly complex logic, introduce a
    ⛔ Don't return Mocks from Mocks⛔
    when(bigObj.getA()).thenReturn(mockA);
    OMG!
    They sBck!

    View Slide

  32. 32 VictorRentea.ro
    🏰
    Constrained Objects
    = data structures that guard their internal consistency by throwing excep$ons,
    (eg required fields)
    Ø Mutable (eg Domain En$$es, Aggregates)
    Ø Immutable❤ (Value Objects)

    View Slide

  33. 33 VictorRentea.ro
    ❷ A group of classes has a
    clear role and manageable complexity
    ❶ A Constrained Object🏰 gets Large
    Object Mother Pa?ern
    TestData.aCustomer(): Customer (valid)
    coupling
    Break Domain En77es
    in separate Bounded Contexts
    packages > modules > microservices
    invoicing | shipping
    è Tes$ng the en$re group with a social unit-test✅
    requires heavier data inputs
    * h1ps://mar/nfowler.com/bliki/ObjectMother.html (2006)
    Same Object Mother used in different ver+cals
    invoicing | shipping
    è Split Object Mother per ver4cal
    InvoicingTestData | ShippingTestData
    Crea$ng valid test data gets cumbersome
    CREEPY
    A large class shared by many tests
    Don't change it. Add methods, or tweak results
    ✅ TestData.charles(): Customer (a persona)

    View Slide

  34. 34 VictorRentea.ro
    Your complex logic directly uses
    External APIs or heavy libraries:
    Unit-tes$ng your logic
    requires understanding the seman$cs of:
    The External API: to populate/assert DTOs
    The Library: to mock it
    ... externalApi.call(apiDetails);
    ... dto.getStrangeField()
    ... Lib.use(mysteriousParam,...)
    Unit Tests should speak your Domain Model
    (mock the Adapter when tes/ng complex domain logic)
    Unit Tests are first-class ci$zens of your project
    #respect your tests
    Agnos7c Domain
    Isolate complex logic from the outside world
    ... clientAdapter.call(domainStuff)
    ... domainObject.getMyField()
    ... libAdapter.use(😊)

    View Slide

  35. 35 VictorRentea.ro
    applica3on / infra
    Value Object
    En0ty
    id
    Domain
    Service
    Domain
    Service
    agnos/c
    domain
    My DTOs
    External
    API
    External
    DTOs
    Client
    External
    Systems
    Applica0on
    Service
    Controller
    Repo
    Clean, Pragma/c Architecture
    at Devoxx Ukraine 2021
    Curious for more?

    IAdapter
    Adapter
    Dependency
    Inversion


    Simplified Onion Architecture
    Interf
    W
    rapper
    f
    Dependency
    Inversion
    Ugly
    Invasive
    Library
    Domain complexity is kept inside
    for easier tes3ng

    View Slide

  36. 36 VictorRentea.ro
    #1
    Unit Tes>ng encourages
    Precise Signatures
    Tailored Data Structures
    Agnos>c Domain

    View Slide

  37. 37 VictorRentea.ro
    #2
    Unit Tes>ng puts design pressure on
    Highly Complex Logic
    🧠

    View Slide

  38. 38 VictorRentea.ro
    class Big {
    f() { //complex
    g();
    }
    g() { //complex
    }
    }
    Inside the same class,
    a complex func$on f()
    calls a complex g()
    g() is complex => unit-tested separately
    When tes$ng f(), can I avoid entering g()?
    Can I mock a local method call?
    class HighLevel {
    LowLevel low;
    f() {//complex
    low.g();
    }
    } class LowLevel {
    g() {/*complex*/}
    }

    Par(al Mock (@Spy)
    Hard to maintain tests:
    Which method is real, which is mocked?🤯
    Split by Layers of Abstrac7on
    (high-level policy vs low-level details)
    Tolerable tes+ng Legacy Code
    If splijng the class doesn't feel right,
    test f() + g() together with bigger tests

    View Slide

  39. 39 VictorRentea.ro
    class HighLevel {
    LowLevel low;
    f() {//complex
    low.g();
    }
    }
    Split by Layers of Abstrac7on
    = ver%cal split of a class
    class LowLevel {
    g() {/*complex*/}
    }

    View Slide

  40. 40 VictorRentea.ro
    class Wide {
    A a;
    B b; //+more dependencies
    f() {..a.a()..}
    g() {..b.b()..}
    }
    @ExtendWith(MockitoExtension)
    class WideTest {
    @Mock A a;
    @Mock B b;
    @InjectMocks Wide wide;
    // 5 tests for f()
    // 4 tests for g()
    }

    Split test class in ComplexFTest, ..G..
    (rule: the before should be used by all tests)
    Complex methods in the same class
    use different sets of dependencies:
    class ComplexF {
    A a;
    f() {
    ..a.a()..
    }
    }
    class ComplexG {
    B b;
    g() {
    ..b.b()..
    }
    }
    @BeforeEach
    void fixture() {
    when(a.a()).then...
    when(b.b()).then...
    }
    Split Unrelated Complexity
    Later: what part of the before
    is used by my failed test?
    ** Mockito (since v2.0) throws UnnecessaryStubbingException if a when..then is not used by a @Test, when using MockitoExtension
    = Unmaintainable Tests
    🤔
    FIXTURE CREEP
    test setup
    DRY
    https://www.davidvlijmincx.com/posts/setting_the_strictness_for_mockito_mocks/
    not used by these tests

    View Slide

  41. 41 VictorRentea.ro
    Split Unrelated Complexity
    class ComplexF {
    A a;
    f() {
    ..a.a()..
    }
    }
    class ComplexG {
    B b;
    g() {
    ..b.b()..
    }
    }
    horizontally ↔

    View Slide

  42. 42 VictorRentea.ro
    ver%cally ↕
    Tests help us to break complexity
    horizontally ↔
    When should we follow those hints?
    clear roles

    View Slide

  43. 43 VictorRentea.ro
    The dawn of Mocks (2004):

    View Slide

  44. 44 VictorRentea.ro
    ver%cally ↕
    Tests help us to break complexity
    horizontally ↔
    clear roles
    #2
    by

    View Slide

  45. 45 VictorRentea.ro
    #3
    Unit Tes>ng promotes
    Func.onal Programming
    pure func%ons & immutable objects

    View Slide

  46. 46 VictorRentea.ro
    1) Has no Side-Effects
    (doesn't change anything)
    INSERT, POST, send message, field changes, files
    2) Returns Same Output for Same Inputs
    (no external source of data)
    GET, SELECT, current 8me, random, …
    Pure Func)on
    aka "Referen?al Transparency"
    Just compute a value
    𝑒𝑔: 𝑀𝑎𝑡ℎ𝑒𝑚𝑎𝑡𝑖𝑐𝑎𝑙 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠: 𝑓(𝑥,𝑦)=𝑥^2+𝑦

    View Slide

  47. 47 VictorRentea.ro
    No Network or files
    No Changes to Data
    No .me/random
    Pure Func)ons
    immutable objects❤
    (a simplified definiJon)

    View Slide

  48. 49 VictorRentea.ro
    a = repo1.findById(..)
    b = repo2.findById(..)
    c = api.call(..)
    🤯complexity🤯
    repo3.save(d);
    mq.send(d.id);
    Complex logic
    using many dependencies
    (eg: computePrice, applyDiscounts)
    Many tests
    using lots of mocks
    when(..).thenReturn(a);
    when(..).thenReturn(b);
    when(..).thenReturn(c);
    prod.complexAndCoupled();
    verify(..).save(captor);
    d = captor.get();
    assertThat(d)...
    verify(..).send(...);
    x15=😖
    ✅ Easier to test, with less mocks
    d = prod.pure(a,b,c);
    assertThat(d)...
    Reduce Coupling of Complex Logic
    ✅ Easier to understand
    D pure(a,b,c) {
    🤯complexity🤯
    return d;
    }

    View Slide

  49. 50 © VictorRentea.ro
    a training by
    Complexity
    Func%onal Core
    Impera%ve Shell / Func%onal Core Segrega%on
    State Muta3on
    DB
    Impera%ve Shell
    API call
    Files
    Dependencies
    Complex Logic

    View Slide

  50. 51 © VictorRentea.ro
    a training by
    Func%onal
    Core
    Impera%ve Shell
    Impera%ve Shell / Func%onal Core Segrega%on
    Extract heaviest complexity
    as pure func5ons

    View Slide

  51. 52 VictorRentea.ro
    method(Mutable order, discounts) {
    ds.applyDiscounts(order, discounts);
    var price = cs.computePrice(order);
    return price;
    }
    ... but you use mutable objects
    Swapping two lines
    causes bugs in produc@on ❌
    despite 4000 ✅ tests
    You have 4.000 unit tests,
    100% test coverage 😲
    👏

    Paranoid Tes@ng
    (verify method call order)
    Immutable Objects
    method(Immutable order, d) {
    var discountedOrder = ds.applyDiscounts(order, d);
    var price = cs.computePrice(discountedOrder);
    return price;
    }
    TEMPORAL COUPLING
    Swapping the two lines
    ❌ breaks compila4on

    View Slide

  52. 54 VictorRentea.ro
    1. Collapse Middle-Man vs "What am I tes,ng here?" Syndrome
    2. Honeycomb Tes7ng Strategy: more Integra,on Tests than Fragile Unit Tests
    3. Precise Signatures: less arguments
    4. Dedicated Data Structures vs Creepy Object Mother
    5. Agnos7c Domain vs using APIs or Libraries in complex logic
    6. Split Complexity by Layers of Abstrac7on ↕ vs Par,al Mocks (@Spy)
    7. Split Unrelated Complexity ↔ vs Fixture Creep (bloated setup)
    8. Clarify Roles, Social Unit Tests vs blindly @Mock all dependencies
    9. Decouple Complexity in Pure Func7ons vs Tests full of mocks
    10.Immutable Objects vs Temporal Coupling
    Design Hints from Tests

    View Slide

  53. 55 VictorRentea.ro
    Testable Design
    is Good Design

    View Slide

  54. 56 VictorRentea.ro
    >meframe for developing your feature
    When do you start wri%ng tests?
    ✅ understand the problem => early ques3ons to biz
    ✅ early design feedback 💎 💎 💎
    ✅ real test coverage => courage to refactor later
    BDD
    (.feature)
    too late
    TDD Good enough
    TDD!

    View Slide

  55. Wri%ng unit tests early
    increases fric%on with careless design

    View Slide

  56. 58 VictorRentea.ro
    Unit Tes%ng Reading Guide (for later)
    1] Classic TDD⭐⭐⭐ (mock-less) https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
    Mock Roles, not Objects ⭐⭐⭐: http://jmock.org/oopsla2004.pdf
    "Is TDD Dead?" https://martinfowler.com/articles/is-tdd-dead/
    Why Most Unit Testing is Waste (James Coplien): https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf
    vs Integrated Tests are a Scam(J Brains): https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam
    2] London TDD⭐⭐⭐ (mockist) https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627
    3] Patterns⭐ https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890
    4] https://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/
    5] (skip through) https://www.amazon.com/Unit-Testing-Principles-Practices-Patterns

    View Slide

  57. Your tests are trying to tell you something ...
    10 design hints you were missing
    Read the article:
    https://victorrentea.ro/blog/design-insights-from-unit-testing/
    About the speaker:
    https://victorrentea.ro
    at è
    Stay connected. Join:

    View Slide