Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Design insights from unit tests @ itkonekt 2023

Design insights from unit tests @ itkonekt 2023

If tests are hard to write, the production design is crappy - goes an old saying. Indeed, writing unit tests gives you one of the most comprehensive, yet brutal, feedback about the design of your production code, but if it comes too late, many developers can’t stand it anymore and they will either stop testing or test more superficially. At the other extreme, others struggle to write contrived, fragile tests full of mocks that end up frustrating more than helping them. This talk reviews the main hints that unit tests provide you, from the most obvious improvements to some of the most subtle design principles.

Victor Rentea

May 21, 2023
Tweet

More Decks by Victor Rentea

Other Decks in Technology

Transcript

  1. Your tests are trying
    to tell you something ...
    10 design hints you were missing

    Read the article:
    https://victorrentea.ro/blog/design-insights-from-unit-testing/
    Ways Victor can help you:
    https://victorrentea.ro
    by Victor Rentea

    View full-size slide

  2. 👉 victorrentea.ro/training-offer
    Hi, I'm Victor Rentea 🇷🇴 PhD(CS)
    Java Champion, 17 years of code, code, code, code, code....
    Consultant & Trainer: 5000 developers of 100+ companies in EU:
    ❤ Clean Code, Architecture, Unit Tes3ng
    🛠 Frameworks: Spring, Hibernate, WebFlux
    ⚡ Java Performance, Secure Coding 🔐
    Conference Speaker – dozens of recorded talks on YouTube
    Founder of European SoMware CraMers (5500+ members)
    🔥 Free monthly webinars, 1 hour a@er work 👉 victorrentea.ro/community
    Past events on youtube.com/vrentea
    Father of 👧👦, servant of a 🐈, weekend gardener 🌼
    h"ps://VictorRentea.ro

    View full-size slide

  3. 263
    From the Agile (2001) Ideology ...
    Emergent Design
    While we keep shipping shit,
    the design of the system design will naturally improve
    (as opposed to large up-front design that caused overengineering in waterfall)
    working so8ware

    View full-size slide

  4. 264
    Wri$ng Unit Tests
    gives you those triggers!
    Emergent Design
    that never emerged
    We need triggers
    to start improving the design!
    I f t e s t s a r e h a r d t o w r i t e ,
    t h e d e s i g n s u c k s

    View full-size slide

  5. 265
    Kent Beck
    Creator of Extreme Programming (XP) in 1999
    the most technical style of Agile
    Inventor of TDD
    Author of JUnit
    Father of Unit Tes3ng

    View full-size slide

  6. 266
    1. Passes all Tests 💪
    2. Expresses Intent = SRP, Domain Names
    2. No Duplica;on of Logic = DRY🌵
    3. Keep it Simple = KISS💋
    Rules of Simple Design
    by Kent Beck
    https://martinfowler.com/bliki/BeckDesignRules.html
    è Design Feedback 💎

    View full-size slide

  7. 270 https://martinfowler.com/articles/practical-test-pyramid.html
    E2E
    Tes1ng Pyramid
    Test a thin slice of behavior
    MOCKS
    Test a group of modules
    Test everything as whole
    ⭐ Deep Edge Case
    ⭐ Cri5cal Business Flow (eg. checkout)
    overlapping is expected
    surface is propor?onal to quan?ty

    View full-size slide

  8. 271
    MOCKS
    💖 / 🤬

    View full-size slide

  9. 272
    Why we 💖 Mocks
    Isolated Tests
    from external systems
    Fast 👑
    no framework, DB, external API
    Test Less Logic
    when tes7ng high complexity 😵💫 è
    Alterna7ves:
    - In-mem DB
    - Testcontainers 🐳
    - WireMock, ..

    View full-size slide

  10. 273
    public computePrices(!!...) {
    !// A
    for (Product product : products) { +1
    !// B
    if (price !== null) { +1
    !// C
    }
    for (Coupon coupon : customer.getCoupons()) { +1
    if (coupon.autoApply() +1
    !&& coupon.isApplicableFor(product, price) +1
    !&& !usedCoupons.contains(coupon)) { +1
    !// D
    }
    }
    }
    return !!...;
    }
    Code Complexity - Cycloma(c
    - Cogni(ve (Sonar)

    View full-size slide

  11. 276
    Why we 💖 Mocks
    Isolated Tests
    from external systems
    Fast 👑
    no framework, DB, external API
    Test Less Logic
    when tes7ng high complexity 😵💫 è
    Alterna7ves:
    - In-mem DB
    - Testcontainers 🐳
    - WireMock, ..
    🤨 Chea&ng
    James Coplien in hEps://rbcs-us.com/documents/Why-Most-Unit-Tes7ng-is-Waste.pdf

    View full-size slide

  12. 277
    Why we 🤬 Mocks
    Uncaught Bugs 😱
    despite 1000s of ✅GREEN tests
    (lock, tap, doors, umbrella)
    Fragile Tests 💔
    that break at any refactoring
    Unreadable Tests 😵💫
    eg. a test using ≥ 5 mocks è

    View full-size slide

  13. 280
    WHAT AM I TESTING HERE ? syndrome

    View full-size slide

  14. 281
    Test has 20 lines full of mocks
    😵💫
    😡
    BURN THE TEST!
    bad cost/benefit ra5o
    HONEYCOMB TESTS
    Integra5on test this!
    WHAT AM I TESTING HERE ?
    syndrome
    Tested prod code has 4 lines
    😩
    f(..) {
    a = api.fetchB(repo1.find(..).getBId());
    d = service.createD(a,b,repo2.find(..));
    repo3.save(d);
    mq.send(d.id);
    }
    g(dto) {
    repo2.save(mapper.fromDto(dto, repo1.find(..)));
    }
    SIMPLIFY PRODUCTION
    "Remove Middle-Man" (inline method)

    View full-size slide

  15. 282 https://martinfowler.com/articles/practical-test-pyramid.html
    End-to-end
    The Tes7ng Pyramid
    For Monoliths

    View full-size slide

  16. 283
    §The Legacy Monolith has
    - Terrible complexity behind only a few entry points
    è Only Unit Tests can cover deep corners of logic è
    §Microservices expose more APIs (test surfaces)
    ... hold less complexity, but more configura7on
    è Test more at API-level (aka Integra7on) = Honeycomb Tes$ng Strategy

    View full-size slide

  17. 284
    Integra3on
    (one microservice)
    Integrated
    (en-re ecosystem)
    Start up all microservices (in dockers / staging)
    Expensive, slow, flaky tests
    Required for business-cri5cal flows (eg. checkout)
    Cover one microservice (eg @SpringBootTest)
    Use for: Every flow of the system ⭐⭐⭐
    Keep tests isolated without mocks
    Cover 1..few classes (Social Unit Testsè)
    Use for: naturally isolated pieces with high complexity.
    @Mock are allowed 🤨
    Honeycomb Tes,ng Strategy
    Testcontainers 🐳
    WireMock
    &co
    Contract
    Tests
    (Pact, SCC)
    DB ES Kafka ...
    API
    many tests on
    one entry point
    h,ps://engineering.atspo8fy.com/2018/01/tes8ng-of-microservices/
    Test DSL
    (helper code)
    Implementa$on Detail
    (a role)
    complexity
    decouple and
    test in isola

    View full-size slide

  18. 286
    Test manageable complexity without mocks
    #0 Integra,on Test
    one flow end-to-end
    The hard part: When you change a 7ny part, understand and test the whole flow 😱
    eg: don't test alone simple mappers (toDto)

    View full-size slide

  19. 287
    Unit Tests give you most Design Feedback 💎
    More Complexity => BeBer Design
    Implementa&on
    Detail Tests
    👍
    MOCKS
    complexity

    View full-size slide

  20. 288
    The Dawn of Mocks
    (2004)

    View full-size slide

  21. 289
    The Dawn of Mocks
    (2004)
    h7p://jmock.org/oopsla2004.pdf

    View full-size slide

  22. 290
    BAD HABIT
    Mock Roles, not Objects
    h=p://jmock.org/oopsla2004.pdf
    You implement a new feature
    > ((click in UI/postman)) > It works > 🎉
    Oups!! I forgot about unit-tests 😱
    ...then you write unit tests
    mocking all the dependencies
    of the prod code you wrote

    Few years later:
    "My tests are fragile and impede refactoring!"
    Contract-Driven Design
    Before mocking a dependency,
    clarify its responsibility
    =
    AZer you mock an API ❄
    changing it = pain
    😭

    View full-size slide

  23. 291
    write
    Social Unit Tests
    for "components" (groups of objects) with clear responsibiliJes
    A B
    ✅ Internal Refactoring won't break such tests
    Instead of fine-grained Solitary Unit Tests tesJng each class in isolaJon,

    View full-size slide

  24. 292
    "Unit Tes5ng means mocking all dependencies of a class"
    - common belief
    WRONG!
    "It's perfectly fine for unit tests to talk to databases and filesystems!"- Ian Cooper in TDD, Where Did It All Go Wrong
    Unit Tes7ng
    = ?
    Robust Unit TesJng requires
    idenJfying responsibiliJes

    View full-size slide

  25. 294
    Unit Tesmethods and data structures
    #1

    View full-size slide

  26. 295
    var bigObj = new BigObj();
    bigObj.setA(a);
    bugObj.setB(b);
    prod.method(bigObj);
    Tests must create bigObj just to pass two inputs🫤
    method(bigObj)
    MUTABLE
    DATA 😱
    in 2023?
    using only 2 of the 15 fields in bigObj
    method(a, b)
    Precise Signatures prod.method(a, b);
    Also, simpler tests:
    Pass only necessary data to func7ons ✅
    when(bigObj.getA()).thenReturn(a);
    ⛔ Don't Mock GeEers ⛔
    ✅ Mock behavior, not data
    prod.horror(new ABC(a, b, c));
    horror(abc)
    Parameter Object
    class ABC {a,b,c}
    When tes7ng highly complex logic, introduce a
    ⛔ Don't return Mocks from Mocks⛔
    when(bigObj.getA()).thenReturn(mockA);

    View full-size slide

  27. 296
    Constrained Objects
    = data structures that throw excep7ons on invalid data
    (eg required fields)
    Ø Mutable (eg Domain En77es, Aggregates)
    Ø Immutable❤ (Value Objects)
    🏰

    View full-size slide

  28. 297
    ❶ A Constrained Object gets Large

    Object Mother PaCern
    TestData.aCustomer(): Customer (valid)
    coupling
    Break Domain En77es
    InvoicingCustomer | ShippingCustomer
    in separate Bounded Contexts
    packages > modules > microservices
    ❷ Integra$on and Social Unit Tests❤
    require heavy data inputs
    * h=ps://marSame Object Mother used in different ver+cals
    invoicing | shipping

    Split Object Mother per ver$cal
    InvoicingTestData | ShippingTestData
    Crea7ng valid test data gets cumbersome
    CREEPY
    A large class shared by many tests
    Don't change it! Just add methods, or tweak results
    ✅ TestData.charles(): Customer (a persona)
    🏰

    View full-size slide

  29. 299
    Tests require detailed understanding of:
    External APIs: to populate/assert DTOs
    The Library: to mock it
    ... externalApi.call(apiDetails);
    ... externalDto.getFooField()
    ... Lib.use(mysteriousParam,...)
    Tests are expressed in your Domain
    #respect your tests
    Agnos7c Domain
    Isolate complex logic from external world
    via Adapters
    ... clientAdapter.call(domainStuff)
    ... domainObject.getMyField()
    ... libWrapper.use(😊)
    Your complex logic directly uses
    External APIs or heavy libraries:

    View full-size slide

  30. 300
    applica3on / infra
    My DTOs
    External
    API
    External
    DTOs
    Client
    External
    Systems
    ApplicaHon
    Service
    Simplified Onion Architecture
    (more in my "Clean Pragmaat Devoxx Ukraine 2021 on YouTube)
    Value Object
    EnHty
    id
    Domain
    Service
    Domain
    Service
    agnos5c
    domain
    Repo
    IAdapter
    Adapter
    Dependency
    Inversion


    IWrapper
    W
    rapper
    Dependency
    Inversion
    Ugly
    Invasive
    Library
    Domain complexity kept inside

    View full-size slide

  31. 301
    #1
    Unit TesPrecise Signatures
    Tailored Data Structures
    Agnos

    View full-size slide

  32. 302
    #2 Unit TesHighly Complex Logic
    🧠

    View full-size slide

  33. 303
    class Big {
    f() { //complex
    g();
    }
    g() { //complex
    }
    }
    Inside the same class,
    a complex func7on f()
    calls a complex g()
    g() is complex => unit-tested separately
    When tes7ng f(), can I avoid entering g()?
    Can I mock a local method call?
    class HighLevel {
    LowLevel low;
    f() {//complex
    low.g();
    }
    } class LowLevel {
    g() {/*complex*/}
    }

    Par(al Mock (@Spy)
    Hard to maintain tests:
    Which method is real, which is mocked?🤯
    Split by Layers of Abstrac7on
    (high-level policy vs low-level details)
    Only when tes?ng Legacy Code
    If splijng the class doesn't feel right,
    test f() + g() together with social unit tests

    View full-size slide

  34. 304
    class HighLevel {
    LowLevel low;
    f() {//complex
    low.g();
    }
    }
    Split by Layers of Abstrac7on
    = ver&cal split of a class
    class LowLevel {
    g() {/*complex*/}
    }
    horizontal è

    View full-size slide

  35. 305
    class Wide {
    A a;
    B b; //+more dependencies
    f() {..a.a()..}
    g() {..b.b()..}
    }
    @ExtendWith(MockitoExtension)
    class WideTest {
    @Mock A a;
    @Mock B b;
    @InjectMocks Wide wide;
    // 5 tests for f()
    // 7 tests for g()
    }

    Split test class in FTest, GTest
    (rule: the @Before should be used by all tests)
    Unrelated complex methods in the same class
    use different sets of dependencies:
    class ComplexF {
    A a;
    f() {
    ..a.a()..
    }
    }
    class ComplexG {
    B b;
    g() {
    ..b.b()..
    }
    }
    @BeforeEach
    void sharedFixture() {
    when(a.a()).then...
    when(b.b()).then...
    }
    Split Unrelated Complexity
    Later: what part of the before
    is used by my failed test?
    ** Mockito (since v2.0) throws UnnecessaryStubbingException if a when..then is not used by a @Test, when using MockitoExtension
    = Unmaintainable Tests
    🤔
    FIXTURE CREEP
    test setup
    DRY
    https://www.davidvlijmincx.com/posts/setting_the_strictness_for_mockito_mocks/
    not used by these tests

    View full-size slide

  36. 306
    Split Unrelated Complexity
    class ComplexF {
    A a;
    f() {
    ..a.a()..
    }
    }
    class ComplexG {
    B b;
    g() {
    ..b.b()..
    }
    }
    horizontally ↔

    View full-size slide

  37. 309
    ver&cally ↕
    Tests help us to break complexity
    horizontally ↔
    #2
    clear roles
    by

    View full-size slide

  38. 310
    #3
    Unit TesFunc,onal Programming
    pure func&ons & immutable objects

    View full-size slide

  39. 313
    1) Has no Side-Effects
    (doesn't change anything)
    INSERT, POST, send message, field changes, files
    2) Returns Same Output for Same Inputs
    (no external data source)
    GET, SELECT, Jme, random, …
    Pure Func)on
    aka "Referen*al Transparency"
    Just compute a value
    𝑒𝑔: 𝑀𝑎𝑡ℎ𝑒𝑚𝑎𝑡𝑖𝑐𝑎𝑙 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠: 𝑓(𝑥,𝑦) = 𝑥2 + 𝑦

    View full-size slide

  40. 314
    No Network or files
    No Changes to Data
    No ,me/random
    Pure Func)ons
    immutable objects❤
    (simplified defini5on)

    View full-size slide

  41. 316
    a = repo1.findById(..)
    b = repo2.findById(..)
    c = api.call(..)
    🤯complexity🤯
    repo3.save(d);
    mq.send(d.id);
    Complex logic
    using many dependencies
    (eg: computePrice, applyDiscounts)
    Many tests
    using lots of mocks
    when(..).thenReturn(a);
    when(..).thenReturn(b);
    when(..).thenReturn(c);
    prod.complexAndCoupled();
    verify(..).save(captor);
    d = captor.get();
    assertThat(d)...
    verify(..).send(...);
    x 15 tests =😖
    ✅ Simpler tests (less mocks)
    d = prod.pure(a,b,c);
    assertThat(d)...
    Reduce Coupling of Complex Logic
    ✅ Simpler produc7on code
    D pure(a,b,c) {
    🤯complexity🤯
    return d;
    }
    5 mocks

    View full-size slide

  42. 317 © VictorRentea.ro
    a training by
    Complexity
    State Muta3on
    DB
    Impera7ve Shell
    API call
    Files
    Dependencies
    Complex Logic
    Func7onal Core

    View full-size slide

  43. 318 © VictorRentea.ro
    a training by
    Complexity
    State Muta3on
    DB
    API call
    Files
    Dependencies
    Complex Logic
    / Segrega7on
    Impera7ve Shell Func7onal Core

    View full-size slide

  44. 319 © VictorRentea.ro
    a training by
    Func&onal
    Core
    Impera&ve Shell
    Extract heaviest complexity
    as pure funcJons
    / Segrega7on
    Impera7ve Shell Func7onal Core

    View full-size slide

  45. 320
    method(Mutable order, discounts) {
    ds.applyDiscounts(order, discounts);
    var price = cs.computePrice(order);
    return price;
    }
    ... but you use mutable objects
    Swapping two lines
    causes bugs in produc7on ❌
    despite 4000 ✅ tests
    You have 4.000 unit tests,
    100% test coverage 😲
    👏

    Paranoid Tes7ng
    (InOrder can verify method call order)
    Immutable Objects
    method(Immutable order, d) {
    var discountedOrder = ds.applyDiscounts(order, d);
    var price = cs.computePrice(discountedOrder);
    return price;
    }
    TEMPORAL COUPLING
    Swapping the two lines
    ❌ breaks compila$on

    View full-size slide

  46. 322
    1. Collapse Middle-Man vs "What am I tes,ng here?" Syndrome or test more:
    2. Honeycomb Tes7ng Strategy: more Integra,on Tests over Fragile Unit Tests
    3. Precise Signatures: simpler arguments, be?er names
    4. Dedicated Data Structures vs Creepy Object Mother
    5. Agnos7c Domain vs using external APIs or Libraries in complex logic
    6. Split Complexity by Layers of Abstrac7on ↕ vs Par,al Mocks (@Spy)
    7. Split Unrelated Complexity ↔ vs Fixture Creep (bloated @Before)
    8. Clarify Roles, Social Unit Tests vs blindly @Mock all dependencies
    9. Decouple Complexity in Pure Func7ons vs Many tests full of mocks
    10.Immutable Objects vs Temporal Coupling
    Design Hints from Tests

    View full-size slide

  47. 323
    Testable Design
    is Good Design

    View full-size slide

  48. 324
    When do you start wri0ng tests?
    ✅ understand the problem => early ques3ons to biz
    ✅ early design feedback 💎 💎 💎
    ✅ test coverage => courage to refactor later
    BDD
    (.feature)
    too late
    TDD Good enough
    TDD!

    View full-size slide

  49. Wri0ng unit tests early
    increases fric0on with careless design

    View full-size slide

  50. 326
    Unit Tes0ng Reading Guide (for later)
    1] Classic TDD⭐⭐⭐ (mock-less) https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
    Mock Roles, not Objects ⭐⭐⭐: http://jmock.org/oopsla2004.pdf
    "Is TDD Dead?" https://martinfowler.com/articles/is-tdd-dead/
    Why Most Unit Testing is Waste (James Coplien): https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf
    vs Integrated Tests are a Scam(J Brains): https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam
    2] London TDD⭐⭐⭐ (mockist) https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627
    3] Patterns⭐ https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890
    4] https://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/
    5] (skip through) https://www.amazon.com/Unit-Testing-Principles-Practices-Patterns

    View full-size slide

  51. Your tests are trying to tell you something ...
    10 design hints you were missing
    Read the article:
    https://victorrentea.ro/blog/design-insights-from-unit-testing/
    Ways Victor can help you::
    https://victorrentea.ro
    Stay connected, and join us:

    View full-size slide