Your Unit Tests are Trying to tell you something @ DevoxxUK23

If tests are hard to write, the production design is crappy – goes an old saying. Indeed, writing unit tests gives you one of the most comprehensive, yet brutal, feedback about the design of your production code, but if it comes too late, many developers can’t stand it anymore and they will either stop testing or test more superficially. At the other extreme, others struggle to write contrived, fragile tests full of mocks that end up frustrating more than helping them. This talk reviews the main hints that unit tests provide you, from the most obvious improvements to some of the most subtle design principles.

Transcript

1. Your tests are trying
   to tell you something ...
   10 design hints you were missing
   ↓
   Read the article:
   https://victorrentea.ro/blog/design-insights-from-unit-testing/
   About the speaker:
   https://victorrentea.ro
   

   View Slide

2. 👉 victorrentea.ro/training-offer
   Hi, I'm Victor Rentea 🇷🇴 PhD(CS)
   Java Champion, 17 years of code, code, code, code, code....
   Consultant & Trainer: 5000 developers of 100+ companies in EU:
   ❤ Clean Code, Architecture, Unit Tes3ng
   🛠 Frameworks: Spring, Hibernate, WebFlux
   ⚡ Java Performance, Secure Coding 🔐
   Conference Speaker – dozens of recorded talks on YouTube
   Founder of European SoMware CraMers (5500+ members)
   🔥 Free monthly webinars, 1 hour a@er work 👉 victorrentea.ro/community
   Past events on youtube.com/vrentea
   Father of 👧👦, servant of a 🐈, weekend gardener 🌼
   h"ps://VictorRentea.ro
   

   View Slide

3. 3 VictorRentea.ro
   From the Agile Ideology ...
   Emergent Design
   while we keep shipping shit working so/ware fast,
   the design of the system will naturally evolve by itself
   (as opposed to large up-front design that caused overengineering in waterfall)
   

   View Slide

4. 4 VictorRentea.ro
   Wri$ng Tests
   tells you
   when to improve the design
   Emergent Design
   that never emerged
   😩
   We need triggers!
   When should I refine the design?
   

   View Slide

5. 5 VictorRentea.ro
   Kent Beck
   Creator of Extreme Programming (XP) in 1999
   the most technical style of Agile
   Inventor of TDD
   Author of JUnit
   Father of Unit Tes3ng
   

   View Slide

6. 6 VictorRentea.ro
   1. Passes all Tests 💪
   2. Expresses Intent = SRP, Domain Names
   2. No Duplica;on of Logic = DRY🌵
   3. Keep it Simple = KISS💋
   Rules of Simple Design
   by Kent Beck
   https://martinfowler.com/bliki/BeckDesignRules.html
   è Design Feedback 💎
   

   View Slide

7. 7 VictorRentea.ro
   

   View Slide

8. 8 VictorRentea.ro
   History of Tes,ng:
   Ice-Cream Cone
   What's this?
   

   View Slide

9. 9 VictorRentea.ro
   Testing Anti-Pattern
   30 manual testers
   🧔🧔🧔🧔 🧔🧔🧔
   🧔🧔🧔🧔🧔 ...
   20% end-to-end coverage
   using GUI robots, Selenium,..
   absent (old frameworks)
   scarce unit-tests, 10% coverage
   Ice-Cream Cone
   History of Tes,ng:
   Reality in many
   Successful Monoliths today:
   

   View Slide

10. 10 VictorRentea.ro
    https://martinfowler.com/articles/practical-test-pyramid.html
    E2E
    Tes1ng Pyramid
    Test a thin slice of behavior
    MOCKS
    Test a group of modules
    Test everything as whole
    ⭐ Deep Edge Case
    ⭐ Cri,cal Business Flow (eg. checkout)
    overlapping is expected
    surface is propor9onal to quan9ty
    

    View Slide

11. 11 VictorRentea.ro
    MOCKS
    💖 / 🤬
    

    View Slide

12. 12 VictorRentea.ro
    Why we 💖 Mocks
    Isolated Tests
    from external systems
    Fast 👑
    no framework, DB, external API
    Test Less Logic
    when tes$ng high complexity 😵💫 è
    Alterna$ves:
    - In-mem DB
    - Testcontainers 🐳
    - WireMock, ..
    🤨 Chea%ng↓
    * James Coplien in h/ps://rbcs-us.com/documents/Why-Most-Unit-Tes?ng-is-Waste.pdf
    

    View Slide

13. 13 VictorRentea.ro
    public computePrices(!!...) {
    !// A
    for (Product product : products) { +1
    !// B
    if (price !== null) { +1
    !// C
    }
    for (Coupon coupon : customer.getCoupons()) { +1
    if (coupon.autoApply() +1
    !&& coupon.isApplicableFor(product, price) +1
    !&& !usedCoupons.contains(coupon)) { +1
    !// D
    }
    }
    }
    return !!...;
    }
    Code Complexity - Cycloma(c
    - Cogni(ve (Sonar)
    

    View Slide

14. 14 VictorRentea.ro
    Logic
    under test
    Tes5ng Complex Code
    Many execu$on paths through code => lots of tests
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    CC=5 CC=6
    f(a) g(b)
    calls
    f() calling g() together have a CC of max ...
    Too Many
    (to cover all branches)
    Too Heavy
    (setup and input data)
    Integra%on Tests for f+g are...
    30
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    Test
    

    View Slide

15. 15 VictorRentea.ro
    MOCK
    

    View Slide

16. 16 VictorRentea.ro
    Why we 🤬 Mocks
    Uncaught Bugs 😱
    despite 1000s of GREEN✅ tests: lock, tap, doors, umbrella 🤣
    Fragile Tests 💔
    that break at any refactoring
    Unreadable Tests 😵💫
    eg. a test using 5+ mocks è
    

    View Slide

17. 17 VictorRentea.ro
    WHAT AM I TESTING HERE ? syndrome
    

    View Slide

18. 18 VictorRentea.ro
    Test has 20 lines full of mocks
    😵💫
    😡
    BURN THE TEST!
    bad cost/benefit ra,o
    HONEYCOMB TESTS
    Integra,on test this!
    WHAT AM I TESTING HERE ?
    syndrome
    Tested prod code has 4 lines
    😩
    f(..) {
    a = api.fetchB(repo1.find(..).getBId());
    d = service.createD(a,b,repo2.find(..));
    repo3.save(d);
    mq.send(d.id);
    }
    g(dto) {
    repo2.save(mapper.fromDto(dto, repo1.find(..)));
    }
    SIMPLIFY PRODUCTION
    Collapse Middle-Man (useless method)
    

    View Slide

19. 19 VictorRentea.ro
    https://martinfowler.com/articles/practical-test-pyramid.html
    End-to-end
    The Tes7ng Pyramid
    For Monoliths
    

    View Slide

20. 20 VictorRentea.ro
    §The Legacy Monolith ("Big ball of mud") has
    - Terrible complexity behind few entry points
    è Many Unit Tests are needed for the deeper corners of logic è
    §Tes(ng Microservices is different
    - Huge complexity in a single microservice = bad practice è Break it !
    è More APIs, hiding a decent amount of complexity (🙏)
    è Possible to test more at the API level = Honeycomb Tes0ng Strategy
    

    View Slide

21. 21 VictorRentea.ro
    Integra4on
    test one microservice endpoint end-to-end
    Integrated
    test the en0re ecosystem
    Implementa4on
    Detail
    Start up all microservices (in dockers / staging)
    Expensive, slow, flaky tests
    Required for business-criJcal flows (eg. checkout)
    Start up one microservice
    Use for: Every flow of the system
    Keep tests isolated without mocks
    Instan0ate several classes.
    Use for: naturally isolated parts of code with high complexity.
    Mocks are tolerated (but sJll avoided):
    Test components with clear roles using social unit tests
    Honeycomb Tes7ng Strategy
    Testcontainers 🐳
    WireMock Contract
    Tests
    (Pact, SCC)
    DB ES Kafka ...
    API
    many tests on
    one entry point
    h,ps://engineering.atspo8fy.com/2018/01/tes8ng-of-microservices/
    complexity
    decouple and
    test in isola/on
    

    View Slide

22. 22 VictorRentea.ro
    Test manageable complexity without mocks
    #0 Integra.on
    test one flow end-to-end
    

    View Slide

23. 23 VictorRentea.ro
    Give you most Design Feedback 💎
    More Complexity => BeDer Design
    Implementa%on
    Detail Tests
    👍
    MOCKS
    

    View Slide

24. 24 VictorRentea.ro
    The dawn of Mocks (2004)
    

    View Slide

25. 25 VictorRentea.ro
    Mock Roles, not Objects
    h"p://jmock.org/oopsla2004.pdf
    The dawn of Mocks (2004)
    

    View Slide

26. 26 VictorRentea.ro
    The dawn of Mocks (2004)
    

    View Slide

27. 27 VictorRentea.ro
    BAD HABIT
    Mock Roles, not Objects
    h1p://jmock.org/oopsla2004.pdf
    You implement a new feature
    > ((click in UI/postman)) > It works > 🎉
    Oups!! I forgot about unit-tests 😱
    ...then you write unit tests
    mocking all the dependencies
    of the prod code you wrote
    ↓
    Few years later:
    "My tests are fragile and impede refactoring!"
    Contract-Driven Design
    Before mocking a dependency,
    clarify its responsibility
    =
    Changing an API you mocked is painful
    😭
    

    View Slide

28. 28 VictorRentea.ro
    write
    Social Unit Tests
    for "components" (groups of objects) with clear responsibili8es
    A B
    ✅ Internal refactoring won't break these tests
    Instead of solitary fine-grained unit tests mocking every dependency,
    

    View Slide

29. 29 VictorRentea.ro
    "Unit Tes=ng means mocking all dependencies of a class"
    - common belief
    WRONG!
    "It's perfectly fine for unit tests to talk to databases and filesystems!"- Ian Cooper in his Talk
    Unit Tes$ng
    = ?
    Robust Unit Tes$ng requires
    iden$fying responsibili$es
    

    View Slide

30. 30 VictorRentea.ro
    Unit Tes>ng tells you to simplify
    methods and data structures
    #1
    

    View Slide

31. 31 VictorRentea.ro
    var bigObj = new BigObj();
    bigObj.setA(a);
    bugObj.setB(b);
    prod.method(bigObj);
    Tests must create bigObj just to pass two inputs🫤
    method(bigObj)
    MUTABLE
    DATA 😱
    in 2023?
    using only 2 of the 15 fields in bigObj
    method(a, b)
    Precise Signatures prod.method(a, b);
    Also, simpler tests:
    Pass only necessary data to func$ons ✅
    when(bigObj.getA()).thenReturn(a);
    ⛔ Don't Mock Ge_ers ⛔
    ✅ Mock behavior, not data
    prod.method(new ABC(a, b, c));
    method(abc)
    Parameter Object
    When tes$ng highly complex logic, introduce a
    ⛔ Don't return Mocks from Mocks⛔
    when(bigObj.getA()).thenReturn(mockA);
    OMG!
    They sBck!
    

    View Slide

32. 32 VictorRentea.ro
    🏰
    Constrained Objects
    = data structures that guard their internal consistency by throwing excep$ons,
    (eg required fields)
    Ø Mutable (eg Domain En$$es, Aggregates)
    Ø Immutable❤ (Value Objects)
    

    View Slide

33. 33 VictorRentea.ro
    ❷ A group of classes has a
    clear role and manageable complexity
    ❶ A Constrained Object🏰 gets Large
    Object Mother Pa?ern
    TestData.aCustomer(): Customer (valid)
    coupling
    Break Domain En77es
    in separate Bounded Contexts
    packages > modules > microservices
    invoicing | shipping
    è Tes$ng the en$re group with a social unit-test✅
    requires heavier data inputs
    * h1ps://mar/nfowler.com/bliki/ObjectMother.html (2006)
    Same Object Mother used in different ver+cals
    invoicing | shipping
    è Split Object Mother per ver4cal
    InvoicingTestData | ShippingTestData
    Crea$ng valid test data gets cumbersome
    CREEPY
    A large class shared by many tests
    Don't change it. Add methods, or tweak results
    ✅ TestData.charles(): Customer (a persona)
    

    View Slide

34. 34 VictorRentea.ro
    Your complex logic directly uses
    External APIs or heavy libraries:
    Unit-tes$ng your logic
    requires understanding the seman$cs of:
    The External API: to populate/assert DTOs
    The Library: to mock it
    ... externalApi.call(apiDetails);
    ... dto.getStrangeField()
    ... Lib.use(mysteriousParam,...)
    Unit Tests should speak your Domain Model
    (mock the Adapter when tes/ng complex domain logic)
    Unit Tests are first-class ci$zens of your project
    #respect your tests
    Agnos7c Domain
    Isolate complex logic from the outside world
    ... clientAdapter.call(domainStuff)
    ... domainObject.getMyField()
    ... libAdapter.use(😊)
    

    View Slide

35. 35 VictorRentea.ro
    applica3on / infra
    Value Object
    En0ty
    id
    Domain
    Service
    Domain
    Service
    agnos/c
    domain
    My DTOs
    External
    API
    External
    DTOs
    Client
    External
    Systems
    Applica0on
    Service
    Controller
    Repo
    Clean, Pragma/c Architecture
    at Devoxx Ukraine 2021
    Curious for more?
    ↓
    IAdapter
    Adapter
    Dependency
    Inversion
    ⛔
    ⛔
    Simplified Onion Architecture
    Interf
    W
    rapper
    f
    Dependency
    Inversion
    Ugly
    Invasive
    Library
    Domain complexity is kept inside
    for easier tes3ng
    

    View Slide

36. 36 VictorRentea.ro
    #1
    Unit Tes>ng encourages
    Precise Signatures
    Tailored Data Structures
    Agnos>c Domain
    

    View Slide

37. 37 VictorRentea.ro
    #2
    Unit Tes>ng puts design pressure on
    Highly Complex Logic
    🧠
    

    View Slide

38. 38 VictorRentea.ro
    class Big {
    f() { //complex
    g();
    }
    g() { //complex
    }
    }
    Inside the same class,
    a complex func$on f()
    calls a complex g()
    g() is complex => unit-tested separately
    When tes$ng f(), can I avoid entering g()?
    Can I mock a local method call?
    class HighLevel {
    LowLevel low;
    f() {//complex
    low.g();
    }
    } class LowLevel {
    g() {/*complex*/}
    }
    ↓
    Par(al Mock (@Spy)
    Hard to maintain tests:
    Which method is real, which is mocked?🤯
    Split by Layers of Abstrac7on
    (high-level policy vs low-level details)
    Tolerable tes+ng Legacy Code
    If splijng the class doesn't feel right,
    test f() + g() together with bigger tests
    

    View Slide

39. 39 VictorRentea.ro
    class HighLevel {
    LowLevel low;
    f() {//complex
    low.g();
    }
    }
    Split by Layers of Abstrac7on
    = ver%cal split of a class
    class LowLevel {
    g() {/*complex*/}
    }
    

    View Slide

40. 40 VictorRentea.ro
    class Wide {
    A a;
    B b; //+more dependencies
    f() {..a.a()..}
    g() {..b.b()..}
    }
    @ExtendWith(MockitoExtension)
    class WideTest {
    @Mock A a;
    @Mock B b;
    @InjectMocks Wide wide;
    // 5 tests for f()
    // 4 tests for g()
    }
    ↓
    Split test class in ComplexFTest, ..G..
    (rule: the before should be used by all tests)
    Complex methods in the same class
    use different sets of dependencies:
    class ComplexF {
    A a;
    f() {
    ..a.a()..
    }
    }
    class ComplexG {
    B b;
    g() {
    ..b.b()..
    }
    }
    @BeforeEach
    void fixture() {
    when(a.a()).then...
    when(b.b()).then...
    }
    Split Unrelated Complexity
    Later: what part of the before
    is used by my failed test?
    ** Mockito (since v2.0) throws UnnecessaryStubbingException if a when..then is not used by a @Test, when using MockitoExtension
    = Unmaintainable Tests
    🤔
    FIXTURE CREEP
    test setup
    DRY
    https://www.davidvlijmincx.com/posts/setting_the_strictness_for_mockito_mocks/
    not used by these tests
    

    View Slide

41. 41 VictorRentea.ro
    Split Unrelated Complexity
    class ComplexF {
    A a;
    f() {
    ..a.a()..
    }
    }
    class ComplexG {
    B b;
    g() {
    ..b.b()..
    }
    }
    horizontally ↔
    

    View Slide

42. 42 VictorRentea.ro
    ver%cally ↕
    Tests help us to break complexity
    horizontally ↔
    When should we follow those hints?
    clear roles
    

    View Slide

43. 43 VictorRentea.ro
    The dawn of Mocks (2004):
    

    View Slide

44. 44 VictorRentea.ro
    ver%cally ↕
    Tests help us to break complexity
    horizontally ↔
    clear roles
    #2
    by
    

    View Slide

45. 45 VictorRentea.ro
    #3
    Unit Tes>ng promotes
    Func.onal Programming
    pure func%ons & immutable objects
    

    View Slide

46. 46 VictorRentea.ro
    1) Has no Side-Effects
    (doesn't change anything)
    INSERT, POST, send message, field changes, files
    2) Returns Same Output for Same Inputs
    (no external source of data)
    GET, SELECT, current 8me, random, …
    Pure Func)on
    aka "Referen?al Transparency"
    Just compute a value
    𝑒𝑔: 𝑀𝑎𝑡ℎ𝑒𝑚𝑎𝑡𝑖𝑐𝑎𝑙 𝐹𝑢𝑛𝑐𝑡𝑖𝑜𝑛𝑠: 𝑓(𝑥,𝑦)=𝑥^2+𝑦
    

    View Slide

47. 47 VictorRentea.ro
    No Network or files
    No Changes to Data
    No .me/random
    Pure Func)ons
    immutable objects❤
    (a simplified definiJon)
    

    View Slide

48. 49 VictorRentea.ro
    a = repo1.findById(..)
    b = repo2.findById(..)
    c = api.call(..)
    🤯complexity🤯
    repo3.save(d);
    mq.send(d.id);
    Complex logic
    using many dependencies
    (eg: computePrice, applyDiscounts)
    Many tests
    using lots of mocks
    when(..).thenReturn(a);
    when(..).thenReturn(b);
    when(..).thenReturn(c);
    prod.complexAndCoupled();
    verify(..).save(captor);
    d = captor.get();
    assertThat(d)...
    verify(..).send(...);
    x15=😖
    ✅ Easier to test, with less mocks
    d = prod.pure(a,b,c);
    assertThat(d)...
    Reduce Coupling of Complex Logic
    ✅ Easier to understand
    D pure(a,b,c) {
    🤯complexity🤯
    return d;
    }
    

    View Slide

49. 50 © VictorRentea.ro
    a training by
    Complexity
    Func%onal Core
    Impera%ve Shell / Func%onal Core Segrega%on
    State Muta3on
    DB
    Impera%ve Shell
    API call
    Files
    Dependencies
    Complex Logic
    

    View Slide

50. 51 © VictorRentea.ro
    a training by
    Func%onal
    Core
    Impera%ve Shell
    Impera%ve Shell / Func%onal Core Segrega%on
    Extract heaviest complexity
    as pure func5ons
    

    View Slide

51. 52 VictorRentea.ro
    method(Mutable order, discounts) {
    ds.applyDiscounts(order, discounts);
    var price = cs.computePrice(order);
    return price;
    }
    ... but you use mutable objects
    Swapping two lines
    causes bugs in produc@on ❌
    despite 4000 ✅ tests
    You have 4.000 unit tests,
    100% test coverage 😲
    👏
    ↓
    Paranoid Tes@ng
    (verify method call order)
    Immutable Objects
    method(Immutable order, d) {
    var discountedOrder = ds.applyDiscounts(order, d);
    var price = cs.computePrice(discountedOrder);
    return price;
    }
    TEMPORAL COUPLING
    Swapping the two lines
    ❌ breaks compila4on
    

    View Slide

52. 54 VictorRentea.ro
    1. Collapse Middle-Man vs "What am I tes,ng here?" Syndrome
    2. Honeycomb Tes7ng Strategy: more Integra,on Tests than Fragile Unit Tests
    3. Precise Signatures: less arguments
    4. Dedicated Data Structures vs Creepy Object Mother
    5. Agnos7c Domain vs using APIs or Libraries in complex logic
    6. Split Complexity by Layers of Abstrac7on ↕ vs Par,al Mocks (@Spy)
    7. Split Unrelated Complexity ↔ vs Fixture Creep (bloated setup)
    8. Clarify Roles, Social Unit Tests vs blindly @Mock all dependencies
    9. Decouple Complexity in Pure Func7ons vs Tests full of mocks
    10.Immutable Objects vs Temporal Coupling
    Design Hints from Tests
    

    View Slide

53. 55 VictorRentea.ro
    Testable Design
    is Good Design
    

    View Slide

54. 56 VictorRentea.ro
    >meframe for developing your feature
    When do you start wri%ng tests?
    ✅ understand the problem => early ques3ons to biz
    ✅ early design feedback 💎 💎 💎
    ✅ real test coverage => courage to refactor later
    BDD
    (.feature)
    too late
    TDD Good enough
    TDD!
    

    View Slide

55. Wri%ng unit tests early
    increases fric%on with careless design
    

    View Slide

56. 58 VictorRentea.ro
    Unit Tes%ng Reading Guide (for later)
    1] Classic TDD⭐⭐⭐ (mock-less) https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530
    Mock Roles, not Objects ⭐⭐⭐: http://jmock.org/oopsla2004.pdf
    "Is TDD Dead?" https://martinfowler.com/articles/is-tdd-dead/
    Why Most Unit Testing is Waste (James Coplien): https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf
    vs Integrated Tests are a Scam(J Brains): https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam
    2] London TDD⭐⭐⭐ (mockist) https://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627
    3] Patterns⭐ https://www.amazon.com/Art-Unit-Testing-examples/dp/1617290890
    4] https://www.amazon.com/xUnit-Test-Patterns-Refactoring-Code/dp/0131495054/
    5] (skip through) https://www.amazon.com/Unit-Testing-Principles-Practices-Patterns
    

    View Slide

57. Your tests are trying to tell you something ...
    10 design hints you were missing
    Read the article:
    https://victorrentea.ro/blog/design-insights-from-unit-testing/
    About the speaker:
    https://victorrentea.ro
    at è
    Stay connected. Join:
    

    View Slide