Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Are Your Tests Slowing You Down?

Trisha Gee
October 03, 2023

Are Your Tests Slowing You Down?

Testing is a Good Thing, right? Especially automated testing. But "Good things come to those who wait" is not something that's going to appeal to the busy developer. You want results, and you want them now. You're in The Zone working on a problem, and the last thing you want is to break your flow wrestling with your testing framework or waiting for the tests to finish running.

More code means more tests. More coverage means more tests. More tests mean more time. Time that you want to spend being productive, creative, innovative. How can you balance the need for quality with the need for speed?

In this talk, Trisha will identify issues that slow down developers when writing, running and debugging tests, and look at tools that can help developers with each of these problems. There will be live coding, analysis of social media poll results, an overview of solutions in this space, "best practice" recommendations, and machine learning will be mentioned at some point.

Trisha Gee

October 03, 2023
Tweet

More Decks by Trisha Gee

Other Decks in Technology

Transcript

  1. Are Your Tests Slowing You Down?
    Trisha Gee

    View full-size slide

  2. https://trishagee.com/books/

    View full-size slide

  3. Gradle is Pioneering DPE
    Developer Productivity Engineering
    is a software development practice
    to maximise developer productivity
    and happiness.

    View full-size slide

  4. Tests Make You More Productive?
    Tests Make You More Productive

    View full-size slide

  5. So it’s OK not to write tests, right?

    View full-size slide

  6. “Testing is Xanax for programmers, it’s an anxiety
    reducing technique.”
    https://youtu.be/guycIP56YeY


    Kent Beck

    View full-size slide

  7. RATED 6+


    because apparently 5-year-olds fall asleep during the presentation


    Some flashing GIF sequences may affect photosensitive viewers


    This program contains product placement

    View full-size slide

  8. “Nothing is ever really lost to us as long as we
    remember it.”
    L.M. Montgomery


    View full-size slide

  9. What slows down your testing?

    View full-size slide

  10. 1. Writing Tests Takes Time

    View full-size slide

  11. “Trisha, you’re actually a tester”
    Liz Keogh (@lunivore)


    View full-size slide

  12. Use Your IDE

    View full-size slide

  13. Buy My Book!

    View full-size slide

  14. AI Will Make You Super Productive

    View full-size slide

  15. Beware, AI lies to you

    View full-size slide

  16. Bring a friend

    View full-size slide

  17. “We recommend that the developers, who are good at
    writing efficient, maintainable code, work together
    with the testers, who are good at specifying test cases,
    to automate tests.”
    Agile Testing Condensed: A Brief Introduction


    Lisa Crispin, Janet Gregory


    View full-size slide

  18. https://bit.ly/3RG85vc

    View full-size slide

  19. “Creativity is a wild mind and a disciplined eye.”
    Dorothy Parker


    View full-size slide

  20. Invest the time in writing to save time troubleshooting

    View full-size slide

  21. 2. Troubleshooting Takes Time

    View full-size slide

  22. What is this test anyway?

    View full-size slide

  23. Is it the test that’s wrong, or
    production code?

    View full-size slide

  24. Let me see…

    View full-size slide

  25. Let me see…

    View full-size slide

  26. Aim for fast feedback and easy
    reproduction

    View full-size slide

  27. “99.9% of the time the best way to troubleshoot a test
    is to re-write it so it’s smaller and faster.”
    Israel Boza Rodriguez (@IsraKaos)


    View full-size slide

  28. I have made the decision to trust you
    Body text


    View full-size slide

  29. I have made the decision to trust you
    Body text


    View full-size slide

  30. Fix your flaky tests!!!

    View full-size slide

  31. https://youtu.be/vHBzZHE4tJ0

    View full-size slide

  32. 3. Running Tests Takes Time

    View full-size slide

  33. How do you speed up your tests?

    View full-size slide

  34. Use a build cache

    View full-size slide

  35. Very effective for multi-module projects

    View full-size slide

  36. No cache: Full build required

    View full-size slide

  37. Build cache: Changing a leaf module

    View full-size slide

  38. Build cache: Changing a module’s implementation

    View full-size slide

  39. Build cache: Changing core implementation

    View full-size slide

  40. Remote Build Cache

    View full-size slide

  41. Run tests in parallel (local)

    View full-size slide

  42. Run tests in parallel (CI)

    View full-size slide

  43. Test Distribution

    View full-size slide

  44. Test Distribution Results
    ‑ ~50%
    ‑ ~50%
    ‑ ~50%
    Doubling the number of executors cuts build time in half

    View full-size slide

  45. Netflix reduced a 62-minute test cycle time down to just under 5 minutes!

    View full-size slide

  46. Let’s use AI!

    View full-size slide

  47. “The test selection procedure selects fewer than a third


    of the tests that would be selected on the basis of build


    dependencies”
    Predictive Test Selection - Meta Research


    https://research.facebook.com/publications/predictive-test-selection/


    Mateusz Machalica, Alex Samylkin, Meredith Porth, Satish Chandra


    View full-size slide

  48. Predictive Test Selection time savings

    View full-size slide

  49. But Trisha, I just have So Many Tests!

    View full-size slide

  50. “Yes, I regularly delete tests that no longer add unique
    value.”
    Angie Jones (@techgirl1908)

    View full-size slide

  51. Like any performance problem,


    focus on the worst-performers first

    View full-size slide

  52. “Once things like databases become a hindrance,
    optimise the testing loop, mock things out to run things
    fast. You can run locally and you don't have all the same
    infra dependencies as someone else.”
    On DevEx, PaaS, and Testing in Production https://bit.ly/3ZA0uQM


    Kelsey Hightower

    View full-size slide

  53. Look out for expensive setup and tear
    down

    View full-size slide

  54. Respect your test code

    View full-size slide

  55. ⬢ Separation of concerns
    ⬢ Encapsulation
    ⬢ SOLID
    ⬢ DRY
    ⬢ Descriptive naming
    ⬢ Modularisation
    ⬢ Designing for performance

    View full-size slide

  56. Tension between these ideals

    View full-size slide

  57. “Tidying is software design addressing you, your
    relationship to your code, and ultimately your
    relationship with yourself…. Tidying is geek self-care.”
    Tidy First?


    Kent Beck


    View full-size slide

  58. Automate as much as possible

    View full-size slide

  59. There are solutions out there designed for exactly this problem

    View full-size slide

  60. There is a time investment in writing tests,


    because there should be thinking involved

    View full-size slide

  61. “Your best and wisest refuge from all troubles is in your
    science.”
    Ada Lovelace


    View full-size slide