Are Your Tests Slowing You Down?

Testing is a Good Thing, right? Especially automated testing. But "Good things come to those who wait" is not something that's going to appeal to the busy developer. You want results, and you want them now. You're in The Zone working on a problem, and the last thing you want is to break your flow wrestling with your testing framework or waiting for the tests to finish running.

More code means more tests. More coverage means more tests. More tests mean more time. Time that you want to spend being productive, creative, innovative. How can you balance the need for quality with the need for speed?

In this talk, Trisha will identify issues that slow down developers when writing, running and debugging tests, and look at tools that can help developers with each of these problems. There will be live coding, analysis of social media poll results, an overview of solutions in this space, "best practice" recommendations, and machine learning will be mentioned at some point.

Transcript

1. Are Your Tests Slowing You Down? Trisha Gee
2. https://trishagee.com/books/
3. Gradle is Pioneering DPE Developer Productivity Engineering is a software development practice to maximise developer productivity and happiness.
5. Tests Make You More Productive? Tests Make You More Productive
10. So it’s OK not to write tests, right?
11. “Testing is Xanax for programmers, it’s an anxiety reducing technique.” https://youtu.be/guycIP56YeY Kent Beck
12. Caveats
13. RATED 6+ because apparently 5-year-olds fall asleep during the presentation Some flashing GIF sequences may affect photosensitive viewers This program contains product placement
14. “Nothing is ever really lost to us as long as we remember it.” L.M. Montgomery
15. What slows down your testing?
16. 1. Writing Tests Takes Time
17. “Trisha, you’re actually a tester” Liz Keogh (@lunivore)
18. Use Your IDE
19. Buy My Book!
21. AI Will Make You Super Productive
24. Beware, AI lies to you
25. Bring a friend
26. “We recommend that the developers, who are good at writing efficient, maintainable code, work together with the testers, who are good at specifying test cases, to automate tests.” Agile Testing Condensed: A Brief Introduction Lisa Crispin, Janet Gregory
27. https://bit.ly/3RG85vc
28. “Creativity is a wild mind and a disciplined eye.” Dorothy Parker
29. Invest the time in writing to save time troubleshooting
30. 2. Troubleshooting Takes Time
31. What is this test anyway?
35. Is it the test that’s wrong, or production code?
36. Let me see…
37. Let me see…
39. Aim for fast feedback and easy reproduction
41. “99.9% of the time the best way to troubleshoot a test is to re-write it so it’s smaller and faster.” Israel Boza Rodriguez (@IsraKaos)
42. Flaky Tests
43. I have made the decision to trust you Body text
44. I have made the decision to trust you Body text
47. Fix your flaky tests!!!
48. https://youtu.be/vHBzZHE4tJ0
49. 3. Running Tests Takes Time
50. How do you speed up your tests?
51. Use a build cache
52. Very effective for multi-module projects
53. No cache: Full build required
54. Build cache: Changing a leaf module
55. Build cache: Changing a module’s implementation
56. Build cache: Changing core implementation
57. Remote Build Cache
58. Run tests in parallel (local)
59. Run tests in parallel (CI)
61. Test Distribution
64. Test Distribution Results ‑ ~50% ‑ ~50% ‑ ~50% Doubling the number of executors cuts build time in half
65. Netflix reduced a 62-minute test cycle time down to just under 5 minutes!
66. Let’s use AI!
67. “The test selection procedure selects fewer than a third of the tests that would be selected on the basis of build dependencies” Predictive Test Selection - Meta Research https://research.facebook.com/publications/predictive-test-selection/ Mateusz Machalica, Alex Samylkin, Meredith Porth, Satish Chandra
68. Predictive Test Selection time savings
69. But Trisha, I just have So Many Tests!
70. “Yes, I regularly delete tests that no longer add unique value.” Angie Jones (@techgirl1908)
74. Like any performance problem, focus on the worst-performers first
76. “Once things like databases become a hindrance, optimise the testing loop, mock things out to run things fast. You can run locally and you don't have all the same infra dependencies as someone else.” On DevEx, PaaS, and Testing in Production https://bit.ly/3ZA0uQM Kelsey Hightower
77. Look out for expensive setup and tear down
80. Respect your test code
84. ⬢ Separation of concerns ⬢ Encapsulation ⬢ SOLID ⬢ DRY ⬢ Descriptive naming ⬢ Modularisation ⬢ Designing for performance
85. Tension between these ideals
86. “Tidying is software design addressing you, your relationship to your code, and ultimately your relationship with yourself…. Tidying is geek self-care.” Tidy First? Kent Beck
87. Summary
88. Automate as much as possible
89. There are solutions out there designed for exactly this problem
90. There is a time investment in writing tests, because there should be thinking involved
91. “Your best and wisest refuge from all troubles is in your science.” Ada Lovelace
92. Thank you!