Maintainable test automation

Test automation. Wat sommigen beschouwen als een enorme enabler/meerwaarde wordt op veel projecten stiefmoederlijk behandelt en aanzien als een "permanente kost".

Voor iedereen die de woorden "Jup, m'n feature is klaar, ik moet enkel de tests nog fixen!" al eens heeft durven uiten, voelt dat het anders/beter zou kunnen maar niet onmiddelijk ziet waar te beginnen.

De sessie behandelt o.a.:
* High-level goals van test automation
* Principes en technieken om test automation ROI interessant te houden
* Nitty-gritty patterns en code-voorbeelden om je geautomatiseerde tests beter onderhoudbaar te maken

Talk goals:
* Insight into what makes "good test automation"
* Provide practical take-aways attendees can immediately start using on the job

Transcript

1. MAINTAINABLE TEST AUTOMATION a.k.a test automation without shooting yourself in the foot
2. Talk goals: • Insight into what makes “good” test automation • Actionable tips you can use at work tomorrow Agenda: • Motivation • Goals of automation • Automation Strategies • Tactics: smells & patterns
6. Iterative and incremental
7. Prerequisites
8. Prerequisites Feedback
10. Prerequisites
11. Prerequisites FAST Feedback
13. “I spend 50% of my time fixing unrelated tests” “Development is done, I just have to get tests green again” “Oh yeah, those tests turn red all the time. Don’t worry!” “Our test suite takes 12 hours to run” “We have separate ‘write tests’ stories on our task board”
14. The eternal trade-off cost(building(T)) + cost(maintaining(T)) < value(T)
15. Make the question a no-brainer! cost(building(T)) + cost(maintaining(T)) < value(T) MINimize this MAXimize this
16. Goals of test automation “Good” test automation: 1. Improves quality 2. Improves understandability 3. Reduces risk 4. Makes it easy to run/write/maintain tests 5. Provides fast feedback
17. Goal 1: improve quality
18. Goal 1: improve quality
19. Goal 1: improve quality
20. Goal 1: improve quality
21. Goal 1: improve quality • External quality: • Correctness • Non-functionals • Internal quality: • Design quality
22. Goal 2: understand system
23. Goal 3: reduce risk • Safety net
24. Goal 4: easy to run/read/write/maintain • FIRST tests • Fast • Isolated • Repeatable • Self-checking • Timely
25. Goal 4: easy to run/read/write/maintain • Expressive, self-contained tests
28. Goal 4: easy to run/read/write/maintain
29. Goal 4: easy to run/read/write/maintain
30. Test Automation Strategy
31. Disclaimer http://blogs.tedneward.com/post/there-is-no-such-thing-as-best-practices-context-matters/
32. What tests to write?
33. What tests to write?
34. What tests to write? Automate the heck out of this Automate + tooling Automate repeated + tooling Needs skill + human brain
35. When to invest? https://www.facebook.com/notes/kent-beck/the-product-development-triathlon/1215075478525314/
36. Ensure testability • What hurts testability most?
37. The hidden/implicit/hardcoded dependency
39. http://alistair.cockburn.us/Hexagonal+architecture
40. Ensure testability Xunit test patterns, Meszaros
41. Ensure testability Xunit test patterns, Meszaros
42. Ensure testability Xunit test patterns, Meszaros
43. Whose job is it anyway?
44. The deep synergy between testability and good design, Michael Feathers https://vimeo.com/15007792 “Tests turn design issues into concrete pain”
45. The deep synergy between testability and good design, Michael Feathers https://vimeo.com/15007792 “Tests turn design issues into concrete pain”
46. Test smells and patterns
47. Smells?
48. Test code smells: an illustrated example
69. • Tests as documentation • Easier to • Read (no noise) • Write (DSL) • Maintain (less fragile)
70. More smells & patterns
71. Test-per-method Programming with GUTs, Kevlin Henney https://www.slideshare.net/Kevlin/programming-with-guts-39293384
72. Test-per-method
73. Test-per-method
74. Test-per-method
75. Test-per-method Focus on behaviour, not implementation
76. Test-per-method
77. Test-per-method
78. Test-per-method • Tests as documentation • Maintainable tests • Test grouping • Per class • Per feature • Per fixture Focus on behaviour, not implementation
79. The “100% coverage” target https://martinfowler.com/bliki/TestCoverage.html
80. Mystery Guest
81. Mystery Guest Fresh fixture Inline Setup Test Data Builders
83. Slow test
84. Slow test
85. Slow test Shared Fixture
86. Slow test Shared Fixture
87. Slow test
88. Fragile test • The 4 sensitivities: • Interface sensitivity • Data sensitivity • Context Sensitivitiy • Behaviour Sensititivy
90. Delta assertions
91. Fresh Fixture
92. Test Stub
93. Further reading http://xunitpatterns.com
96. "Whenever you are tempted to type something into a print statement or a debugger expression, write it as a test instead.“ – Martin Fowler