Seems Good Enough to Me: Working with Testers to Derisk Upgrades

It’s that time again! You have a legacy app running business critical functions and it’s on an old version of Elixir. You need to upgrade to stay safe, have access to the most modern features of the ecosystem, and be current with your community. But a full regression of the app will take days if not weeks and no one can find the time. What’s a team to do?

Jenny Bramble suggests leaning on the deep magic of test methodology and collaboration to determine the minimum viable tests to run in order to ensure your system is preforming as expected. She’ll talk about working through release notes with your testers as well as applying test methodologies to the types of problems we see in upgrades and regressions. You’ll walk away from this talk with a stronger understanding of the ways we can simplify and derisk upgrades and a structure for working with test to make sure that we’re doing the right amount of testing.

Transcript

1. Seems Good Enough to Me @jennydoesthings Working With Testers to Derisk Elixir Upgrades
2. • Director of Quality Engineering, Pap a • Tester by nature and nurtur e • Pronouns are she/he r • I have stickers! Jenny Bramble @jennydoesthings - #ElixirConf2022
3. @jennydoesthings @jennydoesthings I don’t think I need to convince anyone in the room that keeping our ecosystem up to date is a good thing.
4. @jennydoesthings @jennydoesthings What problem do we want to solve?
5. What problem do we want to solve? • Upgrades are risky but necessary . • Most parts of our teams don’t understand how risky upgrades are or are not . • Regression testing takes forever and uses a lot of our resources . • If we mess up, it could be real bad. @jennydoesthings
6. @jennydoesthings
7. @jennydoesthings @jennydoesthings How do we make upgrades less risky?
8. @jennydoesthings @jennydoesthings I have an easy answer.
9. @jennydoesthings @jennydoesthings Do less testing.
10. @jennydoesthings
11. @jennydoesthings @jennydoesthings How do we make upgrades less risky?
12. @jennydoesthings @jennydoesthings I believe we derisk upgrades by testing the smallest set of features that give us confidence to move forward.
13. @jennydoesthings @jennydoesthings That seems obvious, right?
14. @jennydoesthings @jennydoesthings Let’s talk about ‘good enough testing’.
15. Good Enough Testing • Thoughtfully and intentionally craft test scenario s • Detail what we aren’t testing and wh y • Don’t thras h • Don’t toi l • Prioritizes whitewater kayaking and hiking @jennydoesthings
16. @jennydoesthings
17. Good Enough Testing • Thoughtfully and intentionally craft test scenario s • Detail what we aren’t testing and wh y • Don’t thras h • Don’t toi l • Prioritizes work-life balance @jennydoesthings
18. @jennydoesthings @jennydoesthings Testers hold the mental model of the system in our minds.
19. @jennydoesthings @jennydoesthings How do we express that in upgrades?
20. @jennydoesthings @jennydoesthings Spoiler alert: this isn’t going to be a radical shift.
21. @jennydoesthings @jennydoesthings Start with the changelog.
22. @jennydoesthings @jennydoesthings Now communicate it.
23. @jennydoesthings @jennydoesthings For each thing we think will affect us, determine its risk profile.
24. @jennydoesthings
25. @jennydoesthings @jennydoesthings What if we get it right?
26. @jennydoesthings @jennydoesthings How would this express if we missed something?
27. @jennydoesthings @jennydoesthings Do we care?
28. @jennydoesthings @jennydoesthings How will we monitor and observe?
29. @jennydoesthings @jennydoesthings Stop throwing things over the wall…
30. @jennydoesthings @jennydoesthings ….to production.
31. @jennydoesthings @jennydoesthings How will we monitor and observe?
32. @jennydoesthings @jennydoesthings Taadaa, risk profile!
33. @jennydoesthings @jennydoesthings Craft scenarios.
34. @jennydoesthings @jennydoesthings Defend yourselves.
35. @jennydoesthings
36. @jennydoesthings @jennydoesthings Defend yourselves.
37. @jennydoesthings - #ElixirConf2022
38. @jennydoesthings @jennydoesthings This is how we introduce nuance to our testing.
39. @jennydoesthings @jennydoesthings This is how we derisk upgrades.
40. @jennydoesthings @jennydoesthings To recap:
41. Do less testing. • Craft thoughtful, intentional scenarios . • Describe what we aren’t testing and why . • Embrace minimalism. @jennydoesthings
42. Communicate more. • Share your plans . • Defend your plans . • Give others a reason to trust you. @jennydoesthings
43. Questions? @jennydoesthings • jenny.bramble@gmail.com • Twitter: @jennydoesthings
44. Credits This is where you give credit to the ones who are part of this project . • Presentation template by Slidesgo • Icons by Flaticon • Infographics by Freepik • Images created Freepi k • Author introduction slide photo created by Freepi k • Text & Image slide photo created by Freepik.com