The Real Challenges of Software Testing

Shift left testing is not enough to stay on top of the game. You need to combine it with "shift right" to master nowadays challenges (speed of change, complexity, ...)

Transcript

1. The Real Challenges of Software Testing TACON • André Neubauer • 14.09.2022
2. Positioning https://i.insider.com/514205d4eab8ea8b38000000?width=4346
3. Information Age • Discontinuous change • Short product life cycles • Continuous price declines • Little customer loyalty • Technology & data at the core • Based on knowledge work
4. Information Age • Discontinuous change • Short product life cycles • Continuous price declines • Little customer loyalty • Technology & data at the core • Based on knowledge work Speed (time to market) matters
5. Time to market is crucial https://www.tcgen.com/time-to-market/
6. Software “Software is eating the world, in all sectors” … and it just started … “In the future every company will become a software company” (Marc Andreessen, 2011) https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
7. Software is eating the world https://iot-analytics.com/industrial-software-companies/ https://www.worldlocity.com/post/2019/02/20/is-software-really-eating-the-world
8. Software “Software is eating the world, in all sectors” … and it just started … “In the future every company will become a software company” (Marc Andreessen, 2011) Complexity will grow overproportionally https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
9. ML/ AI https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/ “Software Ate The World, Now AI Is Eating Software” (Forbes)
10. GitHub Copilot https://www.encora.com/insights/github-copilot-a-code-autocomplete-tool-on-steroids
11. Software Testing Speed Complexity https://www.youtube.com/watch?v=EnDfTQuivfk Non-deterministic behaviour
12. Enemies of Software Testing • Complexity of environment • Speed of change, Time to market • Non-deterministic behaviour
13. So what?!
14. Software testing paradigms Tradition quality model: • Sequential process with late starting testing Shift left model: • Move testing activities earlier in the software delivery lifecycle to achieve better quality and time to market https://devopedia.org/shift-left
15. Shift left testing principles https://devopedia.org/shift-left • Test everything • Test everytime • Test early • Test continuously • Involve testers earlier • Prevention than detection
16. Test everything? • DS models are hard to test • System testing becomes challenging as complexity increases • Keep in mind: Testing is a risk-based activity.
17. Prevention than detection? • Prevention will result in overtesting high costs and delayed time to market https://www.researchgate.net/publication/342116266_Softwa re_Testing_Techniques_New_Trends
18. Late stage attention to quality • Recreasing attention after early stage • Only minor focus from “Deploy and Release” onwards https://devopedia.org/shift-left
19. Shift left testing is not enough to fight nowadays challenges.
20. So what?!
21. DevOps “[..] shorten the systems development life cycle and provide continuous delivery with high software quality.” Oreilly https://en.wikipedia.org/wiki/DevOps
22. Shift right testing • Making use of continuous delivery and fast release cycle • Focus on detection and mitigation
23. Testing on production?! https://www.washingtontimes.com/news/2019/nov/25/elon-musk-explains-why-cybertrucks-bulletproof-gla/
24. Not about manual testing on production but … https://copyconstruct.medium.com/testing-in-production-the-safe-way-18ca102d0ef
25. A testing approach to fight challenges Unit testing Canary deployment Integration testing Smoke tests KPI tracking Feature flags Interface testing System testing Tracing Log management … testing Requirements engineering Test selection https://en.wikipedia.org/wiki/DevOps
26. Shift left & shift right testing for the win
27. Best of both worlds • Test risk-based • Test early • Test continuously • Involve testers earlier • Have detection and mitigation capabilities • Have a fast release cycle • Important: Balance left and right based on the context
28. What does it mean for Test Automation? • Still: Automate everything Consider: • Smoke tests (UI test subset) • Predictive Test Selection (to stay fast) https://www.launchableinc.com/articles/shift-left-au tomated-tests-with-launchable
29. The Real Challenges of Software Testing TACON • André Neubauer • 14.09.2022 Thanks for listening