Slide 1
Slide 1 text
The Real Challenges of
Software Testing
TACON • André Neubauer • 14.09.2022
Slide 2
Slide 2 text
Positioning
https://i.insider.com/514205d4eab8ea8b38000000?width=4346
Slide 3
Slide 3 text
Information Age
● Discontinuous change
● Short product life cycles
● Continuous price declines
● Little customer loyalty
● Technology & data at the core
● Based on knowledge work
Slide 4
Slide 4 text
Information Age
● Discontinuous change
● Short product life cycles
● Continuous price declines
● Little customer loyalty
● Technology & data at the core
● Based on knowledge work
Speed (time to market) matters
Slide 5
Slide 5 text
Time to market is crucial
https://www.tcgen.com/time-to-market/
Slide 6
Slide 6 text
Software “Software is eating the world, in all
sectors”
… and it just started …
“In the future every company will
become a software company”
(Marc Andreessen, 2011)
https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
Slide 7
Slide 7 text
Software is eating the world
https://iot-analytics.com/industrial-software-companies/ https://www.worldlocity.com/post/2019/02/20/is-software-really-eating-the-world
Slide 8
Slide 8 text
Software “Software is eating the world, in all
sectors”
… and it just started …
“In the future every company will
become a software company”
(Marc Andreessen, 2011)
Complexity will grow overproportionally
https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
Slide 9
Slide 9 text
ML/ AI
https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
“Software Ate The World, Now AI Is
Eating Software”
(Forbes)
Slide 10
Slide 10 text
GitHub Copilot
https://www.encora.com/insights/github-copilot-a-code-autocomplete-tool-on-steroids
Slide 11
Slide 11 text
Software Testing
Speed
Complexity
https://www.youtube.com/watch?v=EnDfTQuivfk
Non-deterministic
behaviour
Slide 12
Slide 12 text
Enemies of Software Testing
● Complexity of environment
● Speed of change, Time to market
● Non-deterministic behaviour
Slide 13
Slide 13 text
So what?!
Slide 14
Slide 14 text
Software testing
paradigms
Tradition quality model:
● Sequential process with late
starting testing
Shift left model:
● Move testing activities earlier in
the software delivery lifecycle
to achieve better quality and
time to market
https://devopedia.org/shift-left
Slide 15
Slide 15 text
Shift left testing
principles
https://devopedia.org/shift-left
● Test everything
● Test everytime
● Test early
● Test continuously
● Involve testers earlier
● Prevention than detection
Slide 16
Slide 16 text
Test everything? ● DS models are hard to test
● System testing becomes
challenging as complexity
increases
● Keep in mind: Testing is a
risk-based activity.
Slide 17
Slide 17 text
Prevention than
detection?
● Prevention will result in
overtesting
high costs and
delayed time to market
https://www.researchgate.net/publication/342116266_Softwa
re_Testing_Techniques_New_Trends
Slide 18
Slide 18 text
Late stage attention
to quality ● Recreasing attention after early
stage
● Only minor focus from “Deploy
and Release” onwards
https://devopedia.org/shift-left
Slide 19
Slide 19 text
Shift left testing is not enough to
fight nowadays challenges.
Slide 20
Slide 20 text
So what?!
Slide 21
Slide 21 text
DevOps
“[..] shorten the systems development
life cycle and provide continuous
delivery with high software quality.”
Oreilly
https://en.wikipedia.org/wiki/DevOps
Slide 22
Slide 22 text
Shift right testing
● Making use of continuous
delivery and fast release cycle
● Focus on detection and
mitigation
Slide 23
Slide 23 text
Testing on production?!
https://www.washingtontimes.com/news/2019/nov/25/elon-musk-explains-why-cybertrucks-bulletproof-gla/
Slide 24
Slide 24 text
Not about manual testing on production but …
https://copyconstruct.medium.com/testing-in-production-the-safe-way-18ca102d0ef
Slide 25
Slide 25 text
A testing approach to fight challenges
Unit testing
Canary deployment
Integration testing
Smoke tests
KPI tracking
Feature flags
Interface testing
System testing
Tracing
Log management
… testing
Requirements
engineering
Test selection
https://en.wikipedia.org/wiki/DevOps
Slide 26
Slide 26 text
Shift left & shift right testing
for the win
Slide 27
Slide 27 text
Best of both worlds
● Test risk-based
● Test early
● Test continuously
● Involve testers earlier
● Have detection and mitigation
capabilities
● Have a fast release cycle
● Important: Balance left and right
based on the context
Slide 28
Slide 28 text
What does it mean for
Test Automation?
● Still: Automate everything
Consider:
● Smoke tests (UI test subset)
● Predictive Test Selection (to
stay fast)
https://www.launchableinc.com/articles/shift-left-au
tomated-tests-with-launchable
Slide 29
Slide 29 text
The Real Challenges of
Software Testing
TACON • André Neubauer • 14.09.2022
Thanks for listening