The Real Challenges of Software Testing

Shift left testing is not enough to stay on top of the game. You need to combine it with "shift right" to master nowadays challenges (speed of change, complexity, ...)

Transcript

1. The Real Challenges of
   Software Testing
   TACON • André Neubauer • 14.09.2022
   

   View Slide

2. Positioning
   https://i.insider.com/514205d4eab8ea8b38000000?width=4346
   

   View Slide

3. Information Age
   ● Discontinuous change
   ● Short product life cycles
   ● Continuous price declines
   ● Little customer loyalty
   ● Technology & data at the core
   ● Based on knowledge work
   

   View Slide

4. Information Age
   ● Discontinuous change
   ● Short product life cycles
   ● Continuous price declines
   ● Little customer loyalty
   ● Technology & data at the core
   ● Based on knowledge work
   Speed (time to market) matters
   

   View Slide

5. Time to market is crucial
   https://www.tcgen.com/time-to-market/
   

   View Slide

6. Software “Software is eating the world, in all
   sectors”
   … and it just started …
   “In the future every company will
   become a software company”
   (Marc Andreessen, 2011)
   https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
   

   View Slide

7. Software is eating the world
   https://iot-analytics.com/industrial-software-companies/ https://www.worldlocity.com/post/2019/02/20/is-software-really-eating-the-world
   

   View Slide

8. Software “Software is eating the world, in all
   sectors”
   … and it just started …
   “In the future every company will
   become a software company”
   (Marc Andreessen, 2011)
   Complexity will grow overproportionally
   https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
   

   View Slide

9. ML/ AI
   https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
   “Software Ate The World, Now AI Is
   Eating Software”
   (Forbes)
   

   View Slide

10. GitHub Copilot
    https://www.encora.com/insights/github-copilot-a-code-autocomplete-tool-on-steroids
    

    View Slide

11. Software Testing
    Speed
    Complexity
    https://www.youtube.com/watch?v=EnDfTQuivfk
    Non-deterministic
    behaviour
    

    View Slide

12. Enemies of Software Testing
    ● Complexity of environment
    ● Speed of change, Time to market
    ● Non-deterministic behaviour
    

    View Slide

13. So what?!
    

    View Slide

14. Software testing
    paradigms
    Tradition quality model:
    ● Sequential process with late
    starting testing
    Shift left model:
    ● Move testing activities earlier in
    the software delivery lifecycle
    to achieve better quality and
    time to market
    https://devopedia.org/shift-left
    

    View Slide

15. Shift left testing
    principles
    https://devopedia.org/shift-left
    ● Test everything
    ● Test everytime
    ● Test early
    ● Test continuously
    ● Involve testers earlier
    ● Prevention than detection
    

    View Slide

16. Test everything? ● DS models are hard to test
    ● System testing becomes
    challenging as complexity
    increases
    ● Keep in mind: Testing is a
    risk-based activity.
    

    View Slide

17. Prevention than
    detection?
    ● Prevention will result in
    overtesting
    high costs and
    delayed time to market
    https://www.researchgate.net/publication/342116266_Softwa
    re_Testing_Techniques_New_Trends
    

    View Slide

18. Late stage attention
    to quality ● Recreasing attention after early
    stage
    ● Only minor focus from “Deploy
    and Release” onwards
    https://devopedia.org/shift-left
    

    View Slide

19. Shift left testing is not enough to
    fight nowadays challenges.
    

    View Slide

20. So what?!
    

    View Slide

21. DevOps
    “[..] shorten the systems development
    life cycle and provide continuous
    delivery with high software quality.”
    Oreilly
    https://en.wikipedia.org/wiki/DevOps
    

    View Slide

22. Shift right testing
    ● Making use of continuous
    delivery and fast release cycle
    ● Focus on detection and
    mitigation
    

    View Slide

23. Testing on production?!
    https://www.washingtontimes.com/news/2019/nov/25/elon-musk-explains-why-cybertrucks-bulletproof-gla/
    

    View Slide

24. Not about manual testing on production but …
    https://copyconstruct.medium.com/testing-in-production-the-safe-way-18ca102d0ef
    

    View Slide

25. A testing approach to fight challenges
    Unit testing
    Canary deployment
    Integration testing
    Smoke tests
    KPI tracking
    Feature flags
    Interface testing
    System testing
    Tracing
    Log management
    … testing
    Requirements
    engineering
    Test selection
    https://en.wikipedia.org/wiki/DevOps
    

    View Slide

26. Shift left & shift right testing
    for the win
    

    View Slide

27. Best of both worlds
    ● Test risk-based
    ● Test early
    ● Test continuously
    ● Involve testers earlier
    ● Have detection and mitigation
    capabilities
    ● Have a fast release cycle
    ● Important: Balance left and right
    based on the context
    

    View Slide

28. What does it mean for
    Test Automation?
    ● Still: Automate everything
    Consider:
    ● Smoke tests (UI test subset)
    ● Predictive Test Selection (to
    stay fast)
    https://www.launchableinc.com/articles/shift-left-au
    tomated-tests-with-launchable
    

    View Slide

29. The Real Challenges of
    Software Testing
    TACON • André Neubauer • 14.09.2022
    Thanks for listening
    

    View Slide