$30 off During Our Annual Pro Sale. View Details »

The Real Challenges of Software Testing

The Real Challenges of Software Testing

Shift left testing is not enough to stay on top of the game. You need to combine it with "shift right" to master nowadays challenges (speed of change, complexity, ...)

André Neubauer

September 14, 2022
Tweet

More Decks by André Neubauer

Other Decks in Technology

Transcript

  1. The Real Challenges of
    Software Testing
    TACON • André Neubauer • 14.09.2022

    View Slide

  2. Positioning
    https://i.insider.com/514205d4eab8ea8b38000000?width=4346

    View Slide

  3. Information Age
    ● Discontinuous change
    ● Short product life cycles
    ● Continuous price declines
    ● Little customer loyalty
    ● Technology & data at the core
    ● Based on knowledge work

    View Slide

  4. Information Age
    ● Discontinuous change
    ● Short product life cycles
    ● Continuous price declines
    ● Little customer loyalty
    ● Technology & data at the core
    ● Based on knowledge work
    Speed (time to market) matters

    View Slide

  5. Time to market is crucial
    https://www.tcgen.com/time-to-market/

    View Slide

  6. Software “Software is eating the world, in all
    sectors”
    … and it just started …
    “In the future every company will
    become a software company”
    (Marc Andreessen, 2011)
    https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/

    View Slide

  7. Software is eating the world
    https://iot-analytics.com/industrial-software-companies/ https://www.worldlocity.com/post/2019/02/20/is-software-really-eating-the-world

    View Slide

  8. Software “Software is eating the world, in all
    sectors”
    … and it just started …
    “In the future every company will
    become a software company”
    (Marc Andreessen, 2011)
    Complexity will grow overproportionally
    https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/

    View Slide

  9. ML/ AI
    https://www.pngitem.com/middle/hxJoxwJ_ai-is-eating-software-hd-png-download/
    “Software Ate The World, Now AI Is
    Eating Software”
    (Forbes)

    View Slide

  10. GitHub Copilot
    https://www.encora.com/insights/github-copilot-a-code-autocomplete-tool-on-steroids

    View Slide

  11. Software Testing
    Speed
    Complexity
    https://www.youtube.com/watch?v=EnDfTQuivfk
    Non-deterministic
    behaviour

    View Slide

  12. Enemies of Software Testing
    ● Complexity of environment
    ● Speed of change, Time to market
    ● Non-deterministic behaviour

    View Slide

  13. So what?!

    View Slide

  14. Software testing
    paradigms
    Tradition quality model:
    ● Sequential process with late
    starting testing
    Shift left model:
    ● Move testing activities earlier in
    the software delivery lifecycle
    to achieve better quality and
    time to market
    https://devopedia.org/shift-left

    View Slide

  15. Shift left testing
    principles
    https://devopedia.org/shift-left
    ● Test everything
    ● Test everytime
    ● Test early
    ● Test continuously
    ● Involve testers earlier
    ● Prevention than detection

    View Slide

  16. Test everything? ● DS models are hard to test
    ● System testing becomes
    challenging as complexity
    increases
    ● Keep in mind: Testing is a
    risk-based activity.

    View Slide

  17. Prevention than
    detection?
    ● Prevention will result in
    overtesting
    high costs and
    delayed time to market
    https://www.researchgate.net/publication/342116266_Softwa
    re_Testing_Techniques_New_Trends

    View Slide

  18. Late stage attention
    to quality ● Recreasing attention after early
    stage
    ● Only minor focus from “Deploy
    and Release” onwards
    https://devopedia.org/shift-left

    View Slide

  19. Shift left testing is not enough to
    fight nowadays challenges.

    View Slide

  20. So what?!

    View Slide

  21. DevOps
    “[..] shorten the systems development
    life cycle and provide continuous
    delivery with high software quality.”
    Oreilly
    https://en.wikipedia.org/wiki/DevOps

    View Slide

  22. Shift right testing
    ● Making use of continuous
    delivery and fast release cycle
    ● Focus on detection and
    mitigation

    View Slide

  23. Testing on production?!
    https://www.washingtontimes.com/news/2019/nov/25/elon-musk-explains-why-cybertrucks-bulletproof-gla/

    View Slide

  24. Not about manual testing on production but …
    https://copyconstruct.medium.com/testing-in-production-the-safe-way-18ca102d0ef

    View Slide

  25. A testing approach to fight challenges
    Unit testing
    Canary deployment
    Integration testing
    Smoke tests
    KPI tracking
    Feature flags
    Interface testing
    System testing
    Tracing
    Log management
    … testing
    Requirements
    engineering
    Test selection
    https://en.wikipedia.org/wiki/DevOps

    View Slide

  26. Shift left & shift right testing
    for the win

    View Slide

  27. Best of both worlds
    ● Test risk-based
    ● Test early
    ● Test continuously
    ● Involve testers earlier
    ● Have detection and mitigation
    capabilities
    ● Have a fast release cycle
    ● Important: Balance left and right
    based on the context

    View Slide

  28. What does it mean for
    Test Automation?
    ● Still: Automate everything
    Consider:
    ● Smoke tests (UI test subset)
    ● Predictive Test Selection (to
    stay fast)
    https://www.launchableinc.com/articles/shift-left-au
    tomated-tests-with-launchable

    View Slide

  29. The Real Challenges of
    Software Testing
    TACON • André Neubauer • 14.09.2022
    Thanks for listening

    View Slide