$30 off During Our Annual Pro Sale. View Details »

Flaky tests - Fighting nightmares

Flaky tests - Fighting nightmares

Ramona Schwering

March 25, 2021
Tweet

More Decks by Ramona Schwering

Other Decks in Programming

Transcript

  1. Flaky Tests
    Fighting Nightmares

    View Slide

  2. Flaky Tests @leichteckig

    View Slide

  3. Flaky Tests @leichteckig

    View Slide

  4. The boy who cried wolf
    Photo by hp koch on Unsplash

    View Slide

  5. Flaky Tests @leichteckig
    –Aesop
    „A liar will not be believed, even when he speaks the truth.“

    View Slide

  6. An ugly pattern emerges
    Photo by Pelly Benassi on Unsplash

    View Slide

  7. Flaky Tests @leichteckig
    –Someone from my development team
    „Oh that test, yeah it sometimes fails, just kick the build off again.“

    View Slide

  8. Spotting the causes
    Photo by Paul Green on Unsplash

    View Slide

  9. Test-sided causes
    Photo by Ferenc Almasi on Unsplash

    View Slide

  10. Flaky Tests @leichteckig
    🗝 📂 🔗 ⏱ 📑

    View Slide

  11. Environment-sided causes
    Photo by Taylor Vick on Unsplash

    View Slide

  12. Flaky Tests @leichteckig
    🚰 🔗 📦

    View Slide

  13. Product-sided causes
    Photo by Markus Spiske on Unsplash

    View Slide

  14. Flaky Tests @leichteckig
    –Sam Saffron
    „In some cases when fixing a flaky tests, the fix is in the app,
    not in the test.“

    View Slide

  15. Photo by Franck V. on Unsplash
    Finding a way out

    View Slide

  16. Start an honest discussion with your team
    Photo by me

    View Slide

  17. Flaky Tests @leichteckig
    🗑 ♻ 🤷 ☣

    View Slide

  18. Run test in a loop
    Photo by Nareeta Martin on Unsplash

    View Slide

  19. Keep tests isolated
    Photo by Ryan Loughlin on Unsplash

    View Slide

  20. Flaky Tests
    Test data management
    @leichteckig

    View Slide

  21. Optimise test structure
    Photo by Kaleidico on Unsplash

    View Slide

  22. Automatic test retries
    Photo by Jim Wilson on Unsplash

    View Slide

  23. Flaky Tests
    Retries in Gitlab
    @leichteckig

    View Slide

  24. Flaky Tests
    Retries in Cypress
    @leichteckig

    View Slide

  25. Add purpose built diagnostic code
    Photo by Agence Olloweb on Unsplash

    View Slide

  26. Flaky Tests
    Add purpose built diagnostic code
    @leichteckig

    View Slide

  27. Matching waiting strategy
    Photo by Phil Hearing on Unsplash

    View Slide

  28. Exit!
    Photo by Michael Jasmund on Unsplash

    View Slide

  29. Thank you!
    Photo by Wilhelm Gunktel on Unsplash Twitter: @leichteckig

    View Slide