Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Flaky tests - Fighting nightmares

Flaky tests - Fighting nightmares

A test can be a liar: Unreliable tests are living nightmares for anyone who writes automated tests or even pays attention to their results. These flaky tests brought me nightmares. To help you get out of this hell or to prevent you from getting into it, I would like to talk to you about these experiences.

36751965e79dca4618cccc3c08efc912?s=128

Ramona Schwering

March 25, 2021
Tweet

Transcript

  1. Flaky Tests Fighting Nightmares

  2. Flaky Tests @leichteckig

  3. Flaky Tests @leichteckig

  4. The boy who cried wolf Photo by hp koch on

    Unsplash
  5. Flaky Tests @leichteckig –Aesop „A liar will not be believed,

    even when he speaks the truth.“
  6. An ugly pattern emerges Photo by Pelly Benassi on Unsplash

  7. Flaky Tests @leichteckig –Someone from my development team „Oh that

    test, yeah it sometimes fails, just kick the build off again.“
  8. Spotting the causes Photo by Paul Green on Unsplash

  9. Test-sided causes Photo by Ferenc Almasi on Unsplash

  10. Flaky Tests @leichteckig 🗝 📂 🔗 ⏱ 📑

  11. Environment-sided causes Photo by Taylor Vick on Unsplash

  12. Flaky Tests @leichteckig 🚰 🔗 📦

  13. Product-sided causes Photo by Markus Spiske on Unsplash

  14. Flaky Tests @leichteckig –Sam Saffron „In some cases when fixing

    a flaky tests, the fix is in the app, not in the test.“
  15. Photo by Franck V. on Unsplash Finding a way out

  16. Start an honest discussion with your team Photo by me

  17. Flaky Tests @leichteckig 🗑 ♻ 🤷 ☣

  18. Run test in a loop Photo by Nareeta Martin on

    Unsplash
  19. Keep tests isolated Photo by Ryan Loughlin on Unsplash

  20. Flaky Tests Test data management @leichteckig

  21. Optimise test structure Photo by Kaleidico on Unsplash

  22. Automatic test retries Photo by Jim Wilson on Unsplash

  23. Flaky Tests Retries in Gitlab @leichteckig

  24. Flaky Tests Retries in Cypress @leichteckig

  25. Add purpose built diagnostic code Photo by Agence Olloweb on

    Unsplash
  26. Flaky Tests Add purpose built diagnostic code @leichteckig

  27. Matching waiting strategy Photo by Phil Hearing on Unsplash

  28. Exit! Photo by Michael Jasmund on Unsplash

  29. Thank you! Photo by Wilhelm Gunktel on Unsplash Twitter: @leichteckig