Flaky tests - Fighting nightmares

Transcript

1. Flaky Tests
   Fighting Nightmares
   

   View Slide

2. Flaky Tests @leichteckig
   

   View Slide

3. Flaky Tests @leichteckig
   

   View Slide

4. The boy who cried wolf
   Photo by hp koch on Unsplash
   

   View Slide

5. Flaky Tests @leichteckig
   –Aesop
   „A liar will not be believed, even when he speaks the truth.“
   

   View Slide

6. An ugly pattern emerges
   Photo by Pelly Benassi on Unsplash
   

   View Slide

7. Flaky Tests @leichteckig
   –Someone from my development team
   „Oh that test, yeah it sometimes fails, just kick the build off again.“
   

   View Slide

8. Spotting the causes
   Photo by Paul Green on Unsplash
   

   View Slide

9. Test-sided causes
   Photo by Ferenc Almasi on Unsplash
   

   View Slide

10. Flaky Tests @leichteckig
    🗝 📂 🔗 ⏱ 📑
    

    View Slide

11. Environment-sided causes
    Photo by Taylor Vick on Unsplash
    

    View Slide

12. Flaky Tests @leichteckig
    🚰 🔗 📦
    

    View Slide

13. Product-sided causes
    Photo by Markus Spiske on Unsplash
    

    View Slide

14. Flaky Tests @leichteckig
    –Sam Saffron
    „In some cases when fixing a flaky tests, the fix is in the app,
    not in the test.“
    

    View Slide

15. Photo by Franck V. on Unsplash
    Finding a way out
    

    View Slide

16. Start an honest discussion with your team
    Photo by me
    

    View Slide

17. Flaky Tests @leichteckig
    🗑 ♻ 🤷 ☣
    

    View Slide

18. Run test in a loop
    Photo by Nareeta Martin on Unsplash
    

    View Slide

19. Keep tests isolated
    Photo by Ryan Loughlin on Unsplash
    

    View Slide

20. Flaky Tests
    Test data management
    @leichteckig
    

    View Slide

21. Optimise test structure
    Photo by Kaleidico on Unsplash
    

    View Slide

22. Automatic test retries
    Photo by Jim Wilson on Unsplash
    

    View Slide

23. Flaky Tests
    Retries in Gitlab
    @leichteckig
    

    View Slide

24. Flaky Tests
    Retries in Cypress
    @leichteckig
    

    View Slide

25. Add purpose built diagnostic code
    Photo by Agence Olloweb on Unsplash
    

    View Slide

26. Flaky Tests
    Add purpose built diagnostic code
    @leichteckig
    

    View Slide

27. Matching waiting strategy
    Photo by Phil Hearing on Unsplash
    

    View Slide

28. Exit!
    Photo by Michael Jasmund on Unsplash
    

    View Slide

29. Thank you!
    Photo by Wilhelm Gunktel on Unsplash Twitter: @leichteckig
    

    View Slide