Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Flaky Tests - Fighting Nightmares

Flaky Tests - Fighting Nightmares

"A liar will not be believed even if he tells the truth" - Aesop

A test can also be such a liar: Unreliable tests are living nightmares for anyone who writes automated tests or even pays attention to their results. Sometimes they pass, sometimes they don't, and therefore do not give a valid statement about the state of the software. In the worst case, they can impair the credibility of the entire test suite. It doesn't matter whether you write unit, integration or end-to-end tests, you can counter them in any way you like.

I've spent hours and hours on such tests, and I want to share my path through them and the lessons learned from them: So that together we can avoid or even get rid of these nightmares.

Ramona Schwering

March 17, 2023
Tweet

More Decks by Ramona Schwering

Other Decks in Programming

Transcript

  1. View Slide

  2. View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. View Slide

  19. View Slide

  20. View Slide

  21. View Slide

  22. View Slide

  23. View Slide

  24. View Slide

  25. View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. View Slide

  30. View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. Cypress._.times(10, () => {
    describe('Description', () => {
    it('runs 10 times', () => {
    //...
    });
    });
    });

    View Slide

  39. View Slide

  40. describe('Customer login', () => {
    let customer = {};
    beforeEach(() => {
    // Set application to clean state
    cy.setInitialState()
    .then(() => {
    // Create test data
    return cy.setFixture('customer');
    })
    }):
    // … tests will start below
    })

    View Slide

  41. View Slide

  42. View Slide

  43. {
    "retries": {
    // Configure retry attempts for `cypress run`
    // Default is 0
    "runMode": 2,
    // Configure retry attempts for `cypress open`
    // Default is 0
    "openMode": 0
    }
    }

    View Slide

  44. View Slide

  45. View Slide

  46. View Slide

  47. View Slide

  48. View Slide

  49. View Slide

  50. View Slide

  51. View Slide