Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Quarantine: Get your confidence back

Quarantine: Get your confidence back

A technique for dealing with non-deterministic tests.

1668868370ee5829339e06031ad0b145?s=128

Robbie Clutton

March 07, 2013
Tweet

More Decks by Robbie Clutton

Other Decks in Technology

Transcript

  1. @robb1e Software Engineer, Pivotal Labs Robbie Clutton Quarantine

  2. @robb1e $ rake ..........F......... $ rake ................... $ rake ....F...........F....

    $ rake ...................
  3. @robb1e You say... Oh, it’s just that flakey test, run

    the build again
  4. @robb1e 3 minutes ago: failed 12 minutes ago: passed 30

    minutes ago: failed 45 minutes ago: failed 2 hours ago: passed
  5. @robb1e You say... Run all build agents, one of them

    will go through
  6. @robb1e

  7. @robb1e Confidence, meet window

  8. @robb1e @quarantine Scenario: Something flakey #.... describe “Flakey”, :quarantine do

    # ...
  9. @robb1e #regular ci build #!/bin/bash cucumber --tags ~@quarantine rspec .

    --tags ~quarantine
  10. @robb1e #quarantine ci build #!/bin/bash cucumber --tags @quarantine rspec .

    --tags quarantine
  11. @robb1e Regular Quarantine Deploy Continuous Integration

  12. @robb1e BUILD NANNY 1. Rewrite 2. Rethink 3. Delete

  13. “You should not be afraid to delete tests that are

    no longer providing value, no matter whether you originally planned to keep them or not. We tend to treat tests as these holy creatures that live blameless, irreproachable lives once they have sprung into existence. Not so. The maintenance required to keep a test running weighs against its value in further development. Sometimes these lines cross, and the test simply becomes a burden on the project. Having the skill and experience to recognize a burdensome test is something we should be bringing to our clients, as well as the fortitude to rewrite it, rethink it, or delete it." - Adam Milligan
  14. @robb1e Request Controller Model View Javascript Decorator Database

  15. @robb1e $ rake ................... $ rake ................... $ rake ...................

    $ rake ...................
  16. @robb1e 3 minutes ago: passed 12 minutes ago: passed 30

    minutes ago: passed 45 minutes ago: passed 2 hours ago: passed
  17. @robb1e Time to get the confidence back

  18. @robb1e Q/A