Slide 1

Slide 1 text

Randomness in Testing Paul Grayson 2013-09-25

Slide 2

Slide 2 text

Normal testing: try edge cases ● Easy when you have a geometric picture:

Slide 3

Slide 3 text

“Fuzz testing” for difficult edges ● What are the “edge cases” here?

Slide 4

Slide 4 text

Fuzz testing kata example

Slide 5

Slide 5 text

Note: fuzz tests generally more simple-minded Things to Fuzz test for: ● exceptions or other bad behavior ● outputs consistent with inputs ● exact answer for a simplified domain (hard!)

Slide 6

Slide 6 text

Fuzz Testing: NOT part of RSpec ● Just use rand in your tests.

Slide 7

Slide 7 text

Test order dependence ● Ideal: tests are independent ● DO NOT use objects created out of the context ● DO NOT break when other objects exist

Slide 8

Slide 8 text

Test order kata example

Slide 9

Slide 9 text

When a test fails... ● Write a test that always fails, or fix the tests ● Re-use the seed to debug rspec --seed 1234

Slide 10

Slide 10 text

Using RSpec seed with rand srand RSpec.configuration.seed + 1

Slide 11

Slide 11 text

Putting it together ● Automatically re-use the seed

Slide 12

Slide 12 text

Other random examples ● Code mutation ● Unavoidable randomness (network, threads) ● Testing random code