The Missing Piece of Testing

So what do you do when you write a test? Usually, you tell stories. You give examples like: for a given input I expect a given output. That’s certainly better than no tests but this approach has drawbacks.

First, you didn’t prove your code works in general – you only told some anecdotes and demonstrated it works for given examples. In other words, using those examples you’re trying to prove there are no bugs. Shouldn’t your tests help find bugs instead?

Second, you only covered some general cases and edge cases you know. What about edge cases you don’t know?

As you can see, even though you’re doing your best, you may still be missing a fundamental piece of testing. A piece that empowers you to describe behavior instead of output, to find bugs more effectively, and to test more cases with less code.

In my talk, I’ll show you how you can leverage the power of property-based testing to complement your testing tool belt and have even more confidence with the software you create.

Transcript

1. The Missing Piece of Testing
2. ✈
3. describe('sort function', () => { it('should sort integers', () => { expect(sort([3, 2])).toEqual([2, 3]) }) })
4. “Testing shows the presence, not the absence of bugs.” Edsger W. Dijkstra
5. describe('sort function', () => { it('should sort integers', () => { expect(sort([3, 2])).toEqual([2, 3]) }) }) }) })
6. describe('sort function', () => { it('should sort integers', () => { }) })
7. describe('sort function', () => { it('should sort integers', () => { generate() }) })
8. describe('sort function', () => { it('should sort integers', () => { generate(array(integer())) }) })
9. describe('sort function', () => { it('should sort integers', () => { generate(array(integer())) }) })
10. describe('sort function', () => { it('should sort integers', () => { generate(array(integer()), (unsortedArr) => {}) }) })
11. describe('sort function', () => { it('should sort integers', () => { generate(array(integer()), (unsortedArr) => { sort(unsortedArr) }) }) })
12. }) }) }) describe('sort function', () => { it('should sort integers', () => { generate(array(integer()), (unsortedArr) => { expect(sort(unsortedArr)).toEqual( /* what */)
13. Properties
20. Type
22. Emptiness
24. Size
26. Shape
27. const sorted = sort(unsortedArr) // type
28. const sorted = sort(unsortedArr) // type expect(Array.isArray(sorted)).toBe(true)
29. const sorted = sort(unsortedArr) // type // emptiness expect(Array.isArray(sorted)).toBe(true)
30. const sorted = sort(unsortedArr) // size
31. const sorted = sort(unsortedArr) // size expect(sorted.length).toBe(unsortedArr.length)
32. const sorted = sort(unsortedArr) // shape
33. const sorted = sort(unsortedArr) // shape sorted.forEach((element) => { expect(typeof element).toBe('number') })
34. const sorted = sort(unsortedArr) // the next element should be greater than the previous one
35. result.forEach((element, i) => { const nextElement = sorted[i + 1] if (nextElement) { expect(nextElement).toBeGreaterThan(element) } }) const sorted = sort(unsortedArr) // the next element should be greater than the previous one
36. const sorted = sort(unsortedArr) // idempotence
37. const sorted = sort(unsortedArr) // idempotence expect(sort(sorted)).toEqual(sorted)
38. describe('sort function', () => { it('should sort integers', () => { generate(array(integer()), (unsortedArr) => { // assertions }) }) })
39. describe('sort function', () => { it('should sort integers', () => { runMultipleTimes( generate(array(integer()), (unsortedArr) => { // assertions }) ) }) })
40. import fc from 'fast-check' describe('sort function', () => { it('should sort integers', () => { runMultipleTimes( generate(array(integer()), (unsortedArr) => { // assertions }) ) }) })
41. import fc from 'fast-check' describe('sort function', () => { it('should sort integers', () => { fc.assert( generate(array(integer()), (unsortedArr) => { // assertions }) ) }) })
42. import fc from 'fast-check' describe('sort function', () => { it('should sort integers', () => { fc.assert( fc.property(array(integer()), (unsortedArr) => { // assertions }) ) }) })
43. import fc from 'fast-check' describe('sort function', () => { it('should sort integers', () => { fc.assert( fc.property(fc.array(fc.integer()), (unsortedArr) => { // assertions }) ) }) })
44. import fc from 'fast-check' describe('sort function', () => { it('should sort integers', () => { fc.assert( fc.property(fc.array(fc.integer()), (unsortedArr) => { // assertions }) ) expect(sort([3, 2])).toEqual([2, 3]) }) })
48. import fc from 'fast-check' describe('sort function', () => { it('should sort integers', () => { fc.assert( fc.property(fc.array(fc.integer()), (unsortedArr) => { // assertions }) ) expect(sort([3, 2])).toEqual([2, 3]) }) })
49. import fc from 'fast-check' describe('sort function', () => { it('should sort integers', () => { fc.assert( fc.property(fc.array(fc.integer()), (unsortedArr) => { // assertions }) ) expect(sort([3, 2])).toEqual([2, 3]) expect(sort([10, 2])).toEqual([2, 10]) }) })
50. 17 steps 33 steps
51. Track B Track A
52. ! React Final Form 100% coverage 3,249 ⭐ 33,402 weekly downloads on npm
53. ! React Final Form Arrays 100% coverage 11,272 weekly downloads on npm
56. 4 already existing issues 6 new bugs
63. Write example-based tests Complement them with property-based test
64. ✈ Fly safely Maciej Myśliński