Setting Your Tests Up to Fail

Slide 1

Slide 1 text

SETTING YOUR TESTS UP TO FAIL @jennydoesthings

Slide 2

Slide 2 text

HI, I’M JENNY HUMAN INTERFACING IS MY FAVORITE THING MY PRONOUNS ARE SHE/HER @jennydoesthings

Slide 3

Slide 3 text

A BIT OF HISTORY FIRST,

Slide 4

Slide 4 text

PAUL GRIZZAFFI TWITTER.COM/PGRIZZAFFI @jennydoesthings

Slide 5

Slide 5 text

I WANT TO WARN YOU AHEAD OF TIME

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

YES.

Slide 8

Slide 8 text

LET’S TALK ABOUT DESIGNING TESTS TO FAIL NOW,

Slide 9

Slide 9 text

TESTING VERIFIES EXPECTED BEHAVIORS POINT 1:

Slide 10

Slide 10 text

I HAVE FOUND A LOT OF DEFECTS @jennydoesthings

Slide 11

Slide 11 text

MANY DEFECTS ARE NEVER FIXED @jennydoesthings

Slide 12

Slide 12 text

THIS MAKES THE BEHAVIOR EXPECTED @jennydoesthings

Slide 13

Slide 13 text

THIS INCLUDES 3RD PARTY APIS @jennydoesthings

Slide 14

Slide 14 text

AND THEN THERE’S COWBOYS @jennydoesthings

Slide 15

Slide 15 text

….MAYBE @jennydoesthings

Slide 16

Slide 16 text

AUTOMATION GIVES INFORMATION POINT 2:

Slide 17

Slide 17 text

AUTOMATION HAS NEVER FOUND A DEFECT @jennydoesthings

Slide 18

Slide 18 text

EVER. @jennydoesthings

Slide 19

Slide 19 text

AUTOMATION HAS NEVER FOUND A DEFECT @jennydoesthings

Slide 20

Slide 20 text

AUTOMATION PROVIDES INFORMATION @jennydoesthings

Slide 21

Slide 21 text

HUMANS DETERMINE WHAT INFORMATION MEANS @jennydoesthings

Slide 22

Slide 22 text

AUTOMATION PROVIDES INFORMATION ABOUT EXPECTED BEHAVIORS POINT 3:

Slide 23

Slide 23 text

TESTS SHOULD PASS ON EXPECTED BEHAVIOR @jennydoesthings

Slide 24

Slide 24 text

TESTS SHOULD PASS ON (SOME) DEFECTS @jennydoesthings

Slide 25

Slide 25 text

BUT WHY

Slide 26

Slide 26 text

KNOW THE STATE OF THE SYSTEM @jennydoesthings

Slide 27

Slide 27 text

LIMITING FAILURE FATIGUE @jennydoesthings

Slide 28

Slide 28 text

INCREASED CONFIDENCE IN OUR SUITE @jennydoesthings

Slide 29

Slide 29 text

TREATING AUTOMATION AS INFORMATION @jennydoesthings

Slide 30

Slide 30 text

I CALL THIS DESIGNING TESTS TO FAIL

Slide 31

Slide 31 text

TESTS ARE WRITTEN TO PASS @jennydoesthings

Slide 32

Slide 32 text

TESTS ARE PLANNED TO FAIL @jennydoesthings

Slide 33

Slide 33 text

DOCUMENT THE BEHAVIOR @jennydoesthings

Slide 34

Slide 34 text

HOW DO YOU DESIGN TESTS TO FAIL

Slide 35

Slide 35 text

ALL HAIL DOCUMENTATION @jennydoesthings

Slide 36

Slide 36 text

SERIOUSLY, DOCUMENT YOUR TESTS @jennydoesthings

Slide 37

Slide 37 text

THE BEHAVIOR HAS CHANGED @jennydoesthings THE TEST IS FLAKY

Slide 38

Slide 38 text

LET’S TALK ABOUT DOCUMENTATION @jennydoesthings

Slide 39

Slide 39 text

DOCUMENTATION

Slide 40

Slide 40 text

CREATE CONTEXT @jennydoesthings

Slide 41

Slide 41 text

START WITH TICKETS @jennydoesthings

Slide 42

Slide 42 text

Defect 1414 @JENNYDOESTHINGS "THE TEST IS IN THE LOGIN GROUP AND CURRENTLY VERIFIES THAT THE NEW PASSWORD IS AT LEAST SIX CHARACTERS. WHEN THIS DEFECT IS RESOLVED, THE TEST WILL NEED TO BE UPDATED TO THE CORRECT VALUE OF EIGHT CHARACTERS."

Slide 43

Slide 43 text

USE CODE COMMENTS @jennydoesthings

Slide 44

Slide 44 text

Your Code @JENNYDOESTHINGS //TODO: REVISIT WHEN DEFECT 1414 IS RESOLVED. TEST WILL NEED TO BE UPDATED FROM 6 CHARS TO 8 CHARS.

Slide 45

Slide 45 text

Xcode @JENNYDOESTHINGS

Slide 46

Slide 46 text

Xcode @JENNYDOESTHINGS

Slide 47

Slide 47 text

Xcode @JENNYDOESTHINGS

Slide 48

Slide 48 text

Your Code @JENNYDOESTHINGS //TODO: REVISIT WHEN DEFECT 1414 IS RESOLVED. TEST WILL NEED TO BE UPDATED FROM 6 CHARS TO 8 CHARS.

Slide 49

Slide 49 text

Android Studio @JENNYDOESTHINGS

Slide 50

Slide 50 text

Android Studio @JENNYDOESTHINGS

Slide 51

Slide 51 text

USE CODE COMMENTS @jennydoesthings

Slide 52

Slide 52 text

REVISIT YOUR TODOS @jennydoesthings

Slide 53

Slide 53 text

CONSIDER THE README @jennydoesthings

Slide 54

Slide 54 text

RESOLVE FAILED TESTS QUICKLY @jennydoesthings

Slide 55

Slide 55 text

IN SUMMARY

Slide 56

Slide 56 text

@JENNYDOESTHINGS WHY DO WE DESIGN TESTS TO FAIL ▸ Know the state of the system ▸ Limit failure fatigue ▸ Increase conﬁdence in our suite ▸ Treat automation as information

Slide 57

Slide 57 text

@JENNYDOESTHINGS HOW DO WE DESIGN TESTS TO FAIL ▸ Write the test to pass on expected behavior ▸ Document your tests ▸ Expect your tests to fail ▸ Resolve failures quickly

Slide 58

Slide 58 text

@JENNYDOESTHINGS FIND ME ONLINE! ▸ jenny.bramble@gmail.com ▸ http://twitter.com/jennydoesthings ▸ https://linkedin.com/in/ jennybramble ▸ http://jennydoesthings.com ▸ Willowtree!