Slide 1
Slide 1 text
WHO WILL TEST
YOUR TESTS?
Shirish Padalkar
VodQA, Pune. November 2014
1
Slide 2
Slide 2 text
ABOUT ME
2
Slide 3
Slide 3 text
ABOUT ME
3
Slide 4
Slide 4 text
ABOUT ME
4
Slide 5
Slide 5 text
WE NEED TO TALK!
5
Slide 6
Slide 6 text
SOME QUESTIONS FIRST
6
https://flic.kr/p/2vgyWN
Slide 7
Slide 7 text
DOES YOUR PROJECT HAVE
AN AUTOMATION TEST SUITE?
7
Slide 8
Slide 8 text
HOW OFTEN YOU NEED TO
CHANGE / REFACTOR
YOUR TEST SUITE / FRAMEWORK?
8
Slide 9
Slide 9 text
DO YOU FEEL
COMFORTABLE
UPDATING YOUR TESTS?
9
Slide 10
Slide 10 text
HOW IS THE
CODE QUALITY
OF YOUR TEST SUITE?
10
Slide 11
Slide 11 text
DO YOU
TRUST
YOUR TESTS AND TEST SUITE?
11
Slide 12
Slide 12 text
LET’S TALK!
12
Slide 13
Slide 13 text
WHY DO WE
WRITE TESTS?
13
https://flic.kr/p/9uZhDS
Slide 14
Slide 14 text
WHY WE WRITE TESTS?
14
TESTS PROVE THAT THE
CODE ACTUALLY WORKS
Slide 15
Slide 15 text
WHY WE WRITE TESTS?
15
TESTS REDUCE BUGS
Slide 16
Slide 16 text
WHY WE WRITE TESTS?
16
TESTS REDUCE
THE COST OF CHANGE
Slide 17
Slide 17 text
WHY WE WRITE TESTS?
17
TESTS ALLOW
REFACTORING
Slide 18
Slide 18 text
WHY WE WRITE TESTS?
18
TESTING HELPS IN
FASTER DEVELOPMENT
Slide 19
Slide 19 text
WHY WE WRITE TESTS?
19
TESTS REDUCE FEAR
Slide 20
Slide 20 text
20
Slide 21
Slide 21 text
21
NOW
WHEN YOU HEAR…
Slide 22
Slide 22 text
Oh, it failed?
Don’t worry. Just re-trigger it! ;)
22
Slide 23
Slide 23 text
Oh, it failed?
Strange; it passed on my machine!
23
Slide 24
Slide 24 text
Oh, it failed?
I know. It fails at 12PM!
24
Slide 25
Slide 25 text
25
http://upload.wikimedia.org/wikipedia/commons/3/3b/Paris_Tuileries_Garden_Facepalm_statue.jpg
Slide 26
Slide 26 text
WHY DO WE
WRITE TESTS?
26
Now, again,
Slide 27
Slide 27 text
WHY WE WRITE TESTS?
27
Tests prove that the code actually works
Tests prove that the code sometimes works
Slide 28
Slide 28 text
WHY WE WRITE TESTS?
28
Tests reduce bugs
Tests itself might have bugs
Slide 29
Slide 29 text
WHY WE WRITE TESTS?
29
Tests reduce the Cost of Change
Change? Are you kidding?
I will need a week to change my framework
and make sure everything works!
Slide 30
Slide 30 text
WHY WE WRITE TESTS?
30
Tests allow refactoring
Slide 31
Slide 31 text
WHY WE WRITE TESTS?
31
Testing helps in faster development.
Testing is slowing me down :(
I am not sure when to stop writing code.
Tests keep randomly failing and succeeding.
Slide 32
Slide 32 text
WHY WE WRITE TESTS?
32
Tests reduce fear
Don’t touch that code and tests.
You will break them!
Slide 33
Slide 33 text
WHY?
33
Slide 34
Slide 34 text
NON-DETERMINISTIC
TESTS
34
A.K.A. “Flaky tests”
Slide 35
Slide 35 text
WHY FLAKY TESTS ARE PROBLEM
35
THEY ARE
HARMFUL TESTS!
Slide 36
Slide 36 text
WHY FLAKY TESTS ARE PROBLEM
36
THEY
BREAK YOUR TRUST!
A red build doesn’t mean anything anymore!
Slide 37
Slide 37 text
WHY FLAKY TESTS ARE PROBLEM
37
THEY DESTROY THE VALUE
OF
AUTOMATED REGRESSION
Slide 38
Slide 38 text
WHY FLAKY TESTS ARE PROBLEM
38
THEY GIVE YOU
ILLUSION OF SUCCESS!
Slide 39
Slide 39 text
WHY FLAKY TESTS ARE PROBLEM
39
THEY GIVE YOU
ILLUSION OF FAILURE!
Slide 40
Slide 40 text
WHY FLAKY TESTS ARE PROBLEM
40
THEY
PROMOTE BAD CULTURE!
Culture of living in failures!
Slide 41
Slide 41 text
CAUSES OF FLAKINESS
41
Slide 42
Slide 42 text
CAUSES OF FLAKINESS
42
LACK OF ISOLATION
Slide 43
Slide 43 text
CAUSES OF FLAKINESS
43
SHARED STATE
IN APPLICATION
Slide 44
Slide 44 text
CAUSES OF FLAKINESS
44
ASYNCHRONOUS
BEHAVIOR
Slide 45
Slide 45 text
CAUSES OF FLAKINESS
45
EXTERNAL
SYSTEM CALLS
Slide 46
Slide 46 text
CAUSES OF FLAKINESS
46
TIME
Slide 47
Slide 47 text
HOW TO
FIX FLAKINESS
47
Slide 48
Slide 48 text
48
Slide 49
Slide 49 text
49
Slide 50
Slide 50 text
50
Slide 51
Slide 51 text
HOW TO FIX FLAKY TESTS
51
DO NOT THINK
BUILD RANDOMLY FAILED.
Slide 52
Slide 52 text
HOW TO FIX FLAKY TESTS
52
THINK
BUILD RANDOMLY PASSED!
Slide 53
Slide 53 text
HOW TO FIX FLAKY TESTS
53
QUARANTINE
Slide 54
Slide 54 text
HOW TO FIX FLAKY TESTS
54
KEEP ONLY N TESTS
IN QUARANTINE
Let’s say, 5 tests at any given point
Slide 55
Slide 55 text
HOW TO FIX FLAKY TESTS
55
ALLOCATE TIME TO
FIX QUARANTINED TESTS
Slide 56
Slide 56 text
HOW TO FIX FLAKY TESTS
56
UNDERSTAND
NATURE OF FLAKINESS
Most flakiness is due to our own mistakes!
Slide 57
Slide 57 text
HOW TO FIX FLAKY TESTS
57
YOU KNOW
THERE MIGHT BE
BUGS IN PRODUCTION CODE
Slide 58
Slide 58 text
HOW TO FIX FLAKY TESTS
58
YOU KNOW
THERE MIGHT BE
BUGS IN PRODUCTION CODE
TEST
Slide 59
Slide 59 text
HOW TO FIX FLAKY TESTS
59
REFACTOR YOUR TESTS!
Slide 60
Slide 60 text
HOW TO FIX FLAKY TESTS
60
RANDOM WAITS
DUPLICATE CODE
GOD OBJECTS
COMPLICATED CODE
Slide 61
Slide 61 text
REMEMBER!
61
Slide 62
Slide 62 text
62
A BUG IS
A TEST YOU FORGOT TO WRITE
Slide 63
Slide 63 text
63
TESTER IS A ROLE
NOT A PERSON
Slide 64
Slide 64 text
64
TESTER IS
NOT A FAILED DEVELOPER
Slide 65
Slide 65 text
65
LOVE YOUR TEST CODE.
Treat your test code as production code!
Slide 66
Slide 66 text
66
DO NOT TOLERATE
BAD TEST CODE
Slide 67
Slide 67 text
67
TALK TO THE TEAM
IF WRITING THE TEST IS HARD
There might a code smell lying somewhere
Slide 68
Slide 68 text
TESTING IS FUN.
DON’T MAKE PEOPLE HATE IT!
68
Slide 69
Slide 69 text
ANY
QUESTIONS?
69
Slide 70
Slide 70 text
RESOURCES
▫︎Eradicating Non-Determinism in Tests - by Martin Fowler
▫︎No more flaky tests on the Go team - by Pavan
Sudarshan
▫︎Top 12 Reasons to Write Unit Tests - by Eric M. Burke
and Brian M. Coyner
▫︎Flaky crusts: Good for pies; bad for test suites
▫︎Creating Maintainable Automated Acceptance Tests - By
Jez humble and Badri
70
Slide 71
Slide 71 text
ThoughtWorks is hiring.
http://www.thoughtworks.com/join
THANK YOU!