[The Conf 2018] Stop hating your test suite

We all know that automated tests are extremely important because, among other reasons, they guarantee code quality and correct application behavior. Despite this, many times we end up with a test suite that, additionally to being hard to understand and maintain, is not trustworthy.
This talk will show you how to recognize and avoid some of the most common problems that make us hate our tests, as well as give you tips on how to improve them.

PS. The gifs don't work on speaker deck, sorry about that )):

Video available at: https://www.infoq.com/br/presentations/stop-hating-your-test-suite/

Transcript

1. @camposmilaa Stop hating your test suite! #TheConfBR @camposmilaa
2. @camposmilaa Stop hating your test suite! #TheConfBR @camposmilaa
3. @camposmilaa Camila Campos Developer @ Creditas Rails Girls organizer Women Dev Summit organizer @camposmilaa on Twitter
4. @camposmilaa Online collateral loans High quality credit Lower interest rates! creditas.com.br vagas.creditas.com.br @CreditasBR
5. @camposmilaa Include women in Tech Rails Workshop Inspiration railsgirls.com.br fb.com/railsgirls.sp @RailsGirlsSP
6. @camposmilaa 4 Communities together Encourage women in Tech Talks & Workshops & Panels womendevsummit.com fb.com/womendevsummit @WomenDevSummit
7. @camposmilaa Stop hating your test suite! @camposmilaa
8. @camposmilaa Do we hate our tests?
9. @camposmilaa WHY do we hate our tests?
10. @camposmilaa WHY do we write tests?
11. @camposmilaa we like tests
12. @camposmilaa we like testing high confidence
13. @camposmilaa we like testing high confidence understanding
14. @camposmilaa we like testing high confidence understanding
15. @camposmilaa high confidence It works
16. @camposmilaa high confidence I can refactor It works
17. @camposmilaa high confidence Didn't break anything I can refactor It works
18. @camposmilaa we like testing high confidence understanding
19. @camposmilaa understanding Easy to use
20. @camposmilaa understanding Easy to modify Easy to use
21. @camposmilaa understanding Guides design Easy to modify Easy to use
22. @camposmilaa test driven development
23. @camposmilaa
24. @camposmilaa
25. @camposmilaa
26. @camposmilaa
27. @camposmilaa test driven design
28. @camposmilaa
29. @camposmilaa WHY do we hate our tests?
30. @camposmilaa 1. 2. 3.
31. @camposmilaa Slow 1. 2. 3.
32. @camposmilaa Slow Complicated 1. 2. 3.
33. @camposmilaa Slow Complicated Unnecessary 1. 2. 3.
34. @camposmilaa Slow Complicated Unnecessary 1. 2. 3.
35. @camposmilaa Time to run vs Quantity of tests Slow tests
36. @camposmilaa Time to run vs Quantity of tests Slow tests
37. @camposmilaa Test pyramid e2e integration unit Slow tests
38. @camposmilaa End to end Slow tests
39. @camposmilaa End to end Slow tests
40. @camposmilaa Unit controller service model model service repository model Slow tests
41. @camposmilaa Unit controller service model model service repository model Slow tests
42. @camposmilaa Unit controller service model model service repository model Slow tests
43. @camposmilaa Unit unit collaborator 2 collaborator 1 Slow tests
44. @camposmilaa Unit unit test double 2 test double 1 collaborator 2 collaborator 1 Slow tests
45. @camposmilaa Unit Slow tests
46. @camposmilaa Unit Slow tests
47. @camposmilaa Integration controller service model model service repository model Slow tests
48. @camposmilaa Test pyramid e2e integration unit Slow tests
49. @camposmilaa Tests at Creditas Slow tests Quantity Time to run End to End 13 ~ 15 min Integration ~ 3k < 3 min Unit ~ 6k ~ 1 min
50. @camposmilaa Test pyramid e2e integration unit Slow tests
51. @camposmilaa Test pyramid e2e integration unit Slow tests
52. @camposmilaa My advice Slow tests
53. @camposmilaa Test pyramid Slow tests
54. @camposmilaa Test pyramid unit Slow tests
55. @camposmilaa Test pyramid e2e unit Slow tests
56. @camposmilaa Test pyramid e2e integration unit Slow tests
57. @camposmilaa Rules and Conventions Data (DB) External dependencies Collaborators Slow tests
58. @camposmilaa Rules and Conventions Slow tests Database External Dependencies Collaborators End to End Yes Yes* Yes Integration Yes No Yes Unit No No No
59. @camposmilaa Test pyramid e2e integration unit Slow tests
60. @camposmilaa Test pyramid e2e integration unit Slow tests When changing a class When testing a feature Before commiting
61. @camposmilaa Slow Complicated Unnecessary 1. 2. 3.
62. @camposmilaa Tests that are hard to read, maintain and/or understand Complicated tests
63. @camposmilaa 1. 2. 3. 4. Unclear Messy Obscure Too smart Complicated tests
64. @camposmilaa 1. 2. 3. 4. Unclear Messy Obscure Too smart Complicated tests
65. @camposmilaa Test is something that guarantees that your "real" code is working Complicated tests > Unclear
66. @camposmilaa Test is code (that guarantees the "real" code is working) Complicated tests > Unclear
67. @camposmilaa Complicated tests > Unclear Test is code untested
68. @camposmilaa
69. @camposmilaa Complicated tests > Unclear Test is code untested
70. @camposmilaa Test is code Complicated tests > Unclear untested Simple
71. @camposmilaa Complicated tests > Unclear Small Simple Test is code untested
72. @camposmilaa Complicated tests > Unclear Easy to read Small Simple Test is code untested
73. @camposmilaa Complicated tests > Unclear Easy to read Small Simple Right to the point Test is code untested
74. @camposmilaa For example Slow tests
75. @camposmilaa it 'sends a message' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
76. @camposmilaa it 'sends a message' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
77. @camposmilaa it 'sends a message' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
78. @camposmilaa it 'sends a message' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
79. @camposmilaa it 'sends a message' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
80. @camposmilaa Bad description it 'sends a message' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
81. @camposmilaa Bad description it 'sends a message' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
82. @camposmilaa Bad description it 'greets a person' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
83. @camposmilaa Bad description it 'greets a person' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
84. @camposmilaa it 'greets a person' do person = Person.new(name: 'John', last_name: 'Doe', age: 27) response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
85. @camposmilaa it 'greets a person' do person = Person.new(name: 'John', last_name: 'Doe', age: 27) response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
86. @camposmilaa it 'greets a person' do person = Person.new(name: 'John', last_name: 'Doe', age: 27) response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
87. @camposmilaa Misleading unimportant stuff it 'greets a person' do person = Person.new(name: 'John', last_name: 'Doe', age: 27) response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
88. @camposmilaa Misleading unimportant stuff it 'greets a person' do person = Person.new(name: 'John') response = Messenger.new(person).greet expect(response).to eq 'Hello, John' end Complicated tests > Unclear
89. @camposmilaa Misleading unimportant stuff it 'greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).greet expect(response).to eq 'Hello, any-name' end Complicated tests > Unclear
90. @camposmilaa 1. 2. 3. 4. Unclear Messy Obscure Too smart Complicated tests
91. @camposmilaa Tests can do 3 things Complicated tests > Messy
92. @camposmilaa Set up everything needed Complicated tests > Messy
93. @camposmilaa Set up everything needed Call whatever is being tested Complicated tests > Messy
94. @camposmilaa Set up everything needed Call whatever is being tested Verify expected behavior Complicated tests > Messy
95. @camposmilaa Complicated tests > Messy A A A
96. @camposmilaa Complicated tests > Messy Arrange A A
97. @camposmilaa Complicated tests > Messy Arrange Act A
98. @camposmilaa Complicated tests > Messy Arrange Act Assert
99. @camposmilaa Setup Complicated tests > Messy Arrange Act Assert
100. @camposmilaa Setup Exercise Complicated tests > Messy Arrange Act Assert
101. @camposmilaa Setup Exercise Verify Complicated tests > Messy Arrange Act Assert
102. @camposmilaa Complicated tests > Messy
103. @camposmilaa 3 things test do it 'greets a person' do person = Person.new(name: 'any-name') # Setup response = Messenger.new(person).greet # Exercise expect(response).to eq 'Hello, any-name' # Verify end Complicated tests > Messy
104. @camposmilaa 3 things test do it 'greets a person' do person = Person.new(name: 'any-name') # Setup response = Messenger.new(person).greet # Exercise expect(response).to eq 'Hello, any-name' # Verify end Complicated tests > Messy
105. @camposmilaa 3 things test do it 'greets a person' do person = Person.new(name: 'any-name') # Setup response = Messenger.new(person).greet # Exercise expect(response).to eq 'Hello, any-name' # Verify end Complicated tests > Messy
106. @camposmilaa it 'greets a person' do person = Person.new(name: 'any-name') # Setup response = Messenger.new(person).greet # Exercise expect(response).to eq 'Hello, any-name' # Verify end 3 things test do Complicated tests > Messy
107. @camposmilaa 3 things test do it 'greets a person' do person = Person.new(name: 'any-name') # Setup response = Messenger.new(person).greet # Exercise expect(response).to eq 'Hello, any-name' # Verify end Complicated tests > Messy
108. @camposmilaa 3 things test do it 'greets a person' do person = Person.new(name: 'any-name') # Setup response = Messenger.new(person).greet # Exercise expect(response).to eq 'Hello, any-name' # Verify end Complicated tests > Messy
109. @camposmilaa 3 things test do it 'sends email to person' do mailer = double('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup expect(mailer).to receive(:notify).with(person.email) # Verify Messenger.new(person, mailer: mailer).send_email # Exercise end Complicated tests > Messy
110. @camposmilaa 3 things test do it 'sends email to person' do mailer = double('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup expect(mailer).to receive(:notify).with(person.email) # Verify Messenger.new(person, mailer: mailer).send_email # Exercise end Complicated tests > Messy
111. @camposmilaa 3 things test do it 'sends email to person' do mailer = double('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup expect(mailer).to receive(:notify).with(person.email) # Verify Messenger.new(person, mailer: mailer).send_email # Exercise end Complicated tests > Messy
112. @camposmilaa 3 things test do it 'sends email to person' do mailer = double('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup expect(mailer).to receive(:notify).with(person.email) # Verify Messenger.new(person, mailer: mailer).send_email # Exercise end Complicated tests > Messy
113. @camposmilaa 3 things test do it 'sends email to person' do mailer = double('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup expect(mailer).to receive(:notify).with(person.email) # Verify Messenger.new(person, mailer: mailer).send_email # Exercise end Complicated tests > Messy
114. @camposmilaa 3 things test do it 'sends email to person' do mailer = double('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup expect(mailer).to receive(:notify).with(person.email) # Verify Messenger.new(person, mailer: mailer).send_email # Exercise end Complicated tests > Messy
115. @camposmilaa 3 things test do it 'sends email to person' do mailer = spy('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup Messenger.new(person, mailer: mailer).send_email # Exercise expect(mailer).to have_received(:notify).with(person.email) # Verify end Complicated tests > Messy
116. @camposmilaa 3 things test do it 'sends email to person' do mailer = spy('my mailer') # Setup person = Person.new(name: 'any-name', email: 'e@mail.com') # Setup Messenger.new(person, mailer: mailer).send_email # Exercise expect(mailer).to have_received(:notify).with(person.email) # Verify end Complicated tests > Messy
117. @camposmilaa 1. 2. 3. 4. Unclear Messy Obscure Too smart Complicated tests
118. @camposmilaa
119. @camposmilaa Obscure tests Complicated tests > Obscure
120. @camposmilaa Obscure tests it 'greets a person' do # ... end it 'gracefully greets a person' do # ... end Complicated tests > Obscure
121. @camposmilaa Obscure tests it 'greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).greet expect(response).to eq 'Hello, any-name' end it 'gracefully greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).fancy_greet expect(response).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
122. @camposmilaa Obscure tests it 'greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).greet expect(response).to eq 'Hello, any-name' end it 'gracefully greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).fancy_greet expect(response).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
123. @camposmilaa Obscure tests it 'greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).greet expect(response).to eq 'Hello, any-name' end it 'gracefully greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).fancy_greet expect(response).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
124. @camposmilaa Obscure tests it 'greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).greet expect(response).to eq 'Hello, any-name' end it 'gracefully greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).fancy_greet expect(response).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
125. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } it 'greets a person' do response = Messenger.new(person).greet expect(response).to eq 'Hello, any-name' end it 'gracefully greets a person' do response = Messenger.new(person).fancy_greet expect(response).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
126. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'greets a person' do expect(messenger.greet).to eq 'Hello, any-name' end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
127. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'greets a person' do expect(messenger.greet).to eq 'Hello, any-name' end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
128. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'greets a person' do expect(messenger.greet).to eq 'Hello, any-name' end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
129. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
130. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'does an entirely different thing' do # .... end it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
131. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'does nothing sometimes' do # .... end it 'does an entirely different thing' do # .... end it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
132. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'does any other thing' do # .... end it 'does nothing sometimes' do # .... end it 'does an entirely different thing' do # .... end it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
133. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'does something else' do # .... end it 'does any other thing' do # .... end it 'does nothing sometimes' do # .... end it 'does an entirely different thing' do # .... end it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
134. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'does something' do # .... end it 'does something else' do # .... end it 'does any other thing' do # .... end it 'does nothing sometimes' do # .... end it 'does an entirely different thing' do # .... end it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
135. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'does something' do # .... end it 'does something else' do # .... end it 'does any other thing' do # .... end it 'does nothing sometimes' do # .... end it 'does an entirely different thing' do # .... end it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
136. @camposmilaa Obscure tests let(:person) { Person.new(name: 'any-name') } let(:messenger) { Messenger.new(person) } it 'does something' do # .... end it 'does something else' do # .... end it 'does any other thing' do # .... end it 'does nothing sometimes' do # .... end it 'does an entirely different thing' do # .... end it 'greets a person' do expect(messenger.greet).to eq "Hello, #{person.name}" end it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
137. @camposmilaa Mystery Guest it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
138. @camposmilaa Mystery Guest it 'gracefully greets a person' do expect(messenger.fancy_greet).to eq "Good afternoon, ma'am #{person.name}!" end Complicated tests > Obscure
139. @camposmilaa Mystery Guest it 'greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).greet expect(response).to eq 'Hello, any-name' end it 'gracefully greets a person' do person = Person.new(name: 'any-name') response = Messenger.new(person).fancy_greet expect(response).to eq "Good afternoon, ma'am any-name!" end Complicated tests > Obscure
140. @camposmilaa 1. 2. 3. 4. Unclear Messy Obscure Too smart Complicated tests
141. @camposmilaa Real code needs some refactor Complicated tests > Too smart
142. @camposmilaa Refactor time! Complex/Big setup
143. @camposmilaa Refactor time! Complex/Big setup Too many dependencies
144. @camposmilaa Refactor time! Complex/Big setup Multiple verifications Too many dependencies
145. @camposmilaa Refactor time! Complex/Big setup Multiple verifications Too many dependencies Too many responsibilities
146. @camposmilaa Refactor time! Complex/Big setup Multiple verifications Multiple "exercise" steps Too many dependencies Too many responsibilities
147. @camposmilaa Refactor time! Complex/Big setup Multiple verifications Multiple "exercise" steps Too many dependencies Too many responsibilities Complex API
148. @camposmilaa Refactor time! Complex/Big setup Multiple verifications Complex code on test (ex. Loops, private method calls) Multiple "exercise" steps Too many dependencies Too many responsibilities Complex API
149. @camposmilaa Refactor time! Multiple "exercise" steps Complex/Big setup Multiple verifications Complex code on test (ex. Loops, private method calls) Multiple "exercise" steps Too many dependencies Too many responsibilities Complex code Complex API
150. @camposmilaa Slow Complicated Unnecessary 1. 2. 3.
151. @camposmilaa Are intermittent Unnecessary tests
152. @camposmilaa Are intermittent Are order dependent Unnecessary tests
153. @camposmilaa Are intermittent Are order dependent Are fragile Unnecessary tests
154. @camposmilaa Redundant tests controller service model repository Unnecessary tests
155. @camposmilaa Redundant tests controller service model repository A B Unnecessary tests
156. @camposmilaa Redundant tests controller service model repository A B A B Unnecessary tests
157. @camposmilaa Redundant tests controller service model repository A B A B A B Unnecessary tests
158. @camposmilaa Redundant tests controller service model repository A B A B A B C Unnecessary tests
159. @camposmilaa Redundant tests controller service model repository A B A B A B C C Unnecessary tests
160. @camposmilaa Redundant tests controller service model repository A B A B A B C C C Unnecessary tests
161. @camposmilaa Unnecessary tests Redundant tests
162. @camposmilaa Mock everything Unnecessary tests Redundant tests
163. @camposmilaa Redundant tests Fewer tests Mock everything Unnecessary tests
164. @camposmilaa Slow Complicated Unnecessary 1. 2. 3.
165. @camposmilaa Slow Complicated Unnecessary 1. 2. 3.
166. @camposmilaa
167. @camposmilaa Bad tests are better than none
168. @camposmilaa Bad tests are better than none No tests are better than bad ones
169. @camposmilaa Thank you! bit.ly/theconf-camila camilacamposramos camposmilaa