BDD is not about Testing

BDD started as a way to teach TDD to programmers who kept getting hung up on the idea they were writing tests. Fast-forward a decade or so and it seems BDD scenario automation tools have invaded the world of acceptance testing like Japanese knotweed. All around I see teams harming themselves writing awkward, verbose tests using slow, cumbersome tools like Cucumber and SpecFlow, and acting as though BDD is some kind of testing approach.

Part of the problem is that once you have an automated BDD scenario and you've written the software to satisfy it, it can look seductively like a test. From there it is a short step to thinking of these scenario automation tools as testing software, and the rest is frustrating, repetitive history.

This long-overdue talk explores the relationship between BDD scenarios and acceptance tests, and suggests some strategies for avoiding the pain of BDD-as-test automation.

Transcript

1. Behaviour-Driven Development is not about Testing Dan North @tastapod
2. Motivation
3. Motivation Testers feeling “left behind” by Agile
4. Motivation Testers feeling “left behind” by Agile Testers writing hundreds of “BDDs”
5. Motivation Testers feeling “left behind” by Agile Testers writing hundreds of “BDDs” Hiring for “automation testers”
6. Motivation Testers feeling “left behind” by Agile Testers writing hundreds of “BDDs” Hiring for “automation testers” Generally poor training around “agile testing”
7. Motivation Testers feeling “left behind” by Agile Testers writing hundreds of “BDDs” Hiring for “automation testers” Generally poor training around “agile testing” Testers unsettled about the role of SET/SDET
8. tl;dr
9. tl;dr 1. Programmers don’t understand testing
10. tl;dr 1. Programmers don’t understand testing 2. Agile doesn’t understand testing
11. tl;dr 1. Programmers don’t understand testing 2. Agile doesn’t understand testing 3. BDD is appealing to testers
12. tl;dr 1. Programmers don’t understand testing 2. Agile doesn’t understand testing 3. BDD is appealing to testers 4. BDD is not about testing
13. We need to talk about testing…
14. Why do we have testing?
15. Why do we have testing? Mandated by the SDLC
16. Why do we have testing? Mandated by the SDLC Programmers see testing as an irritation
17. Why do we have testing? Mandated by the SDLC Programmers see testing as an irritation Project managers see testing as a source of risk
18. Why do we have testing? Mandated by the SDLC Programmers see testing as an irritation Project managers see testing as a source of risk Testing is always the thing that gets squeezed
19. Why do we have testing? Mandated by the SDLC Programmers see testing as an irritation Project managers see testing as a source of risk Testing is always the thing that gets squeezed Non-differentiating, non-critical
20. Purpose Alignment model Business Critical Differentiating
21. Purpose Alignment model Business Critical Differentiating “Who cares?”
22. Purpose Alignment model Business Critical Differentiating “Who cares?” Parity
23. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Parity
24. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner “Invest and Excel” Parity
25. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner “Invest and Excel” Parity Testing!
26. The goal of testing…
27. The goal of testing… “to find information”?
28. The goal of testing… “to find information”? “to communicate”?
29. The goal of testing… “to find information”? “to communicate”? to enable us to “reasonably believe that the probability is low that the product still has important undiscovered problems”?
30. –Kent Beck "I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence." http://stackoverflow.com/a/153565/632259
31. The goal of testing…
32. The goal of testing… to increase confidence for stakeholders through evidence
33. The goal of testing… to increase confidence for stakeholders through evidence
34. The goal of testing… to increase confidence for stakeholders through evidence
35. The goal of testing… to increase confidence for stakeholders through evidence
36. A good tester has 3 superpowers… to increase confidence for stakeholders through evidence
37. A good tester has 3 superpowers… to increase confidence for stakeholders through evidence Empathy
38. A good tester has 3 superpowers… to increase confidence for stakeholders through evidence Empathy Ingenuity
39. A good tester has 3 superpowers… to increase confidence for stakeholders through evidence Empathy Balance Ingenuity
40. 1. Programmers don’t understand testing
47. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing! “Invest and Excel” Parity
48. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing! “Invest and Excel” Parity
49. 2. Agile doesn’t understand testing
50. Agile doesn’t understand testing
51. Agile doesn’t understand testing Agile methods were invented by programmers
52. Agile doesn’t understand testing Agile methods were invented by programmers Testing is implied but never explicit
53. Agile doesn’t understand testing Agile methods were invented by programmers Testing is implied but never explicit The tester role is missing or poorly defined
54. Agile Testing Training…
55. Agile Testing Training…
56. Agile Testing Training…
57. Agile Testing Training…
58. Agile Testing Training…
59. Agile Testing Training…
60. Agile Testing Training… Certified Agile Tester (CAT)
61. Agile Testing Training… Certified Agile Tester (CAT)
62. Agile Testing Training… Certified Agile Tester (CAT)
63. Agile Testing Training… Certified Agile Tester (CAT)
64. Agile Testing Training… Certified Agile Tester (CAT)
65. Agile Testing Training… Certified Agile Tester (CAT)
66. Agile Testing Training… Certified Agile Tester (CAT)
67. Agile Testing Training… Certified Agile Tester (CAT)
68. Agile Testing Training… Certified Agile Tester (CAT)
69. Agile Testing Training… Certified Agile Tester (CAT)
70. Agile Testing Training… Certified Agile Tester (CAT) WAT?
71. 3. BDD is appealing to testers
72. BDD is appealing to testers…
73. BDD is appealing to testers… We acknowledge multiple stakeholders
74. BDD is appealing to testers… We acknowledge multiple stakeholders We discuss acceptance criteria from the outset
75. BDD is appealing to testers… We acknowledge multiple stakeholders We discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders!
76. BDD is appealing to testers… We acknowledge multiple stakeholders We discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders! We increase confidence for stakeholders through evidence
77. BDD is appealing to testers… We acknowledge multiple stakeholders We discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders! We increase confidence for stakeholders through evidence It’s not surprising BDD often starts with testers
78. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing! “Invest and Excel” Parity
79. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing! “Invest and Excel” Parity
80. 4. BDD is not about testing!
81. The lifecycle of a scenario
82. The lifecycle of a scenario Starts as enabling constraint to guide the design
83. The lifecycle of a scenario Starts as enabling constraint to guide the design Then: checks* the solution
84. The lifecycle of a scenario Starts as enabling constraint to guide the design Then: checks* the solution May be useful as documentation
85. The lifecycle of a scenario Starts as enabling constraint to guide the design Then: checks* the solution May be useful as documentation May form the basis of automated tests
86. The lifecycle of a scenario Starts as enabling constraint to guide the design Then: checks* the solution May be useful as documentation May form the basis of automated tests *check: (vt) inspect, investigate, examine, verify
87. Scenario as the basis of tests
88. Scenario as the basis of tests What general case is this an example of?
89. Scenario as the basis of tests What general case is this an example of? How could I gain confidence in the general case?
90. Scenario as the basis of tests What general case is this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need?
91. Scenario as the basis of tests What general case is this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need? Which stakeholders have we not yet served?
92. Scenario as the basis of tests What general case is this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need? Which stakeholders have we not yet served? Scenarios are necessary but not sufficient for testing
93. What does a good automated test look like?
94. What does a good automated test look like? Intention-revealing name
95. What does a good automated test look like? Intention-revealing name Intention-revealing errors
96. What does a good automated test look like? Intention-revealing name Intention-revealing errors Consistent language
97. What does a good automated test look like? Intention-revealing name Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent
98. What does a good automated test look like? Intention-revealing name Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
99. What does a typical gherkin scenario look like? Intention-revealing name Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
100. What does a typical gherkin scenario look like? Intention-revealing name Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
101. What does a typical gherkin scenario look like? Intention-revealing name Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
102. What does a typical gherkin scenario look like? Intention-revealing name Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action” ??
103. What does a typical gherkin scenario look like? Intention-revealing name Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action” ?? ??
104. Good scenarios != Good tests
105. BDD is not about testing, however…
106. BDD is not about testing, however… Testing is complementary to BDD
107. BDD is not about testing, however… Testing is complementary to BDD Testing is the sufficient to BDD’s necessary
108. BDD is not about testing, however… Testing is complementary to BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery
109. BDD is not about testing, however… Testing is complementary to BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery Testability starts with design
110. BDD is not about testing, however… Testing is complementary to BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery Testability starts with design software that matters, for “people whose lives you touch”
111. Thank you! @tastapod http://dannorth.net Enjoy BDDx 2016