BDD is not about Testing

BDD started as a way to teach TDD to programmers who kept getting hung up on the idea they were writing tests. Fast-forward a decade or so and it seems BDD scenario automation tools have invaded the world of acceptance testing like Japanese knotweed. All around I see teams harming themselves writing awkward, verbose tests using slow, cumbersome tools like Cucumber and SpecFlow, and acting as though BDD is some kind of testing approach.

Part of the problem is that once you have an automated BDD scenario and you've written the software to satisfy it, it can look seductively like a test. From there it is a short step to thinking of these scenario automation tools as testing software, and the rest is frustrating, repetitive history.

This long-overdue talk explores the relationship between BDD scenarios and acceptance tests, and suggests some strategies for avoiding the pain of BDD-as-test automation.

Transcript

1. Behaviour-Driven Development
   is not about Testing
   Dan North
   @tastapod
   

   View Slide

2. Motivation
   

   View Slide

3. Motivation
   Testers feeling “left behind” by Agile
   

   View Slide

4. Motivation
   Testers feeling “left behind” by Agile
   Testers writing hundreds of “BDDs”
   

   View Slide

5. Motivation
   Testers feeling “left behind” by Agile
   Testers writing hundreds of “BDDs”
   Hiring for “automation testers”
   

   View Slide

6. Motivation
   Testers feeling “left behind” by Agile
   Testers writing hundreds of “BDDs”
   Hiring for “automation testers”
   Generally poor training around “agile testing”
   

   View Slide

7. Motivation
   Testers feeling “left behind” by Agile
   Testers writing hundreds of “BDDs”
   Hiring for “automation testers”
   Generally poor training around “agile testing”
   Testers unsettled about the role of SET/SDET
   

   View Slide

8. tl;dr
   

   View Slide

9. tl;dr
   1. Programmers don’t understand testing
   

   View Slide

10. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing
    

    View Slide

11. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing
    3. BDD is appealing to testers
    

    View Slide

12. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing
    3. BDD is appealing to testers
    4. BDD is not about testing
    

    View Slide

13. We need to talk about testing…
    

    View Slide

14. Why do we have testing?
    

    View Slide

15. Why do we have testing?
    Mandated by the SDLC
    

    View Slide

16. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    

    View Slide

17. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk
    

    View Slide

18. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk
    Testing is always the thing that gets squeezed
    

    View Slide

19. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk
    Testing is always the thing that gets squeezed
    Non-differentiating, non-critical
    

    View Slide

20. Purpose Alignment model
    Business Critical
    Differentiating
    

    View Slide

21. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    

    View Slide

22. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?” Parity
    

    View Slide

23. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Parity
    

    View Slide

24. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner “Invest and Excel”
    Parity
    

    View Slide

25. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner “Invest and Excel”
    Parity
    Testing!
    

    View Slide

26. The goal of testing…
    

    View Slide

27. The goal of testing…
    “to find information”?
    

    View Slide

28. The goal of testing…
    “to find information”?
    “to communicate”?
    

    View Slide

29. The goal of testing…
    “to find information”?
    “to communicate”?
    to enable us to “reasonably believe that the probability is low that
    the product still has important undiscovered problems”?
    

    View Slide

30. –Kent Beck
    "I get paid for code that works, not for tests, so
    my philosophy is to test as little as possible to
    reach a given level of confidence."
    http://stackoverflow.com/a/153565/632259
    

    View Slide

31. The goal of testing…
    

    View Slide

32. The goal of testing…
    to increase confidence for stakeholders through evidence
    

    View Slide

33. The goal of testing…
    to increase confidence for stakeholders through evidence
    

    View Slide

34. The goal of testing…
    to increase confidence for stakeholders through evidence
    

    View Slide

35. The goal of testing…
    to increase confidence for stakeholders through evidence
    

    View Slide

36. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    

    View Slide

37. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy
    

    View Slide

38. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy
    Ingenuity
    

    View Slide

39. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy
    Balance
    Ingenuity
    

    View Slide

40. 1. Programmers don’t understand testing
    

    View Slide

41. View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity
    

    View Slide

48. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity
    

    View Slide

49. 2. Agile doesn’t understand testing
    

    View Slide

50. Agile doesn’t understand testing
    

    View Slide

51. Agile doesn’t understand testing
    Agile methods were invented by programmers
    

    View Slide

52. Agile doesn’t understand testing
    Agile methods were invented by programmers
    Testing is implied but never explicit
    

    View Slide

53. Agile doesn’t understand testing
    Agile methods were invented by programmers
    Testing is implied but never explicit
    The tester role is missing or poorly defined
    

    View Slide

54. Agile Testing Training…
    

    View Slide

55. Agile Testing Training…
    

    View Slide

56. Agile Testing Training…
    

    View Slide

57. Agile Testing Training…
    

    View Slide

58. Agile Testing Training…
    

    View Slide

59. Agile Testing Training…
    

    View Slide

60. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

61. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

62. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

63. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

64. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

65. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

66. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

67. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

68. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

69. Agile Testing Training…
    Certified Agile Tester (CAT)
    

    View Slide

70. Agile Testing Training…
    Certified Agile Tester (CAT)
    WAT?
    

    View Slide

71. 3. BDD is appealing to testers
    

    View Slide

72. BDD is appealing to testers…
    

    View Slide

73. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    

    View Slide

74. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    

    View Slide

75. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!
    

    View Slide

76. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!
    We increase confidence for stakeholders through evidence
    

    View Slide

77. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!
    We increase confidence for stakeholders through evidence
    It’s not surprising BDD often starts with testers
    

    View Slide

78. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity
    

    View Slide

79. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity
    

    View Slide

80. 4. BDD is not about testing!
    

    View Slide

81. The lifecycle of a scenario
    

    View Slide

82. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    

    View Slide

83. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    

    View Slide

84. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation
    

    View Slide

85. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation
    May form the basis of automated tests
    

    View Slide

86. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation
    May form the basis of automated tests
    *check: (vt) inspect, investigate, examine, verify
    

    View Slide

87. Scenario as the basis of tests
    

    View Slide

88. Scenario as the basis of tests
    What general case is this an example of?
    

    View Slide

89. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    

    View Slide

90. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?
    

    View Slide

91. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?
    Which stakeholders have we not yet served?
    

    View Slide

92. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?
    Which stakeholders have we not yet served?
    Scenarios are necessary but not sufficient for testing
    

    View Slide

93. What does a good automated test look like?
    

    View Slide

94. What does a good automated test look like?
    Intention-revealing name
    

    View Slide

95. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    

    View Slide

96. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    

    View Slide

97. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    

    View Slide

98. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”
    

    View Slide

99. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”
    

    View Slide

100. What does a typical gherkin scenario look like?
     Intention-revealing name
     Intention-revealing errors
     Consistent language
     The intent, the whole intent and nothing but the intent
     Fast! “Close to the action”
     

     View Slide

101. What does a typical gherkin scenario look like?
     Intention-revealing name
     Intention-revealing errors
     Consistent language
     The intent, the whole intent and nothing but the intent
     Fast! “Close to the action”
     

     View Slide

102. What does a typical gherkin scenario look like?
     Intention-revealing name
     Intention-revealing errors
     Consistent language
     The intent, the whole intent and nothing but the intent
     Fast! “Close to the action”
     ??
     

     View Slide

103. What does a typical gherkin scenario look like?
     Intention-revealing name
     Intention-revealing errors
     Consistent language
     The intent, the whole intent and nothing but the intent
     Fast! “Close to the action”
     ??
     ??
     

     View Slide

104. Good scenarios != Good tests
     

     View Slide

105. BDD is not about testing, however…
     

     View Slide

106. BDD is not about testing, however…
     Testing is complementary to BDD
     

     View Slide

107. BDD is not about testing, however…
     Testing is complementary to BDD
     Testing is the sufficient to BDD’s necessary
     

     View Slide

108. BDD is not about testing, however…
     Testing is complementary to BDD
     Testing is the sufficient to BDD’s necessary
     Test thinking is critical to successful delivery
     

     View Slide

109. BDD is not about testing, however…
     Testing is complementary to BDD
     Testing is the sufficient to BDD’s necessary
     Test thinking is critical to successful delivery
     Testability starts with design
     

     View Slide

110. BDD is not about testing, however…
     Testing is complementary to BDD
     Testing is the sufficient to BDD’s necessary
     Test thinking is critical to successful delivery
     Testability starts with design
     software that matters, for “people whose lives you touch”
     

     View Slide

111. Thank you!
     @tastapod
     http://dannorth.net
     Enjoy BDDx 2016
     

     View Slide