BDD is not about Testing

BDD is not about Testing

BDD started as a way to teach TDD to programmers who kept getting hung up on the idea they were writing tests. Fast-forward a decade or so and it seems BDD scenario automation tools have invaded the world of acceptance testing like Japanese knotweed. All around I see teams harming themselves writing awkward, verbose tests using slow, cumbersome tools like Cucumber and SpecFlow, and acting as though BDD is some kind of testing approach.

Part of the problem is that once you have an automated BDD scenario and you've written the software to satisfy it, it can look seductively like a test. From there it is a short step to thinking of these scenario automation tools as testing software, and the rest is frustrating, repetitive history.

This long-overdue talk explores the relationship between BDD scenarios and acceptance tests, and suggests some strategies for avoiding the pain of BDD-as-test automation.

08145ecb1ce091d9dd3c328ea2a707fb?s=128

Daniel Terhorst-North

November 10, 2016
Tweet

Transcript

  1. Behaviour-Driven Development is not about Testing Dan North @tastapod

  2. Motivation

  3. Motivation Testers feeling “left behind” by Agile

  4. Motivation Testers feeling “left behind” by Agile Testers writing hundreds

    of “BDDs”
  5. Motivation Testers feeling “left behind” by Agile Testers writing hundreds

    of “BDDs” Hiring for “automation testers”
  6. Motivation Testers feeling “left behind” by Agile Testers writing hundreds

    of “BDDs” Hiring for “automation testers” Generally poor training around “agile testing”
  7. Motivation Testers feeling “left behind” by Agile Testers writing hundreds

    of “BDDs” Hiring for “automation testers” Generally poor training around “agile testing” Testers unsettled about the role of SET/SDET
  8. tl;dr

  9. tl;dr 1. Programmers don’t understand testing

  10. tl;dr 1. Programmers don’t understand testing 2. Agile doesn’t understand

    testing
  11. tl;dr 1. Programmers don’t understand testing 2. Agile doesn’t understand

    testing 3. BDD is appealing to testers
  12. tl;dr 1. Programmers don’t understand testing 2. Agile doesn’t understand

    testing 3. BDD is appealing to testers 4. BDD is not about testing
  13. We need to talk about testing…

  14. Why do we have testing?

  15. Why do we have testing? Mandated by the SDLC

  16. Why do we have testing? Mandated by the SDLC Programmers

    see testing as an irritation
  17. Why do we have testing? Mandated by the SDLC Programmers

    see testing as an irritation Project managers see testing as a source of risk
  18. Why do we have testing? Mandated by the SDLC Programmers

    see testing as an irritation Project managers see testing as a source of risk Testing is always the thing that gets squeezed
  19. Why do we have testing? Mandated by the SDLC Programmers

    see testing as an irritation Project managers see testing as a source of risk Testing is always the thing that gets squeezed Non-differentiating, non-critical
  20. Purpose Alignment model Business Critical Differentiating

  21. Purpose Alignment model Business Critical Differentiating “Who cares?”

  22. Purpose Alignment model Business Critical Differentiating “Who cares?” Parity

  23. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Parity

  24. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner “Invest

    and Excel” Parity
  25. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner “Invest

    and Excel” Parity Testing!
  26. The goal of testing…

  27. The goal of testing… “to find information”?

  28. The goal of testing… “to find information”? “to communicate”?

  29. The goal of testing… “to find information”? “to communicate”? to

    enable us to “reasonably believe that the probability is low that the product still has important undiscovered problems”?
  30. –Kent Beck "I get paid for code that works, not

    for tests, so my philosophy is to test as little as possible to reach a given level of confidence." http://stackoverflow.com/a/153565/632259
  31. The goal of testing…

  32. The goal of testing… to increase confidence for stakeholders through

    evidence
  33. The goal of testing… to increase confidence for stakeholders through

    evidence
  34. The goal of testing… to increase confidence for stakeholders through

    evidence
  35. The goal of testing… to increase confidence for stakeholders through

    evidence
  36. A good tester has 3 superpowers… to increase confidence for

    stakeholders through evidence
  37. A good tester has 3 superpowers… to increase confidence for

    stakeholders through evidence Empathy
  38. A good tester has 3 superpowers… to increase confidence for

    stakeholders through evidence Empathy Ingenuity
  39. A good tester has 3 superpowers… to increase confidence for

    stakeholders through evidence Empathy Balance Ingenuity
  40. 1. Programmers don’t understand testing

  41. None
  42. None
  43. None
  44. None
  45. None
  46. None
  47. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing!

    “Invest and Excel” Parity
  48. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing!

    “Invest and Excel” Parity
  49. 2. Agile doesn’t understand testing

  50. Agile doesn’t understand testing

  51. Agile doesn’t understand testing Agile methods were invented by programmers

  52. Agile doesn’t understand testing Agile methods were invented by programmers

    Testing is implied but never explicit
  53. Agile doesn’t understand testing Agile methods were invented by programmers

    Testing is implied but never explicit The tester role is missing or poorly defined
  54. Agile Testing Training…

  55. Agile Testing Training…

  56. Agile Testing Training…

  57. Agile Testing Training…

  58. Agile Testing Training…

  59. Agile Testing Training…

  60. Agile Testing Training… Certified Agile Tester (CAT)

  61. Agile Testing Training… Certified Agile Tester (CAT)

  62. Agile Testing Training… Certified Agile Tester (CAT)

  63. Agile Testing Training… Certified Agile Tester (CAT)

  64. Agile Testing Training… Certified Agile Tester (CAT)

  65. Agile Testing Training… Certified Agile Tester (CAT)

  66. Agile Testing Training… Certified Agile Tester (CAT)

  67. Agile Testing Training… Certified Agile Tester (CAT)

  68. Agile Testing Training… Certified Agile Tester (CAT)

  69. Agile Testing Training… Certified Agile Tester (CAT)

  70. Agile Testing Training… Certified Agile Tester (CAT) WAT?

  71. 3. BDD is appealing to testers

  72. BDD is appealing to testers…

  73. BDD is appealing to testers… We acknowledge multiple stakeholders

  74. BDD is appealing to testers… We acknowledge multiple stakeholders We

    discuss acceptance criteria from the outset
  75. BDD is appealing to testers… We acknowledge multiple stakeholders We

    discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders!
  76. BDD is appealing to testers… We acknowledge multiple stakeholders We

    discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders! We increase confidence for stakeholders through evidence
  77. BDD is appealing to testers… We acknowledge multiple stakeholders We

    discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders! We increase confidence for stakeholders through evidence It’s not surprising BDD often starts with testers
  78. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing!

    “Invest and Excel” Parity
  79. Purpose Alignment model Business Critical Differentiating “Who cares?” Partner Testing!

    “Invest and Excel” Parity
  80. 4. BDD is not about testing!

  81. The lifecycle of a scenario

  82. The lifecycle of a scenario Starts as enabling constraint to

    guide the design
  83. The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution
  84. The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution May be useful as documentation
  85. The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution May be useful as documentation May form the basis of automated tests
  86. The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution May be useful as documentation May form the basis of automated tests *check: (vt) inspect, investigate, examine, verify
  87. Scenario as the basis of tests

  88. Scenario as the basis of tests What general case is

    this an example of?
  89. Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case?
  90. Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need?
  91. Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need? Which stakeholders have we not yet served?
  92. Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need? Which stakeholders have we not yet served? Scenarios are necessary but not sufficient for testing
  93. What does a good automated test look like?

  94. What does a good automated test look like? Intention-revealing name

  95. What does a good automated test look like? Intention-revealing name

    Intention-revealing errors
  96. What does a good automated test look like? Intention-revealing name

    Intention-revealing errors Consistent language
  97. What does a good automated test look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent
  98. What does a good automated test look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  99. What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  100. What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  101. What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  102. What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action” ??
  103. What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action” ?? ??
  104. Good scenarios != Good tests

  105. BDD is not about testing, however…

  106. BDD is not about testing, however… Testing is complementary to

    BDD
  107. BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary
  108. BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery
  109. BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery Testability starts with design
  110. BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery Testability starts with design software that matters, for “people whose lives you touch”
  111. Thank you! @tastapod http://dannorth.net Enjoy BDDx 2016