BDD is not about Testing

BDD is not about Testing

BDD started as a way to teach TDD to programmers who kept getting hung up on the idea they were writing tests. Fast-forward a decade or so and it seems BDD scenario automation tools have invaded the world of acceptance testing like Japanese knotweed. All around I see teams harming themselves writing awkward, verbose tests using slow, cumbersome tools like Cucumber and SpecFlow, and acting as though BDD is some kind of testing approach.

Part of the problem is that once you have an automated BDD scenario and you've written the software to satisfy it, it can look seductively like a test. From there it is a short step to thinking of these scenario automation tools as testing software, and the rest is frustrating, repetitive history.

This long-overdue talk explores the relationship between BDD scenarios and acceptance tests, and suggests some strategies for avoiding the pain of BDD-as-test automation.

08145ecb1ce091d9dd3c328ea2a707fb?s=128

Daniel Terhorst-North

November 10, 2016
Tweet

Transcript

  1. 5.

    Motivation Testers feeling “left behind” by Agile Testers writing hundreds

    of “BDDs” Hiring for “automation testers”
  2. 6.

    Motivation Testers feeling “left behind” by Agile Testers writing hundreds

    of “BDDs” Hiring for “automation testers” Generally poor training around “agile testing”
  3. 7.

    Motivation Testers feeling “left behind” by Agile Testers writing hundreds

    of “BDDs” Hiring for “automation testers” Generally poor training around “agile testing” Testers unsettled about the role of SET/SDET
  4. 8.
  5. 12.

    tl;dr 1. Programmers don’t understand testing 2. Agile doesn’t understand

    testing 3. BDD is appealing to testers 4. BDD is not about testing
  6. 17.

    Why do we have testing? Mandated by the SDLC Programmers

    see testing as an irritation Project managers see testing as a source of risk
  7. 18.

    Why do we have testing? Mandated by the SDLC Programmers

    see testing as an irritation Project managers see testing as a source of risk Testing is always the thing that gets squeezed
  8. 19.

    Why do we have testing? Mandated by the SDLC Programmers

    see testing as an irritation Project managers see testing as a source of risk Testing is always the thing that gets squeezed Non-differentiating, non-critical
  9. 29.

    The goal of testing… “to find information”? “to communicate”? to

    enable us to “reasonably believe that the probability is low that the product still has important undiscovered problems”?
  10. 30.

    –Kent Beck "I get paid for code that works, not

    for tests, so my philosophy is to test as little as possible to reach a given level of confidence." http://stackoverflow.com/a/153565/632259
  11. 37.
  12. 38.

    A good tester has 3 superpowers… to increase confidence for

    stakeholders through evidence Empathy Ingenuity
  13. 39.

    A good tester has 3 superpowers… to increase confidence for

    stakeholders through evidence Empathy Balance Ingenuity
  14. 41.
  15. 42.
  16. 43.
  17. 44.
  18. 45.
  19. 46.
  20. 53.

    Agile doesn’t understand testing Agile methods were invented by programmers

    Testing is implied but never explicit The tester role is missing or poorly defined
  21. 74.
  22. 75.

    BDD is appealing to testers… We acknowledge multiple stakeholders We

    discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders!
  23. 76.

    BDD is appealing to testers… We acknowledge multiple stakeholders We

    discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders! We increase confidence for stakeholders through evidence
  24. 77.

    BDD is appealing to testers… We acknowledge multiple stakeholders We

    discuss acceptance criteria from the outset Everything happens from the perspective of the stakeholders! We increase confidence for stakeholders through evidence It’s not surprising BDD often starts with testers
  25. 83.

    The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution
  26. 84.

    The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution May be useful as documentation
  27. 85.

    The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution May be useful as documentation May form the basis of automated tests
  28. 86.

    The lifecycle of a scenario Starts as enabling constraint to

    guide the design Then: checks* the solution May be useful as documentation May form the basis of automated tests *check: (vt) inspect, investigate, examine, verify
  29. 89.

    Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case?
  30. 90.

    Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need?
  31. 91.

    Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need? Which stakeholders have we not yet served?
  32. 92.

    Scenario as the basis of tests What general case is

    this an example of? How could I gain confidence in the general case? Which stakeholder does this serve? What else do they need? Which stakeholders have we not yet served? Scenarios are necessary but not sufficient for testing
  33. 96.

    What does a good automated test look like? Intention-revealing name

    Intention-revealing errors Consistent language
  34. 97.

    What does a good automated test look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent
  35. 98.

    What does a good automated test look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  36. 99.

    What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  37. 100.

    What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  38. 101.

    What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action”
  39. 102.

    What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action” ??
  40. 103.

    What does a typical gherkin scenario look like? Intention-revealing name

    Intention-revealing errors Consistent language The intent, the whole intent and nothing but the intent Fast! “Close to the action” ?? ??
  41. 107.

    BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary
  42. 108.

    BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery
  43. 109.

    BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery Testability starts with design
  44. 110.

    BDD is not about testing, however… Testing is complementary to

    BDD Testing is the sufficient to BDD’s necessary Test thinking is critical to successful delivery Testability starts with design software that matters, for “people whose lives you touch”