Upgrade to Pro — share decks privately, control downloads, hide ads and more …

BDD is not about Testing

BDD is not about Testing

BDD started as a way to teach TDD to programmers who kept getting hung up on the idea they were writing tests. Fast-forward a decade or so and it seems BDD scenario automation tools have invaded the world of acceptance testing like Japanese knotweed. All around I see teams harming themselves writing awkward, verbose tests using slow, cumbersome tools like Cucumber and SpecFlow, and acting as though BDD is some kind of testing approach.

Part of the problem is that once you have an automated BDD scenario and you've written the software to satisfy it, it can look seductively like a test. From there it is a short step to thinking of these scenario automation tools as testing software, and the rest is frustrating, repetitive history.

This long-overdue talk explores the relationship between BDD scenarios and acceptance tests, and suggests some strategies for avoiding the pain of BDD-as-test automation.

Daniel Terhorst-North

November 10, 2016
Tweet

More Decks by Daniel Terhorst-North

Other Decks in Programming

Transcript

  1. Behaviour-Driven Development
    is not about Testing
    Dan North
    @tastapod

    View full-size slide

  2. Motivation
    Testers feeling “left behind” by Agile

    View full-size slide

  3. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”

    View full-size slide

  4. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”
    Hiring for “automation testers”

    View full-size slide

  5. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”
    Hiring for “automation testers”
    Generally poor training around “agile testing”

    View full-size slide

  6. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”
    Hiring for “automation testers”
    Generally poor training around “agile testing”
    Testers unsettled about the role of SET/SDET

    View full-size slide

  7. tl;dr
    1. Programmers don’t understand testing

    View full-size slide

  8. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing

    View full-size slide

  9. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing
    3. BDD is appealing to testers

    View full-size slide

  10. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing
    3. BDD is appealing to testers
    4. BDD is not about testing

    View full-size slide

  11. We need to talk about testing…

    View full-size slide

  12. Why do we have testing?

    View full-size slide

  13. Why do we have testing?
    Mandated by the SDLC

    View full-size slide

  14. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation

    View full-size slide

  15. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk

    View full-size slide

  16. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk
    Testing is always the thing that gets squeezed

    View full-size slide

  17. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk
    Testing is always the thing that gets squeezed
    Non-differentiating, non-critical

    View full-size slide

  18. Purpose Alignment model
    Business Critical
    Differentiating

    View full-size slide

  19. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”

    View full-size slide

  20. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?” Parity

    View full-size slide

  21. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Parity

    View full-size slide

  22. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner “Invest and Excel”
    Parity

    View full-size slide

  23. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner “Invest and Excel”
    Parity
    Testing!

    View full-size slide

  24. The goal of testing…

    View full-size slide

  25. The goal of testing…
    “to find information”?

    View full-size slide

  26. The goal of testing…
    “to find information”?
    “to communicate”?

    View full-size slide

  27. The goal of testing…
    “to find information”?
    “to communicate”?
    to enable us to “reasonably believe that the probability is low that
    the product still has important undiscovered problems”?

    View full-size slide

  28. –Kent Beck
    "I get paid for code that works, not for tests, so
    my philosophy is to test as little as possible to
    reach a given level of confidence."
    http://stackoverflow.com/a/153565/632259

    View full-size slide

  29. The goal of testing…

    View full-size slide

  30. The goal of testing…
    to increase confidence for stakeholders through evidence

    View full-size slide

  31. The goal of testing…
    to increase confidence for stakeholders through evidence

    View full-size slide

  32. The goal of testing…
    to increase confidence for stakeholders through evidence

    View full-size slide

  33. The goal of testing…
    to increase confidence for stakeholders through evidence

    View full-size slide

  34. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence

    View full-size slide

  35. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy

    View full-size slide

  36. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy
    Ingenuity

    View full-size slide

  37. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy
    Balance
    Ingenuity

    View full-size slide

  38. 1. Programmers don’t understand testing

    View full-size slide

  39. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View full-size slide

  40. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View full-size slide

  41. 2. Agile doesn’t understand testing

    View full-size slide

  42. Agile doesn’t understand testing

    View full-size slide

  43. Agile doesn’t understand testing
    Agile methods were invented by programmers

    View full-size slide

  44. Agile doesn’t understand testing
    Agile methods were invented by programmers
    Testing is implied but never explicit

    View full-size slide

  45. Agile doesn’t understand testing
    Agile methods were invented by programmers
    Testing is implied but never explicit
    The tester role is missing or poorly defined

    View full-size slide

  46. Agile Testing Training…

    View full-size slide

  47. Agile Testing Training…

    View full-size slide

  48. Agile Testing Training…

    View full-size slide

  49. Agile Testing Training…

    View full-size slide

  50. Agile Testing Training…

    View full-size slide

  51. Agile Testing Training…

    View full-size slide

  52. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  53. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  54. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  55. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  56. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  57. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  58. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  59. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  60. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  61. Agile Testing Training…
    Certified Agile Tester (CAT)

    View full-size slide

  62. Agile Testing Training…
    Certified Agile Tester (CAT)
    WAT?

    View full-size slide

  63. 3. BDD is appealing to testers

    View full-size slide

  64. BDD is appealing to testers…

    View full-size slide

  65. BDD is appealing to testers…
    We acknowledge multiple stakeholders

    View full-size slide

  66. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset

    View full-size slide

  67. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!

    View full-size slide

  68. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!
    We increase confidence for stakeholders through evidence

    View full-size slide

  69. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!
    We increase confidence for stakeholders through evidence
    It’s not surprising BDD often starts with testers

    View full-size slide

  70. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View full-size slide

  71. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View full-size slide

  72. 4. BDD is not about testing!

    View full-size slide

  73. The lifecycle of a scenario

    View full-size slide

  74. The lifecycle of a scenario
    Starts as enabling constraint to guide the design

    View full-size slide

  75. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution

    View full-size slide

  76. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation

    View full-size slide

  77. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation
    May form the basis of automated tests

    View full-size slide

  78. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation
    May form the basis of automated tests
    *check: (vt) inspect, investigate, examine, verify

    View full-size slide

  79. Scenario as the basis of tests

    View full-size slide

  80. Scenario as the basis of tests
    What general case is this an example of?

    View full-size slide

  81. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?

    View full-size slide

  82. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?

    View full-size slide

  83. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?
    Which stakeholders have we not yet served?

    View full-size slide

  84. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?
    Which stakeholders have we not yet served?
    Scenarios are necessary but not sufficient for testing

    View full-size slide

  85. What does a good automated test look like?

    View full-size slide

  86. What does a good automated test look like?
    Intention-revealing name

    View full-size slide

  87. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors

    View full-size slide

  88. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language

    View full-size slide

  89. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent

    View full-size slide

  90. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View full-size slide

  91. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View full-size slide

  92. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View full-size slide

  93. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View full-size slide

  94. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”
    ??

    View full-size slide

  95. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”
    ??
    ??

    View full-size slide

  96. Good scenarios != Good tests

    View full-size slide

  97. BDD is not about testing, however…

    View full-size slide

  98. BDD is not about testing, however…
    Testing is complementary to BDD

    View full-size slide

  99. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary

    View full-size slide

  100. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary
    Test thinking is critical to successful delivery

    View full-size slide

  101. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary
    Test thinking is critical to successful delivery
    Testability starts with design

    View full-size slide

  102. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary
    Test thinking is critical to successful delivery
    Testability starts with design
    software that matters, for “people whose lives you touch”

    View full-size slide

  103. Thank you!
    @tastapod
    http://dannorth.net
    Enjoy BDDx 2016

    View full-size slide