Upgrade to Pro — share decks privately, control downloads, hide ads and more …

BDD is not about Testing

BDD is not about Testing

BDD started as a way to teach TDD to programmers who kept getting hung up on the idea they were writing tests. Fast-forward a decade or so and it seems BDD scenario automation tools have invaded the world of acceptance testing like Japanese knotweed. All around I see teams harming themselves writing awkward, verbose tests using slow, cumbersome tools like Cucumber and SpecFlow, and acting as though BDD is some kind of testing approach.

Part of the problem is that once you have an automated BDD scenario and you've written the software to satisfy it, it can look seductively like a test. From there it is a short step to thinking of these scenario automation tools as testing software, and the rest is frustrating, repetitive history.

This long-overdue talk explores the relationship between BDD scenarios and acceptance tests, and suggests some strategies for avoiding the pain of BDD-as-test automation.

Daniel Terhorst-North
PRO

November 10, 2016
Tweet

More Decks by Daniel Terhorst-North

Other Decks in Programming

Transcript

  1. Behaviour-Driven Development
    is not about Testing
    Dan North
    @tastapod

    View Slide

  2. Motivation

    View Slide

  3. Motivation
    Testers feeling “left behind” by Agile

    View Slide

  4. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”

    View Slide

  5. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”
    Hiring for “automation testers”

    View Slide

  6. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”
    Hiring for “automation testers”
    Generally poor training around “agile testing”

    View Slide

  7. Motivation
    Testers feeling “left behind” by Agile
    Testers writing hundreds of “BDDs”
    Hiring for “automation testers”
    Generally poor training around “agile testing”
    Testers unsettled about the role of SET/SDET

    View Slide

  8. tl;dr

    View Slide

  9. tl;dr
    1. Programmers don’t understand testing

    View Slide

  10. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing

    View Slide

  11. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing
    3. BDD is appealing to testers

    View Slide

  12. tl;dr
    1. Programmers don’t understand testing
    2. Agile doesn’t understand testing
    3. BDD is appealing to testers
    4. BDD is not about testing

    View Slide

  13. We need to talk about testing…

    View Slide

  14. Why do we have testing?

    View Slide

  15. Why do we have testing?
    Mandated by the SDLC

    View Slide

  16. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation

    View Slide

  17. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk

    View Slide

  18. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk
    Testing is always the thing that gets squeezed

    View Slide

  19. Why do we have testing?
    Mandated by the SDLC
    Programmers see testing as an irritation
    Project managers see testing as a source of risk
    Testing is always the thing that gets squeezed
    Non-differentiating, non-critical

    View Slide

  20. Purpose Alignment model
    Business Critical
    Differentiating

    View Slide

  21. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”

    View Slide

  22. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?” Parity

    View Slide

  23. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Parity

    View Slide

  24. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner “Invest and Excel”
    Parity

    View Slide

  25. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner “Invest and Excel”
    Parity
    Testing!

    View Slide

  26. The goal of testing…

    View Slide

  27. The goal of testing…
    “to find information”?

    View Slide

  28. The goal of testing…
    “to find information”?
    “to communicate”?

    View Slide

  29. The goal of testing…
    “to find information”?
    “to communicate”?
    to enable us to “reasonably believe that the probability is low that
    the product still has important undiscovered problems”?

    View Slide

  30. –Kent Beck
    "I get paid for code that works, not for tests, so
    my philosophy is to test as little as possible to
    reach a given level of confidence."
    http://stackoverflow.com/a/153565/632259

    View Slide

  31. The goal of testing…

    View Slide

  32. The goal of testing…
    to increase confidence for stakeholders through evidence

    View Slide

  33. The goal of testing…
    to increase confidence for stakeholders through evidence

    View Slide

  34. The goal of testing…
    to increase confidence for stakeholders through evidence

    View Slide

  35. The goal of testing…
    to increase confidence for stakeholders through evidence

    View Slide

  36. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence

    View Slide

  37. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy

    View Slide

  38. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy
    Ingenuity

    View Slide

  39. A good tester has 3 superpowers…
    to increase confidence for stakeholders through evidence
    Empathy
    Balance
    Ingenuity

    View Slide

  40. 1. Programmers don’t understand testing

    View Slide

  41. View Slide

  42. View Slide

  43. View Slide

  44. View Slide

  45. View Slide

  46. View Slide

  47. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View Slide

  48. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View Slide

  49. 2. Agile doesn’t understand testing

    View Slide

  50. Agile doesn’t understand testing

    View Slide

  51. Agile doesn’t understand testing
    Agile methods were invented by programmers

    View Slide

  52. Agile doesn’t understand testing
    Agile methods were invented by programmers
    Testing is implied but never explicit

    View Slide

  53. Agile doesn’t understand testing
    Agile methods were invented by programmers
    Testing is implied but never explicit
    The tester role is missing or poorly defined

    View Slide

  54. Agile Testing Training…

    View Slide

  55. Agile Testing Training…

    View Slide

  56. Agile Testing Training…

    View Slide

  57. Agile Testing Training…

    View Slide

  58. Agile Testing Training…

    View Slide

  59. Agile Testing Training…

    View Slide

  60. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  61. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  62. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  63. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  64. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  65. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  66. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  67. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  68. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  69. Agile Testing Training…
    Certified Agile Tester (CAT)

    View Slide

  70. Agile Testing Training…
    Certified Agile Tester (CAT)
    WAT?

    View Slide

  71. 3. BDD is appealing to testers

    View Slide

  72. BDD is appealing to testers…

    View Slide

  73. BDD is appealing to testers…
    We acknowledge multiple stakeholders

    View Slide

  74. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset

    View Slide

  75. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!

    View Slide

  76. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!
    We increase confidence for stakeholders through evidence

    View Slide

  77. BDD is appealing to testers…
    We acknowledge multiple stakeholders
    We discuss acceptance criteria from the outset
    Everything happens from the perspective of the stakeholders!
    We increase confidence for stakeholders through evidence
    It’s not surprising BDD often starts with testers

    View Slide

  78. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View Slide

  79. Purpose Alignment model
    Business Critical
    Differentiating
    “Who cares?”
    Partner
    Testing!
    “Invest and Excel”
    Parity

    View Slide

  80. 4. BDD is not about testing!

    View Slide

  81. The lifecycle of a scenario

    View Slide

  82. The lifecycle of a scenario
    Starts as enabling constraint to guide the design

    View Slide

  83. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution

    View Slide

  84. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation

    View Slide

  85. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation
    May form the basis of automated tests

    View Slide

  86. The lifecycle of a scenario
    Starts as enabling constraint to guide the design
    Then: checks* the solution
    May be useful as documentation
    May form the basis of automated tests
    *check: (vt) inspect, investigate, examine, verify

    View Slide

  87. Scenario as the basis of tests

    View Slide

  88. Scenario as the basis of tests
    What general case is this an example of?

    View Slide

  89. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?

    View Slide

  90. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?

    View Slide

  91. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?
    Which stakeholders have we not yet served?

    View Slide

  92. Scenario as the basis of tests
    What general case is this an example of?
    How could I gain confidence in the general case?
    Which stakeholder does this serve? What else do they need?
    Which stakeholders have we not yet served?
    Scenarios are necessary but not sufficient for testing

    View Slide

  93. What does a good automated test look like?

    View Slide

  94. What does a good automated test look like?
    Intention-revealing name

    View Slide

  95. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors

    View Slide

  96. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language

    View Slide

  97. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent

    View Slide

  98. What does a good automated test look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View Slide

  99. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View Slide

  100. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View Slide

  101. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”

    View Slide

  102. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”
    ??

    View Slide

  103. What does a typical gherkin scenario look like?
    Intention-revealing name
    Intention-revealing errors
    Consistent language
    The intent, the whole intent and nothing but the intent
    Fast! “Close to the action”
    ??
    ??

    View Slide

  104. Good scenarios != Good tests

    View Slide

  105. BDD is not about testing, however…

    View Slide

  106. BDD is not about testing, however…
    Testing is complementary to BDD

    View Slide

  107. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary

    View Slide

  108. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary
    Test thinking is critical to successful delivery

    View Slide

  109. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary
    Test thinking is critical to successful delivery
    Testability starts with design

    View Slide

  110. BDD is not about testing, however…
    Testing is complementary to BDD
    Testing is the sufficient to BDD’s necessary
    Test thinking is critical to successful delivery
    Testability starts with design
    software that matters, for “people whose lives you touch”

    View Slide

  111. Thank you!
    @tastapod
    http://dannorth.net
    Enjoy BDDx 2016

    View Slide