Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Sustainable BDD

Sustainable BDD

The BDD hype cycle is over. Recently, there’s been a lot of backlash against popular BDD libraries like Cucumber. Some developers blame their test frameworks for brittle test suites and long build times. Others go so far as to claim that acceptance testing is simply not sustainable, period. In this talk, we’ll do some root cause analysis of this phenomenon with shocking results - it’s not the test framework, it’s not the methodology, it’s you. You’ve abused your test framework, you’ve cargo-culted the methodology, and now you’re feeling the pain. We’ll show you a way out of the mess you’ve made. We’ll discuss the main problems BDD was intended to solve. We’ll show you how to groom your test suite into journey, functional, integration, and unit tests in order to address build times. We’ll teach how to mitigate against brittleness and flickers, and how to let your tests reveal the intent of the application and actually become the executable documentation we’ve been waiting for.

Robbie Clutton

February 23, 2013
Tweet

More Decks by Robbie Clutton

Other Decks in Technology

Transcript

  1. WTF BDD 1. Brittle Tests 2. Dev-written Acceptance Criteria 3.

    Unreadable Tests 4. Flickering Tests 5. Slow Tests
  2. FEATURE: SIGN UP Give me an example. "User opens app

    for the first time, provides their email, desired username, and desired password, and is immediately logged in and allowed to start using the application."
  3. FEATURE: SIGN UP Give me an example. • Valid username,

    email, password • Blank username, email, or password
  4. FEATURE: SIGN UP Give me an example. • Valid username,

    email, password • Blank username, email, or password • Non-matching Password
  5. FEATURE: SIGN UP Give me an example. • Valid username,

    email, password • Blank username, email, or password • Non-matching Password • Password too weak
  6. FEATURE: SIGN UP Give me an example. • Valid username,

    email, password • Blank username, email, or password • Non-matching Password • Password too weak • Email address invalid
  7. FEATURE: SIGN UP Give me an example. • Valid username,

    email, password • Blank username, email, or password • Non-matching Password • Password too weak • Email address invalid • Username unavailable
  8. FEATURE: ACCOUNT RECOVERY Background: User forgot password • User remembers

    email • User remembers username • User forgot everything
  9. FEATURE: SIGN UP Give me an example. • Valid username,

    email, password • Blank username, email, or password • Non-matching Password • Password too weak • Email address invalid • Username unavailable • User provides someone else’s email
  10. FEATURE: EMAIL VERIFICATION Give me an example. • Verification link

    followed • Fraud link followed • No links ever followed
  11. FEATURE: SIGN IN Give me an example. • Existing username

    / password • Unknown username • Wrong password
  12. APP AS STATE MACHINE Every feature of your application is

    a state transition. GIVEN (start state) WHEN (event) THEN (state transition)
  13. WHY GHERKIN? 1. State Diagrams for the Masses 2. Copy/Paste

    from Tracker to editor 3. Living / Executable Documentation
  14. BRITTLE A single change in the application causes many tests

    to break, and you are forced to visit each and every test and fix each one individually.
  15. Scenario: Valid Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/bob" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." When I click "Log out" And I visit "/" And I fill in "username" with "alice" And I fill in "password" with "password" And I click "Log in" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters."
  16. Scenario: Valid Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/bob" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." When I click "Log out" And I visit "/" And I fill in "username" with "alice" And I fill in "password" with "password" And I click "Log in" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Login
  17. Scenario: Valid Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/bob" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." When I click "Log out" And I visit "/" And I fill in "username" with "alice" And I fill in "password" with "password" And I click "Log in" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Tweet
  18. Scenario: Overlong Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" And I click "Submit" Then I should see "Tweet too long"
  19. Scenario: Overlong Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" And I click "Submit" Then I should see "Tweet too long" Tweet
  20. Scenario: Overlong Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" And I click "Submit" Then I should see "Tweet too long" Login
  21. Scenario: Duplicate Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see “Your tweet is a duplicate” When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters."
  22. Scenario: Duplicate Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see “Your tweet is a duplicate” When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Login
  23. Scenario: Duplicate Tweet Given there is a user "bob" with

    password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click “Submit” Then I should see “Your tweet is a duplicate” When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Tweet
  24. feature “Tweet” do scenario "valid tweet" do create :user, username:

    "bob", password: "password" create :user, username: "alice", password: "password" visit "/" fill_in "Username", with: "bob" fill_in "Password", with: "password" click_button "Log In" fill_in "Tweet", with: "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." click_button "Submit" page.should have_content "Your tweet was submitted" visit "/bob" page.should have_content "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." click_link "Log out" visit "/" fill_in "Username", with: "alice" fill_in "Password", with: "password" click_button "Log in" page.should have_content "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." end end
  25. BIG RATS LEAVE BIG POOS click_button "Tweet" click_link_or_button "Tweet" click_on

    "Tweet" click_on "#tweet" find("#tweet").click page.execute_script("$('#tweet').click()")
  26. BIG RATS LEAVE BIG POOS “Now, whenever you tweet, pop

    up a modal that forces the user to enter a CAPTCHA.”
  27. BIG RATS LEAVE BIG POOS module WebHelpers def tweet(message) fill_in

    “Message”, with: message click_on “Tweet” end end
  28. BIG RATS LEAVE BIG POOS #rspec RSpec.configure do |c| c.include

    WebHelpers, type: :request end #cucumber World WebHelpers
  29. Feature: Tweet Scenario: Valid Tweet Given Bob has authenticated When

    Bob submits a valid tweet Then Alice can see Bob’s tweet And Bob should see that tweet in his timeline Scenario: Overlong Tweet #.... Scenario: Duplicate tweet #....
  30. BIG RATS LEAVE BIG POOS Given /^Bob has authenticated$/ do

    visit “/” fill_in “Username”, with: “Bob” fill_in “Password”, with: “password” click_on “Sign In” end
  31. BIG RATS LEAVE BIG POOS module WebHelpers #... def bob

    @bob ||= create :user, name: “Bob” end #... end
  32. BIG RATS LEAVE BIG POOS When /^Bob submits a valid

    Tweet$/ do click_on “Tweet” within(“#submit_tweet_modal”) do fill_in “Captcha”, with: captcha click_on “Tweet” end end
  33. BIG RATS LEAVE BIG POOS When /^Bob submits a valid

    Tweet$/ do tweet “Hello, world” end
  34. BIG RATS LEAVE BIG POOS Then /^Alice can see Bob’s

    Tweet$/ do click_on “Sign Out” fill_in “Username”, with: “Alice” fill_in “Password”, with: “password” click_on “Sign In” visit “/alice” page.should have_content “Hello, world” end
  35. BIG RATS LEAVE BIG POOS Then /^Alice can see Bob’s

    Tweet$/ do login alice alice.should see_tweet “Hello, world” end
  36. BIG RATS LEAVE BIG POOS feature “Tweet” do scenario “Valid

    Tweet” do login bob tweet “hello, world” login alice alice.should see “hello, world” end end
  37. Given I click on “Sign In” And I fill in

    “Username” with “Bob” And I fill in “Password” with “password” And I click on “Submit” #....
  38. NO ONE READS THE CUKES 1. They’re unreadable 2. They’re

    not exposed anywhere 3. You haven’t needed documentation
  39. $ time rake real 18m0.926s imagined ETERNITY visit root_path click_on

    “Some Link” sleep 10 page.should have_content “FOO”
  40. $ time rake real 18m0.926s imagined ETERNITY visit root_path click_on

    “Some Link” wait_for { page.has_selector? (“.foo_container”) } page.should have_content “Foo”
  41. $ time rake real 18m0.926s imagined ETERNITY visit root_path click_on

    “Some Link” wait_for { page.should have_selector(“.foo_container”) } page.should have_content “Foo” Whoops
  42. $ time rake real 18m0.926s imagined ETERNITY visit root_path click_on

    “Some Link” page.should have_content “Foo”
  43. $ time rake “You should not be afraid to delete

    tests that are no longer providing value, no matter whether you originally planned to keep them or not. We tend to treat tests as these holy creatures that live blameless, irreproachable lives once they have sprung into existence. Not so. The maintenance required to keep a test running weighs against its value in further development. Sometimes these lines cross, and the test simply becomes a burden on the project. Having the skill and experience to recognize a burdensome test is something we should be bringing to our clients, as well as the fortitude to rewrite it, rethink it, or delete it." - ADAM MILLIGAN
  44. $ time rake Feature: Sign Up Scenario: Valid username, email,

    password Scenario: Blank username, email, or password Scenario: Non-matching Password Scenario: Password too weak Scenario: Email address invalid Scenario: Username unavailable Scenario: User provides someone else’s email
  45. $ time rake Feature: Sign Up Scenario: Valid username, email,

    password Scenario: Blank username, email, password Scenario: Non-matching Password Scenario: Password too weak Scenario: Email address invalid Scenario: Username unavailable Scenario: User provides someone else’s email
  46. $ time rake Feature: Sign Up Scenario: Valid username, email,

    password Scenario: Blank username, email, password Scenario: Non-matching Password Scenario: Password too weak Scenario: Email address invalid Scenario: Username unavailable Scenario: User provides someone else’s email
  47. $ time rake Feature: Sign Up Scenario: Valid username, email,

    password Scenario: Blank username, email, password @ci Scenario: Non-matching Password ....
  48. $ time rake Feature: Sign Up Scenario: Valid username, email,

    password Scenario: Blank username, email, password @no-ui Scenario: Non-matching Password ....
  49. $ time rake module DomainHelpers attr_reader :current_user def authenticate @current_user

    = create :user end def tweet(message) current_user.tweet message end #... end
  50. Q/A