Tweet
Tweet

Slide 1

Slide 1 text

SUSTAINABLE BDD Matt Parker Robbie Clutton Software Engineers, Pivotal Labs

Slide 2

Slide 2 text

HYPE CYCLE

Slide 3

Slide 3 text

WTF BDD 1. Brittle Tests 2. Dev-written Acceptance Criteria 3. Unreadable Tests 4. Flickering Tests 5. Slow Tests

Slide 4

Slide 4 text

CARGO CULTING

Slide 5

Slide 5 text

HISTORY In the beginning... there was the unit test. And lo it was good.

Slide 6

Slide 6 text

HISTORY sUnit -> xUnit

Slide 7

Slide 7 text

HISTORY And then there was Dan North.

Slide 8

Slide 8 text

SPECIFICATION BY EXAMPLE

Slide 9

Slide 9 text

FEATURE: SIGN UP

Slide 10

Slide 10 text

FEATURE: SIGN UP Give me an example.

Slide 11

Slide 11 text

FEATURE: SIGN UP Give me an example. "User opens app for the first time, provides their email, desired username, and desired password, and is immediately logged in and allowed to start using the application."

Slide 12

Slide 12 text

FEATURE: SIGN UP Give me an example. • Valid username, email, password

Slide 13

Slide 13 text

FEATURE: SIGN UP Give me an example. • Valid username, email, password • Blank username, email, or password

Slide 14

Slide 14 text

FEATURE: SIGN UP Give me an example. • Valid username, email, password • Blank username, email, or password • Non-matching Password

Slide 15

Slide 15 text

FEATURE: SIGN UP Give me an example. • Valid username, email, password • Blank username, email, or password • Non-matching Password • Password too weak

Slide 16

Slide 16 text

FEATURE: SIGN UP Give me an example. • Valid username, email, password • Blank username, email, or password • Non-matching Password • Password too weak • Email address invalid

Slide 17

Slide 17 text

FEATURE: SIGN UP Give me an example. • Valid username, email, password • Blank username, email, or password • Non-matching Password • Password too weak • Email address invalid • Username unavailable

Slide 18

Slide 18 text

FEATURE: ACCOUNT RECOVERY Background: User forgot password • User remembers email • User remembers username • User forgot everything

Slide 19

Slide 19 text

FEATURE: SIGN UP Give me an example. • Valid username, email, password • Blank username, email, or password • Non-matching Password • Password too weak • Email address invalid • Username unavailable • User provides someone else’s email

Slide 20

Slide 20 text

FEATURE: EMAIL VERIFICATION Give me an example. • Verification link followed • Fraud link followed • No links ever followed

Slide 21

Slide 21 text

FEATURE: SIGN IN Give me an example. • Existing username / password • Unknown username • Wrong password

Slide 22

Slide 22 text

HIDDEN COMPLEXITY 1 sentence => 4 features => 17 scenarios

Slide 23

Slide 23 text

HISTORY JBehave

Slide 24

Slide 24 text

HISTORY Business Value Feature Scenario

Slide 25

Slide 25 text

HISTORY RBehave

Slide 26

Slide 26 text

HISTORY Story Runner / RSpec

Slide 27

Slide 27 text

HISTORY Cucumber

Slide 28

Slide 28 text

WHY GHERKIN? 1. State Diagrams for the Masses

Slide 29

Slide 29 text

APP AS STATE MACHINE Every feature of your application is a state transition. GIVEN (start state) WHEN (event) THEN (state transition)

Slide 30

Slide 30 text

WHY GHERKIN? 1. State Diagrams for the Masses 2. Copy/Paste from Tracker to editor

Slide 31

Slide 31 text

WHY GHERKIN? 1. State Diagrams for the Masses 2. Copy/Paste from Tracker to editor 3. Living / Executable Documentation

Slide 32

Slide 32 text

HISTORY Spinach, Turnip, Steak, Bacon, Filet, Lemon, ......................

Slide 33

Slide 33 text

HISTORY 10 build shiny new tool 20 make same mistakes 30 goto 10

Slide 34

Slide 34 text

BRITTLE A single change in the application causes many tests to break

Slide 35

Slide 35 text

BRITTLE A single change in the application causes many tests to break, and you are forced to visit each and every test and fix each one individually.

Slide 36

Slide 36 text

Brittle tests typically don’t reveal intent.

Slide 37

Slide 37 text

Feature: Tweet Scenario: Valid Tweet #.... Scenario: Overlong Tweet #.... Scenario: Duplicate tweet #....

Slide 38

Slide 38 text

Scenario: Valid Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/bob" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." When I click "Log out" And I visit "/" And I fill in "username" with "alice" And I fill in "password" with "password" And I click "Log in" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters."

Slide 39

Slide 39 text

Scenario: Valid Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/bob" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." When I click "Log out" And I visit "/" And I fill in "username" with "alice" And I fill in "password" with "password" And I click "Log in" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Login

Slide 40

Slide 40 text

Scenario: Valid Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/bob" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." When I click "Log out" And I visit "/" And I fill in "username" with "alice" And I fill in "password" with "password" And I click "Log in" Then I should see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Tweet

Slide 41

Slide 41 text

Scenario: Overlong Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" And I click "Submit" Then I should see "Tweet too long"

Slide 42

Slide 42 text

Scenario: Overlong Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" And I click "Submit" Then I should see "Tweet too long" Tweet

Slide 43

Slide 43 text

Scenario: Overlong Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" And I click "Submit" Then I should see "Tweet too long" Login

Slide 44

Slide 44 text

Scenario: Duplicate Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see “Your tweet is a duplicate” When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters."

Slide 45

Slide 45 text

Scenario: Duplicate Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see “Your tweet is a duplicate” When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Login

Slide 46

Slide 46 text

Scenario: Duplicate Tweet Given there is a user "bob" with password "password" And there is a user "alice" with password "password" that follows "bob" And I visit "/" And I fill in "username" with "bob" And I fill in "password" with "password" And I click "Log in" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click "Submit" Then I should see "Your tweet was submitted" When I visit "/" When I fill in "Tweet" with "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." And I click “Submit” Then I should see “Your tweet is a duplicate” When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise OH NOEZ IM TOO LONG NOOOOOOOOOOOOOOOOOOOOOO" When I visit "/bob" Then I should not see "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." Tweet

Slide 47

Slide 47 text

feature “Tweet” do scenario "valid tweet" do create :user, username: "bob", password: "password" create :user, username: "alice", password: "password" visit "/" fill_in "Username", with: "bob" fill_in "Password", with: "password" click_button "Log In" fill_in "Tweet", with: "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." click_button "Submit" page.should have_content "Your tweet was submitted" visit "/bob" page.should have_content "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." click_link "Log out" visit "/" fill_in "Username", with: "alice" fill_in "Password", with: "password" click_button "Log in" page.should have_content "this is a test tweet noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise noise haha I'm less than 140 characters." end end

Slide 48

Slide 48 text

BIG RATS LEAVE BIG POOS

Slide 49

Slide 49 text

BIG RATS LEAVE BIG POOS click_button "Tweet"

Slide 50

Slide 50 text

BIG RATS LEAVE BIG POOS click_button "Tweet" click_link_or_button "Tweet"

Slide 51

Slide 51 text

BIG RATS LEAVE BIG POOS click_button "Tweet" click_link_or_button "Tweet" click_on "Tweet"

Slide 52

Slide 52 text

BIG RATS LEAVE BIG POOS click_button "Tweet" click_link_or_button "Tweet" click_on "Tweet" click_on "#tweet"

Slide 53

Slide 53 text

BIG RATS LEAVE BIG POOS click_button "Tweet" click_link_or_button "Tweet" click_on "Tweet" click_on "#tweet" find("#tweet").click

Slide 54

Slide 54 text

BIG RATS LEAVE BIG POOS click_button "Tweet" click_link_or_button "Tweet" click_on "Tweet" click_on "#tweet" find("#tweet").click page.execute_script("$('#tweet').click()")

Slide 55

Slide 55 text

BIG RATS LEAVE BIG POOS “Now, whenever you tweet, pop up a modal that forces the user to enter a CAPTCHA.”

Slide 56

Slide 56 text

BIG RATS LEAVE BIG POOS FFFFF...FFF....F....FFFFFFFFFFFFF.........

Slide 57

Slide 57 text

BIG RATS LEAVE BIG POOS module WebHelpers def tweet(message) fill_in “Message”, with: message click_on “Tweet” end end

Slide 58

Slide 58 text

BIG RATS LEAVE BIG POOS #rspec RSpec.configure do |c| c.include WebHelpers, type: :request end #cucumber World WebHelpers

Slide 59

Slide 59 text

Feature: Tweet Scenario: Valid Tweet Given Bob has authenticated When Bob submits a valid tweet Then Alice can see Bob’s tweet And Bob should see that tweet in his timeline Scenario: Overlong Tweet #.... Scenario: Duplicate tweet #....

Slide 60

Slide 60 text

BIG RATS LEAVE BIG POOS Given /^Bob has authenticated$/ do visit “/” fill_in “Username”, with: “Bob” fill_in “Password”, with: “password” click_on “Sign In” end

Slide 61

Slide 61 text

BIG RATS LEAVE BIG POOS Given /^Bob has authenticated$/ do login bob end

Slide 62

Slide 62 text

BIG RATS LEAVE BIG POOS module WebHelpers #... def bob @bob ||= create :user, name: “Bob” end #... end

Slide 63

Slide 63 text

BIG RATS LEAVE BIG POOS When /^Bob submits a valid Tweet$/ do click_on “Tweet” within(“#submit_tweet_modal”) do fill_in “Captcha”, with: captcha click_on “Tweet” end end

Slide 64

Slide 64 text

BIG RATS LEAVE BIG POOS When /^Bob submits a valid Tweet$/ do tweet “Hello, world” end

Slide 65

Slide 65 text

BIG RATS LEAVE BIG POOS Then /^Alice can see Bob’s Tweet$/ do click_on “Sign Out” fill_in “Username”, with: “Alice” fill_in “Password”, with: “password” click_on “Sign In” visit “/alice” page.should have_content “Hello, world” end

Slide 66

Slide 66 text

BIG RATS LEAVE BIG POOS Then /^Alice can see Bob’s Tweet$/ do login alice alice.should see_tweet “Hello, world” end

Slide 67

Slide 67 text

BIG RATS LEAVE BIG POOS feature “Tweet” do scenario “Valid Tweet” do login bob tweet “hello, world” login alice alice.should see “hello, world” end end

Slide 68

Slide 68 text

PRODUCT OWNERS DON’T COLLABORATE ON ACCEPTANCE CRITERIA

Slide 69

Slide 69 text

PRODUCT OWNERS DON’T COLLABORATE ON ACCEPTANCE CRITERIA 1. CARGO CULT

Slide 70

Slide 70 text

PRODUCT OWNERS DON’T COLLABORATE ON ACCEPTANCE CRITERIA 1. CARGO CULT 2. It’s hard

Slide 71

Slide 71 text

PRODUCT OWNERS DON’T COLLABORATE ON ACCEPTANCE CRITERIA 1. CARGO CULT 2. It’s hard 3. You’re annoying them

Slide 72

Slide 72 text

Given I click on “Sign In” And I fill in “Username” with “Bob” And I fill in “Password” with “password” And I click on “Submit” #....

Slide 73

Slide 73 text

Given Bob logged in

Slide 74

Slide 74 text

NO ONE READS THE CUKES

Slide 75

Slide 75 text

NO ONE READS THE CUKES 1. They’re unreadable

Slide 76

Slide 76 text

NO ONE READS THE CUKES 1. They’re unreadable 2. They’re not exposed anywhere

Slide 77

Slide 77 text

$ cucumber -f html

Slide 78

Slide 78 text

$ relish push myproj

Slide 79

Slide 79 text

$ wally push \ $PROJ features/

Slide 80

Slide 80 text

NO ONE READS THE CUKES 1. They’re unreadable 2. They’re not exposed anywhere 3. You haven’t needed documentation

Slide 81

Slide 81 text

$ time rake

Slide 82

Slide 82 text

$ time rake real 18m0.926s

Slide 83

Slide 83 text

$ time rake real 18m0.926s imagined ETERNITY

Slide 84

Slide 84 text

Slow tests find a way of getting run less often.

Slide 85

Slide 85 text

$ time rake real 18m0.926s imagined ETERNITY visit root_path click_on “Some Link” sleep 10 page.should have_content “FOO”

Slide 86

Slide 86 text

$ rake ..........F......... $ rake ................... $ rake ....F...........F.... $ rake ...................

Slide 87

Slide 87 text

$ time rake real 18m0.926s imagined ETERNITY visit root_path click_on “Some Link” wait_for { page.has_selector? (“.foo_container”) } page.should have_content “Foo”

Slide 88

Slide 88 text

$ time rake real 18m0.926s imagined ETERNITY visit root_path click_on “Some Link” wait_for { page.should have_selector(“.foo_container”) } page.should have_content “Foo” Whoops

Slide 89

Slide 89 text

$ time rake real 18m0.926s imagined ETERNITY visit root_path click_on “Some Link” page.should have_content “Foo”

Slide 90

Slide 90 text

Q IS FOR QUARANTINE @quarantine Scenario: Something flakey #....

Slide 91

Slide 91 text

Q IS FOR QUARANTINE #regular ci build #!/bin/bash cucumber --tags ~@quarantine

Slide 92

Slide 92 text

Q IS FOR QUARANTINE #quarantine ci build #!/bin/bash cucumber --tags @quarantine

Slide 93

Slide 93 text

BUILD NANNY 1. Timebox 2. Rethink 3. Delete

Slide 94

Slide 94 text

Photo by Kris Hicks

Slide 95

Slide 95 text

$ time rake “You should not be afraid to delete tests that are no longer providing value, no matter whether you originally planned to keep them or not. We tend to treat tests as these holy creatures that live blameless, irreproachable lives once they have sprung into existence. Not so. The maintenance required to keep a test running weighs against its value in further development. Sometimes these lines cross, and the test simply becomes a burden on the project. Having the skill and experience to recognize a burdensome test is something we should be bringing to our clients, as well as the fortitude to rewrite it, rethink it, or delete it." - ADAM MILLIGAN

Slide 96

Slide 96 text

$ time rake

Slide 97

Slide 97 text

$ time rake real 9m0.926s

Slide 98

Slide 98 text

$ time rake real 9m0.926s imagined ETERNITY

Slide 99

Slide 99 text

Why are you testing the login functionality 50 times?

Slide 100

Slide 100 text

1. Journey 2. Functional

Slide 101

Slide 101 text

$ time rake real 9m0.926s imagined ETERNITY @authenticated Feature: Publishing Widgets Scenario: ....

Slide 102

Slide 102 text

$ time rake real 9m0.926s imagined ETERNITY Before(“@authenticated”) do stub_authentication end

Slide 103

Slide 103 text

$ time rake

Slide 104

Slide 104 text

$ time rake real 4m0.926s

Slide 105

Slide 105 text

$ time rake real 4m0.926s imagined ETERNITY

Slide 106

Slide 106 text

$ time rake Feature: Sign Up Scenario: Valid username, email, password Scenario: Blank username, email, or password Scenario: Non-matching Password Scenario: Password too weak Scenario: Email address invalid Scenario: Username unavailable Scenario: User provides someone else’s email

Slide 107

Slide 107 text

$ time rake Feature: Sign Up Scenario: Valid username, email, password Scenario: Blank username, email, password Scenario: Non-matching Password Scenario: Password too weak Scenario: Email address invalid Scenario: Username unavailable Scenario: User provides someone else’s email

Slide 108

Slide 108 text

$ time rake Feature: Sign Up Scenario: Valid username, email, password Scenario: Blank username, email, password Scenario: Non-matching Password Scenario: Password too weak Scenario: Email address invalid Scenario: Username unavailable Scenario: User provides someone else’s email

Slide 109

Slide 109 text

$ time rake Feature: Sign Up Scenario: Valid username, email, password Scenario: Blank username, email, password @ci Scenario: Non-matching Password ....

Slide 110

Slide 110 text

$ time rake #.cucumber.yml default: --tags ~@ci

Slide 111

Slide 111 text

$ time rake Feature: Sign Up Scenario: Valid username, email, password Scenario: Blank username, email, password @no-ui Scenario: Non-matching Password ....

Slide 112

Slide 112 text

$ time rake Before “@no-ui” do extend DomainHelpers end

Slide 113

Slide 113 text

$ time rake module DomainHelpers attr_reader :current_user def authenticate @current_user = create :user end def tweet(message) current_user.tweet message end #... end

Slide 114

Slide 114 text

$ time rake

Slide 115

Slide 115 text

$ time rake real 30.926s

Slide 116

Slide 116 text

1. FIGHT PAIN 2. REVEAL INTENT 3. BDD LIKE YOU MEAN IT

Slide 117

Slide 117 text

Q/A