Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Breaking up (with) your test suite

Breaking up (with) your test suite

As presented at Ancient City Ruby 2014.

Please send questions & feedback to [email protected]!

Justin Searls

April 03, 2014
Tweet

More Decks by Justin Searls

Other Decks in Programming

Transcript

  1. Breaking up
    (with) your
    test suite
    poa asd

    View full-size slide

  2. My name is Justin Searls
    Please tweet me @searls &
    Say [email protected]

    View full-size slide

  3. Any questions?
    Tweet @searls during the talk
    Favorite question gets a book!

    View full-size slide

  4. Any questions?
    Tweet @searls during the talk
    Favorite question gets a book!

    View full-size slide

  5. This talk is about
    z aPPOOIU

    View full-size slide

  6. This talk is about
    z aPPOOIU
    relationships

    View full-size slide

  7. I my
    customers
    ias al

    View full-size slide

  8. I coding
    zffrttyu z

    View full-size slide

  9. I testing
    aske zc

    View full-size slide

  10. We don't test for
    testing's sake
    rRth asd

    View full-size slide

  11. I get paid for code that
    works, not for tests, so
    my philosophy is to test
    as little as possible to
    reach a given level of
    confidence.
    - Kent Beck
    asdz ty

    View full-size slide

  12. We don't code for
    code's sake, either
    mmmmznxdfr lr

    View full-size slide

  13. Writing code is a costly
    way to solve problems
    oiz eu

    View full-size slide

  14. Too often, we cheat on our
    customers with our code
    c d aoiey

    View full-size slide

  15. How to cheat on
    your customers:
    ae ts

    View full-size slide

  16. How to cheat on
    your customers:
    ae ts
    1. Uncritically assume your
    favorite way to write code
    applies to every situation

    View full-size slide

  17. How to cheat on
    your customers:
    ae ts
    1. Uncritically assume your
    favorite way to write code
    applies to every situation
    2. There is no #2

    View full-size slide

  18. evh zc
    Cargo-culting
    as a service

    View full-size slide

  19. We must challenge our biases
    in each new context to grow
    fh ywe

    View full-size slide

  20. iO ZOe
    infatuation

    View full-size slide

  21. xmt iRosXc
    Remember
    TATFT?

    View full-size slide

  22. TDD literally
    rescued me
    from the pit
    of despair.

    View full-size slide

  23. lzxcm zxcmlzcl lzxckz zxczc zxjcozcjoe AOO asodjaosd encz caosdjojo jojo
    6 months later
    ~ ~

    View full-size slide

  24. Why does
    every refactor
    break 6 tests?!

    View full-size slide

  25. Why does the
    build take nine
    hours to run?!

    View full-size slide

  26. Why are people
    mocking methods
    on the object
    they're testing?!

    View full-size slide

  27. Why can't we
    get anything
    done anymore?

    View full-size slide

  28. asodo zoxjcz aoo ueue yyyz cxjk zxcczx
    and eventually
    ~ ~

    View full-size slide

  29. the break-up

    View full-size slide

  30. the break-up

    View full-size slide

  31. $
    the break-up

    View full-size slide

  32. $
    the break-up
    rm -rf spec features

    View full-size slide

  33. jn poi
    introspection

    View full-size slide

  34. qwer bvt
    Why test?

    View full-size slide

  35. !


    !



    !








    !



    !



    !!

    View full-size slide

  36. !


    !



    !



    !


    !









    !!

    !



    !
    !




    !

    View full-size slide

  37. !


    !



    !



    !


    !









    !!

    !



    !
    !




    !

    Confidence

    View full-size slide

  38. !


    !



    !



    !


    !









    !!



    !









    !!

    View full-size slide

  39. !


    !



    !



    !


    !









    !!

    Understanding


    !









    !!

    View full-size slide

  40. mni pl
    Single
    Responsibility
    Testing

    View full-size slide

  41. describe "Panda" do
    end

    View full-size slide

  42. describe "Panda" do
    let(:bamboo) { double("bamboo").as_null_object }
    end

    View full-size slide

  43. describe "Panda" do
    let(:bamboo) { double("bamboo").as_null_object }
    subject { Panda.new(bamboo) }
    end

    View full-size slide

  44. describe "Panda" do
    let(:bamboo) { double("bamboo").as_null_object }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    end
    end

    View full-size slide

  45. describe "Panda" do
    let(:bamboo) { double("bamboo").as_null_object }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    subject.eat!
    end
    end

    View full-size slide

  46. describe "Panda" do
    let(:bamboo) { double("bamboo").as_null_object }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    end

    View full-size slide

  47. describe "Panda" do
    let(:bamboo) { double("bamboo").as_null_object }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    end

    View full-size slide

  48. asdj zoxc mmr qu psoi rnzc vnkl la
    later, fixing a bug
    ~ ~

    View full-size slide

  49. describe "Panda" do
    let(:bamboo) { double("bamboo").as_null_object }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    end

    View full-size slide

  50. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    end

    View full-size slide

  51. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    end

    View full-size slide

  52. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    end
    end

    View full-size slide

  53. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end

    View full-size slide

  54. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end

    View full-size slide

  55. asdj zoxc mmr qu psoi rnzc vnkl la
    one day, I find it
    ~ ~

    View full-size slide

  56. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end

    View full-size slide

  57. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end
    ?
    ? ?

    View full-size slide

  58. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end
    ?
    ? ?

    View full-size slide

  59. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end
    ?
    ? ?
    Why does this test exist?

    View full-size slide

  60. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end
    ?
    ? ?
    Why does this test exist?
    What type of test is this?

    View full-size slide

  61. describe "Panda" do
    let(:bamboo) { Bamboo.find(34) }
    subject { Panda.new(bamboo) }
    !
    it "munches bamboo" do
    allow(bamboo).to receive(:munch)
    subject.eat!
    expect(bamboo).to have_received(:munch)
    end
    !
    it "returns bamboo id" do
    expect(subject.eat!).to eq(34)
    end
    end
    ?
    ? ?
    Why does this test exist?
    How should tests be written?
    What type of test is this?

    View full-size slide

  62. nini lkre
    We should know
    these answers before
    opening the test!

    View full-size slide

  63. drt byby
    The purpose, rules, and
    structure of each test should
    be immediately clear

    View full-size slide

  64. kpo qw
    A single test can't address
    all of our motivations

    View full-size slide

  65. zxc eqr
    Such tests are unclear:

    View full-size slide

  66. zxc eqr
    • purpose is hazy
    Such tests are unclear:

    View full-size slide

  67. zxc eqr
    • purpose is hazy
    • rules are debatable
    Such tests are unclear:

    View full-size slide

  68. zxc eqr
    • purpose is hazy
    • rules are debatable
    • structure is ad hoc
    Such tests are unclear:

    View full-size slide

  69. tyu fd
    Unclear tests
    cost money

    View full-size slide

  70. awe voOa
    Unclear tests must
    constantly have their:

    View full-size slide

  71. awe voOa
    Unclear tests must
    constantly have their:
    • purpose rediscovered

    View full-size slide

  72. awe voOa
    Unclear tests must
    constantly have their:
    • purpose rediscovered
    • rules renegotiated

    View full-size slide

  73. awe voOa
    Unclear tests must
    constantly have their:
    • purpose rediscovered
    • rules renegotiated
    • structure reorganized

    View full-size slide

  74. xmcu peo
    None of those
    activities have value!

    View full-size slide

  75. ae r
    Every suite should
    promote at most
    type of
    confidence

    View full-size slide

  76. ae r
    Every suite should
    promote at most
    type of
    confidence

    View full-size slide

  77. c z
    Every suite should
    promote at most
    type of
    understanding

    View full-size slide

  78. c z
    Every suite should
    promote at most
    type of
    understanding

    View full-size slide

  79. suite design
    zxcm nz

    View full-size slide

  80. oeiz jdk
    Omakase Rails

    View full-size slide

  81. oeiz jdk
    Omakase Rails
    MiniTest

    View full-size slide

  82. oeiz jdk
    Omakase Rails
    Views MiniTest

    View full-size slide

  83. oeiz jdk
    Omakase Rails
    Views MiniTest
    MiniTest

    View full-size slide

  84. oeiz jdk
    Omakase Rails
    Controllers
    Views MiniTest
    MiniTest

    View full-size slide

  85. oeiz jdk
    Omakase Rails
    Controllers
    Views MiniTest
    MiniTest
    MiniTest

    View full-size slide

  86. oeiz jdk
    Omakase Rails
    Controllers
    Views
    Models
    MiniTest
    MiniTest
    MiniTest

    View full-size slide

  87. oeiz jdk
    Omakase Rails
    Controllers
    Views
    Models
    MiniTest
    MiniTest
    MiniTest

    View full-size slide

  88. ri xc
    Prime Rails

    View full-size slide

  89. ri xc
    Prime Rails
    Cucumber

    View full-size slide

  90. ri xc
    Prime Rails
    UI Cucumber

    View full-size slide

  91. ri xc
    Prime Rails
    RSpec
    UI Cucumber

    View full-size slide

  92. ri xc
    Prime Rails
    Views RSpec
    UI Cucumber

    View full-size slide

  93. ri xc
    Prime Rails
    Views RSpec
    RSpec
    UI Cucumber

    View full-size slide

  94. ri xc
    Prime Rails
    Controllers
    Views RSpec
    RSpec
    UI Cucumber

    View full-size slide

  95. ri xc
    Prime Rails
    Controllers
    Views RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  96. ri xc
    Prime Rails
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  97. ri xc
    Prime Rails
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  98. zpo qw
    "Fast Specs" Rails
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  99. zpo qw
    "Fast Specs" Rails
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber
    RSpec

    View full-size slide

  100. POROs
    zpo qw
    "Fast Specs" Rails
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber
    RSpec

    View full-size slide

  101. What's the
    problem?
    rRth asd

    View full-size slide

  102. Redundant test
    coverage
    rRth asd

    View full-size slide

  103. ri xc
    One model change
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  104. ri xc
    One model change
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  105. ri xc
    One model change
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  106. ri xc
    One model change
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  107. ri xc
    One model change
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  108. ri xc
    One model change
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  109. ri xc
    One model change
    Controllers
    Views
    Models
    RSpec
    RSpec
    RSpec
    UI Cucumber

    View full-size slide

  110. Which leads to:
    rRth asd

    View full-size slide

  111. ji cnzm
    Controllers
    Views
    Models
    UI RSpec
    (›°□°ʣ›ớ ᵲᴸᵲ Rails

    View full-size slide

  112. architecture
    jd qr

    View full-size slide

  113. +
    -
    router
    web
    app
    #1
    web
    app
    #2
    walrus
    data
    panda
    data
    cage
    control
    service

    View full-size slide

  114. +
    -
    router
    walrus
    data
    panda
    data
    web
    front-
    end
    web
    back-
    end
    web app #1
    cage
    control
    service

    View full-size slide

  115. +
    -
    walrus
    data
    web backend
    panda
    data
    router
    Feed Zoo
    Animals
    Build Feed
    Plan
    Provide
    Meal
    Fetch
    Schedule
    Plan Meal
    Animal
    Adapter
    Compute
    Portions
    cage
    control
    service
    Release
    Food
    Cage
    Adapter

    View full-size slide

  116. +
    -
    class ProvidesMeal
    def initialize...
    !
    def provide(plan)
    @releases_food.release(
    @computes_portion.compute(plan)
    )
    end
    end
    provides_meal.rb

    View full-size slide

  117. suite design
    slp er

    View full-size slide

  118. +
    -
    class ProvidesMeal
    def initialize...
    !
    def provide(plan)
    @releases_food.release(
    @computes_portion.compute(plan)
    )
    end
    end
    provides_meal.rb

    View full-size slide

  119. +
    -
    walrus
    data
    web backend
    panda
    data
    router
    Feed Zoo
    Animals
    Build Feed
    Plan
    Provide
    Meal
    Fetch
    Schedule
    Plan Meal
    Animal
    Adapter
    Compute
    Portions
    cage
    control
    service
    Release
    Food
    Cage
    Adapter

    View full-size slide

  120. +
    -
    router
    walrus
    data
    panda
    data
    web
    front-
    end
    web
    back-
    end
    web app #1
    cage
    control
    service

    View full-size slide

  121. +
    -
    router
    web
    app
    #1
    web
    app
    #2
    walrus
    data
    panda
    data
    cage
    control
    service

    View full-size slide

  122. The greatest
    test of all:
    dk rrt

    View full-size slide

  123. Does our software
    serve its purpose?
    qwe rtb

    View full-size slide

  124. Wait… what
    is its purpose?
    zl dfr

    View full-size slide

  125. Generating
    revenue?
    asd er

    View full-size slide

  126. Cutting
    costs?
    eit dfp

    View full-size slide

  127. Limiting our
    liability?
    xcg ke

    View full-size slide

  128. Funding
    non-profits?
    xd asdasdrg

    View full-size slide

  129. Can we write a test
    for those outcomes?
    zc er

    View full-size slide

  130. Maybe!*
    zc er

    View full-size slide

  131. Maybe!*
    zc er
    *But not in this talk.

    View full-size slide

  132. Smoke tests
    oi fg
    app

    View full-size slide

  133. Acceptance tests
    zxc cxoije
    app

    View full-size slide

  134. Feature tests
    rv cm
    app

    View full-size slide

  135. End-to-end tests
    zoc qwe
    app

    View full-size slide

  136. Integration tests
    xu vll
    app

    View full-size slide

  137. Integration tests
    xu vll
    app

    View full-size slide

  138. as cjhf
    app
    Smoke
    Acceptance
    Feature
    End-to-end

    View full-size slide

  139. a cv
    app
    Smoke
    Acceptance
    Feature
    End-to-end

    View full-size slide

  140. ed cx
    app
    S
    A
    F
    E

    View full-size slide

  141. zfh ie
    app
    SAFE tests

    View full-size slide

  142. The test's "user"

    View full-size slide

  143. Confidence sought

    View full-size slide

  144. Understanding gained

    View full-size slide

  145. Warning Signs

    View full-size slide

  146. A real-world user
    SAFE

    View full-size slide

  147. web apps
    app
    SAFE

    View full-size slide

  148. http services
    app
    $ curl http://
    SAFE

    View full-size slide

  149. Application works when
    it's all glued together
    SAFE

    View full-size slide

  150. "How simple is
    our product?"
    SAFE

    View full-size slide

  151. Can't fit your tests
    in 30 minutes?
    SAFE

    View full-size slide

  152. Can't fit your tests
    in 30 minutes?
    SAFE
    The app is complex

    View full-size slide

  153. SAFE
    Can't write a test
    in 30 minutes?

    View full-size slide

  154. SAFE
    Can't write a test
    in 30 minutes?
    The app is complicated

    View full-size slide

  155. SAFE tests shouldn't
    know internal APIs
    SAFE

    View full-size slide

  156. Web tests should bind to
    user-visible text, not markup
    SAFE

    View full-size slide

  157. Enforce a fixed-time budget
    SAFE

    View full-size slide

  158. Time-box the suite
    5 minutes
    5 minutes
    SAFE

    View full-size slide

  159. Time-box the suite
    5 minutes
    5 minutes
    monthly
    financials
    SAFE

    View full-size slide

  160. Time-box the suite
    5 minutes
    5 minutes
    monthly
    financials
    search
    SAFE

    View full-size slide

  161. Time-box the suite
    5 minutes
    5 minutes
    monthly
    financials
    process
    order
    search
    SAFE

    View full-size slide

  162. Time-box the suite
    5 minutes
    5 minutes
    monthly
    financials
    process
    order ad
    words
    search
    SAFE

    View full-size slide

  163. Time-box the suite
    5 minutes
    5 minutes
    sign-up
    monthly
    financials
    process
    order ad
    words
    search
    SAFE

    View full-size slide

  164. Time-box the suite
    5 minutes
    5 minutes
    sign-up
    monthly
    financials
    process
    order ad
    words
    search
    daily
    deals
    SAFE

    View full-size slide

  165. Time-box the suite
    5 minutes
    5 minutes
    sign-up
    monthly
    financials
    process
    order ad
    words
    search
    daily
    deals
    ?
    SAFE

    View full-size slide

  166. SAFE
    Failures due to refactors
    are false negatives

    View full-size slide

  167. SAFE
    Beware: human intuition
    overvalues "realistic" tests

    View full-size slide

  168. SAFE
    Building multiple releases
    & branches is expensive

    View full-size slide

  169. SAFE
    Superlinear Slowdown

    View full-size slide

  170. SAFE
    Superlinear Slowdown
    More
    tests
    Slower
    build

    View full-size slide

  171. SAFE
    Superlinear Slowdown

    View full-size slide

  172. SAFE
    Superlinear Slowdown
    Bigger
    System
    Slower
    build

    View full-size slide

  173. !


    !



    !








    !



    !



    !!

    SAFE

    View full-size slide

  174. !


    !



    !








    !



    !



    !!

    SAFE

    View full-size slide

  175. !


    !



    !








    !



    !



    !!

    SAFE

    View full-size slide

  176. +
    -
    router
    web
    app
    #1
    web
    app
    #2
    walrus
    data
    panda
    data
    cage
    control
    service

    View full-size slide

  177. +
    -
    router
    walrus
    data
    panda
    data
    web
    front-
    end
    web
    back-
    end
    web app #1
    cage
    control
    service

    View full-size slide

  178. Good SAFE tests
    are accurate
    ef zc

    View full-size slide

  179. But they're
    also expensive
    asd tg

    View full-size slide

  180. And they don't inform
    our public API's design
    ef gi

    View full-size slide

  181. Every library or service you write
    is consumed by somebody
    tg gz

    View full-size slide

  182. Verify behavior & demonstrate
    usage with Consumption tests
    rRth asd

    View full-size slide

  183. Your 's consumer
    Consumption

    View full-size slide

  184. Your 's consumer
    Consumption
    service

    View full-size slide

  185. Your 's consumer
    Consumption
    library

    View full-size slide

  186. Your 's consumer
    Consumption
    user interface

    View full-size slide

  187. Your 's consumer
    Consumption
    repository

    View full-size slide

  188. Verifies behavior you're
    directly responsible for
    Consumption

    View full-size slide

  189. Answers:
    "Is it usable?"
    Consumption

    View full-size slide

  190. If it's hard to test,
    then it's hard to use
    Consumption

    View full-size slide

  191. Module boundaries should be
    meaningful beyond testing
    Consumption

    View full-size slide

  192. Consumption
    Fakes all external dependencies

    View full-size slide

  193. Consumption
    Fakes all external dependencies
    (i.e. can run without
    access to other services)

    View full-size slide

  194. Consumption
    Fakes all external dependencies
    (i.e. can run without
    access to other services)
    (e.g. in Travis CI)

    View full-size slide

  195. Exercises public,
    not private APIs
    Consumption

    View full-size slide

  196. Organized by consumers'
    desired outcomes
    Consumption

    View full-size slide

  197. Keep consumption
    tests really fast
    Consumption

    View full-size slide

  198. faked
    services
    Consumption
    no good
    reason to
    be slow

    View full-size slide

  199. The only suite that says
    you just broke something
    Consumption

    View full-size slide

  200. !


    !



    !








    !



    !



    !!

    Consumption

    View full-size slide

  201. !


    !



    !








    !



    !



    !!

    Consumption

    View full-size slide

  202. !


    !



    !








    !



    !



    !!

    Consumption

    View full-size slide

  203. +
    -
    router
    walrus
    data
    panda
    data
    web
    front-
    end
    web
    back-
    end
    web app #1
    cage
    control
    service

    View full-size slide

  204. +
    -
    router
    web
    app
    #1
    web
    app
    #2
    walrus
    data
    panda
    data
    cage
    control
    service

    View full-size slide

  205. "If my tests fake out
    WalrusService
    how can I trust that
    GET /walruses
    still returns walruses?"
    xc r

    View full-size slide

  206. Why not test across
    service boundaries?
    q od

    View full-size slide

  207. Inter-service tests are easy!
    cd fe

    View full-size slide

  208. Inter-service tests are easy!
    cd fe
    1. Start by not writing them

    View full-size slide

  209. Inter-service tests are easy!
    cd fe
    1. Start by not writing them
    2. Continue to not write them

    View full-size slide

  210. Inter-service tests are easy!
    cd fe
    1. Start by not writing them
    2. Continue to not write them
    3. You're already done. Easy!

    View full-size slide

  211. qwe sd
    Inter-service tests are tempting,
    but they're .

    View full-size slide

  212. Inter-service tests are tempting,
    but they're .
    hard to set up
    qwe sd

    View full-size slide

  213. Inter-service tests are tempting,
    but they're .
    slow to run
    qwe sd

    View full-size slide

  214. Inter-service tests are tempting,
    but they're .
    redundant coverage
    qwe sd

    View full-size slide

  215. The root question
    indicates a lack of trust
    aie wq

    View full-size slide

  216. Default to trusting your
    depended-on services
    zxc see

    View full-size slide

  217. ... but sometimes trust
    isn't appropriate!
    poae fg

    View full-size slide

  218. Sometimes our usage is
    abnormal or unsupported
    xcg asd

    View full-size slide

  219. Sometimes other teams
    demonstrate blind-spots
    iocv hgt

    View full-size slide

  220. When trust isn't enough,
    try writing Contract tests
    zx rrr

    View full-size slide

  221. The "user" is us!
    Contract

    View full-size slide

  222. Test is written to
    represent your interests
    in somebody else's repo
    Contract

    View full-size slide

  223. Your needs become part
    of their Consumption suite
    Contract

    View full-size slide

  224. If they break your test,
    they know to call you!
    Contract

    View full-size slide

  225. Speedier Feedback
    Contract
    fast
    slow

    View full-size slide

  226. Speedier Feedback
    Contract
    production
    release
    fast
    slow

    View full-size slide

  227. Speedier Feedback
    Contract
    production
    release
    git push
    fast
    slow

    View full-size slide

  228. Dependencies that we own
    behave how we need them to
    Contract

    View full-size slide

  229. We learn: "is this service
    a good fit for us?"
    Contract

    View full-size slide

  230. Frequent failures may
    reveal differing priorities
    Contract

    View full-size slide

  231. The maintainer learns:
    "are people using our service
    as we expect (or desire)?"
    Contract

    View full-size slide

  232. Maybe this isn't
    the best service
    for them
    Contract

    View full-size slide

  233. Maybe this isn't
    the best service
    period.
    Contract
    ,

    View full-size slide

  234. Written just like
    consumption tests
    Contract

    View full-size slide

  235. Committed into the repo
    of the depended-on service
    Contract

    View full-size slide

  236. Maintainers can lower barrier of
    entry by providing conveniences
    Contract

    View full-size slide

  237. Not a replacement
    for human interaction
    Contract

    View full-size slide

  238. !


    !



    !








    !



    !



    !!

    Contract

    View full-size slide

  239. !


    !



    !








    !



    !



    !!

    Contract

    View full-size slide

  240. !


    !



    !








    !



    !



    !!

    Contract

    View full-size slide

  241. +
    -
    router
    web
    app
    #1
    web
    app
    #2
    walrus
    data
    panda
    data
    cage
    control
    service

    View full-size slide

  242. +
    -
    router
    walrus
    data
    panda
    data
    web
    front-
    end
    web
    back-
    end
    web app #1
    cage
    control
    service

    View full-size slide

  243. +
    -
    walrus
    data
    web backend
    panda
    data
    router
    Feed Zoo
    Animals
    Build Feed
    Plan
    Provide
    Meal
    Fetch
    Schedule
    Plan Meal
    Animal
    Adapter
    Compute
    Portions
    cage
    control
    service
    Release
    Food
    Cage
    Adapter

    View full-size slide

  244. +
    -
    provides_meal.rb
    class ProvidesMeal
    def initialize...
    !
    def provide(plan)
    @releases_food.release(
    @computes_portion.compute(plan)
    )
    end
    end

    View full-size slide

  245. What about TDD?
    zxc ei

    View full-size slide

  246. "What's the point?"
    eq rf

    View full-size slide

  247. "We already have
    lots of tests!"
    xc fe

    View full-size slide

  248. So far, no feedback
    about code's design
    ei kt

    View full-size slide

  249. You already have a
    definition for "unit" test
    eq xc

    View full-size slide

  250. I wrote about this
    a few months ago
    xi x

    View full-size slide

  251. I wrote about this
    a few months ago
    xi x
    (just Google "tdd failure")

    View full-size slide

  252. It even elicited a
    response from Uncle Bob!
    qw rf

    View full-size slide

  253. I will refute his arguments
    and utterly destroy his
    conclusions. And then, once
    I'm done salting the ground
    where he used to live, ...l tell
    you why I completely agree
    with him.
    - Uncle Bob
    asdz ty

    View full-size slide

  254. I will refute his arguments
    and utterly destroy his
    conclusions. And then, once
    I'm done salting the ground
    where he used to live, I'll tell
    you why I completely agree
    with him.
    - Uncle Bob
    asdz ty

    View full-size slide

  255. There's no
    "one true TDD"
    iz rgr

    View full-size slide

  256. The principal value of TDD
    in this model is discovery
    vj sdf

    View full-size slide

  257. Discovery of tiny, boring,
    consistent units that break
    big, scary problems down
    into small, manageable ones
    ri gh

    View full-size slide

  258. The first caller of
    a new method
    Discovery

    View full-size slide

  259. Concerned primarily
    with inputs and output
    (or side effect)
    Discovery

    View full-size slide

  260. Also concerned with
    basic code design details
    Discovery

    View full-size slide

  261. Inputs & Outcome
    Discovery
    Feed Zoo
    Animals

    View full-size slide

  262. Inputs & Outcome
    Discovery
    Feed Zoo
    Animals
    (walrus, cheese)

    View full-size slide

  263. Inputs & Outcome
    Discovery
    Feed Zoo
    Animals
    (walrus, cheese) Walrus has
    cheese

    View full-size slide

  264. Discovery
    Now what?!

    View full-size slide

  265. Discovery
    describe FeedsAnimals do
    end

    View full-size slide

  266. Discovery
    describe FeedsAnimals do
    describe "#feed" do
    end
    end

    View full-size slide

  267. Discovery
    describe FeedsAnimals do
    describe "#feed" do
    expect(my_face).to look_like(ಠ_ಠ)
    end
    end

    View full-size slide

  268. Discovery
    Feed Zoo
    Animals
    Code by wishful thinking!

    View full-size slide

  269. Discovery
    Build Feed
    Plan
    Provide
    Meal
    Feed Zoo
    Animals
    Code by wishful thinking!

    View full-size slide

  270. Discovery
    The test is our sounding board
    Build Feed
    Plan
    Provide
    Meal
    Feed Zoo
    Animals

    View full-size slide

  271. Discovery
    describe FeedsAnimals do
    !
    !
    !
    end

    View full-size slide

  272. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    !
    !
    end

    View full-size slide

  273. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    Given(:provides_meal) { gimme(ProvidesMeal) }
    !
    !
    end

    View full-size slide

  274. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    Given(:provides_meal) { gimme(ProvidesMeal) }
    !
    subject { FeedsAnimals.new(builds_feed_plan, provides_meal) }
    !
    end

    View full-size slide

  275. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    Given(:provides_meal) { gimme(ProvidesMeal) }
    !
    subject { FeedsAnimals.new(builds_feed_plan, provides_meal) }
    !
    describe "#feed" do
    end
    end

    View full-size slide

  276. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    Given(:provides_meal) { gimme(ProvidesMeal) }
    !
    subject { FeedsAnimals.new(builds_feed_plan, provides_meal) }
    !
    describe "#feed" do
    When(:result) { subject.feed(:walrus, :cheese) }
    end
    end

    View full-size slide

  277. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    Given(:provides_meal) { gimme(ProvidesMeal) }
    !
    subject { FeedsAnimals.new(builds_feed_plan, provides_meal) }
    !
    describe "#feed" do
    When(:result) { subject.feed(:walrus, :cheese) }
    Then { verify(provides_meal).provide(:cheese, :plan) }
    end
    end

    View full-size slide

  278. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    Given(:provides_meal) { gimme(ProvidesMeal) }
    !
    subject { FeedsAnimals.new(builds_feed_plan, provides_meal) }
    !
    describe "#feed" do
    Given { give(builds_feed_plan).build(:walrus) {:plan}}
    When(:result) { subject.feed(:walrus, :cheese) }
    Then { verify(provides_meal).provide(:cheese, :plan) }
    end
    end

    View full-size slide

  279. Discovery
    describe FeedsAnimals do
    Given(:builds_feed_plan) { gimme(BuildsFeedPlan) }
    Given(:provides_meal) { gimme(ProvidesMeal) }
    !
    subject { FeedsAnimals.new(builds_feed_plan, provides_meal) }
    !
    describe "#feed" do
    Given { give(builds_feed_plan).build(:walrus) {:plan}}
    When(:result) { subject.feed(:walrus, :cheese) }
    Then { verify(provides_meal).provide(:cheese, :plan) }
    end
    end

    View full-size slide

  280. Discovery
    Now I have two problems!
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  281. Discovery
    Now I have two problems!
    Build Feed
    Plan
    Provide
    Meal


    View full-size slide

  282. Correct behavior of
    logical leaf nodes
    Discovery

    View full-size slide

  283. (a.k.a. pure, first-
    order functions)
    Discovery

    View full-size slide

  284. Discovery
    Feed Zoo
    Animals
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  285. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  286. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  287. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  288. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    ✔️
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  289. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    ✔️
    Compute
    Portions
    Release
    Food
    ✔️
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  290. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    ✔️
    Compute
    Portions
    Release
    Food
    ✔️
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  291. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    ✔️
    Compute
    Portions
    Release
    Food
    ✔️
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  292. Small things
    Discovery

    View full-size slide

  293. Small things
    Discovery
    (Free SRP!)

    View full-size slide

  294. Separation of Roles
    Discovery

    View full-size slide

  295. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal

    View full-size slide

  296. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal
    Command

    View full-size slide

  297. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal
    Command
    Query

    View full-size slide

  298. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal
    Command
    Query Command

    View full-size slide

  299. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal
    Command
    Query
    Query
    Command

    View full-size slide

  300. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal
    Command
    Query
    Query Logic
    Command

    View full-size slide

  301. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal
    Command
    Query
    Query Logic Logic
    Command

    View full-size slide

  302. Discovery
    Feed Zoo
    Animals
    Fetch
    Schedule
    Plan Meal
    Compute
    Portions
    Release
    Food
    Build Feed
    Plan
    Provide
    Meal
    Command
    Query
    Query Logic Logic Command
    Command

    View full-size slide

  303. Command & Query tests discover
    dependencies with test doubles
    Discovery

    View full-size slide

  304. Discovery
    Logic tests discover
    implementation by usage

    View full-size slide

  305. Discovery
    (i.e. no test doubles)
    Logic tests discover
    implementation by usage

    View full-size slide

  306. Commands and Queries
    contain very little logic
    Discovery

    View full-size slide

  307. Pain is good!
    Discovery

    View full-size slide

  308. Discovery tests yield
    small, disposable units
    Discovery

    View full-size slide

  309. Discovery tests yield
    small, disposable units
    Discovery
    ...so don't be afraid
    to throw them away!

    View full-size slide

  310. If requirements change, trash
    a minimal sub-tree of units
    and drive out a new solution
    Discovery

    View full-size slide

  311. Re-use is
    overrated
    Discovery

    View full-size slide

  312. Extract refactors
    are a smell
    Discovery

    View full-size slide

  313. Frameworks will
    fight you!
    Discovery

    View full-size slide

  314. Frameworks & TDD try to
    solve the same problems
    Discovery

    View full-size slide

  315. Frameworks provide
    an orderly structure
    Discovery

    View full-size slide

  316. TDD is an approach
    for discovering an
    orderly structure
    Discovery

    View full-size slide

  317. Discovery testing is at odds
    with most app frameworks
    Discovery

    View full-size slide

  318. !


    !



    !








    !



    !



    !!

    Discovery

    View full-size slide

  319. !


    !



    !








    !



    !



    !!

    Discovery

    View full-size slide

  320. !


    !



    !








    !



    !



    !!

    Discovery

    View full-size slide

  321. +
    -
    provides_meal.rb
    class ProvidesMeal
    def initialize...
    !
    def provide(plan)
    @releases_food.release(
    @computes_portion.compute(plan)
    )
    end
    end

    View full-size slide

  322. +
    -
    walrus
    data
    web backend
    panda
    data
    router
    Feed Zoo
    Animals
    Build Feed
    Plan
    Provide
    Meal
    Fetch
    Schedule
    Plan Meal
    Animal
    Adapter
    Compute
    Portions
    cage
    control
    service
    Release
    Food
    Cage
    Adapter

    View full-size slide

  323. "Wow, that's a lot
    of !"
    zxc tg
    mock objects

    View full-size slide

  324. "Wow, that's a lot
    of !"
    fake stuff
    zxc tg

    View full-size slide

  325. "Wow, that's a lot
    of !"
    test doubles
    zxc tg

    View full-size slide

  326. "Aren't those many
    test doubles a pain?"
    ad er

    View full-size slide

  327. They can be!
    asd xc

    View full-size slide

  328. Note: only use doubles
    when testing collaborators
    g rg

    View full-size slide

  329. Note: better test double
    libraries are better
    gr cz

    View full-size slide

  330. kdasd fk
    Discovery tests try to
    improve dependencies' APIs

    View full-size slide

  331. kg er
    But we can't change
    3rd party APIs

    View full-size slide

  332. er zxc
    Mocking what you don't
    own leads to useless pain

    View full-size slide

  333. Wrap 3rd party API calls in
    adapters that you do own
    uc vt

    View full-size slide

  334. Wrap 3rd party API calls in
    adapters that you do own
    uc vt
    (And mock those adapters instead)

    View full-size slide

  335. Typically, adapters
    shouldn't need tests
    zx bt

    View full-size slide

  336. Most tests exercise features
    that make use 3rd party APIs
    jd hn

    View full-size slide

  337. It sometimes make sense to test
    code we don't own separately
    jd hn

    View full-size slide

  338. When you want to respond to 3rd party
    failures differently, write Adapter tests
    jd hn

    View full-size slide

  339. Your application, pondering its
    relationship with a 3rd party API
    Adapter

    View full-size slide

  340. Exercises your adapter API under
    realistic-enough circumstances
    Adapter

    View full-size slide

  341. Adapter
    Tests of libraries warn
    of unsafe upgrades

    View full-size slide

  342. Adapter
    Tests of network services warn
    of outages & breaking changes

    View full-size slide

  343. Adapters (& tests) specify
    how you depend on a thing
    Adapter

    View full-size slide

  344. Establishes boundaries, prevents
    3rd party API references from
    seeping throughout your app
    Adapter

    View full-size slide

  345. Drastically reduces the cost of
    replacing dependencies later
    Adapter

    View full-size slide

  346. Only test adapters
    with good cause
    Adapter

    View full-size slide

  347. "Don't test the
    framework."
    Adapter

    View full-size slide

  348. Adapter test suites of
    network code can be
    tricky to run in CI
    Adapter

    View full-size slide

  349. It's easier to test 3rd party
    services from your SAFE suite
    Adapter

    View full-size slide

  350. All adapter tests are
    likely to be slow in ways
    outside your control
    Adapter

    View full-size slide

  351. !


    !



    !








    !



    !



    !!

    Adapter

    View full-size slide

  352. !


    !



    !








    !



    !



    !!

    Adapter

    View full-size slide

  353. !


    !



    !








    !



    !



    !!

    Adapter

    View full-size slide

  354. And that's just
    way to break
    up your test suite
    asd

    View full-size slide

  355. And that's just
    way to break
    up your test suite
    asd

    View full-size slide

  356. My name is Justin Searls
    Please tweet me @searls &
    Say [email protected]

    View full-size slide

  357. Like everyone, we're hiring!
    Just [email protected]

    View full-size slide

  358. Please say hello if your team
    could use our team's help #

    View full-size slide

  359. Quotation Marks designed by Horacio Bella from the thenounproject.com
    !
    Happy designed by Michael Hourigan from the thenounproject.com
    !
    Tired designed by Aha-Soft from the thenounproject.com
    !
    Indifference designed by Paul F. from the thenounproject.com
    !
    Mad designed by Aha-Soft from the thenounproject.com
    !
    Sad designed by Simple Icons from the thenounproject.com
    !
    Marker designed by Jeff Seevers from the thenounproject.com
    !
    Database designed by Grant Fisher from the thenounproject.com
    !
    Cloud designed by Thomas Hirter from the thenounproject.com
    !
    Safety Net designed by irene hoffman from the thenounproject.com
    !
    Light Bulb designed by Olivier Guin from the thenounproject.com
    !
    Scale designed by Stephanie Wauters from the thenounproject.com
    !
    Warning designed by Lorena Salagre from the thenounproject.com
    !
    Reload designed by Mateo Zlatar from the thenounproject.com
    Noun project attribution

    View full-size slide