Testing 201, or: Great Expectations

Testing 201, or: Great Expectations

Ruby developers are a testy bunch. Who else writes more tests that we do? Which makes sense! Our language allows us freedom, and the opportunity to abuse it.

Most of us learned to test informally, from existing projects or tutorials. Sadly, that often leads us to the false conclusion that is all about verifying that our code works. Would you believe that’s only the third most important reason to test?

Let’s take a non-dogmatic look at the less obvious roles for testing with some real examples. We can get so much more from testing.

A9704266587836f7e784235e5073b93e?s=128

Joseph Mastey

August 04, 2020
Tweet

Transcript

  1. TESTING 201 OR: GREAT EXPECTATIONS JOE MASTEY, AUGUST 2020

  2. LOTS OF RSPEC, FACTORY BOT, AND RAILS

  3. EVERYONE IS DOING THEIR BEST

  4. HEURISTICS, NOT RULES

  5. PROLOGUE: THE TROUBLE WITH A SIMPLE TEST

  6. class CreateShipmentHistoryReport attr_accessor :user, :shipments def initialize(user) @user = user

    @shipments = user.shipments end def process shipments.sort_by(&:created_at) data = shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name, shipment.product.name ] end file = CSV.open(tmp_filename) do |csv| csv << shipments.first.to_json.keys + ['user', 'product'] data.map do |row| csv << row end end persist_csv_to_s3(file) notify_user(user.email) file end end
  7. class CreateShipmentHistoryReport attr_accessor :user, :shipments def initialize(user) @user = user

    @shipments = user.shipments end
  8. def process shipments.sort_by(&:created_at) data = shipments.map do |shipment| shipment.to_json.values +

    [ shipment.user.name, shipment.product.name ] end # continued below...
  9. # ... process continued file = CSV.open(tmp_filename) do |csv| csv

    << shipments.first.to_json.keys + ['user', 'product'] data.map do |row| csv << row end end persist_csv_to_s3(file) notify_user(user.email) file end
  10. class CreateShipmentHistoryReport attr_accessor :user, :shipments def initialize(user) @user = user

    @shipments = user.shipments end def process shipments.order('created_at desc') data = shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name, shipment.product.name ] end file = CSV.open(tmp_filename) do |csv| csv << shipments.first.to_json.keys + ['user', 'product'] data.map do |row| csv << row end end persist_csv_to_s3(file) notify_user(user.email) file end end
  11. describe CreateShipmentHistoryReport do describe "#process" do # ... it "generates

    an array of strings" do expect(csv_data.length).to eq(shipments.length) expect(csv_data).to all(be_an(Array)) end it "matches the expected output" do expect(csv_data).to eq(output_records) end end end
  12. let(:subject) { described_class.new(user).process } let(:csv_data) { CSV.parse(subject) }

  13. let(:user) { create(:user, :signup_complete) } let(:subscription) { create(:subscription) } before

    do user.subscriptions << subscription expect(user) .to receive(:shipments) .and_return(shipments) end
  14. let(:shipments) do [ shipment_1, shipment_2, shipment_3, shipment_4 ] end

  15. let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) {

    create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) }
  16. let(:product_1) { create(:product) } let(:product_2) { create(:product) } let(:product_3) {

    create(:product) } let(:product_4) { create(:product) }
  17. let(:output_records) do [ [shipment_4.created_on, shipment_4.completed_on, user.name, product_4.name], [shipment_3.created_on, shipment_3.completed_on, user.name,

    product_3.name], [shipment_2.created_on, shipment_2.completed_on, user.name, product_2.name], [shipment_1.created_on, shipment_1.completed_on, user.name, product_1.name], ] end
  18. before do expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true) expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true)

    end
  19. it "matches the expected output" do expect(csv_data).to eq(output_records) end

  20. describe CreateShipmentHistoryReport do describe "#process" do let(:user) { create(:user, :signup_complete)

    } let(:subscription) { create(:subscription) } let(:product_1) { create(:product) } let(:product_2) { create(:product) } let(:product_3) { create(:product) } let(:product_4) { create(:product) } let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) { create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) } let(:subject) { described_class.new(user).process } let(:csv_data) { CSV.parse(subject) } let(:shipments) do [ shipment_1, shipment_2, shipment_3, shipment_4 ] end let(:output_records) do [ [shipment_4.created_on, shipment_4.completed_on, user.name, product_4.name], [shipment_3.created_on, shipment_3.completed_on, user.name, product_3.name], [shipment_2.created_on, shipment_2.completed_on, user.name, product_2.name], [shipment_1.created_on, shipment_1.completed_on, user.name, product_1.name], ] end before do user.subscriptions << subscription expect(user) .to receive(:shipments) .and_return(shipments) expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true) expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true) end it "generates an array of strings" do expect(csv_data.length).to eq(shipments.length) expect(csv_data).to all(be_an(Array)) end it "matches the expected output" do expect(csv_data).to eq(output_records) end end end
  21. FOCUSING ON THE WRONG THINGS

  22. THE 3 ROLES OF TESTING (IN ORDER)

  23. 1. DESIGN FEEDBACK 2. DOCUMENTATION 3. VERIFICATION

  24. TESTS AS DESIGN FEEDBACK

  25. A NOTE ON TDD

  26. CODE THAT’S DIFFICULT TO TEST IS TRYING TO TELL YOU

    SOMETHING IMPORTANT
  27. IT’S EASIER TO WORK WITH OBJECTS IN SMALL CHUNKS.

  28. let(:csv_data) { CSV.parse(subject) } before do expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true)

    expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true) end
  29. def data(shipments: @shipments) shipments.sort_by!(&:created_at) shipments.map do |shipment| shipment.to_json.values + [

    shipment.user.name, shipment.product.name ] end end def headers shipments.first.to_json.keys + ['user', 'product'] end
  30. file = CSV.open(tmp_filename) do |csv| csv << headers data(shipments).map {

    |row| csv << row } end
  31. it "matches the expected output" do subject = described_class.new(user) expect(subject.data).to

    eq(output_records) end
  32. before do # expect_any_instance_of(described_class) # .to receive(:save_to_s3) # .and_return(true) #

    expect_any_instance_of(described_class) # .to receive(:email_user) # .and_return(true) end
  33. before do expect(user) .to receive(:shipments) .and_return(shipments) end

  34. class CreateShipmentHistoryReport def initialize(user, shipments = user.shipments) @user = user

    @shipments = shipments end end
  35. before do # expect(user) # .to receive(:shipments) # .and_return(shipments) end

    described_class.new(user, shipments)
  36. it "matches the expected output" do subject = described_class.new(user, shipments)

    expect(subject.data).to eq(output_records) end
  37. def data(shipments) shipments.sort_by!(&:created_at) shipments.map do |shipment| shipment.to_json.values + [ shipment.user.name,

    shipment.product.name ] end end def headers shipments.first.to_json.keys + ['user', 'product'] end
  38. def serialize(shipment) { created_on: shipment.created_on, completed_on: shipment.completed_on, user: shipment.user.name, product:

    shipment.product.name, } end
  39. def data(shipments) shipments.sort_by!(&:created_at) shipments.map { |shipment| serialize(shipment) } end def

    headers serialize(shipments.first).keys end
  40. describe "#serialize" do it "serializes some shipment and user data"

    do shipment = create(:shipment) report = described_class.new(user, []) result = report.serialize(shipment) expect(result).to eq({ created_on: shipment.created_on, completed_on: shipment.completed_on, user: shipment.user.name, product: shipment.product.name, }) end end
  41. # let(:output_records) do # [ # [shipment_4.created_on, shipment_4.completed_on, # user.name,

    product_4.name], # [shipment_3.created_on, shipment_3.completed_on, # user.name, product_3.name], # [shipment_2.created_on, shipment_2.completed_on, # user.name, product_2.name], # [shipment_1.created_on, shipment_1.completed_on, # user.name, product_1.name], # ] # end
  42. describe "ordering" do it "reorders shipments by their creation date"

    do report = described_class.new(user,[shipment_1, shipment_2]) result = report.data expect(result.map(&:first)).to eq( [shipment_2.created_on, shipment_1.created_on] ) end end
  43. CREATE AS FEW RECORDS AS YOU CAN (AND ONLY RELATED

    ONES)
  44. let(:user) { create(:user, :signup_complete) } let(:subscription) { create(:subscription) } before

    do user.subscriptions << subscription end
  45. Factory AR records AR queries create(:user) 1 10 create(:meal) 5

    41 create(:full_menu) 94 584 create(:weekly_basket) 104 644 create(:user, :with_order_history) 379 2336
  46. let(:user) do instance_double(User, email: "test@test.com", shipments: shipments) end

  47. let(:user) { User.new(email: "test@test.com", name: "Joe") }

  48. describe "ordering" do it "reorders shipments by their creation date"

    do report = described_class.new(user, [shipment_1, shipment_2]) result = report.data expect(result.map(&:first)).to eq( [shipment_2.created_on, shipment_1.created_on] ) end end
  49. PAY ATTENTION WHEN CLASSES DO TOO MUCH

  50. def process def serialize(shipment) def data(shipments) def headers def save_to_s3(file)

    def notify_user(email)
  51. TESTS AS DOCUMENTATION

  52. OPTIMIZE TESTS FOR HUMAN UNDERSTANDING

  53. it "matches the expected output" do expect(csv_data).to eq(output_records) end

  54. describe "ordering" do it "reorders shipments by their creation date"

    do old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) result = report.data expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
  55. it "serializes users as passed into the service" do users

    = create_list(:user, 3) response = subject.new(User.all).process response = response[0][4] expect(response).to eq(users.first.name) end
  56. it "serializes users as passed into the service" do users

    = create_list(:user, 3) serialized_users = subject.new.serialize(users) response = serialized_users.first.full_name expect(response).to eq(users.first.name) end
  57. PAY ATTENTION TO SIMILARITY AND DIFFERENCE

  58. let(:product_1) { create(:product) } let(:product_2) { create(:product) } let(:product_3) {

    create(:product) } let(:product_4) { create(:product) }
  59. let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) {

    create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) }
  60. let(:output_records) do [ [shipment_4.created_on, shipment_4.completed_on, user.name, product_4.name], [shipment_3.created_on, shipment_3.completed_on, user.name,

    product_3.name], [shipment_2.created_on, shipment_2.completed_on, user.name, product_2.name], [shipment_1.created_on, shipment_1.completed_on, user.name, product_1.name], ] end
  61. def create_shipment(date:) product = create(:product) create(:shipment, product: product, created_at: date)

    end
  62. describe "ordering" do it "reorders shipments by their creation date"

    do old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) result = report.data expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
  63. # let(:product_1) { create(:product) } # let(:product_2) { create(:product) }

    # let(:product_3) { create(:product) } # let(:product_4) { create(:product) } # let!(:shipment_1) { create(:shipment, product: product_1, # created_at: 2.days.ago) } # let!(:shipment_2) { create(:shipment, product: product_2, # created_at: 3.days.ago) } # let!(:shipment_3) { create(:shipment, product: product_3, # created_at: 4.days.ago) } # let!(:shipment_4) { create(:shipment, product: product_4, # created_at: 5.days.ago) } # # let(:shipments) do # [ # shipment_1, # shipment_2, # shipment_3, # shipment_4 # ] # end # let(:output_records) do # [ # [shipment_4.created_on, shipment_4.completed_on, # user.name, product_4.name], # [shipment_3.created_on, shipment_3.completed_on, # user.name, product_3.name], # [shipment_2.created_on, shipment_2.completed_on, # user.name, product_2.name], # [shipment_1.created_on, shipment_1.completed_on, # user.name, product_1.name], # ] # end
  64. TRY TO STICK TO A GIVEN/ WHEN/THEN STRUCTURE

  65. describe "ordering" do it "reorders shipments by their creation date"

    do old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) result = report.data expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
  66. describe "ordering" do it "reorders shipments by their creation date"

    do # Given old = create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) # When report = described_class.new(user, [new, old]) result = report.data # Then expect(result.map(&:first)).to eq( [old.created_on, new.created_on] ) end end
  67. it "reorders shipments by their creation date" do old =

    create_shipment(date: 9.days.ago) new = create_shipment(date: 2.days.ago) report = described_class.new(user, [new, old]) expect(report.data.map(&:first)).to eq([old, new]) end
  68. before do expect_any_instance_of(described_class) .to receive(:save_to_s3) .and_return(true) expect_any_instance_of(described_class) .to receive(:email_user) .and_return(true)

    end
  69. USING G/W/T EMPHASIZES SAMENESS AND DIFFERENCE.

  70. it "returns order units summed, divided by units per shipper"

    do create_store_order(count_units: 4) create_store_order(count_units: 8) subject = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(3) end
  71. it "returns a ceiling rounded value" do create_store_order(count_units: 7) subject

    = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(2) end
  72. let!(:shipment_1) { create(:shipment, product: product_1, created_at: 2.days.ago) } let!(:shipment_2) {

    create(:shipment, product: product_2, created_at: 3.days.ago) } let!(:shipment_3) { create(:shipment, product: product_3, created_at: 4.days.ago) } let!(:shipment_4) { create(:shipment, product: product_4, created_at: 5.days.ago) }
  73. TESTS DON’T NEED TO BE (TOO) DRY

  74. it "returns order units summed, divided by units per shipper"

    do create_store_order(count_units: 4) create_store_order(count_units: 8) subject = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(3) end it "returns a ceiling rounded value" do create_store_order(count_units: 7) subject = described_class.new(store_orders: store_orders) total_shippers = subject.total_shipper_count expect(total_shippers).to eq(2) end
  75. describe "#total_shipper_count" do subject { described_class.new(store_orders: StoreOrder.all) } context "even

    division" do let!(:order1) { create_store_order(count_units: 4) } let!(:order2) { create_store_order(count_units: 8) } let!(:expected_total) { 3 } # ... lots of other tests ... it "returns order units summed, divided by units per shipper" do expect(subject.total_shipper_count).to eq(expected_total) end end end
  76. context "rounding" do let!(:order) { create_store_order(count_units: 7) } let!(:expected_total) {

    2 } # ... lots of other tests ... it "returns order units summed, divided by units per shipper" do expect(subject.total_shipper_count).to eq(expected_total) end end
  77. IF YOUR TESTS ARE SUPER DUPLICATIVE, THAT’S ACTUALLY DESIGN FEEDBACK

  78. SAY WHAT YOU MEAN, LITERALLY

  79. def user_summary(user) "(#{user.id}) #{user.full_name}, #{user.role}" end

  80. it "returns a user summary for printing" do user =

    User.new(id: 5, full_name: "Dave G", role: :admin) expected = "(#{user.id}) #{user.full_name}, #{user.role}" summary = user_summary(user) expect(summary).to eq(expected) end
  81. it "returns a user summary for printing" do user =

    User.new(id: 5, full_name: "Dave G", role: :admin) summary = user_summary(user) expect(summary).to eq("(5) Dave G, admin") end
  82. LEVERAGE YOUR TOOLS TO HELP GUIDE READERS

  83. describe "#process" do it “has a correct return value" end

    describe "#process" do it "returns the number of records correctly persisted" end
  84. describe CreateShipmentHistoryReport do describe "#sorted_shipments" do it "returns an empty

    array when there are no shipments" end end
  85. rspec ./spec/services/create_shipment_history_report_spec.rb --format=‘documentation’ CreateShipmentHistoryReport#sorted_shipments returns an empty array when there

    are no shipments
  86. it “picks randomly, but correctly” do double1 = double double2

    = double result = [double1, double2].sample expect(result).to eq(double1) end # expected: #<Double (anonymous)> # got: #<Double (anonymous)>
  87. it “picks randomly, but correctly” do double1 = double("first record")

    double2 = double("second record”) result = [double1, double2].sample expect(result).to eq(double1) end # expected: #<Double "first record"> # got: #<Double "second record">
  88. it "checks result size" do arr = [1,2,3] response =

    arr.length expect(response > 3).to be_true end # expected true # got false
  89. it "checks result size" do arr = [1,2,3] response =

    arr.length expect(response).to be > 3 end # expected: > 3 # got: 3
  90. it "checks validity of a record" do thing = double(valid?:

    true) expect(thing.valid?).to be_false end # expected: false # got: true
  91. it "checks validity of a record" do thing = double("order",

    valid?: true) expect(thing).not_to be_valid end # expected `#<Double "order">.valid?` to return false, got true
  92. TESTS AS VERIFICATION

  93. TEST WHAT’S COMPLICATED, RISKY, OR CHEAP

  94. YOU CAN’T HAVE 100% COVERAGE

  95. DON’T TEST WHAT YOU DON’T OWN

  96. class JobLog < ActiveRecord::Base validate :status, presence: true end it

    { should validate_presence_of(:status) }
  97. class JobLog < ActiveRecord::Base validate :failure_message, presence: true, if: :failed?

    end it "requires failure_message for failures” do job = JobLog.new(status: :failed) expect(job).to validate_presence_of(:failure_message) job = JobLog.new(status: :completed) expect(job).not_to validate_presence_of(:failure_message) end
  98. DON’T BE AFRAID TO DELETE YOUR TESTS

  99. it "generates an array of strings” do expect(csv_data.length).to eq(shipments.length) expect(csv_data).to

    all(be_an(Array)) end
  100. BEWARE OF OVER-MOCKING

  101. def user_summary(user) "(#{user.id}) #{user.full_name}, #{user.role}" end it "returns a user

    summary for printing, bad" do user = double(id: 5, full_name: "Dave G", role: :admin) response = user_summary(user) expect(response).to eq("(5) Dave G, admin") end
  102. it "returns a user summary for printing, better" do user

    = instance_double(User, id: 5, full_name: "Dave G", role: :admin) response = user_summary(user) expect(response).to eq("(5) Dave G, admin") end
  103. it "returns a user summary for printing, better" do user

    = User.new(id: 5, full_name: "Dave G", role: :admin) expect(user_summary(user)).to eq("(5) Dave G, admin") end
  104. IN SUMMARY!

  105. IT GETS EASIER WITH PRACTICE

  106. 2. WRITE TESTS FOR HUMANS 3. VERIFICATION COMES LAST 1.

    USE TESTS TO DRIVE DESIGN
  107. THANKS!