Share
Embed
Identifying Flaky Tests
Ways to identify and avoid writing flaky tests.
Stephanie Viccari
December 21, 2020
Transcript
13 slides
-
W at s F ak T st? A s a
s e f w n d c . 2 -
P ob em w th a y es s 4
T r u 4 T w u t 4 T u n u t 3 -
L ak S at - G ob l ar ab
es $global_variable = 1 RSpec.describe "mutating a global variable" do it "reads from a global variable" do expect($global_variable).to eq(1) end it "updates the global variable" do $global_variable = 2 expect($global_variable).to eq(2) end end # These tests fail when the second test runs first. # Global variables are best avoided because they break encapsulation. 6 -
L ak S at - C ch d al es
module Clearance class Configuration def initialize @routes = true end def routes_enabled? @routes end end end RSpec.describe Clearance::Configuration do it "is true by default" do expect(Clearance.configuration.routes_enabled?).to be true end it "is false when routes are set to false" do # this test setup is setting state on `Clearance.config` and # unless properly reset after each test. Running these tests # out of order will cause the 'true by default' test to fail Clearance.config { |config| config.routes = false } expect(Clearance.configuration.routes_enabled?).to be false end end 7 -
O de d pe de cy class Book < ApplicationRecord
def self.published where.not(publication_date: nil) end end Rspec.describe Book do describe ".published" do it "returns only published books" do book1 = create(:book, :published) book2 = create(:book, :published) book3 = create(:book, :draft) expect(Book.published).to eq [book1, book2] end end end 8 -
O de d pe de cy - R fa to
class Book < ApplicationRecord def self.published where.not(publication_date: nil) end end Rspec.describe Book do describe ".published" do it "returns only published books" do book1 = create(:book, :published) book2 = create(:book, :published) book3 = create(:book, :draft) expect(Book.published).to match_array [book1, book2] expect(Book.published).to contain_exactly(book1, book2) end end end 9 -
T me Rspec.describe Venue do describe ".upcoming_concerts" do it "returns
concerts today or in the future" do today = DateTime.current yesterday_concert = create(:concert, start_time: today - 1.day) todays_concert = create(:concert, start_time: today) tomorrows_concert = create(:concert, start_time: today + 1.day) upcoming_concert_names = Venue.upcoming_concerts.map(&:name) expect(upcoming_concerts).to match_array( [todays_concert.name, tomorrows_concert.name] ) end end end 1 -
T me - F ee e im w th im
Co Rspec.describe Venue do describe ".upcoming_concerts" do it "returns concerts today or in the future" do Timecop.freeze do today = DateTime.current yesterday_concert = create(:concert, start_time: today - 1.day) todays_concert = create(:concert, start_time: today) todays_concert = create(:concert, start_time: today + 1.day) upcoming_concert_names = venue.upcoming_concerts.map(&:name) expect(upcoming_concerts).to match_array( [todays_concert.name, tomorrows_concert.name] ) end end end end 1 -
T me - P ec si n # Ruby Time
objects maintain greater precision than the database. # use the `be_within` matcher to compare timestamps expect(database_record.updated_at).to be_within(1.second).of(ruby_object.timestamp) 1 -
F rt er ea in T - A e a
p s m t : h ://g .c /t e e /t I 's A m (Z ): h ://t b .c /b /i -a -t -z 1