Slide 1

Slide 1 text

21 december 2015 RSpec 3.3, 3.4
 new features @ngtk

Slide 2

Slide 2 text

New Features And Changes • Unique IDs (3.3) • only-failures option (3.3) • next-failure option (3.3) • Bisect (3.3, 3.4) • aggregate_failures API (3.3) • Improved Failure Output (3.3, 3.4) • with_captures in match matcher (3.4) • [Rails] have_enqueued_job matcher (3.4) 2

Slide 3

Slide 3 text

Unique IDs (3.3) • What: • ͋ΔexampleΛಛఆͰ͖ΔIDΛಋೖ • `$ rspec foo_spec.rb[1:3]` • Why: • Ҏલ͸ߦ൪߸ͰexampleΛಛఆ • exampleͷ௥ՃҎ֎ͷมߋ͕͋ͬͨ࣌ ʹҰҙʹͳΒͳ͍ • zshͩͱׅ֯ހ [] ͕࢖͑ͳ͍ͷͰγϯάϧ ΫΦʔςʔγϣϯͰғΉඞཁ༗Γ 3

Slide 4

Slide 4 text

foo_spec.rb 1 require 'spec_helper' 2 3 describe PracticeRspec::Foo do # [1] 4 describe '.yeah' do # [1:1] 5 it { expect(described_class.yeah).to eq true } # [1:1:1] 6 end 7 8 describe '.hello' do # [1:2] 9 it { expect(described_class.hello).to eq "world"} # [1:2:1] 10 end 11 end 4

Slide 5

Slide 5 text

shell (bash) $ bundle exec rspec spec/practice_rspec/foo_spec.rb[1:2:1] Run options: include {:ids=>{“./spec/practice_rspec/foo_spec.rb"=>["1:2:1"]}} PracticeRspec::Foo .hello should eq "world" Finished in 0.00177 seconds (files took 0.13821 seconds to load) 1 example, 0 failures 5

Slide 6

Slide 6 text

[Core] only-failures option (3.3) • Why: ࣦഊͨ͠exampleΛ੒ޭͤ͞Δϑϩʔͷվળ • What: • `$ rspec . —only-failures` • લճࣦഊͨ͠exampleͷΈΛ࣮ߦ͢ΔΦϓγϣϯ • How: • લճ࣮ࢪͨ͠ςετ݁ՌΛϑΝΠϧͰอ࣋ • ผ్ઃఆͷඞཁ༗Γ 6

Slide 7

Slide 7 text

spec_helper.rb 4 RSpec.configure do |c| 5 # ςετ݁ՌΛอ࣋͢ΔϑΝΠϧύε 6 c.example_status_persistence_file_path = "spec/examples.txt" 7 c.run_all_when_everything_filtered = true 8 end 7

Slide 8

Slide 8 text

examples.txt 1 example_id | status | run_time | 2 ---------------------------------------- | ------ | --------------- | 3 ./spec/practice_rspec/foo_spec.rb[1:1:1] | passed | 0.00065 seconds | 4 ./spec/practice_rspec/foo_spec.rb[1:2:1] | passed | 0.00019 seconds | 5 ./spec/practice_rspec_spec.rb[1:1] | passed | 0.00116 seconds | 6 ./spec/practice_rspec_spec.rb[1:2] | failed | 0.02164 seconds | 8

Slide 9

Slide 9 text

shell $ rspec spec --only-failures Run options: include {:last_run_status=>"failed"} PracticeRspec does something useful Finished in 0.00103 seconds (files took 0.1165 seconds to load) 1 example, 0 failures 9

Slide 10

Slide 10 text

examples.txt (all passed) 1 example_id | status | run_time | 2 ---------------------------------------- | ------ | --------------- | 3 ./spec/practice_rspec/foo_spec.rb[1:1:1] | passed | 0.00065 seconds | 4 ./spec/practice_rspec/foo_spec.rb[1:2:1] | passed | 0.00019 seconds | 5 ./spec/practice_rspec_spec.rb[1:1] | passed | 0.00116 seconds | 6 ./spec/practice_rspec_spec.rb[1:2] | passed | 0.00061 seconds | 10

Slide 11

Slide 11 text

[Core] next-failure option (3.3) • Why: • ࣦഊͨ͠exampleΛ੒ޭͤ͞Δϑϩʔͷվળ • What: • $ rspec —next-failure • only-failuresΦϓγϣϯͱfail-fastΦϓγϣϯͱorderΦϓγϣϯ(defined)
 Λಉ࣌ʹ෇͚ͨ݁Ռͱಉ͡ 11

Slide 12

Slide 12 text

[Core] next-failure option (3.3) • only-failuresΦϓγϣϯ: ࣦഊͨ͠ςετͷΈΛ࣮ߦ • fail-fastΦϓγϣϯ: ςετʹམͪͨ࣌఺ͰςετΛऴྃ • orderΦϓγϣϯ: exampleͷ࣮ߦॱ൪ɺorderedͷ৔߹͸ఆٛॱ(্͔Βॱʹ࣮ߦ) 12

Slide 13

Slide 13 text

[Core] next-failure option (3.3) • ͢Ͱʹམ͍ͪͯΔexampleͷΈΛ࣮ߦ͠ɺςετ͕མͪͨ࣌఺ͰςετΛऴྃ • ࣍ʹରԠ͢Δ΂͖exampleͷ݁ՌΛදࣔ • ࣮ߦ͞ΕΔexample͕ͳ͘ͳͬͨ࣌఺Ͱ͢΂ͯͷςετ͕௨͍ͬͯΔ͜ͱʹͳΔ 13

Slide 14

Slide 14 text

Bisect (3.3) • Why: • order randomΦϓγϣϯ΍seedΦϓγϣϯͳͲͰ࣮ߦॱংʹΑࣦͬͯഊ͢ΔexampleΛ ൃݟ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳ͕ͬͨɺͲͷexampleʹґଘ͍ͯ͠Δͷ͔͸Θ͔Βͳ͍ • What: • ࠶ݱ͢ΔύλʔϯΛ୳ࡧͯ͠ίϚϯυͱͯ͠ฦ͢ • $ rspec —bisect —seed 2256 14

Slide 15

Slide 15 text

Bisect (3.3) • Why: • order randomΦϓγϣϯ΍seedΦϓγϣϯͳͲͰ࣮ߦॱংʹΑࣦͬͯഊ͢ΔexampleΛ ൃݟ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳ͕ͬͨɺͲͷexampleʹґଘ͍ͯ͠Δͷ͔͸Θ͔Βͳ͍ • What: • ࠶ݱ͢ΔύλʔϯΛ୳ࡧͯ͠ίϚϯυͱͯ͠ฦ͢ • $ rspec —bisect —seed 2256 15

Slide 16

Slide 16 text

lib/calculator.rb 1 class Calculator 2 def self.add(x, y) 3 x + y 4 end 5 end 16

Slide 17

Slide 17 text

spec/calculator_1_spec.rb 1 require 'calculator' 2 3 RSpec.describe "Calculator" do 4 it 'adds numbers' do 5 expect(Calculator.add(1, 2)).to eq(3) 6 end 7 end 17

Slide 18

Slide 18 text

spec/calculator_[2..9]_spec.rb • All passed examples 18

Slide 19

Slide 19 text

spec/calculator_10_spec.rb 1 require 'calculator' 2 3 RSpec.describe "Monkey patched Calculator" do 4 it 'does screwy math' do 5 # monkey patching Calculator affects examples that are 6 # executed after this one! 7 def Calculator.add(x, y) 8 x - y 9 end 10 11 expect(Calculator.add(5, 10)).to eq(-5) 12 end 13 end 19

Slide 20

Slide 20 text

passed case (seed: 1412) $ rspec --order random Randomized with seed 1412 Calculator adds numbers # … examples … Monkey patched Calculator does screwy math # … examples … Finished in 0.00421 seconds (files took 0.12455 seconds to load) 6 examples, 0 failures Randomized with seed 1412 20

Slide 21

Slide 21 text

failed case (seed: 52712) $ rspec --order random Randomized with seed 52712 Monkey patched Calculator does screwy math Calculator adds numbers (FAILED - 1) Failures: 1) Calculator adds numbers Failure/Error: expect(Calculator.add(1, 2)).to eq(3) expected: 3 got: -1 (compared using ==) # ./spec/calculator_1_spec.rb:5:in `block (2 levels) in ' 6 examples, 1 failure Failed examples: rspec ./spec/calculator_1_spec.rb:4 # Calculator adds numbers Randomized with seed 52712 21

Slide 22

Slide 22 text

failed case (seed: 52712) $ rspec --bisect --seed 52712 Bisect started using options: "--seed 52712" Running suite to find failures... (0.25752 seconds) Starting bisect with 1 failing example and 5 non-failing examples. Checking that failure(s) are order-dependent... failure appears to be order-dependent Round 1: bisecting over non-failing examples 1-5 .. ignoring examples 4-5 (0.44252 seconds) Round 2: bisecting over non-failing examples 1-3 . ignoring examples 1-2 (0.22256 seconds) Bisect complete! Reduced necessary non-failing examples from 5 to 1 in 0.88248 seconds. The minimal reproduction command is: rspec './spec/calculator_10_spec.rb[1:1]' './spec/calculator_1_spec.rb[1:1]' --seed 52712 22

Slide 23

Slide 23 text

minimal reproduction rspec './spec/calculator_10_spec.rb[1:1]' './spec/calculator_1_spec.rb[1:1]' --seed 52712 Run options: include {:ids=>{"./spec/calculator_10_spec.rb"=>["1:1"], "./spec/calculator_1_spec.rb"=>["1:1"]}} Randomized with seed 52712 Monkey patched Calculator does screwy math Calculator adds numbers (FAILED - 1) Failures: 1) Calculator adds numbers Failure/Error: expect(Calculator.add(1, 2)).to eq(3) expected: 3 got: -1 (compared using ==) # ./spec/calculator_1_spec.rb:5:in `block (2 levels) in ' Finished in 0.02526 seconds (files took 0.20786 seconds to load) 2 examples, 1 failure Failed examples: rspec ./spec/calculator_1_spec.rb:4 # Calculator adds numbers Randomized with seed 52712 23

Slide 24

Slide 24 text

Bisect (3.3) 1. ͋ΔseedͰ(ͷΈ)མͪΔςετ͕ݟ͔ͭΔ 2. མͪͨςετͷseed͔ΒbisectΛ૸ΒͤΔ 3. bisectͷ݁Ռ͔ΒfailureͷݪҼʹͳ͍ͬͯΔexample͕Θ͔Δ 24

Slide 25

Slide 25 text

Bisect Algorithm Improvements (3.4) “Wow. Started running an RSpec bisect around 5pm yesterday. Sat down at my computer this morning and it was still going.” @geeksam “@myronmarston @urbanautomaton Thanks! Running against HEAD, bisect completed in ~20min. Minimal repro consists of 23 examples (1 failing).” @geeksam 25

Slide 26

Slide 26 text

Bisect Algorithm Improvements (3.4) • https://github.com/rspec/rspec-core/pull/1997 26

Slide 27

Slide 27 text

aggregate_failures API (3.3) • Why: ଎͞ͳͲͷཧ༝ʹexpectationΛ·ͱΊͨࡍʹ͢΂ͯͷexpectationͷ݁ՌΛ஌Γ͍ͨ • What: aggregate_failuresϒϩοΫͰ͘͘Δ͜ͱͰ͢΂ͯͷexpectationͷ݁ՌΛදࣔ 27

Slide 28

Slide 28 text

client_spec.rb (independent) 3 RSpec.describe Client do 4 let(:response) { Client.make_request } 5 6 it "returns a 200 response" do 7 expect(response.status).to eq(200) 8 end 9 10 it "indicates the response body is JSON" do 11 expect(response.headers).to include("Content-Type" => "application/json") 12 end 13 14 it "returns a success message" do 15 expect(response.body).to eq('{"message":"Success"}') 16 end 17 end 28

Slide 29

Slide 29 text

client_spec.rb (independent) • “one expectation per example”ͱ͍͏ݪଇ௨ΓʹͳΔ • example͝ͱʹbefore͕૸ΔͷͰ஗͍ 29

Slide 30

Slide 30 text

client_spec.rb (combined) 4 RSpec.describe Client do 5 it "returns a successful JSON response" do 6 response = Client.make_request 7 8 expect(response.status).to eq(200) 9 expect(response.headers).to include("Content-Type" => "applicati
 on/json") 10 expect(response.body).to eq('{"message":"Success"}') 11 end 12 end 30

Slide 31

Slide 31 text

client_spec.rb (combined) • ·ͱΊͯ1ճ͔͠before͕૸Βͳ͍෼͚ͩ଎͍ • ઌͷexpectation͕མͪͨ৔߹ʹޙͷexpectationͷ݁Ռ͕෼͔Βͳ͍ 31

Slide 32

Slide 32 text

client_spec.rb (aggregate_failures) 4 RSpec.describe Client do 5 it "returns a successful JSON response" do 6 response = Client.make_request 7 8 aggregate_failures "testing response" do 9 expect(response.status).to eq(200) 10 expect(response.headers).to include("Content-Type" => "application/json") 11 expect(response.body).to eq('{"message":"Success"}') 12 end 13 end 14 end 32

Slide 33

Slide 33 text

shell Client returns a successful JSON response (FAILED - 1) Failures: 1) Client returns a successful JSON response Got 3 failures from failure aggregation block "testing response". # ./spec/client_aggregate_failures_spec.rb:8:in `block (2 levels) in ' 1.1) Failure/Error: expect(response.status).to eq(200) expected: 200 got: 500 (compared using ==) # ./spec/client_aggregate_failures_spec.rb:9:in `block (3 levels) in ' 1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json") expected {} to include {"Content-Type" => "application/json"} Diff: @@ -1,2 +1 @@ -[{"Content-Type"=>"application/json"}] # ./spec/client_aggregate_failures_spec.rb:10:in `block (3 levels) in ' 1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}') expected: "{\"message\":\"Success\"}" got: "" (compared using ==) # ./spec/client_aggregate_failures_spec.rb:11:in `block (3 levels) in ' 33

Slide 34

Slide 34 text

Improved Failure Output (3.3, 3.4) • TimeͷsubsecondͷdiffΛදࣔ (3.3) • MockͰҾ਺ͷdiffΛදࣔ (3.3) • ෳ਺ߦͰͷexpectationʹରԠ (3.4) • coderayʹΑΔγϯλοΫεϋΠϥΠτʹରԠ (3.4) • productίʔυͰൃੜͨ͠ΤϥʔͷελοΫτϨʔεΛදࣔ (3.4) 34

Slide 35

Slide 35 text

with_captures in match matcher (3.4) 6 year_regex = /(\d{4})\-(\d{2})\-(\d{2})/ 7 expect(year_regex).to match("2015-12-25").with_captures("2015", "12", “25”) 35

Slide 36

Slide 36 text

with_captures in match matcher (3.4) 36 11 year_regex = /(?\d{4})\-(?\d{2})\-(?\d{2})/ 12 expect(year_regex).to match("2015-12-25").with_captures( 13 year: "2015", 14 month: "12", 15 day: "25" 16 )

Slide 37

Slide 37 text

[Rails] have_enqueued_job matcher (3.4) 4 expect { 5 HeavyLiftingJob.perform_later 6 }.to have_enqueued_job 37

Slide 38

Slide 38 text

[Rails] have_enqueued_job matcher (3.4) 8 expect { 9 HelloJob.perform_later 10 HeavyLiftingJob.perform_later 11 }.to have_enqueued_job(HelloJob).exactly(:once) 38

Slide 39

Slide 39 text

[Rails] have_enqueued_job matcher (3.4) 13 expect { 14 HelloJob.perform_later 15 HelloJob.perform_later 16 HelloJob.perform_later 17 }.to have_enqueued_job(HelloJob).at_least(2).times 39

Slide 40

Slide 40 text

[Rails] have_enqueued_job matcher (3.4) 19 expect { 20 HelloJob.perform_later 21 }.to have_enqueued_job(HelloJob).at_most(:twice) 40

Slide 41

Slide 41 text

[Rails] have_enqueued_job matcher (3.4) 23 expect { 24 HelloJob.perform_later 25 HeavyLiftingJob.perform_later 26 }.to have_enqueued_job(HelloJob).and have_enqueued_job(HeavyLiftingJob) 41

Slide 42

Slide 42 text

[Rails] have_enqueued_job matcher (3.4) 28 expect { 29 HelloJob.set(wait_until: Date.tomorrow.noon, queue: “low”).perform_l ater(42) 30 }.to have_enqueued_job.with(42).on_queue("low").at(Date.tomorrow.noon) 42

Slide 43

Slide 43 text

New Features And Changes • Unique IDs (3.3) • only-failures option (3.3) • next-failure option (3.3) • Bisect (3.3, 3.4) • aggregate_failures API (3.3) • Improved Failure Output (3.3, 3.4) • with_captures in match matcher (3.4) • [Rails] have_enqueued_job matcher (3.4) 43

Slide 44

Slide 44 text

Happy Testing! 44