RSpec 3.3, 3.4 New Features

New features of RSpec 3.3 and 3.4

Transcript

1. 21 december 2015 RSpec 3.3, 3.4
 new features @ngtk
2. New Features And Changes • Unique IDs (3.3) • only-failures option (3.3) • next-failure option (3.3) • Bisect (3.3, 3.4) • aggregate_failures API (3.3) • Improved Failure Output (3.3, 3.4) • with_captures in match matcher (3.4) • [Rails] have_enqueued_job matcher (3.4) 2
3. Unique IDs (3.3) • What: • ͋ΔexampleΛಛఆͰ͖ΔIDΛಋೖ • `$ rspec foo_spec.rb[1:3]` • Why: • Ҏલ͸ߦ൪߸ͰexampleΛಛఆ • exampleͷ௥ՃҎ֎ͷมߋ͕͋ͬͨ࣌ ʹҰҙʹͳΒͳ͍ • zshͩͱׅ֯ހ [] ͕࢖͑ͳ͍ͷͰγϯάϧ ΫΦʔςʔγϣϯͰғΉඞཁ༗Γ 3
4. foo_spec.rb 1 require 'spec_helper' 2 3 describe PracticeRspec::Foo do # [1] 4 describe '.yeah' do # [1:1] 5 it { expect(described_class.yeah).to eq true } # [1:1:1] 6 end 7 8 describe '.hello' do # [1:2] 9 it { expect(described_class.hello).to eq "world"} # [1:2:1] 10 end 11 end 4
5. shell (bash) $ bundle exec rspec spec/practice_rspec/foo_spec.rb[1:2:1] Run options: include {:ids=>{“./spec/practice_rspec/foo_spec.rb"=>["1:2:1"]}} PracticeRspec::Foo .hello should eq "world" Finished in 0.00177 seconds (files took 0.13821 seconds to load) 1 example, 0 failures 5
6. [Core] only-failures option (3.3) • Why: ࣦഊͨ͠exampleΛ੒ޭͤ͞Δϑϩʔͷվળ • What: • `$ rspec . —only-failures` • લճࣦഊͨ͠exampleͷΈΛ࣮ߦ͢ΔΦϓγϣϯ • How: • લճ࣮ࢪͨ͠ςετ݁ՌΛϑΝΠϧͰอ࣋ • ผ్ઃఆͷඞཁ༗Γ 6
7. spec_helper.rb 4 RSpec.configure do |c| 5 # ςετ݁ՌΛอ࣋͢ΔϑΝΠϧύε 6 c.example_status_persistence_file_path = "spec/examples.txt" 7 c.run_all_when_everything_filtered = true 8 end 7
8. examples.txt 1 example_id | status | run_time | 2 ---------------------------------------- | ------ | --------------- | 3 ./spec/practice_rspec/foo_spec.rb[1:1:1] | passed | 0.00065 seconds | 4 ./spec/practice_rspec/foo_spec.rb[1:2:1] | passed | 0.00019 seconds | 5 ./spec/practice_rspec_spec.rb[1:1] | passed | 0.00116 seconds | 6 ./spec/practice_rspec_spec.rb[1:2] | failed | 0.02164 seconds | 8
9. shell $ rspec spec --only-failures Run options: include {:last_run_status=>"failed"} PracticeRspec does something useful Finished in 0.00103 seconds (files took 0.1165 seconds to load) 1 example, 0 failures 9
10. examples.txt (all passed) 1 example_id | status | run_time | 2 ---------------------------------------- | ------ | --------------- | 3 ./spec/practice_rspec/foo_spec.rb[1:1:1] | passed | 0.00065 seconds | 4 ./spec/practice_rspec/foo_spec.rb[1:2:1] | passed | 0.00019 seconds | 5 ./spec/practice_rspec_spec.rb[1:1] | passed | 0.00116 seconds | 6 ./spec/practice_rspec_spec.rb[1:2] | passed | 0.00061 seconds | 10
11. [Core] next-failure option (3.3) • Why: • ࣦഊͨ͠exampleΛ੒ޭͤ͞Δϑϩʔͷվળ • What: • $ rspec —next-failure • only-failuresΦϓγϣϯͱfail-fastΦϓγϣϯͱorderΦϓγϣϯ(defined)
 Λಉ࣌ʹ෇͚ͨ݁Ռͱಉ͡ 11
12. [Core] next-failure option (3.3) • only-failuresΦϓγϣϯ: ࣦഊͨ͠ςετͷΈΛ࣮ߦ • fail-fastΦϓγϣϯ: ςετʹམͪͨ࣌఺ͰςετΛऴྃ • orderΦϓγϣϯ: exampleͷ࣮ߦॱ൪ɺorderedͷ৔߹͸ఆٛॱ(্͔Βॱʹ࣮ߦ) 12
13. [Core] next-failure option (3.3) • ͢Ͱʹམ͍ͪͯΔexampleͷΈΛ࣮ߦ͠ɺςετ͕མͪͨ࣌఺ͰςετΛऴྃ • ࣍ʹରԠ͢Δ΂͖exampleͷ݁ՌΛදࣔ • ࣮ߦ͞ΕΔexample͕ͳ͘ͳͬͨ࣌఺Ͱ͢΂ͯͷςετ͕௨͍ͬͯΔ͜ͱʹͳΔ 13
14. Bisect (3.3) • Why: • order randomΦϓγϣϯ΍seedΦϓγϣϯͳͲͰ࣮ߦॱংʹΑࣦͬͯഊ͢ΔexampleΛ ൃݟ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳ͕ͬͨɺͲͷexampleʹґଘ͍ͯ͠Δͷ͔͸Θ͔Βͳ͍ • What: • ࠶ݱ͢ΔύλʔϯΛ୳ࡧͯ͠ίϚϯυͱͯ͠ฦ͢ • $ rspec —bisect —seed 2256 14
15. Bisect (3.3) • Why: • order randomΦϓγϣϯ΍seedΦϓγϣϯͳͲͰ࣮ߦॱংʹΑࣦͬͯഊ͢ΔexampleΛ ൃݟ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳ͕ͬͨɺͲͷexampleʹґଘ͍ͯ͠Δͷ͔͸Θ͔Βͳ͍ • What: • ࠶ݱ͢ΔύλʔϯΛ୳ࡧͯ͠ίϚϯυͱͯ͠ฦ͢ • $ rspec —bisect —seed 2256 15
16. lib/calculator.rb 1 class Calculator 2 def self.add(x, y) 3 x + y 4 end 5 end 16
17. spec/calculator_1_spec.rb 1 require 'calculator' 2 3 RSpec.describe "Calculator" do 4 it 'adds numbers' do 5 expect(Calculator.add(1, 2)).to eq(3) 6 end 7 end 17
18. spec/calculator_[2..9]_spec.rb • All passed examples 18
19. spec/calculator_10_spec.rb 1 require 'calculator' 2 3 RSpec.describe "Monkey patched Calculator" do 4 it 'does screwy math' do 5 # monkey patching Calculator affects examples that are 6 # executed after this one! 7 def Calculator.add(x, y) 8 x - y 9 end 10 11 expect(Calculator.add(5, 10)).to eq(-5) 12 end 13 end 19
20. passed case (seed: 1412) $ rspec --order random Randomized with seed 1412 Calculator adds numbers # … examples … Monkey patched Calculator does screwy math # … examples … Finished in 0.00421 seconds (files took 0.12455 seconds to load) 6 examples, 0 failures Randomized with seed 1412 20
21. failed case (seed: 52712) $ rspec --order random Randomized with seed 52712 Monkey patched Calculator does screwy math Calculator adds numbers (FAILED - 1) Failures: 1) Calculator adds numbers Failure/Error: expect(Calculator.add(1, 2)).to eq(3) expected: 3 got: -1 (compared using ==) # ./spec/calculator_1_spec.rb:5:in `block (2 levels) in <top (required)>' 6 examples, 1 failure Failed examples: rspec ./spec/calculator_1_spec.rb:4 # Calculator adds numbers Randomized with seed 52712 21
22. failed case (seed: 52712) $ rspec --bisect --seed 52712 Bisect started using options: "--seed 52712" Running suite to find failures... (0.25752 seconds) Starting bisect with 1 failing example and 5 non-failing examples. Checking that failure(s) are order-dependent... failure appears to be order-dependent Round 1: bisecting over non-failing examples 1-5 .. ignoring examples 4-5 (0.44252 seconds) Round 2: bisecting over non-failing examples 1-3 . ignoring examples 1-2 (0.22256 seconds) Bisect complete! Reduced necessary non-failing examples from 5 to 1 in 0.88248 seconds. The minimal reproduction command is: rspec './spec/calculator_10_spec.rb[1:1]' './spec/calculator_1_spec.rb[1:1]' --seed 52712 22
23. minimal reproduction rspec './spec/calculator_10_spec.rb[1:1]' './spec/calculator_1_spec.rb[1:1]' --seed 52712 Run options: include {:ids=>{"./spec/calculator_10_spec.rb"=>["1:1"], "./spec/calculator_1_spec.rb"=>["1:1"]}} Randomized with seed 52712 Monkey patched Calculator does screwy math Calculator adds numbers (FAILED - 1) Failures: 1) Calculator adds numbers Failure/Error: expect(Calculator.add(1, 2)).to eq(3) expected: 3 got: -1 (compared using ==) # ./spec/calculator_1_spec.rb:5:in `block (2 levels) in <top (required)>' Finished in 0.02526 seconds (files took 0.20786 seconds to load) 2 examples, 1 failure Failed examples: rspec ./spec/calculator_1_spec.rb:4 # Calculator adds numbers Randomized with seed 52712 23
24. Bisect (3.3) 1. ͋ΔseedͰ(ͷΈ)མͪΔςετ͕ݟ͔ͭΔ 2. མͪͨςετͷseed͔ΒbisectΛ૸ΒͤΔ 3. bisectͷ݁Ռ͔ΒfailureͷݪҼʹͳ͍ͬͯΔexample͕Θ͔Δ 24
25. Bisect Algorithm Improvements (3.4) “Wow. Started running an RSpec bisect around 5pm yesterday. Sat down at my computer this morning and it was still going.” @geeksam “@myronmarston @urbanautomaton Thanks! Running against HEAD, bisect completed in ~20min. Minimal repro consists of 23 examples (1 failing).” @geeksam 25
26. Bisect Algorithm Improvements (3.4) • https://github.com/rspec/rspec-core/pull/1997 26
27. aggregate_failures API (3.3) • Why: ଎͞ͳͲͷཧ༝ʹexpectationΛ·ͱΊͨࡍʹ͢΂ͯͷexpectationͷ݁ՌΛ஌Γ͍ͨ • What: aggregate_failuresϒϩοΫͰ͘͘Δ͜ͱͰ͢΂ͯͷexpectationͷ݁ՌΛදࣔ 27
28. client_spec.rb (independent) 3 RSpec.describe Client do 4 let(:response) { Client.make_request } 5 6 it "returns a 200 response" do 7 expect(response.status).to eq(200) 8 end 9 10 it "indicates the response body is JSON" do 11 expect(response.headers).to include("Content-Type" => "application/json") 12 end 13 14 it "returns a success message" do 15 expect(response.body).to eq('{"message":"Success"}') 16 end 17 end 28
29. client_spec.rb (independent) • “one expectation per example”ͱ͍͏ݪଇ௨ΓʹͳΔ • example͝ͱʹbefore͕૸ΔͷͰ஗͍ 29
30. client_spec.rb (combined) 4 RSpec.describe Client do 5 it "returns a successful JSON response" do 6 response = Client.make_request 7 8 expect(response.status).to eq(200) 9 expect(response.headers).to include("Content-Type" => "applicati
 on/json") 10 expect(response.body).to eq('{"message":"Success"}') 11 end 12 end 30
31. client_spec.rb (combined) • ·ͱΊͯ1ճ͔͠before͕૸Βͳ͍෼͚ͩ଎͍ • ઌͷexpectation͕མͪͨ৔߹ʹޙͷexpectationͷ݁Ռ͕෼͔Βͳ͍ 31
32. client_spec.rb (aggregate_failures) 4 RSpec.describe Client do 5 it "returns a successful JSON response" do 6 response = Client.make_request 7 8 aggregate_failures "testing response" do 9 expect(response.status).to eq(200) 10 expect(response.headers).to include("Content-Type" => "application/json") 11 expect(response.body).to eq('{"message":"Success"}') 12 end 13 end 14 end 32
33. shell Client returns a successful JSON response (FAILED - 1) Failures: 1) Client returns a successful JSON response Got 3 failures from failure aggregation block "testing response". # ./spec/client_aggregate_failures_spec.rb:8:in `block (2 levels) in <top (required)>' 1.1) Failure/Error: expect(response.status).to eq(200) expected: 200 got: 500 (compared using ==) # ./spec/client_aggregate_failures_spec.rb:9:in `block (3 levels) in <top (required)>' 1.2) Failure/Error: expect(response.headers).to include("Content-Type" => "application/json") expected {} to include {"Content-Type" => "application/json"} Diff: @@ -1,2 +1 @@ -[{"Content-Type"=>"application/json"}] # ./spec/client_aggregate_failures_spec.rb:10:in `block (3 levels) in <top (required)>' 1.3) Failure/Error: expect(response.body).to eq('{"message":"Success"}') expected: "{\"message\":\"Success\"}" got: "" (compared using ==) # ./spec/client_aggregate_failures_spec.rb:11:in `block (3 levels) in <top (required)>' 33
34. Improved Failure Output (3.3, 3.4) • TimeͷsubsecondͷdiffΛදࣔ (3.3) • MockͰҾ਺ͷdiffΛදࣔ (3.3) • ෳ਺ߦͰͷexpectationʹରԠ (3.4) • coderayʹΑΔγϯλοΫεϋΠϥΠτʹରԠ (3.4) • productίʔυͰൃੜͨ͠ΤϥʔͷελοΫτϨʔεΛදࣔ (3.4) 34
35. with_captures in match matcher (3.4) 6 year_regex = /(\d{4})\-(\d{2})\-(\d{2})/ 7 expect(year_regex).to match("2015-12-25").with_captures("2015", "12", “25”) 35
36. with_captures in match matcher (3.4) 36 11 year_regex = /(?<year>\d{4})\-(?<month>\d{2})\-(?<day>\d{2})/ 12 expect(year_regex).to match("2015-12-25").with_captures( 13 year: "2015", 14 month: "12", 15 day: "25" 16 )
37. [Rails] have_enqueued_job matcher (3.4) 4 expect { 5 HeavyLiftingJob.perform_later 6 }.to have_enqueued_job 37
38. [Rails] have_enqueued_job matcher (3.4) 8 expect { 9 HelloJob.perform_later 10 HeavyLiftingJob.perform_later 11 }.to have_enqueued_job(HelloJob).exactly(:once) 38
39. [Rails] have_enqueued_job matcher (3.4) 13 expect { 14 HelloJob.perform_later 15 HelloJob.perform_later 16 HelloJob.perform_later 17 }.to have_enqueued_job(HelloJob).at_least(2).times 39
40. [Rails] have_enqueued_job matcher (3.4) 19 expect { 20 HelloJob.perform_later 21 }.to have_enqueued_job(HelloJob).at_most(:twice) 40
41. [Rails] have_enqueued_job matcher (3.4) 23 expect { 24 HelloJob.perform_later 25 HeavyLiftingJob.perform_later 26 }.to have_enqueued_job(HelloJob).and have_enqueued_job(HeavyLiftingJob) 41
42. [Rails] have_enqueued_job matcher (3.4) 28 expect { 29 HelloJob.set(wait_until: Date.tomorrow.noon, queue: “low”).perform_l ater(42) 30 }.to have_enqueued_job.with(42).on_queue("low").at(Date.tomorrow.noon) 42
43. New Features And Changes • Unique IDs (3.3) • only-failures option (3.3) • next-failure option (3.3) • Bisect (3.3, 3.4) • aggregate_failures API (3.3) • Improved Failure Output (3.3, 3.4) • with_captures in match matcher (3.4) • [Rails] have_enqueued_job matcher (3.4) 43
44. Happy Testing! 44