Best tools for automated testing in Ruby

Ruby is a language known for its love of automated testing. It has an awesome set of tools to fill all your needs. Which ones are the best or to avoid? What are some good and bad practices in general in testing? I’ll tell you which tools I’m using now, after years of working on multiple apps with different tools. After this talk, you’ll have a great toolset to maintain a useful test suite over time.

Talk given at ConFoo 2016, in Montreal.

https://confoo.ca/en/2016/session/best-tools-for-automated-testing-in-ruby

Transcript

1. metrics.watch | freeGoogleAnalyticsCourse.com Best tools for automated testing in Ruby Jean-Philippe Boily @jipiboily | jipiboily.com Founder of Metrics Watch http://metrics.watch
2. metrics.watch | freeGoogleAnalyticsCourse.com whoami Jean-Philippe “JP” Boily Founder of Metrics Watch SaaS consultant Software Engineer with experience working remotely for US- based SaaS startups Last one was Rainforest QA (YC backed)
3. metrics.watch | freeGoogleAnalyticsCourse.com Metrics Watch Alerts for Google Analytics in near real-time. Support for custom metrics, filters, alert templates, etc. Check us out! http://metrics.watch PS: I have stickers!
4. metrics.watch | freeGoogleAnalyticsCourse.com Let’s talk about testing!
5. RSpec or MiniTest?
6. metrics.watch | freeGoogleAnalyticsCourse.com RSpec, or MiniTest? MiniTest comes with Ruby RSpec is a gem, NOT built-in in Ruby. It’s a DSL.
7. metrics.watch | freeGoogleAnalyticsCourse.com MiniTest Plain Ruby require "minitest/autorun" class TestMeme < Minitest::Test def setup @meme = Meme.new end def test_that_kitty_can_eat assert_equal "OHAI!", @meme.i_can_has_cheezburger? end end
8. metrics.watch | freeGoogleAnalyticsCourse.com MiniTest Spec require "minitest/autorun" describe Meme do before do @meme = Meme.new end describe "when asked about cheeseburgers" do it "must respond positively" do @meme.i_can_has_cheezburger?.must_equal "OHAI!" end end end
9. metrics.watch | freeGoogleAnalyticsCourse.com RSpec DSL that reads like English. Ships with color output & easy to re-run just the failures.
10. metrics.watch | freeGoogleAnalyticsCourse.com Color &failures
11. metrics.watch | freeGoogleAnalyticsCourse.com It just works Reads really well Works really well Has passed the test of time
12. RSpec!
13. metrics.watch | freeGoogleAnalyticsCourse.com The basics require ‘rails_helper' describe Watcher, :type => :model do describe '#alert' do it 'does something' do expect(true).to eq true end it { expect(true).to eq true } end end
14. metrics.watch | freeGoogleAnalyticsCourse.com describe Describe wraps tests, and other describe blocks Can have nested describe
15. metrics.watch | freeGoogleAnalyticsCourse.com it Definition of an example
16. metrics.watch | freeGoogleAnalyticsCourse.com expect What other frameworks call assertions
17. metrics.watch | freeGoogleAnalyticsCourse.com The basics: review require ‘rails_helper' describe Watcher, :type => :model do describe '#alert' do it 'does something' do expect(true).to eq true end it { expect(true).to eq true } end end
18. metrics.watch | freeGoogleAnalyticsCourse.com context Technically same as describe. Different semantic: they define… (drum roll) context! (SURPRISED,HUH?) describe Watcher, :type => :model do describe '#alert' do subject { Watcher.new } context 'when invalid' do it { is_expected.not_to be_valid } end end end
19. metrics.watch | freeGoogleAnalyticsCourse.com before & after setup & tear down describe '#alert' do before do @watcher = Watcher.new(threshold: 11) end after { reset_database } it { expect(@watcher.threshold).to eq 11 } end
20. metrics.watch | freeGoogleAnalyticsCourse.com let & let! memorized helper method `let` is lazy- evaluated, `let!` forces the evaluation describe '#alert' do let(:watcher) { Watcher.new(threshold: 11)} it { expect(watcher.threshold).to eq 11 } end
21. metrics.watch | freeGoogleAnalyticsCourse.com subject & is_expected DRY test with same subject, use “subject” describe Watcher, :type => :model do describe '#alert' do subject { Watcher.new(name:’:wave:') } it { is_expected.to be_new } end end describe Watcher, :type => :model do describe '#alert' do it { expect(Watcher.new(name:’:wave:')).to be_new } end end
22. metrics.watch | freeGoogleAnalyticsCourse.com More expectations .not_to eq .to eq .to be .to be_valid (or any method finishing with a ?) .to change{Watcher.count}.by(1) .to raise_error .to start_with
23. metrics.watch | freeGoogleAnalyticsCourse.com rspec-mock Stubs: allow(watcher).to receive(:alert) Message expectations expect(watcher).to receive(:alert) and much more…
24. metrics.watch | freeGoogleAnalyticsCourse.com Be careful with mocking & stubbing!
25. metrics.watch | freeGoogleAnalyticsCourse.com
26. metrics.watch | freeGoogleAnalyticsCourse.com (spec || rails)_helpers Configure & require stuff. Configure RSpec and other tools.
27. metrics.watch | freeGoogleAnalyticsCourse.com Best practices betterspecs.org
28. Great tools
29. metrics.watch | freeGoogleAnalyticsCourse.com factory_girl Help create test data. FactoryGirl.define do factory :watcher do metric_name "ga:sessions" threshold 100 end end RSpec.describe Watcher, :type => :model do describe '#alert' do let(:watcher) { create(:watcher) } it { expect(watcher.threshold).to eq 11 } end end https://github.com/thoughtbot/factory_girl
30. metrics.watch | freeGoogleAnalyticsCourse.com database_cleaner Keeps your database clean for new test data. RSpec.configure do |config| config.use_transactional_fixtures = false config.before(:suite) do DatabaseCleaner.clean_with(:transaction) end config.before(:each) do DatabaseCleaner.start end config.after(:each) do DatabaseCleaner.clean end end https://github.com/DatabaseCleaner/database_cleaner
31. metrics.watch | freeGoogleAnalyticsCourse.com guard & friends guard guard-rspec terminal-notifier-guard
32. metrics.watch | freeGoogleAnalyticsCourse.com guard Watch your file and do stuff when they change. Highly configurable. https://github.com/guard/guard
33. metrics.watch | freeGoogleAnalyticsCourse.com guard-rspec Run your Ruby specs when saving. Just the one that is related. Can keep running the failing ones. https://github.com/guard/guard-rspec
34. metrics.watch | freeGoogleAnalyticsCourse.com terminal-notifier-guard
35. metrics.watch | freeGoogleAnalyticsCourse.com spring Rails application preloader. It just works™.
36. metrics.watch | freeGoogleAnalyticsCourse.com VCR
37. metrics.watch | freeGoogleAnalyticsCourse.com I’m half kidding…
38. metrics.watch | freeGoogleAnalyticsCourse.com VCR “Record your test suite's HTTP interactions and replay them during future test runs for fast, deterministic, accurate tests.” Records them in cassettes. I’m NOT kidding.
39. metrics.watch | freeGoogleAnalyticsCourse.com VCR Great stuff! Much better than manually mocking. Now RSpec AND MiniTest! https://github.com/vcr/vcr
40. metrics.watch | freeGoogleAnalyticsCourse.com Timecop A "time travel” & "time freezing” gem describe "some set of tests to mock" do before do Timecop.freeze(Time.local(1990)) end after do Timecop.return end it "should do blah blah blah" do end end
41. Concepts
42. metrics.watch | freeGoogleAnalyticsCourse.com Test Driven Development (TDD) Red, green, refactor!
43. metrics.watch | freeGoogleAnalyticsCourse.com TDD: red class Watcher end describe Watcher do it { expect(Watcher.alert?).to eq(true) } end
44. metrics.watch | freeGoogleAnalyticsCourse.com TDD: green class Watcher def alert? true end end describe Watcher do it { expect(Watcher.alert?).to eq(true) } end
45. metrics.watch | freeGoogleAnalyticsCourse.com TDD: refactor class Watcher def alert? value > threshold end end describe Watcher do it { expect(Watcher.alert?).to eq(true) } end
46. metrics.watch | freeGoogleAnalyticsCourse.com TDD When should I TDD, or not?
47. metrics.watch | freeGoogleAnalyticsCourse.com Coverage Percentage of code…that runs during tests. Not that is properly tested! Beware of coverage! I find it useless.
48. metrics.watch | freeGoogleAnalyticsCourse.com CI, CD & CQA Continuous Integration Continuous Deployment Continuous QA (!!!)
49. metrics.watch | freeGoogleAnalyticsCourse.com CI, CD & CQA CI & CD: Circle CI, Codeship, Drone, Jenkins. Continuous QA: Rainforest QA
50. Conclusion?
51. metrics.watch | freeGoogleAnalyticsCourse.com Key takeaways •Use RSpec and betterspecs.org •Know your options & tools •TDD, CI, CD, CQA = win. Get as close. •Improve your workflow •Learn to trust your tests
52. GET AWESOME GIFTS ConFooLove.com • Slides for my two talks at ConFoo • testing toolbox cheatsheet • early invite to join my free email course about distributed teams, for pre-launch freeGAcourse.com • free course “How to get the most out of Google Analytics” • longer trial for Metrics Watch (1 month instead of 14 days)