Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tests should be fast: why and how?

110cd4dcaed019b3bd9809e541afdeb6?s=47 Masafumi Okura
September 04, 2019

Tests should be fast: why and how?

Speed is power, fast is good. But how about tests? Here I explain why speed of tests is important and how to improve it.

110cd4dcaed019b3bd9809e541afdeb6?s=128

Masafumi Okura

September 04, 2019
Tweet

Transcript

  1. Tests should be fast: why and how? Event: Tokyo Rubyist

    Meetup Date: 2019-09-04 Speaker: OKURA Masafumi
  2. Tests should be fast.

  3. Any opposition?✋

  4. Because: Fast is good.

  5. That’s all, thank you!

  6. None
  7. Introduction: Why should tests be faster?

  8. Short answer: It’s a low hanging fruit.

  9. Long answer: Making tests faster is easier than making production

    code faster.
  10. To make production code faster, we: • Measure performance with

    some specific tools such as “rack- mini-profiler” or “ruby-prof”. • Reduce N+1 queries with “bullet”. • Try to make Ruby code faster but it’s not a bottleneck. • Sometimes spend lots of time for small improvements.
  11. To make tests faster, we: • Profile tests with “—profile”

    option (if we use RSpec). • Look into slow tests and find the causes (described later). • Improve slow tests and win! • (If we struggle to find a bottleneck, “test-prof” gem helps us)!
  12. But you say: What’s bad with slow tests?

  13. Short answer: Fast is good.

  14. Long answer: Tests are critical in our feedback cycle, so

    faster tests mean faster feedback.
  15. Imagine: software development without tests.

  16. Software development without tests • We often miss errors/bugs before

    shipping. • We cannot do any refactoring because it causes bugs. • Fixing a bug causes more bugs including regressions. • These mean we develop software without any feedback.
  17. Tests are virtually the only way to get feedback often

    enough.
  18. Next: How can we make tests faster?

  19. Before getting started, one thing to say:

  20. None
  21. Capybara is really slow, so are feature tests and system

    tests.
  22. However: they are all good and cannot be removed.

  23. Removing capybara is not an option here.

  24. Strategy to make tests faster 1. Measure and profile using

    “profile” option and “test-prof” gem. 2. Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec). 3. Removing things your tests don’t need (optimizations and compilations).
  25. Strategy to make tests faster 1. Measure and profile using

    “profile” option and “test- prof” gem. 2. Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec). 3. Removing things your tests don’t need (optimizations and compilations).
  26. Measure and profile using “profile” option and “test-prof” gem

  27. “profile” option

  28. Profile option is: • Available natively for RSpec and as

    a plugin for Minitest • Used to show top 10 (default) slowest test cases. • The easiest way to know where’s slow in your test suite.
  29. “test-prof” gem

  30. “test-prof” is: • A collection of tools to profile your

    test suite. • Integrated with well-known general Ruby profiling tools such as “ruby-prof” and “stackprof”. • Used by many companies including GitLab and Dev.to.
  31. Example: Measuring time on before/let • Set RD_PROF=1 and run

    RSpec as usual • Get the output like below:
  32. Strategy to make tests faster 1. Measure and profile using

    “profile” option and “test-prof” gem. 2. Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec). 3. Removing things your tests don’t need (optimizations and compilations).
  33. Make slow tests faster taking care of setup (before hook)

    and database interactions (typically “let” in RSpec)
  34. Question: How many minutes of the test suite are spent

    on before and let?
  35. Answer: Most of the time. (In my case, 5:50 out

    of 7:35 are spent on before and let)
  36. Measured with “test-prof” gem.

  37. From here we go refactoring.

  38. Notes on reducing before • Too much DRYness is the

    enemy here. Duplicate code from before block to it block and cut redundant parts. • Don’t mimic login through web interaction. Set cookie directly. • Use tags effectively. Load something only with certain tags.
  39. Notes on reducing let • Maybe you should use instance

    variables instead of let.
  40. Notes on reducing let • Maybe you should use instance

    variables instead of let.
  41. Notes on reducing let • If we use more than

    5 lets in one example, maybe too many lets (one of the exceptions is search). • Prefer build over create, prefer build_stubbed over build. • Use let! only when required.
  42. Strategy to make tests faster 1. Measure and profile using

    “profile” option and “test-prof” gem. 2. Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec). 3. Removing things your tests don’t need (optimizations and compilations).
  43. Removing things your tests don’t need (optimizations and compilations)

  44. Real story:

  45. spec/support/stripe.rb require 'fake_stripe' RSpec.configure do |config| config.before(:each) do FakeStripe.stub_stripe end

    end
  46. How many minutes does it take? (Hint: minute, not second)

  47. 9 minutes.

  48. Image optimization:

  49. Several minutes (I forgot exact number)

  50. Compiling with Webpacker

  51. Compile once # config/webpacker.yml test: <<: *default compile: false #

    spec/rails_helper.rb RSpec.configure do |config| config.before(:suite) do Webpacker.compile end end
  52. A few minutes.

  53. Remove these for fast tests • Redundant `before each` call

    (or use some technique to load code only when required). • Redundant asset optimizations. • Webpacker compilation.
  54. Recap

  55. Strategy to make tests faster 1. Measure and profile using

    “profile” option and “test-prof” gem. 2. Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec). 3. Removing things your tests don’t need (optimizations and compilations).
  56. One more thing:

  57. Who am I?

  58. pp self • Name: OKURA Masafumi • Software developer for

    several years using Ruby and Rails • Vimmer (and a staff of VimConf 2019!) • Freelance now, seeking international jobs!
  59. References • https://relishapp.com/rspec/rspec-core/docs/configuration/ profile-examples • https://github.com/palkan/test-prof