Tests should be fast: why and how?

Slide 1

Slide 1 text

Tests should be fast: why and how? Event: Tokyo Rubyist Meetup Date: 2019-09-04 Speaker: OKURA Masafumi

Slide 2

Slide 2 text

Tests should be fast.

Slide 3

Slide 3 text

Any opposition?✋

Slide 4

Slide 4 text

Because: Fast is good.

Slide 5

Slide 5 text

That’s all, thank you!

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Introduction: Why should tests be faster?

Slide 8

Slide 8 text

Short answer: It’s a low hanging fruit.

Slide 9

Slide 9 text

Long answer: Making tests faster is easier than making production code faster.

Slide 10

Slide 10 text

To make production code faster, we: • Measure performance with some speciﬁc tools such as “rack- mini-proﬁler” or “ruby-prof”. • Reduce N+1 queries with “bullet”. • Try to make Ruby code faster but it’s not a bottleneck. • Sometimes spend lots of time for small improvements.

Slide 11

Slide 11 text

To make tests faster, we: • Profile tests with “—profile” option (if we use RSpec). • Look into slow tests and find the causes (described later). • Improve slow tests and win! • (If we struggle to find a bottleneck, “test-prof” gem helps us)!

Slide 12

Slide 12 text

But you say: What’s bad with slow tests?

Slide 13

Slide 13 text

Short answer: Fast is good.

Slide 14

Slide 14 text

Long answer: Tests are critical in our feedback cycle, so faster tests mean faster feedback.

Slide 15

Slide 15 text

Imagine: software development without tests.

Slide 16

Slide 16 text

Software development without tests • We often miss errors/bugs before shipping. • We cannot do any refactoring because it causes bugs. • Fixing a bug causes more bugs including regressions. • These mean we develop software without any feedback.

Slide 17

Slide 17 text

Tests are virtually the only way to get feedback often enough.

Slide 18

Slide 18 text

Next: How can we make tests faster?

Slide 19

Slide 19 text

Before getting started, one thing to say:

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

Capybara is really slow, so are feature tests and system tests.

Slide 22

Slide 22 text

However: they are all good and cannot be removed.

Slide 23

Slide 23 text

Removing capybara is not an option here.

Slide 24

Slide 24 text

Strategy to make tests faster 1. Measure and proﬁle using “proﬁle” option and “test-prof” gem. 2. Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec). 3. Removing things your tests don’t need (optimizations and compilations).

Slide 25

Slide 25 text

Strategy to make tests faster 1. Measure and proﬁle using “proﬁle” option and “test- prof” gem. 2. Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec). 3. Removing things your tests don’t need (optimizations and compilations).

Slide 26

Slide 26 text

Measure and proﬁle using “proﬁle” option and “test-prof” gem

Slide 27

Slide 27 text

“proﬁle” option

Slide 28

Slide 28 text

Proﬁle option is: • Available natively for RSpec and as a plugin for Minitest • Used to show top 10 (default) slowest test cases. • The easiest way to know where’s slow in your test suite.

Slide 29

Slide 29 text

“test-prof” gem

Slide 30

Slide 30 text

“test-prof” is: • A collection of tools to proﬁle your test suite. • Integrated with well-known general Ruby proﬁling tools such as “ruby-prof” and “stackprof”. • Used by many companies including GitLab and Dev.to.

Slide 31

Slide 31 text

Example: Measuring time on before/let • Set RD_PROF=1 and run RSpec as usual • Get the output like below:

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Make slow tests faster taking care of setup (before hook) and database interactions (typically “let” in RSpec)

Slide 34

Slide 34 text

Question: How many minutes of the test suite are spent on before and let?

Slide 35

Slide 35 text

Answer: Most of the time. (In my case, 5:50 out of 7:35 are spent on before and let)

Slide 36

Slide 36 text

Measured with “test-prof” gem.

Slide 37

Slide 37 text

From here we go refactoring.

Slide 38

Slide 38 text

Notes on reducing before • Too much DRYness is the enemy here. Duplicate code from before block to it block and cut redundant parts. • Don’t mimic login through web interaction. Set cookie directly. • Use tags effectively. Load something only with certain tags.

Slide 39

Slide 39 text

Notes on reducing let • Maybe you should use instance variables instead of let.

Slide 40

Slide 40 text

Notes on reducing let • Maybe you should use instance variables instead of let.

Slide 41

Slide 41 text

Notes on reducing let • If we use more than 5 lets in one example, maybe too many lets (one of the exceptions is search). • Prefer build over create, prefer build_stubbed over build. • Use let! only when required.

Slide 42

Slide 42 text

Slide 43

Slide 43 text

Removing things your tests don’t need (optimizations and compilations)

Slide 44

Slide 44 text

Real story:

Slide 45

Slide 45 text

spec/support/stripe.rb require 'fake_stripe' RSpec.configure do |config| config.before(:each) do FakeStripe.stub_stripe end end

Slide 46

Slide 46 text

How many minutes does it take? (Hint: minute, not second)

Slide 47

Slide 47 text

9 minutes.

Slide 48

Slide 48 text

Image optimization:

Slide 49

Slide 49 text

Several minutes (I forgot exact number)

Slide 50

Slide 50 text

Compiling with Webpacker

Slide 51

Slide 51 text

Compile once # config/webpacker.yml test: <<: *default compile: false # spec/rails_helper.rb RSpec.configure do |config| config.before(:suite) do Webpacker.compile end end

Slide 52

Slide 52 text

A few minutes.

Slide 53

Slide 53 text

Remove these for fast tests • Redundant `before each` call (or use some technique to load code only when required). • Redundant asset optimizations. • Webpacker compilation.

Slide 54

Slide 54 text

Recap

Slide 55

Slide 55 text

Slide 56

Slide 56 text

One more thing:

Slide 57

Slide 57 text

Who am I?

Slide 58

Slide 58 text

pp self • Name: OKURA Masafumi • Software developer for several years using Ruby and Rails • Vimmer (and a staff of VimConf 2019!) • Freelance now, seeking international jobs!

Slide 59

Slide 59 text

References • https://relishapp.com/rspec/rspec-core/docs/conﬁguration/ proﬁle-examples • https://github.com/palkan/test-prof