How Airbnb Made Specs Run 2.5x Faster

You're a developer on a large, legacy Rails project with thousands of specs. You hate waiting for them to run. If you sit and watch the dots go by, you're wasting time doing nothing. But if you go on Hacker News to do some other work, you lose your flow and it takes you too long to get back to it. It's a lose-lose. Is it impossible to speed up specs on a large Rails project? At Airbnb, two engineers made our specs run 2.5x faster in a three-day hackathon project. We did this without rewriting any specs. Multiple engineers had told us they had already tried to speed up the specs and they didn't think there was much left that could be done. Learn what we did to diagnose and improve spec performance.

Transcript

1. Slow Specs Don’t Belong Anywhere Brian Morearty with help from Clint Kelly
2. BELONG ANYWHERE
3. Problem:
4. Waiting for Specs
5. Choices:
6. 1. Multitask.
7. MY SPECS ARE RUNNING
8. 2. Wait for the dots.
10. Need a Solution
11. I Know:
12. More Fastness
13. Goal:
14. Twice as Fast
15. How Solve?
16. Hackathon Project
17. Results:
18. 2.5x as Fast
19. A.k.a. 61% Less Time
21. (not counting app bootup)
22. More Info:
23. Focus Was On Vagrant
24. Also Speeds Up Solano
25. Changed Spec Infrastructure. Not Specs.
26. All Specs Pass.
27. How Find Slow?
28. 1. Perftools.rb
29. /usr/bin/ruby1.9.1 Total samples: 36234 Focusing on: 36234 Dropped nodes with <= 181 abs(samples) Dropped edges with <= 36 samples BasicObject#instance_exec 14 (0.0%) of 19941 (55.0%) 9 5 0 2 Array#map 44 (0.1%) of 18478 (51.0%) 4 2 3 FactoryGirl FactoryRunner#run 2 (0.0%) of 15382 (42.5%) 8 6 9 8 FactoryGirl E v a l u a t o r # a s s o c i a t i o n 0 (0.0%) of 4791 (13.2%) 6 4 6 4 C l a s s # r u n 0 (0.0%) of 19363 (53.4%) R S p e c C o r e R u n n e r # r u n 0 (0.0%) of 18658 (51.5%) 1 8 5 2 5 2 5 5 7 9 C l a s s # r u n _ e x a m p l e s 0 (0.0%) of 18111 (50.0%) 1 7 9 9 4 R S p e c C o r e R e p o r t e r # r e p o r t 0 (0.0%) 1 7 4 6 8 4 3 4 6 3 RSpec Core Example#run 1 (0.0%) of 18281 (50.5%) 1 8 2 0 0 4 2 5 R S p e c C o r e Example#with_around_each_hooks 0 (0.0%) of 18360 (50.7%) R S p e c C o r e E x a m p l e # r u n _ b e f o r e _ e a c h 0 (0.0%) 8 9 1 0 1 8 2 6 4 1 8 1 1 1 1 7 6 3 4 garbage_collector 15558 (42.9%) Module#instrument 1 (0.0%) of 15379 (42.4%) 2 4 4 6 9 9 1 2 2 FactoryGirl S t r a t e g y C r e a t e # a s s o c i a t i o n 0 (0.0%) of 4684 (12.9%) 6 3 9 0 6 3 9 1
30. 2. Poor Man’s Profiler
32. 3. Edumacated Hunches
34. How Fix Slow?
35. Tune the GC
36. 1.61x as fast. 0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%# Before# Tune#GC# Op9mize#user#factory# Op9mize#index# enforcer# UserEcrea9on# background#jobs# Disable#upshot#in# specs# Bonus:#backport# `pluck`#
37. Optimize The :user Factory
38. 1.28x as fast. 0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%# Before# Tune#GC# Op9mize#user#factory# Op9mize#index# enforcer# UserEcrea9on# background#jobs# Disable#upshot#in# specs# Bonus:#backport# `pluck`#
39. Optimize IndexEnforcer
40. 1.06x as fast. 0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%# Before# Tune#GC# Op9mize#user#factory# Op9mize#index# enforcer# UserEcrea9on# background#jobs# Disable#upshot#in# specs# Bonus:#backport# `pluck`#
41. Move User-Creation Side-Effects to Background Job
42. 1.10x as fast. 0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%# Before# Tune#GC# Op9mize#user#factory# Op9mize#index# enforcer# UserEcrea9on# background#jobs# Disable#upshot#in# specs# Bonus:#backport# `pluck`#
43. Disable SQL Logging to StatsD in Specs
44. 1.06x as fast. 0%# 10%# 20%# 30%# 40%# 50%# 60%# 70%# 80%# 90%# 100%# Before# Tune#GC# Op9mize#user#factory# Op9mize#index# enforcer# UserEcrea9on# background#jobs# Disable#upshot#in# specs# Bonus:#backport# `pluck`#
45. Quentions?