Slide 1
Slide 1 text
Kir Shatrov, RailsConf 2015
Building a toolkit to detect
performance regressions
in Ruby on Rails core
Slide 2
Slide 2 text
@kirs @kirshatrov
!"
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
Catching
Performance Regressions
in Rails
Slide 5
Slide 5 text
What’s a
performance regression?
Slide 6
Slide 6 text
What’s a
performance regression
in Rails?
Slide 7
Slide 7 text
Page load
250ms → 4s
Slide 8
Slide 8 text
Case Study
Slide 9
Slide 9 text
4.2rc1
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
2 times slower
Slide 12
Slide 12 text
4.2rc3
Slide 13
Slide 13 text
Faster framework = Faster app
Slide 14
Slide 14 text
Examples from Rails commits
Slide 15
Slide 15 text
Metrics
Slide 16
Slide 16 text
Timing & Allocations
Slide 17
Slide 17 text
Faster Methods = Faster App
Slide 18
Slide 18 text
start = Time.now.to_f
sleep 5
ends = Time.now
puts "it took #{ends - start}"
Slide 19
Slide 19 text
Allocations = GC work
Less allocations = less GC work
Slide 20
Slide 20 text
before = GC.stat[:total_allocated_object]
10_000.times do
{ "key" => "value" }
end
after = GC.stat[:total_allocated_object]
puts "allocated: #{after - before}"
Slide 21
Slide 21 text
Basic primitives
Slide 22
Slide 22 text
Examples from Rails commits
String
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
No content
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
No content
Slide 27
Slide 27 text
No content
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
No content
Slide 30
Slide 30 text
No content
Slide 31
Slide 31 text
No content
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
Examples from Rails commits
Hash
Slide 34
Slide 34 text
No content
Slide 35
Slide 35 text
No content
Slide 36
Slide 36 text
No content
Slide 37
Slide 37 text
No content
Slide 38
Slide 38 text
No content
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
Optimize only “hot” code
Slide 41
Slide 41 text
Track the changes
Slide 42
Slide 42 text
The Idea to build a service
Slide 43
Slide 43 text
Track it with benchmarks
Slide 44
Slide 44 text
Any existing benchmarks?
Slide 45
Slide 45 text
No content
Slide 46
Slide 46 text
Setup Discourse
Populate records
Start Unicorn with RAILS_ENV=production
Make requests with Apache Bench to the list of endpoints
Print timings and memory usage
Slide 47
Slide 47 text
Components to benchmark
activerecord
activemodel
actionview
actioncontroller
activesupport
Slide 48
Slide 48 text
3.2 – 4.0 – 4.1 – 4.2
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
No content
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
No content
Slide 53
Slide 53 text
ActiveRecord Finders
require_relative 'support/activerecord_base.rb'
require_relative 'support/benchmark_rails.rb'
User.create!(name: 'kir', email: '[email protected]')
m = Benchmark.rails(100, "activerecord/#{db_adapter}/finders") do
User.find(1)
end
puts m.to_json
Slide 54
Slide 54 text
Disable GC is necessary
Warm up the code
Make N iterations
Calculate the mean timing and object allocations
Slide 55
Slide 55 text
ActiveRecord::Base#create
require_relative 'support/activerecord_base.rb'
require_relative 'support/benchmark_rails.rb'
fields = {
name: "Kir",
notes: "Note",
created_at: Date.today
}
Benchmark.rails(1000, "activerecord/#{db_adapter}/create") do
Exhibit.create(fields)
end
Slide 56
Slide 56 text
Correct benchmarks
Slide 57
Slide 57 text
Full app benchmark
Slide 58
Slide 58 text
Full app benchmark
before
Slide 59
Slide 59 text
stackprof by Aman Gupta
github.com/tmm1/stackprof
Slide 60
Slide 60 text
4.2
4.1
Slide 61
Slide 61 text
config/environments/production.rb
Slide 62
Slide 62 text
Full app benchmark
after
Slide 63
Slide 63 text
✅ Benchmarks
Slide 64
Slide 64 text
No content
Slide 65
Slide 65 text
No content
Slide 66
Slide 66 text
The Service
Slide 67
Slide 67 text
What do we want?
See benchmark for every commit
See benchmark for every PR
Report to PR author (“activerecord became 2% slower”)
See charts
Slide 68
Slide 68 text
railsperf.evilmartians.io
The prototype of the service. Built in January
Slide 69
Slide 69 text
rubybench.org
by Sam Saffron and Guo Xiang Tan “Alan”
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
3.6GHz Intel® Xeon® E3-1270 v3 Quad-Core
4 x 8GB Micron ECC
Samsung SSD 845DC EVO - 240 GB
Sponsored hardware
Slide 72
Slide 72 text
ruby-bench + rails
Slide 73
Slide 73 text
github.com/ruby/ruby
webhook
web app
docker server
benchmark
suite
Slide 74
Slide 74 text
No content
Slide 75
Slide 75 text
github.com/ruby-bench
web app docker files benchmark suite
Slide 76
Slide 76 text
github.com/ruby-bench
web app docker files benchmark suite
Slide 77
Slide 77 text
github.com/ruby-bench
web app docker files benchmark suite
Slide 78
Slide 78 text
github.com/ruby-bench
web app docker files benchmark suite
Slide 79
Slide 79 text
Results
Slide 80
Slide 80 text
Running builds for Rails
Slide 81
Slide 81 text
250k builds for Ruby
Slide 82
Slide 82 text
MRI performance regression
caught
Slide 83
Slide 83 text
What’s next?
Slide 84
Slide 84 text
What’s next
Pull Request benchmarks
Weekly reports
JRuby support
Slide 85
Slide 85 text
Overview
What is a performance regression
How to track them
How to solve them
How to write benchmarks
Automate tracking!
Slide 86
Slide 86 text
How to track your app performance?
Study how do Ruby objects work and how to treat them
Write a Discourse-like benchmark and run it on Travis
Subscribe to Rails Weekly
Slide 87
Slide 87 text
Ruby Under a Microscope
Slide 88
Slide 88 text
http://bit.ly/writing-fast-ruby
Slide 89
Slide 89 text
How to track your app performance?
Study how do Ruby objects work and how to treat them
Write a Discourse-like benchmark and run it on Travis
Subscribe to Rails Weekly
Slide 90
Slide 90 text
Setup the app
Populate records
Start Unicorn with RAILS_ENV=production
Make requests with Apache Bench to the list of endpoints
Print timings and memory usage
Slide 91
Slide 91 text
How to track your app performance?
Study how do Ruby objects work and how to treat them
Write a Discourse-like benchmark and run it on Travis
Subscribe to Rails Weekly
Slide 92
Slide 92 text
bit.ly/railsperf-gh
github.com/rubybench
bit.ly/railsperf-static
Slide 93
Slide 93 text
Thanks!
@kirs
@kirshatrov
@evilmartians
evilmartians.com