Save 37% off PRO during our Black Friday Sale! »

Building a toolkit to detect performance regressions in Ruby on Rails core

Building a toolkit to detect performance regressions in Ruby on Rails core

My talk from RailsConf 2015.

C40db80709604a6868c769aca6367fb7?s=128

Kir Shatrov

April 22, 2015
Tweet

Transcript

  1. Kir Shatrov, RailsConf 2015 Building a toolkit to detect performance

    regressions in Ruby on Rails core
  2. @kirs @kirshatrov !"

  3. None
  4. Catching Performance Regressions in Rails

  5. What’s a performance regression?

  6. What’s a performance regression in Rails?

  7. Page load 250ms → 4s

  8. Case Study

  9. 4.2rc1

  10. None
  11. 2 times slower

  12. 4.2rc3

  13. Faster framework = Faster app

  14. Examples from Rails commits

  15. Metrics

  16. Timing & Allocations

  17. Faster Methods = Faster App

  18. start = Time.now.to_f sleep 5 ends = Time.now puts "it

    took #{ends - start}"
  19. Allocations = GC work Less allocations = less GC work

  20. before = GC.stat[:total_allocated_object] 10_000.times do { "key" => "value" }

    end after = GC.stat[:total_allocated_object] puts "allocated: #{after - before}"
  21. Basic primitives

  22. Examples from Rails commits String

  23. None
  24. None
  25. None
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. Examples from Rails commits Hash

  34. None
  35. None
  36. None
  37. None
  38. None
  39. None
  40. Optimize only “hot” code

  41. Track the changes

  42. The Idea to build a service

  43. Track it with benchmarks

  44. Any existing benchmarks?

  45. None
  46. Setup Discourse Populate records Start Unicorn with RAILS_ENV=production Make requests

    with Apache Bench to the list of endpoints Print timings and memory usage
  47. Components to benchmark activerecord activemodel actionview actioncontroller activesupport

  48. 3.2 – 4.0 – 4.1 – 4.2

  49. None
  50. None
  51. None
  52. None
  53. ActiveRecord Finders require_relative 'support/activerecord_base.rb' require_relative 'support/benchmark_rails.rb' User.create!(name: 'kir', email: 'shatrov@me.com')

    m = Benchmark.rails(100, "activerecord/#{db_adapter}/finders") do User.find(1) end puts m.to_json
  54. Disable GC is necessary Warm up the code Make N

    iterations Calculate the mean timing and object allocations
  55. ActiveRecord::Base#create require_relative 'support/activerecord_base.rb' require_relative 'support/benchmark_rails.rb' fields = { name: "Kir",

    notes: "Note", created_at: Date.today } Benchmark.rails(1000, "activerecord/#{db_adapter}/create") do Exhibit.create(fields) end
  56. Correct benchmarks

  57. Full app benchmark

  58. Full app benchmark before

  59. stackprof by Aman Gupta github.com/tmm1/stackprof

  60. 4.2 4.1

  61. config/environments/production.rb

  62. Full app benchmark after

  63. ✅ Benchmarks

  64. None
  65. None
  66. The Service

  67. What do we want? See benchmark for every commit See

    benchmark for every PR Report to PR author (“activerecord became 2% slower”) See charts
  68. railsperf.evilmartians.io The prototype of the service. Built in January

  69. rubybench.org by Sam Saffron and Guo Xiang Tan “Alan”

  70. None
  71. 3.6GHz Intel® Xeon® E3-1270 v3 Quad-Core 4 x 8GB Micron

    ECC Samsung SSD 845DC EVO - 240 GB Sponsored hardware
  72. ruby-bench + rails

  73. github.com/ruby/ruby webhook web app docker server benchmark suite

  74. None
  75. github.com/ruby-bench web app docker files benchmark suite

  76. github.com/ruby-bench web app docker files benchmark suite

  77. github.com/ruby-bench web app docker files benchmark suite

  78. github.com/ruby-bench web app docker files benchmark suite

  79. Results

  80. Running builds for Rails

  81. 250k builds for Ruby

  82. MRI performance regression caught

  83. What’s next?

  84. What’s next Pull Request benchmarks Weekly reports JRuby support

  85. Overview What is a performance regression How to track them

    How to solve them How to write benchmarks Automate tracking!
  86. How to track your app performance? Study how do Ruby

    objects work and how to treat them Write a Discourse-like benchmark and run it on Travis Subscribe to Rails Weekly
  87. Ruby Under a Microscope

  88. http://bit.ly/writing-fast-ruby

  89. How to track your app performance? Study how do Ruby

    objects work and how to treat them Write a Discourse-like benchmark and run it on Travis Subscribe to Rails Weekly
  90. Setup the app Populate records Start Unicorn with RAILS_ENV=production Make

    requests with Apache Bench to the list of endpoints Print timings and memory usage
  91. How to track your app performance? Study how do Ruby

    objects work and how to treat them Write a Discourse-like benchmark and run it on Travis Subscribe to Rails Weekly
  92. bit.ly/railsperf-gh github.com/rubybench bit.ly/railsperf-static

  93. Thanks! @kirs @kirshatrov @evilmartians evilmartians.com