Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a toolkit to detect performance regressions in Ruby on Rails core

Building a toolkit to detect performance regressions in Ruby on Rails core

My talk from RailsConf 2015.

Kir Shatrov

April 22, 2015
Tweet

More Decks by Kir Shatrov

Other Decks in Programming

Transcript

  1. Kir Shatrov, RailsConf 2015
    Building a toolkit to detect
    performance regressions
    in Ruby on Rails core

    View full-size slide

  2. @kirs @kirshatrov
    !"

    View full-size slide

  3. Catching
    Performance Regressions
    in Rails

    View full-size slide

  4. What’s a
    performance regression?

    View full-size slide

  5. What’s a
    performance regression
    in Rails?

    View full-size slide

  6. Page load
    250ms → 4s

    View full-size slide

  7. 2 times slower

    View full-size slide

  8. Faster framework = Faster app

    View full-size slide

  9. Examples from Rails commits

    View full-size slide

  10. Timing & Allocations

    View full-size slide

  11. Faster Methods = Faster App

    View full-size slide

  12. start = Time.now.to_f
    sleep 5
    ends = Time.now
    puts "it took #{ends - start}"

    View full-size slide

  13. Allocations = GC work
    Less allocations = less GC work

    View full-size slide

  14. before = GC.stat[:total_allocated_object]
    10_000.times do
    { "key" => "value" }
    end
    after = GC.stat[:total_allocated_object]
    puts "allocated: #{after - before}"

    View full-size slide

  15. Basic primitives

    View full-size slide

  16. Examples from Rails commits
    String

    View full-size slide

  17. Examples from Rails commits
    Hash

    View full-size slide

  18. Optimize only “hot” code

    View full-size slide

  19. Track the changes

    View full-size slide

  20. The Idea to build a service

    View full-size slide

  21. Track it with benchmarks

    View full-size slide

  22. Any existing benchmarks?

    View full-size slide

  23. Setup Discourse
    Populate records
    Start Unicorn with RAILS_ENV=production
    Make requests with Apache Bench to the list of endpoints
    Print timings and memory usage

    View full-size slide

  24. Components to benchmark
    activerecord
    activemodel
    actionview
    actioncontroller
    activesupport

    View full-size slide

  25. 3.2 – 4.0 – 4.1 – 4.2

    View full-size slide

  26. ActiveRecord Finders
    require_relative 'support/activerecord_base.rb'
    require_relative 'support/benchmark_rails.rb'
    User.create!(name: 'kir', email: '[email protected]')
    m = Benchmark.rails(100, "activerecord/#{db_adapter}/finders") do
    User.find(1)
    end
    puts m.to_json

    View full-size slide

  27. Disable GC is necessary
    Warm up the code
    Make N iterations
    Calculate the mean timing and object allocations

    View full-size slide

  28. ActiveRecord::Base#create
    require_relative 'support/activerecord_base.rb'
    require_relative 'support/benchmark_rails.rb'
    fields = {
    name: "Kir",
    notes: "Note",
    created_at: Date.today
    }
    Benchmark.rails(1000, "activerecord/#{db_adapter}/create") do
    Exhibit.create(fields)
    end

    View full-size slide

  29. Correct benchmarks

    View full-size slide

  30. Full app benchmark

    View full-size slide

  31. Full app benchmark
    before

    View full-size slide

  32. stackprof by Aman Gupta
    github.com/tmm1/stackprof

    View full-size slide

  33. config/environments/production.rb

    View full-size slide

  34. Full app benchmark
    after

    View full-size slide

  35. ✅ Benchmarks

    View full-size slide

  36. What do we want?
    See benchmark for every commit
    See benchmark for every PR
    Report to PR author (“activerecord became 2% slower”)
    See charts

    View full-size slide

  37. railsperf.evilmartians.io
    The prototype of the service. Built in January

    View full-size slide

  38. rubybench.org
    by Sam Saffron and Guo Xiang Tan “Alan”

    View full-size slide

  39. 3.6GHz Intel® Xeon® E3-1270 v3 Quad-Core
    4 x 8GB Micron ECC
    Samsung SSD 845DC EVO - 240 GB
    Sponsored hardware

    View full-size slide

  40. ruby-bench + rails

    View full-size slide

  41. github.com/ruby/ruby
    webhook
    web app
    docker server
    benchmark
    suite

    View full-size slide

  42. github.com/ruby-bench
    web app docker files benchmark suite

    View full-size slide

  43. github.com/ruby-bench
    web app docker files benchmark suite

    View full-size slide

  44. github.com/ruby-bench
    web app docker files benchmark suite

    View full-size slide

  45. github.com/ruby-bench
    web app docker files benchmark suite

    View full-size slide

  46. Running builds for Rails

    View full-size slide

  47. 250k builds for Ruby

    View full-size slide

  48. MRI performance regression
    caught

    View full-size slide

  49. What’s next?

    View full-size slide

  50. What’s next
    Pull Request benchmarks
    Weekly reports
    JRuby support

    View full-size slide

  51. Overview
    What is a performance regression
    How to track them
    How to solve them
    How to write benchmarks
    Automate tracking!

    View full-size slide

  52. How to track your app performance?
    Study how do Ruby objects work and how to treat them
    Write a Discourse-like benchmark and run it on Travis
    Subscribe to Rails Weekly

    View full-size slide

  53. Ruby Under a Microscope

    View full-size slide

  54. http://bit.ly/writing-fast-ruby

    View full-size slide

  55. How to track your app performance?
    Study how do Ruby objects work and how to treat them
    Write a Discourse-like benchmark and run it on Travis
    Subscribe to Rails Weekly

    View full-size slide

  56. Setup the app
    Populate records
    Start Unicorn with RAILS_ENV=production
    Make requests with Apache Bench to the list of endpoints
    Print timings and memory usage

    View full-size slide

  57. How to track your app performance?
    Study how do Ruby objects work and how to treat them
    Write a Discourse-like benchmark and run it on Travis
    Subscribe to Rails Weekly

    View full-size slide

  58. bit.ly/railsperf-gh
    github.com/rubybench
    bit.ly/railsperf-static

    View full-size slide

  59. Thanks!
    @kirs
    @kirshatrov
    @evilmartians
    evilmartians.com

    View full-size slide