Building a toolkit to detect performance regressions in Ruby on Rails core

Transcript

1. Kir Shatrov, RailsConf 2015 Building a toolkit to detect performance

   regressions in Ruby on Rails core

2. @kirs @kirshatrov !"

3. None

4. Catching Performance Regressions in Rails

5. What’s a performance regression?

6. What’s a performance regression in Rails?

7. Page load 250ms → 4s

8. Case Study

9. 4.2rc1

10. None

11. 2 times slower

12. 4.2rc3

13. Faster framework = Faster app

14. Examples from Rails commits

15. Metrics

16. Timing & Allocations

17. Faster Methods = Faster App

18. start = Time.now.to_f sleep 5 ends = Time.now puts "it

    took #{ends - start}"

19. Allocations = GC work Less allocations = less GC work

20. before = GC.stat[:total_allocated_object] 10_000.times do { "key" => "value" }

    end after = GC.stat[:total_allocated_object] puts "allocated: #{after - before}"

21. Basic primitives

22. Examples from Rails commits String

23. None
24. None
25. None
26. None
27. None
28. None
29. None
30. None
31. None
32. None

33. Examples from Rails commits Hash

34. None
35. None
36. None
37. None
38. None
39. None

40. Optimize only “hot” code

41. Track the changes

42. The Idea to build a service

43. Track it with benchmarks

44. Any existing benchmarks?

45. None

46. Setup Discourse Populate records Start Unicorn with RAILS_ENV=production Make requests

    with Apache Bench to the list of endpoints Print timings and memory usage

47. Components to benchmark activerecord activemodel actionview actioncontroller activesupport

48. 3.2 – 4.0 – 4.1 – 4.2

49. None
50. None
51. None
52. None

53. ActiveRecord Finders require_relative 'support/activerecord_base.rb' require_relative 'support/benchmark_rails.rb' User.create!(name: 'kir', email: 'shatrov@me.com')

    m = Benchmark.rails(100, "activerecord/#{db_adapter}/finders") do User.find(1) end puts m.to_json

54. Disable GC is necessary Warm up the code Make N

    iterations Calculate the mean timing and object allocations

55. ActiveRecord::Base#create require_relative 'support/activerecord_base.rb' require_relative 'support/benchmark_rails.rb' fields = { name: "Kir",

    notes: "Note", created_at: Date.today } Benchmark.rails(1000, "activerecord/#{db_adapter}/create") do Exhibit.create(fields) end

56. Correct benchmarks

57. Full app benchmark

58. Full app benchmark before

59. stackprof by Aman Gupta github.com/tmm1/stackprof

60. 4.2 4.1

61. config/environments/production.rb

62. Full app benchmark after

63. ✅ Benchmarks

64. None
65. None

66. The Service

67. What do we want? See benchmark for every commit See

    benchmark for every PR Report to PR author (“activerecord became 2% slower”) See charts

68. railsperf.evilmartians.io The prototype of the service. Built in January

69. rubybench.org by Sam Saffron and Guo Xiang Tan “Alan”

70. None

71. 3.6GHz Intel® Xeon® E3-1270 v3 Quad-Core 4 x 8GB Micron

    ECC Samsung SSD 845DC EVO - 240 GB Sponsored hardware

72. ruby-bench + rails

73. github.com/ruby/ruby webhook web app docker server benchmark suite

74. None

75. github.com/ruby-bench web app docker files benchmark suite

76. github.com/ruby-bench web app docker files benchmark suite

77. github.com/ruby-bench web app docker files benchmark suite

78. github.com/ruby-bench web app docker files benchmark suite

79. Results

80. Running builds for Rails

81. 250k builds for Ruby

82. MRI performance regression caught

83. What’s next?

84. What’s next Pull Request benchmarks Weekly reports JRuby support

85. Overview What is a performance regression How to track them

    How to solve them How to write benchmarks Automate tracking!

86. How to track your app performance? Study how do Ruby

    objects work and how to treat them Write a Discourse-like benchmark and run it on Travis Subscribe to Rails Weekly

87. Ruby Under a Microscope

88. http://bit.ly/writing-fast-ruby

89. How to track your app performance? Study how do Ruby

    objects work and how to treat them Write a Discourse-like benchmark and run it on Travis Subscribe to Rails Weekly

90. Setup the app Populate records Start Unicorn with RAILS_ENV=production Make

    requests with Apache Bench to the list of endpoints Print timings and memory usage

91. How to track your app performance? Study how do Ruby

    objects work and how to treat them Write a Discourse-like benchmark and run it on Travis Subscribe to Rails Weekly

92. bit.ly/railsperf-gh github.com/rubybench bit.ly/railsperf-static

93. Thanks! @kirs @kirshatrov @evilmartians evilmartians.com