RailsConf 2015 Keynote

Transcript

1. View Slide

2. View Slide

3. Click to add title
   Click to add subtitle
   

   View Slide

4. Why Windows?
   

   View Slide

5. Kent Beck!!!
   

   View Slide

6. W
   IN
   DO
   W
   S
   

   View Slide

7. No emoji
   

   View Slide

8. !"#$
   

   View Slide

9. Welcome To Rails Conf!!!
   

   View Slide

10. Aaron Patterson
    

    View Slide

11. @tenderlove
    

    View Slide

12. View Slide

13. Thanks!!!
    

    View Slide

14. View Slide

15. View Slide

16. Ruby Central
    

    View Slide

17. @eileencodes
    

    View Slide

18. @_ko1
    

    View Slide

19. Thank You!!! <3
    

    View Slide

20. Cat photos
    

    View Slide

21. View Slide

22. View Slide

23. View Slide

24. Extreme Programming
    

    View Slide

25. View Slide

26. Last Year:
    Thought Leader in Training
    

    View Slide

27. I Graduated!!
    

    View Slide

28. Degree in Thought Leadering
    from Online University
    

    View Slide

29. Ruby
    

    View Slide

30. Rails
    

    View Slide

31. Technoligy
    

    View Slide

32. Rails
    

    View Slide

33. Computers
    

    View Slide

34. Rails
    

    View Slide

35. Synergy
    

    View Slide

36. Rails
    

    View Slide

37. Innovation
    

    View Slide

38. Rails
    

    View Slide

39. Future
    

    View Slide

40. Rails
    

    View Slide

41. Words
    

    View Slide

42. Trolling DHH
    

    View Slide

43. "TDD is dead"
    

    View Slide

44. SOA is dead
    

    View Slide

45. Turbo Links 3: RJS
    

    View Slide

46. I should announce something.
    

    View Slide

47. Backpack
    

    View Slide

48. Lightweight
    

    View Slide

49. Action Fanny Pack
    

    View Slide

50. Pink
    

    View Slide

51. Performance!
    vroom!!!
    

    View Slide

52. Rails 5?
    

    View Slide

53. Adequate Development
    

    View Slide

54. Automatic Parallel Testing
    

    View Slide

55. Data Base
    Test Process Test Process Test Process
    

    View Slide

56. Data Base
    Test Process Test Process Test Process
    Data Base Data Base
    Data Base
    

    View Slide

57. Automatic
    

    View Slide

58. Transparent
    

    View Slide

59. Caching Compiled Views
    

    View Slide

60. Today: lazy compile
    

    View Slide

61. Get lock
    Template Compiled?
    Compile
    Save
    Unlock
    Writes to
    memory
    

    View Slide

62. Threaded Servers: lock
    

    View Slide

63. Forking servers:
    waste memory
    

    View Slide

64. Challenges
    

    View Slide

65. Too many templates!
    

    View Slide

66. Rendering with Locals
    

    View Slide

67. Template source
    eval("
    local_variable = variables_hash[:local_variable]
    # rest of the template
    ")
    

    View Slide

68. Integration Tests
    

    View Slide

69. Controller Test
    class ControllerTest < ActionController::TestCase
    test "index" do
    get :index
    assert_equal 200, response.status
    end
    end
    

    View Slide

70. Integration Test
    class IntegrationTest < ActionDispatch::IntegrationTest
    test "index" do
    get '/documents'
    assert_equal 200, response.status
    end
    end
    

    View Slide

71. Why do we write
    controller tests?
    

    View Slide

72. Because integration tests
    are slow?
    

    View Slide

73. If tests are slow,
    is the website slow?
    

    View Slide

74. Rails 5?: Fast Tests
    

    View Slide

75. Controller / Integration:
    What’s the difference?
    

    View Slide

76. Controller:
    no middleware
    

    View Slide

77. Integration:
    has middleware
    

    View Slide

78. Rails 5?: Delete controller
    tests
    *We won’t actually delete them
    

    View Slide

79. Why So Slow?
    

    View Slide

80. How slow?
    

    View Slide

81. Benchmark
    Benchmark.ips do |bm|
    bm.report 'INDEX: Integration Test' do
    Minitest.run_one_method(IntegrationTest, 'test_index')
    end
    bm.report 'INDEX: Functional Test' do
    Minitest.run_one_method(ControllerTest, 'test_index')
    end
    bm.compare!
    end
    Controller
    Integration
    Compare
    

    View Slide

82. Result
    Calculating -------------------------------------
    INDEX: Integration Test
    15.000 i/100ms
    INDEX: Functional Test
    39.000 i/100ms
    -------------------------------------------------
    INDEX: Integration Test
    158.019 (± 8.2%) i/s - 795.000
    INDEX: Functional Test
    390.853 (± 9.0%) i/s - 1.950k
    Comparison:
    INDEX: Functional Test: 390.9 i/s
    INDEX: Integration Test: 158.0 i/s - 2.47x slower
    Comparison
    

    View Slide

83. What is slow?
    

    View Slide

84. stackprof
    

    View Slide

85. Controller Bottleneck
    Minitest.run_one_method(ControllerTest, 'test_index')
    StackProf.run(mode: :cpu, out: 'stackprof.dump') do
    3000.times do
    Minitest.run_one_method(ControllerTest,
    'test_index')
    end
    end
    

    View Slide

86. CPU time vs
    Wall time
    

    View Slide

87. `sleep` is slow, but doesn’t
    use CPU
    

    View Slide

88. View Stack
    $ stackprof stackprof.dump
    

    View Slide

89. Stack
    ==================================
    Mode: cpu(1000)
    Samples: 3280 (0.00% miss rate)
    GC: 134 (4.09%)
    ==================================
    TOTAL (pct) SAMPLES (pct) FRAME
    3146 (95.9%) 1740 (53.0%)
    Minitest::Runnable.on_signal
    483 (14.7%) 483 (14.7%)
    Dependencies#search_for_file
    207 (6.3%) 191 (5.8%) SQLite3::Database#prepare
    95 (2.9%) 95 (2.9%) Inflector#underscore
    47 (1.4%) 46 (1.4%) TestRequest#initialize
    

    View Slide

90. 53.0% in
    Minitest::Runnable.on_signal
    

    View Slide

91. on_signal
    def self.on_signal name, action # :nodoc:
    supported = SIGNALS[name]
    old_trap = trap name do
    old_trap.call if old_trap.respond_to? :call
    action.call
    end if supported
    yield
    ensure
    trap name, old_trap if supported
    end
    

    View Slide

92. JUST DELETE IT
    

    View Slide

93. on_signal
    def self.on_signal name, action
    yield
    end
    

    View Slide

94. Results
    ==================================
    Mode: cpu(1000)
    Samples: 4246 (0.00% miss rate)
    GC: 386 (9.09%)
    ==================================
    TOTAL (pct) SAMPLES (pct) FRAME
    220 (5.2%) 185 (4.4%)
    SQLite3::Database#prepare
    148 (3.5%) 131 (3.1%) TestRequest#initialize
    113 (2.7%) 113 (2.7%) Event#initialize
    105 (2.5%) 105 (2.5%)
    PerThreadRegistry#instance
    

    View Slide

95. 50% faster!
    

    View Slide

96. ☺
    

    View Slide

97. Time the process
    $ time ruby benchmark.rb
    real 0m12.358s
    user 0m11.732s
    sys 0m0.498s
    $ time ruby benchmark.rb
    real 0m12.055s
    user 0m11.462s
    sys 0m0.475s
    Before
    After
    

    View Slide

98. &
    

    View Slide

99. CPU bound
    

    View Slide

100. Wall should be similar to
     CPU
     

     View Slide

101. Stack (CPU)
     ==================================
     Mode: cpu(1000)
     Samples: 3280 (0.00% miss rate)
     GC: 134 (4.09%)
     ==================================
     TOTAL (pct) SAMPLES (pct) FRAME
     3146 (95.9%) 1740 (53.0%)
     Minitest::Runnable.on_signal
     483 (14.7%) 483 (14.7%)
     Dependencies#search_for_file
     207 (6.3%) 191 (5.8%) SQLite3::Database#prepare
     95 (2.9%) 95 (2.9%) Inflector#underscore
     47 (1.4%) 46 (1.4%) TestRequest#initialize
     

     View Slide

102. Stack (WALL)
     ==================================
     Mode: wall(1000)
     Samples: 8797 (0.11% miss rate)
     GC: 535 (6.08%)
     ==================================
     TOTAL (pct) SAMPLES (pct) FRAME
     286 (3.3%) 286 (3.3%) PerThreadRegistry#instance
     233 (2.6%) 233 (2.6%) Formatter#non_recursive
     336 (3.8%) 227 (2.6%) Class#class_attribute
     209 (2.4%) 209 (2.4%)
     NonConcurrentCacheBackend#[]
     198 (2.3%) 198 (2.3%) ActiveRecord::Base.logger
     

     View Slide

103. '
     

     View Slide

104. Even profilers have bugs!
     

     View Slide

105. Something useful
     

     View Slide

106. Integration test profiles.
     

     View Slide

107. Stack (WALL)
     ==================================
     Mode: wall(1000)
     Samples: 20168 (2.95% miss rate)
     GC: 1247 (6.18%)
     ==================================
     TOTAL (pct) SAMPLES (pct) FRAME
     6401 (31.7%) 5622 (27.9%) block in Module#delegate
     495 (2.5%) 495 (2.5%)
     PerThreadRegistry#instance
     341 (1.7%) 341 (1.7%)
     NonConcurrentCacheBackend#[]
     317 (1.6%) 317 (1.6%) Set#include?
     

     View Slide

108. Calling `delegate` is slow.
     

     View Slide

109. Patch
     - delegate :url_for, :optimize_routes_generation?, to: '@_routes'
     + def url_for(options)
     + @_routes.url_for(options)
     + end
     +
     + def optimize_routes_generation?
     + @_routes.optimize_routes_generation?
     + end
     +
     

     View Slide

110. Check our progress
     

     View Slide

111. 2.47x slower
     

     View Slide

112. Result
     Calculating -------------------------------------
     INDEX: Integration Test
     23.000 i/100ms
     INDEX: Functional Test
     37.000 i/100ms
     -------------------------------------------------
     INDEX: Integration Test
     251.487 (±12.7%) i/s - 1.242k
     INDEX: Functional Test
     365.112 (±11.0%) i/s - 1.813k
     Comparison:
     INDEX: Functional Test: 365.1 i/s
     INDEX: Integration Test: 251.5 i/s - 1.45x slower
     Comparison
     

     View Slide

113. 2.47x to 1.45x
     

     View Slide

114. Verify
     $ time ruby test.rb
     real 0m26.821s
     user 0m25.277s
     sys 0m0.903s
     $ time ruby test.rb
     real 0m16.632s
     user 0m15.709s
     sys 0m0.638s
     Before
     After
     

     View Slide

115. Even profilers have bugs
     

     View Slide

116. Always measure
     

     View Slide

117. Verify results
     

     View Slide

118. Today: 10% slower
     

     View Slide

119. But faster than controller
     tests in previous versions
     

     View Slide

120. Soup two Nuts
     Performance
     

     View Slide

121. bundler
     

     View Slide

122. Binstubs
     

     View Slide

123. bin stub source code
     #!/Users/aaron/.rbenv/versions/ruby-trunk/bin/ruby
     #
     # This file was generated by RubyGems.
     #
     # The application 'bundler' is installed as part of a gem, and
     # this file is here to facilitate running it.
     #
     # some unimportant stuff happens here!
     gem 'bundler', version
     load Gem.bin_path('bundler', 'bundle', version)
     

     View Slide

124. gem
     

     View Slide

125. require
     

     View Slide

126. What does
     `gem` do?
     

     View Slide

127. Finds a gemspec
     

     View Slide

128. Puts a path on
     $LOAD_PATH
     

     View Slide

129. Load Path Manipulation
     > x = $LOAD_PATH.dup; nil
     => nil
     > gem 'bundler'
     => true
     > $LOAD_PATH - x
     => ["/Users/aaron/.rbenv/versions/ruby-trunk/lib/ruby/gems/2.3.0/gems/
     bundler-1.9.2/lib"]
     

     View Slide

130. What Happens
     When File is Required?
     

     View Slide

131. Is File in
     $LOAD_PATH?
     Load it!
     Is file in
     Gemspec?
     Add gemspec to
     $LOAD_PATH
     

     View Slide

132. RubyGems performance
     

     View Slide

133. Test Methodology
     

     View Slide

134. Require One File
     And Increase Gems
     

     View Slide

135. Activate one Gem
     And Increase Gems
     

     View Slide

136. time to require one file
     Time in seconds
     0
     0.08
     0.16
     0.24
     0.32
     Number of Gems on the system
     0 100 200 300 400 500 600 700 800 900 1000
     best
     worst
     

     View Slide

137. allocations to require one file
     Number of allocations
     0
     22500
     45000
     67500
     90000
     Number of Gems on the system
     0 100 200 300 400 500 600 700 800 900 1000
     best
     worst
     

     View Slide

138. time to activate one gem
     Time in seconds
     0
     0.06
     0.12
     0.18
     0.24
     Number of Gems on the system
     0 100 200 300 400 500 600 700 800 900 1000
     best
     worst
     

     View Slide

139. allocations to activate one gem
     Number of allocations
     0
     10000
     20000
     30000
     40000
     Number of Gems on the system
     0 100 200 300 400 500 600 700 800 900 1000
     

     View Slide

140. Bundler Performance
     

     View Slide

141. Test Method
     

     View Slide

142. One Gem in Gemfile
     Increase Gems on System
     

     View Slide

143. Increase Gems in Gemfile
     Constant Gems on System
     

     View Slide

144. *NO DEPENDENCIES
     

     View Slide

145. *ALL LOCAL
     

     View Slide

146. Vary System Gems
     

     View Slide

147. `bundle update` with one gem
     Time in Seconds
     0
     0.2
     0.4
     0.6
     0.8
     Number of Gems on the system
     100 200 300 400 500 600 700 800 900 1000
     

     View Slide

148. `bundle exec` with one gem
     Time in Seconds
     0
     0.2
     0.4
     0.6
     0.8
     Number of Gems on the system
     100 200 300 400 500 600 700 800 900 1000
     

     View Slide

149. Vary Gems in Gemfile
     

     View Slide

150. `bundle exec` time
     Time in seconds
     1.44
     1.45
     1.46
     1.47
     1.48
     Number of gems in the gemfile
     0 100 200 300 400 500 600 700 800 900 1000
     y = 0.0008x + 1.4603
     R² = 0.1323
     

     View Slide

151. Bundle Exec Time Breakdown (ms)
     Other?
     750
     Spec Load
     710
     

     View Slide

152. `bundle update` time
     Time in Seconds
     0
     17.5
     35
     52.5
     70
     Number of Gems in the Gemfile
     0 100 200 300 400 500 600 700 800 900 1000
     y = 0.6984x2 - 2.9466x + 13.856
     R² = 0.999
     

     View Slide

153. Thanks Excel
     

     View Slide

154. View Slide

155. View Slide

156. View Slide

157. R Code - Fit curve
     > bundler run_bundle_updats.csv",head=TRUE,sep=",")
     > fit2 TRUE))
     > xx > plot(bundler$gems, bundler$time)
     > lines(xx, predict(fit2, data.frame(x = xx)), col="red")
     

     View Slide

158. View Slide

159. R Code - Coefficients
     > coefficients(fit2)
     (Intercept) poly(bundler$gems, 2, raw = TRUE)1 poly(bundler$gems, 2, raw = TRUE)2
     1.160827e+01 -1.549806e-02 6.984136e-05
     > summary(fit2)$r.squared
     [1] 0.9989536
     >
     

     View Slide

160. View Slide

161. Make Predictions
     > second_order + coefs + res + return(res)
     + }
     > second_order(0, fit2)
     (Intercept)
     11.60827
     > second_order(10000, fit2)
     (Intercept)
     6840.763
     empty gemfile
     10k gemfile
     

     View Slide

162. How to Fix?
     

     View Slide

163. What do we know?
     

     View Slide

164. bin stub source code
     #!/Users/aaron/.rbenv/versions/ruby-trunk/bin/ruby
     #
     # This file was generated by RubyGems.
     #
     # The application 'bundler' is installed as part of a gem, and
     # this file is here to facilitate running it.
     #
     # some unimportant stuff happens here!
     gem 'bundler', version
     load Gem.bin_path('bundler', 'bundle', version)
     gem
     name
     

     View Slide

165. Gemfile.lock
     GEM
     specs:
     pkg0 (1)
     PLATFORMS
     ruby
     DEPENDENCIES
     pkg0
     gem name and version!
     

     View Slide

166. Spec loading code today
     def get_spec name
     spec_cache = Dir[File.join(dir, "*.gemspec")]
     spec_cache.find { |spec| spec.name == name }
     end
     

     View Slide

167. bundle -v against 100k gems
     [aaron@TC rubygems (master)]$ time GEM_HOME=`pwd`/100k_gems
     GEM_PATH=`pwd`/100k_gems ruby -I lib 100k_gems/bin/bundle -v
     Bundler version 1.9.4
     real 0m50.045s
     user 0m15.815s
     sys 0m11.967s
     

     View Slide

168. Spec loading code I want
     def get_spec name
     spec_cache = Dir[File.join(dir, "#{name}-*.gemspec")]
     spec_cache.find { |spec| spec.name == name }
     end
     

     View Slide

169. bundle -v against 100k gems
     [aaron@TC rubygems (specs)]$ time GEM_HOME=`pwd`/100k_gems
     GEM_PATH=`pwd`/100k_gems ruby -I lib 100k_gems/bin/bundle -v
     Bundler version 1.9.4
     real 0m0.632s
     user 0m0.479s
     sys 0m0.124s
     

     View Slide

170. Still O(n)
     

     View Slide

171. I think we can make it O(1)
     

     View Slide

172. Future Work:
     Make `require` O(1)
     

     View Slide

173. Don’t recalculate
     static data
     

     View Slide

174. Take measurements
     

     View Slide

175. Question intuitions
     

     View Slide

176. I have stickers!
     

     View Slide

177. THANK YOU!!!
     

     View Slide