RailsConf 2016 Keynote

This is my keynote from RailsConf 2016.

Transcript

1. How people build software ! " Make Rails Great Again A @tenderlove Guide to Things
2. How people build software ! HELLO!!!! 2 !
3. How people build software ! Aaron Patterson 3 !
4. How people build software ! @tenderlove 4 !
6. How people build software ! Ruby Core Team Rails Core Team 6 !
7. How people build software ! Ruby Security Team Rails Security Team 7 !
8. How people build software ! PGP Fingerprint: 4CE9 1B75 A798 28E8 6B1A A8BB 9531 70BC B4FF AFC6 8 !
9. How people build software ! #2 Committer 9 !
11. How people build software ! GitHub 11 !
12. How people build software ! Le Git 12 !
13. How people build software ! GitHug 13 !
14. Plain old slides
15. Sponsored Talk
16. Brought to you by @ihavenotea
21. I have stickers of my cats
23. My Job at GitHub
24. GitHub: How people build software
25. Bringing GitHub Application Development to Everyone
26. Rails Core Develompent
27. Ruby Core Development
28. Buy our products!
29. My Career Goals
30. I wanted to get rich.
31. Why?
32. So that I could do what I want. (Which is making Rails great again)
33. Become a Cog
34. I want to be a cog!
35. New in Rails 5
36. Rails is in it’s Prime 5
37. No more XML situps
38. JSON burpees
39. Threading
40. Race Condition VROOOOM!!!!
41. Le Mans is just Nascar for Europe Follow @ juliancheal today! (He gave me this joke) Advertisement:
42. Ruby Drama
43. I am extremely angry at Jeremy
44. Friday, April 22, 2016 7:57 pm
46. JEREMY!!!
49. $5
50. $5 footlong
51. WHAT EVEN IS THIS?
52. Ruby on Rails The mainstream }The Surplus of Slides
53. I ❤ u JD!
54. Major Changes in ActiveRecord
55. I want to talk about SQL
56. So I’ve prepared a statement Follow @ kerrizor today! ❤ Advertisement:
57. ApplicationRecord
58. Appealing to Startups
59. Application Generation [aaron@TC rails (master)]$ ruby railties/exe/rails new ~/git/omglolwut2 --dev --skip-bundle create create README.md create Rakefile create config.ru create .gitignore create Gemfile create app create app/assets/config/manifest.js create app/assets/javascripts/application.js create app/assets/javascripts/cable.js create app/assets/stylesheets/application.css create app/channels/application_cable/channel.rb create app/channels/application_cable/connection.rb create app/controllers/application_controller.rb create app/helpers/application_helper.rb create app/jobs/application_job.rb create app/mailers/application_mailer.rb create app/models/application_record.rb create app/models/business.rb create app/views/layouts/application.html.erb create app/views/layouts/mailer.html.erb create app/views/layouts/mailer.text.erb create app/assets/images/.keep create app/assets/javascripts/channels create app/assets/javascripts/channels/.keep create app/controllers/concerns/.keep create app/models/concerns/.keep create bin create bin/bundle create bin/rails create bin/rake create bin/setup create bin/update create config create config/routes.rb create config/application.rb create config/environment.rb create config/secrets.yml create config/cable.yml create config/puma.rb om glolwut2
60. Application Generation create app/controllers/application_controller.rb create app/helpers/application_helper.rb create app/jobs/application_job.rb create app/mailers/application_mailer.rb create app/models/application_record.rb create app/models/business.rb create app/views/layouts/application.html.erb create app/views/layouts/mailer.html.erb
61. Business Model [aaron@TC rails (master)]$ cd ~/git/omglolwut2 [aaron@TC omglolwut2]$ cat app/models/business.rb class Business < ApplicationRecord # Business model end [aaron@TC omglolwut2]$
62. This is a HackerNews compliant business model
63. Surprises!
64. PHP templates
68. We need to go further
70. Lets get down to business
71. What have you done for me lately?
72. Yes, but what have you done for me *lately*?
73. Performance
74. SURPRISE!!! GitHub: You can put code there! Advertisement:
75. I ❤ Performance
76. Topics Boot Time Performance Run Time Performance Memory Efficiency
77. Performance is always about tradeoffs
78. Memory for speed
79. Concurrency for memory
80. Complexity for memory
81. Performance is never free
82. Always understand your constraints.
83. Boot Time Performance leftpad-ng.js: for all your padding needs, but only on the left side Advertisement:
84. bundle exec rails s RubyGems Bundler Rails Rails
85. bundle exec rails s RubyGems Bundler Rails Rails
86. Speeding up Ruby speeds up everything.
87. But that doesn’t mean we should write slow code.
88. Two Optimizations Ruby (sort of) RubyGems
89. Run an empty program [aaron@TC ~]$ ruby -v ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin15] [aaron@TC ~]$ time ruby -e' ' real 0m0.101s user 0m0.061s sys 0m0.032s OMG RUBY IS SLOW
90. Without RubyGems [aaron@TC ~]$ ruby -v ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin15] [aaron@TC ~]$ time ruby -e' ' real 0m0.101s user 0m0.061s sys 0m0.032s [aaron@TC ~]$ time ruby --disable-gems -e' ' real 0m0.051s user 0m0.022s sys 0m0.023s
91. Measurement by Elimination
92. Placing blame is difficult git blame: for all your blame placing needs Advertisement:
93. gem_prelude.rb
94. Gem Prelude if defined?(Gem) require 'rubygems.rb' begin require 'did_you_mean' rescue LoadError end if defined?(DidYouMean) end Load Rubygems
95. Why load RubyGems? [aaron@TC ~]$ ruby -e'require "rubygems"; require "rails"' [aaron@TC ~]$ ruby -e'require "rails"' Bad old days of Ruby 1.8 Ruby 1.9+
96. Gem Prelude if defined?(Gem) require 'rubygems.rb' begin require 'did_you_mean' rescue LoadError end if defined?(DidYouMean) end Load Rubygems did_you_mean New in Ruby 2.3!
97. Did you mean? [aaron@TC ~]$ ruby -e'Object.new.object_ip' -e:1:in `<main>': undefined method `object_ip' for #<Object: 0x007fa7938b4fd8> (NoMethodError) Did you mean? object_id [aaron@TC ~]$ I see you’re trying to call a method!
98. did_you_mean is a gem
99. Gem Prelude if defined?(Gem) require 'rubygems.rb' begin require 'did_you_mean' rescue LoadError end if defined?(DidYouMean) end did_you_mean
100. # RubyGems's require: TL;DR alias :original_require :require def require file original_require file rescue LoadError found_gem = all_gems.find do |gem| gem.contains?(file) || gem.contains?(file + ".rb") || gem.contains?(file + ".so") # .bundle on OS X, .dll on Windows end if found_gem found_gem.activate # mutates $LOAD_PATH original_require file else raise "idk lol" end end RubyGems’ Require O(3N) or just O(N)
101. The more gems installed, the slower `require` gets
102. The more gems installed, the slower `ruby` gets.
103. Watch the O(N) $ sudo dtrace -q -n \ 'syscall::stat*:entry { printf("%s\n", copyinstr(arg0)); }' \ -c`rbenv which ruby`" -e '\ '"
104. Run it! [aaron@TC ~]$ sudo dtrace -q -n 'syscall::stat*:entry { printf("%s\n", copyinstr(arg0)); }' -c`rbenv which ruby`" -e '\ '" | wc -l 298 [aaron@TC ~]$ sudo dtrace -q -n 'syscall::stat*:entry { printf("%s\n", copyinstr(arg0)); }' -c`rbenv which ruby`" -- disable-did_you_mean -e '\ '" | wc -l 12 [aaron@TC ~]$ sudo dtrace -q -n 'syscall::stat*:entry { printf("%s\n", copyinstr(arg0)); }' -c`rbenv which ruby`" -- disable-gems -e '\ '" | wc -l 5 [aaron@TC ~]$ With RubyGems Without did_you_mean Without Both
105. File Stats actioncable-5.0.0.beta2/lib/did_you_mean actioncable-5.0.0.beta2/lib/did_you_mean.rb actioncable-5.0.0.beta2/lib/did_you_mean.bundle actionmailer-5.0.0.beta2/lib/did_you_mean actionmailer-5.0.0.beta2/lib/did_you_mean.rb actionmailer-5.0.0.beta2/lib/did_you_mean.bundle actionpack-5.0.0.beta2/lib/did_you_mean actionpack-5.0.0.beta2/lib/did_you_mean.rb actionpack-5.0.0.beta2/lib/did_you_mean.bundle actionview-5.0.0.beta2/lib/did_you_mean actionview-5.0.0.beta2/lib/did_you_mean.rb actionview-5.0.0.beta2/lib/did_you_mean.bundle activejob-5.0.0.beta2/lib/did_you_mean activejob-5.0.0.beta2/lib/did_you_mean.rb activejob-5.0.0.beta2/lib/did_you_mean.bundle
106. File Stats actioncable-5.0.0.beta2/lib/did_you_mean actioncable-5.0.0.beta2/lib/did_you_mean.rb actioncable-5.0.0.beta2/lib/did_you_mean.bundle actionmailer-5.0.0.beta2/lib/did_you_mean actionmailer-5.0.0.beta2/lib/did_you_mean.rb actionmailer-5.0.0.beta2/lib/did_you_mean.bundle actionpack-5.0.0.beta2/lib/did_you_mean actionpack-5.0.0.beta2/lib/did_you_mean.rb actionpack-5.0.0.beta2/lib/did_you_mean.bundle actionview-5.0.0.beta2/lib/did_you_mean actionview-5.0.0.beta2/lib/did_you_mean.rb actionview-5.0.0.beta2/lib/did_you_mean.bundle activejob-5.0.0.beta2/lib/did_you_mean activejob-5.0.0.beta2/lib/did_you_mean.rb activejob-5.0.0.beta2/lib/did_you_mean.bundle Why stat file with no extension?
107. GOOD NEWS EVERYONE
108. "d" is pretty early in the alphabet!
109. To speed up Rails, let us rename it to AAARails
110. Improvement
111. Use `gem` $ cat test.rb require "did_you_mean" $ sudo sh count_syscalls.sh | wc -l 299 $ cat test.rb gem "did_you_mean" require "did_you_mean" $ sudo sh count_syscalls.sh | wc -l 16 Bare `require` `gem` + `require`
112. Why this is faster: def gem gem_name spec = load_specfile(gem_name + ".gemspec") spec.activate # mutates the load path end # RubyGems's require: TL;DR alias :original_require :require def require file original_require file rescue LoadError found_gem = all_gems.find do |gem| gem.contains?(file) || gem.contains?(file + ".rb") || gem.contains?(file + ".so") # .dylib on OS X, .dll on Windows end if found_gem found_gem.activate # mutates $LOAD_PATH original_require file else raise "idk lol" end end O(1) O(1)* *Not actually O(1)
113. Fix gem_prelude.rb diff --git a/gem_prelude.rb b/gem_prelude.rb index 3f171d1..be9c419 100644 --- a/gem_prelude.rb +++ b/gem_prelude.rb @@ -1,7 +1,8 @@ if defined?(Gem) require 'rubygems.rb' begin + gem 'did_you_mean' require 'did_you_mean' - rescue LoadError + rescue Gem::LoadError, LoadError end if defined?(DidYouMean) end
114. Bare Require: O(N) Gem + Require: O(1)
115. Tradeoffs: Complexity for Speed
116. Complexity does have overhead
117. bundle exec rails s RubyGems Bundler Rails Rails Probably want the bundler gem
118. `bundle` before RG 2.5.2 [aaron@TC ~]$ tail -3 `rbenv which bundler` gem 'bundler', version load Gem.bin_path('bundler', 'bundler', version) [aaron@TC ~]$ O(1)
119. RG >= 2.5.2, < 2.6.2 [aaron@TC ~]$ tail -3 `rbenv which bundler` end load Gem.bin_path('bundler', 'bundler', version) [aaron@TC ~]$ (N)
120. bundle exec rails s RubyGems Bundler Rails Rails
121. New Technology!!!!* *Not actually new
122. Startup Time 12% 13% 34% 41% Compilation Execution Searching GC These Are Not Actual Times
123. How does GC impact? require 'benchmark/ips' # GC.disable # Uncomment to measure GC Benchmark.ips do |x| x.report("something") do call_some_method end end
124. Startup Time 12% 13% 34% 41% Compilation Execution Searching GC These Are Not Actual Times
125. https://github.com/byroot/ bootscale This link brought to you by GitHub dot com Advertisement:
126. Startup Time 13% 39% 47% Compilation Execution GC These Are Not Actual Times
127. Startup Time 46% 54% Compilation Execution These Are Not Actual Times
128. Lets tackle compilation!
129. Go to @_ko1’s talk
130. that was yesterday…
131. Program Flow Source Code Byte Code Execution Cache this!
132. Program Flow Source Code Byte Code Execution Read Byte Code from File Byte Code Execution
133. Compilation Script input_file = ARGV[0] output_file = input_file + "c" insns = RubyVM::InstructionSequence.compile_file input_file File.binwrite output_file, insns.to_binary New in Ruby 2.3!
134. Compilation Example $ cat hello.rb class Hello def hello! puts "hello" end end $ ruby compile.rb hello.rb $ cat hello.rbc YARB? LX?x86_64-darwin15*0*3 +3DD*???????**0*3 $t|????????||T? <class:Hello>Ehello!EhelloEHelloEputsEcore#define_methodx????8Qj?
135. 1st half is done
136. Load & Execute Byte Code compiled_file = ARGV[0] binary_data = File.binread compiled_file insns = RubyVM::InstructionSequence.load_from_binary binary_data insns.eval Hello.new.hello! New in Ruby 2.3!
137. Run it $ ruby load_compiled_file.rb hello.rbc hello
138. 2nd half is done
139. 100% done, but Rails isn’t faster
140. I always give 110%
141. Integrating with `require`
142. Require Process # a.rb require 'b' # b.rb require 'c' # c.rb require 'b'
143. Dependencies a.rb b.rb c.rb
144. Require Process Start to require 'b' Acquire lock for "b" Start to require 'c' Acquire lock for "c" Try to require 'b' Already locked for "b" Finish require 'c' Finish require 'b' Release lock for "c" Release lock for "b"
145. There are many rules, and I don’t want to figure them out.
146. Load File Hook set_load_required_file_func ->(full_file) { if can_handle?(full_file) # do something true else # do something else false end } require 'foo' I want foo.rb! /path/to/foo.rb
147. https://github.com/tenderlove/ ruby/tree/binload ADVERTISING!!
148. Load Compiled Code if ENV['COMP'] set_load_required_file_func ->(full_path) { fname = full_path + "o" if File.exist? fname insns = RubyVM::InstructionSequence.load_from_binary File.binread fname insns.eval true else insns = RubyVM::InstructionSequence.compile_file full_path File.binwrite fname, insns.to_binary false end } end https://gist.github.com/tenderlove/f6f4189db2eb748802b21a8b4fd52c99
149. Compiled vs Non-Compiled [aaron@TC omglolwut (master)]$ time RUBYOPT='-I. -rx' bin/rails db:migrate real 0m1.804s user 0m1.487s sys 0m0.414s [aaron@TC omglolwut (master)]$ time COMP=1 RUBYOPT='-I. -rx' bin/rails db:migrate real 0m1.273s user 0m1.002s sys 0m0.374s Before (not compiled) After (compiled)
150. ~30% faster Try mechanical keyboards today! Advertisement:
151. The more code, the bigger the impact.
152. Future Work Upstream new callback Compile code on `gem install` Cache invalidation If you like "cache invalidation", try "naming things"! Advertisement:
153. Runtime Performance
154. Polymorphic Inline Caching
155. Inline Caching class Hello def bar; end end def foo(object) object.bar end foo Hello.new foo Hello.new `object` is Hello, where is foo? `object` is Hello, I know where foo is. HIT! Cache Contents Key Value [Hello, foo] method source
156. Inline Caching class Hello def bar; end end class World def bar; end end def foo(object) object.bar end foo Hello.new foo World.new `object` is Hello, where is foo? MISS `object` is World, where is foo? MISS Cache Contents Key Value [Hello, foo] method source
157. Cache size: 1
158. Monomorphic Inline Cache
159. Cache size: 2 class Hello def bar; end end class World def bar; end end def foo(object) object.bar end foo Hello.new foo World.new foo Hello.new foo World.new `object` is Hello, where is foo? MISS `object` is World, where is foo? MISS `object` is Hello, I know foo. HIT `object` is World, I know foo. HIT Cache Contents Key Value [Hello, foo] method source [World, foo] method source
160. Cache size 2 (or more)
161. Polymorphic Inline Cachine
162. Pays off when many call sites "see" many types.
163. Implemented here: https://github.com/tenderlove/ruby/tree/PIC_expand
164. TL;DR: Didn’t help
165. ~3% of calls saw 2 types Test App: https://github.com/manageiq/manageiq Current Company Previous Company
166. Tradeoff: Complexity / Memory For Speed
167. Call site with 1600 types
168. Class Creation Code class Hello end def foo(object) object.bar end 2.times { hello = Hello.new hello.instance_eval do end } Creates a new subclass of Hello Always sees an anonymous class
169. What class is this? class Hello end var = Hello.new Ancestors BasicObject Kernel Object Hello #<Class blah>
170. Singleton Uses • instance_eval • singleton_class • def my_obj.some_method; end
171. Given "I’m a Rails Dev" When "Presented with this info" Then "Why should I care?" Brought to you by rspec-given. It’s a Given! Advertisement:
172. Some libraries do this
173. Don’t write code like that
174. If you think you have to
175. Just don’t. This slide brought to you by singleton classes Advertisement:
176. If you really really really must Brought to you by eigenclasses Advertisement:
177. We can speed it up.
178. Singleton classes with no new methods are equal to the superclass Your ad here! Contact @tenderlove for details Advertisement:
179. Patch to Copy $ git diff trunk class.c diff --git a/class.c b/class.c index 0261838..65428be 100644 --- a/class.c +++ b/class.c @@ -1591,7 +1591,9 @@ singleton_class_of(VALUE obj) klass = RBASIC(obj)->klass; if (!(FL_TEST(klass, FL_SINGLETON) && rb_ivar_get(klass, id_attached) == obj)) { + rb_serial_t serial = RCLASS_SERIAL(klass); klass = rb_make_metaclass(obj, klass); + RCLASS_SERIAL(klass) = serial; } if (OBJ_TAINTED(obj)) { https://github.com/tenderlove/ruby/tree/singleton-serial
180. Benchmark class C1 def m; 1; end end o1 = C1.new o2 = C1.new o2.singleton_class i = 0 while i<6_000_000 # benchmark loop 2 o = (i % 2 == 0) ? o1 : o2 o.m; o.m; o.m; o.m; o.m; o.m; o.m; o.m i += 1 end
181. Run it! [aaron@TC ruby (trunk)]$ time ./ruby benchmark/ bm_vm2_poly_singleton.rb real 0m2.510s user 0m2.453s sys 0m0.024s [aaron@TC ruby (singleton-serial)]$ time ./ruby benchmark/ bm_vm2_poly_singleton.rb real 0m1.380s user 0m1.319s sys 0m0.020s
182. 45% faster!
183. Trade off: Complexity for Speed
184. Memory Efficiency
185. Copy on Write (a.k.a. CoW) Milk! It’s what’s for dinner Advertisement:
186. Heap Layout Optimizations
187. Copy on Write Parent Process Child Process Child Process Memory
188. Copy on Write Parent Process Child Process Child Process Memory Memory
189. "Page Fault"
190. OS copies some of the memory
191. Ruby Memory Layout Pages Object Object Object Object Object Object Object Object Object Object Object Object
192. GC Happens Pages Object Object Object Object Object Object Object Object Object Object FREE! FREE! FREE! FREE! Object Object
193. Objects Don’t Move
194. Swiss Cheese Memory Pages Object FREE! FREE! Object FREE! FREE! Object Object Object Object Object Object
195. CoW wastefulness Pages Object FREE! FREE! Object FREE! FREE! Object Object Object Object Object Object Object Parent Child Copied to Child
196. OS Page Size 4k Ruby Page Size 16k Ruby Object Size 40 bytes
197. 1 Ruby Page: ~400 Objects
198. 1 Ruby Page: 4 OS Pages
199. 1 OS Page: ~100 Ruby Objects
200. We’re probably copying too many objects. Try JRuby: it’s Ruby with a J Advertisement:
201. This only impacts Ruby code that forks.
202. Unicorn Ugh. This is the web server we use.
203. Sideways Ruby Page
204. Sideways Ruby Page Imaginary Ruby Page (Slab)
205. Pages dedicated to "probably old" objects
206. What are "probably old" objects?
207. Probably old class Foo end module Bar end BAZ = "OMGOMG!!"
208. Tradeoffs?
209. Complexity for memory?
210. How complex?
211. How much memory?
212. Tools for introspection
213. github.com/tenderlove/heapfrag Sponsored link:
215. Wrap it up!!
216. 5 lessons
217. What do I value? What should I measure How can I measure it Does it provide shareholder value?
219. 1. What do I value?
220. 2. What should I measure?
221. 3. How can I measure it?
222. 4. Does it provide shareholder value? ActionCable: it’s a thing!! Advertisement:
223. Make Rails Great Again!