RailsConf 2014 Keynote

My RailsConf 2014 keynote slides.

Transcript

1. Welcome To Rail Sconf! @tenderlove
2. "My mom has the Ferrari of sewing machines" — @heddle317
3. "Dad Gummit" — Bill Dance
4. Aaron Patterson
5. @tenderlove
8. "Tenderlove’s Seal of Approval" — Nathan Long @sleeplessgeek
9. Ruby Core
10. Rails Core
11. AT&T, AT&T logo and all AT&T related marks are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies.
12. Job Titles • 2011: Corey Haines • 2012: Senior Facebook Integration Engineer Elect • 2013: Senior Software Architect
13. This Year: Thought Leader, In Training
15. Action Shots
18. Two Cats
19. SeaTac! YouTube! FaceBook! Instagram
20. Gorbachev! Puff Puff! Thunderhorse! The Third @gorbypuff
22. @tenderlove
23. My Tweets 3% 5% 2% 90% Puns Hugs Cat Photos Tech Stuff
25. Chicago Stuff
26. Da Bears
27. Da Bulls
28. Mike Ditka
29. Last Year…
30. ☑Adequate Everything. Adequately.
31. F- F- F- F- F- F- F- F- F- F- F-
32. Awkward Situations
33. This story is rated PG-13 for adult situations
34. Rails is 10 years old!!
37. ☑ Naming
38. ☑ Caching
39. ☑ Dependencies
40. "The genius behind … npm is that it solves dependencies, a longtime problem in computer science." — Npm, Inc. COO Rod Boothby http://venturebeat.com/2014/02/11/former-node-leader-takes-big-money-launches-node-startup/
41. Get your act together, RubyGems.
42. What’s left?
43. P = NP
44. |N| = 1
45. I think we’re done here.
46. Anti-Keynote
47. Coverage
48. Speed
49. Metrics
50. Hard Science fing ^
51. Things I’ve Learned At RailsConf
53. Listen To Me. ! Science is good. TDD is Great.
56. Troll Fast Troll Furious
58. Great Workout For Your Eye Muscles.
59. Eyeroll Muscles
60. OMG HOW TO TEST? class Person < ActiveRecord::Base def birthyear? Date.today.year == birthyear end end
61. Pass in today class Person < ActiveRecord::Base def birthyear?(today = Date.today) today.year == birthyear end end ! assert person.birthyear?(Date.today) STAHP
62. omg so wow! class Person < ActiveRecord::Base def birthyear? Date.today.year == birthyear end end ! person = Person.new birthyear: Date.today.year assert person.birthyear? ! person = Person.new birthyear: 1980 assert_not person.birthyear?
63. No New API Necessary!
64. Constructors are pretty cool!
65. Make Construction Easy
66. Controversial Opinions
67. I am a Software Engineer
68. I am not a "Software Writer"
69. [Code] beauty is in the eye of the beholder
70. Science is important. I can’t believe I actually had to say this.
71. Mutable State
72. Rails.has_many :bugs — @a_matsuda
73. Inconsistent Values sp = ActionController::Parameters.new({ foo:[Object.new] }) ! sp.fetch(:foo).equal?(sp[:foo]) # => false sp.fetch(:foo).equal?(sp[:foo]) # => true lolw ut
75. Memory Leaks params = ActionController::Parameters.new({ foo:[Object.new] }) ! loop do params.fetch(:foo) params.delete :foo params[:foo] = [Object.new] end
76. Yikes!
77. Leak Source def convert_value_to_parameters(value) if value.is_a?(Array) && !converted_arrays.member? (value) converted = value.map { |_| convert_value_to_parameters(_) } converted_arrays << converted converted elsif value.is_a?(Parameters) || !value.is_a?(Hash) value else self.class.new(value) end end Mutation
78. class HashWithIndifferentAccess < Hash end ! class Parameters < HashWithIndifferentAccess end
80. Inconsistent Return Person.create name: "teelo green" ! relation = Person.group(:id) ! relation.size # => {1=>1} relation.to_a # => [#<Person id: 1, name: "teelo">] relation.size # => 1 wut? wut?
81. Cause def size loaded? ? @records.length : count(:all) end Executes! G RO UP Returns # of records
83. wtf SQL courtesy of @eileencodes
84. Guess the SQL! author = authors :david 100.times { |i| author.thinking_posts.create!(:body => i.to_s) } author.thinking_posts.delete_all
85. Output SQL DELETE FROM \"posts\" WHERE \"posts\".\"author_id\" = ?
86. Guess the SQL! author = authors :david 100.times { |i| author.thinking_posts.create!(:body => i.to_s) } author.thinking_posts.inspect author.thinking_posts.delete_all Inspect first
87. Output SQL DELETE FROM \"posts\" WHERE \"posts\".\"author_id \" = ? AND \"posts\".\"id\" IN (112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211)
88. Cause if loaded? || dependent == :destroy delete_or_destroy(load_target, dependent) else delete_records(:all, dependent) end
90. Avoid Subclassing Built-in’s
91. Be Conservative About Your API
92. Be Extremely Careful With Mutations
93. I Ruby
94. AdequateRecord
95. What is "Adequate Record"?
96. What people are saying:
97. "It’s like ActiveRecord, but more Adequate" — Aaron Patterson, Rails Core Team
98. "It’s alright. I’ve seen better, but this is fine." — Jeremy Kemper, Rails Core Team
99. "It seems to work" — Yehuda Katz, Ember Core Team
100. "Sometimes good enough is good enough" — Josh Susser, Ruby Rogue
101. "Huh? What is that?" — Yukihiro Matz, Creator of Ruby
102. Why "Adequate"?
103. Humility.
104. Anti-Marketing
105. Why a fork?
106. I don’t want new features.
107. Adequate History
108. 3 years.
109. Adequate?
110. How ActiveRecord works.
111. ActiveRecord::Relation Post.find(10) ARel::SQL::Node Database ActiveRecord::Base SELECT * FROM …
112. Adequate Change: Part 1
113. Bind Parameter Introduction
114. Post.find(1) Post.find(3) Post.find(5) SELECT * FROM posts WHERE id = 1 SELECT * FROM posts WHERE id = 3 SELECT * FROM posts WHERE id = 5
115. Post.find(1) Post.find(3) Post.find(5) SELECT * FROM posts WHERE id = ? [id, 1] SELECT * FROM posts WHERE id = ? [id, 3] SELECT * FROM posts WHERE id = ? [id, 5]
116. Separate static and dynamic content
117. Adequate Theory
118. ActiveRecord::Relation Post.find(10) ARel::SQL::Node Database ActiveRecord::Base SELECT * FROM … Cacheable
119. Adequate Change: Part 2
120. Code Decoupling
121. def add_constraints(scope) tables = construct_tables ! chain.each_with_index do |reflection, i| table, foreign_table = tables.shift, tables.first ! if reflection.source_macro == :has_and_belongs_to_many join_table = tables.shift ! scope = scope.joins(join( join_table, table[reflection.association_primary_key]. eq(join_table[reflection.association_foreign_key]) )) ! table, foreign_table = join_table, tables.first end ! if reflection.source_macro == :belongs_to if reflection.options[:polymorphic] key = reflection.association_primary_key(klass) else key = reflection.association_primary_key end ! foreign_key = reflection.foreign_key else key = reflection.foreign_key foreign_key = reflection.active_record_primary_key end ! if reflection == chain.last bind_val = bind scope, table.table_name, key.to_s, owner[foreign_key] scope = scope.where(table[key].eq(bind_val)) ! if reflection.type value = owner.class.base_class.name bind_val = bind scope, table.table_name, reflection.type.to_s, value scope = scope.where(table[reflection.type].eq(bind_val)) end else constraint = table[key].eq(foreign_table[foreign_key]) ! if reflection.type type = chain[i + 1].klass.base_class.name constraint = constraint.and(table[reflection.type].eq(type)) end ! scope = scope.joins(join(foreign_table, constraint)) end ! # Exclude the scope of the association itself, because that # was already merged in the #scope method. (scope_chain[i] - [self.reflection.scope]).each do |scope_chain_item| item = eval_scope(reflection.klass, scope_chain_item) ! scope.includes! item.includes_values scope.where_values += item.where_values end end ! scope end Dynamic Dynamic Dynamic
122. One Method to Rule Them All, And In Legacy Code Bind Them.
123. rm habtm R ails 4.1
124. habtm.is_a?(hm:t) #=> true
125. commit 88c009377851912c60fd16ec4bfab3001ac2cf9f Author: Aaron Patterson <aaron.patterson@gmail.com> Date: Wed Oct 2 15:53:56 2013 -0700 ! remove HABTM special cases from associations classes ! diff --git a/activerecord/lib/active_record/associations/association_scope.rb b/activerecord/lib/active_record/associations/ association_scope.rb index 8027acf..d862a5f 100644 --- a/activerecord/lib/active_record/associations/association_scope.rb +++ b/activerecord/lib/active_record/associations/association_scope.rb @@ -44,18 +44,6 @@ module ActiveRecord chain.each_with_index do |reflection, i| table, foreign_table = tables.shift, tables.first - if reflection.source_macro == :has_and_belongs_to_many - join_table = tables.shift - - scope = scope.joins(join( - join_table, - table[reflection.association_primary_key]. - eq(join_table[reflection.association_foreign_key]) - )) - - table, foreign_table = join_table, tables.first - end - if reflection.source_macro == :belongs_to if reflection.options[:polymorphic] key = reflection.association_primary_key(self.klass) diff --git a/activerecord/lib/active_record/associations/join_dependency.rb b/activerecord/lib/active_record/associations/ join_dependency.rb index 5aa17e5..fa212f3 100644 --- a/activerecord/lib/active_record/associations/join_dependency.rb +++ b/activerecord/lib/active_record/associations/join_dependency.rb @@ -216,7 +216,7 @@ module ActiveRecord else association = join_part.instantiate(row) unless row[join_part.aliased_primary_key].nil? case macro - when :has_many, :has_and_belongs_to_many + when :has_many other = record.association(join_part.reflection.name) other.loaded! other.target.push(association) if association diff --git a/activerecord/lib/active_record/associations/join_dependency/join_association.rb b/activerecord/lib/
127. Adequate Change: Part 3
128. Introduce a cache
129. Cache Code Example cache = ActiveRecord::StatementCache.new do |params| Person.where(name: params.bind).limit(1) end ! cache.execute ["Aaron"] cache.execute ["Ebi"] AR::Relation Execute
130. Cache Object Internals #<ActiveRecord::StatementCache:0x007fb8522a09b0 @bind_map= #<BindMap:0x007fb8522a2490 @bind_values= [[#<SQLite3Column:0x007fb852384f98>, #<Substitute:0x007fb8522c35a0>]], @indexes=[0]>, @query_builder= #<Query:0x007fb8522a09d8 @sql= "SELECT \"people\".* FROM \"people\" WHERE \"people\".\"name\" = ? LIMIT 1">> Binds C om piled SQ L
131. Update Internals, Cache Relation Objects
132. Things That Use Relations • Post.find() • Post.find_by_* • has_many • has_many :through • has_and_belongs_to_many • belongs_to All Cacheable
133. Example Implementation
134. Person.find(id) s = find_by_statement_cache.synchronize { find_by_statement_cache[key] ||= StatementCache.create(connection) { |params| where(primary_key => params.bind).limit(1) } } record = s.execute([id], self, connection).first R elation! (static) Parameter! (dynamic)
135. Fun Facts!
136. Ways to call `Post.find` • Post.find(1) • Post.find([1]) • Post.find([[1]]) • Post.find({1 => 1}) • Post.find(post) • Post.find(1) { … } • scoping { Post.find(1) } Cacheable wtf WHAT DO THESE EVEN MEAN???
138. Rejection Code def find(*ids) # We don't have cache keys for this stuff yet return super unless ids.length == 1 return super if block_given? || primary_key.nil? || default_scopes.any? || columns_hash.include?(inheritance_column) || ids.first.kind_of?(Array)
139. Measuring Performance
140. benchmark/ips
141. benchmark Benchmark.bm do |x| x.report('some test') { N.times { some_test }} end How big?
142. Output user system total real some test 0.000000 0.000000 0.000000 ( 0.000098)
143. benchmark/ips Benchmark.ips do |x| x.report('some test') { some_test } end
144. Output Calculating ------------------------------------- some test 114186 i/100ms ------------------------------------------------- some test 7677966.1 (±3.0%) i/s - 38366496 in 5.002452s
145. Sttdev is important
146. GC.stat()
147. Total allocations GC.stat(:total_allocated_object)
148. Measure Allocations Person.find id before = GC.stat(:total_allocated_object) N.times { Person.find id } after = GC.stat(:total_allocated_object) puts (after - before) / N W arm up Benchmark Objects / Call Count O bjs C ount O bjs
149. Performance of AdequateRecord
150. Post.find() Post.find_by_name()
151. `Post.find` Calls Per Second over Time Calls Per Second (higher is better) 0 4000 8000 12000 16000 2-3-stable 3-0-stable 3-1-stable 3-2-stable 4-0-stable 4-1-stable master adequaterecord SQLite3 MySQL2 MySQL PostgreSQL
152. `Post.find_by_name` Calls Per Second over Time Calls Per Second (higher is better) 0 3500 7000 10500 14000 2-3-stable 3-0-stable 3-1-stable 3-2-stable 4-0-stable 4-1-stable master adequaterecord SQLite3 MySQL2 MySQL PostgreSQL
153. Calls Per Second (MySQL) Calls Per Second (higher is better) 0 2000 4000 6000 8000 2-3-stable 3-0-stable 3-1-stable 3-2-stable 4-0-stable 4-1-stable master adequaterecord find by id find by name
154. % Faster than 4-1-stable 0.00% 45.00% 90.00% 135.00% 180.00% SQLite3 MySQL2 MySQL PostgreSQL find by id find by name
155. % Faster than 2-3-stable 0.00% 35.00% 70.00% 105.00% 140.00% SQLite3 MySQL PostgreSQL find by id find by name
156. Objects Allocated Per Call (find by id) Objects Allocated (lower is better) 0 75 150 225 300 2-3-stable 3-0-stable 3-1-stable 3-2-stable 4-0-stable 4-1-stable master adequate SQLite3 MySQL2 MySQL PostgreSQL
157. Object Allocated Per Call (find by name) Objects Allocated (lower is better) 0 75 150 225 300 2-3-stable 3-0-stable 3-1-stable 3-2-stable 4-0-stable 4-1-stable master adequate SQLite3 MySQL2 MySQL PostgreSQL
158. 70% Fewer Objects Compared to 4-1-stable
159. 55% Fewer Objects Compared to 2-3-stable
160. belongs_to
161. belongs_to calls per second Calls Per Second (higher is better) 0 3000 6000 9000 12000 2-3-stable 3-0-stable 3-1-stable 3-2-stable 4-0-stable 4-1-stable master adequaterecord SQLite3 MySQL MySQL2 PostgreSQL
162. belongs_to Percent Faster 0.00% 45.00% 90.00% 135.00% 180.00% SQLite3 MySQL MySQL2 PostgreSQL 2-3-stable 4-1-stable
163. has_many has_many :through
164. `has_many` Call Speed Over Time Calls Per Second (higher is better) 0 2750 5500 8250 11000 2-3-stable 3-0-stable 4-1-stable master adequaterecord SQlite3 MySQL MySQL2 PostgreSQL
165. `hm:t` Call Speed Over Time Calls Per Second (higher is better) 0 3000 6000 9000 12000 2-3-stable 3-0-stable 4-1-stable master adequaterecord SQLite3 MySQL MySQL2 PostgreSQL
166. Percent Faster than 2-3-stable 0% 25% 50% 75% 100% SQLite3 MySQL PostgreSQL has_many hm:t
167. Percent Faster than 4-1-stable 0% 65% 130% 195% 260% SQLite3 MySQL MySQL2 PostgreSQL has_many hm:t
168. has_many :through growth
169. Growth Test class A < ActiveRecord::Base has_many :bs has_many :cs, :through => :bs end ! class B < ActiveRecord::Base belongs_to :a has_many :cs end ! class C < ActiveRecord::Base belongs_to :b end
170. Growth Test class A < ActiveRecord::Base has_many :bs has_many :cs, :through => :bs has_many :ds, :through => :cs end ! class B < ActiveRecord::Base belongs_to :a has_many :cs end ! class C < ActiveRecord::Base belongs_to :b has_many :ds end ! class D < ActiveRecord::Base belongs_to :b end as we add more,! what’s the speed?
171. Time Taken to Call hm:t 100k times Seconds Taken (lower is better) 0 40 80 120 160 Nuber of hm:t associations 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 y = 0.5783x + 5.9815 R² = 0.9971 y = 7.448x + 16.506 R² = 0.9974 4-1-stable adequate AWWWWW YEAH, CONSTANT TIME!
172. TL;DR: ~100% faster
173. TL;DR: 9001% better
174. Adequate Pricing Model
175. $2.99 + in-app purchases. Act! Now!
176. Adequate Challenges
177. Trade Memory For Speed
178. Total Cache Size = count(find_by_*) * Size
179. Raw Relations
180. Example Controller class PeopleController def index @people = Person.where(name: params[:name]).to_a end end
181. Can we cache it?
182. Experimental Branch Person.where(name: params[:name]).to_a Iterations Per Second (higher is better) 0 2000 4000 6000 8000 where(:name) Master Experimental Branch
183. ~30% faster
184. Key Calculation def cached_query if [joins_values, having_values, group_values, order_values, select_values, ].any?(&:any?) return yield end ! if limit_value || offset_value || distinct_value || from_value || lock_value return yield end ! if where_values.length != bind_values.length return yield end ! key = if bind_values.any? "bv_#{bind_values.map { |bv| bv.first.name }.join}" else :find_all_stuff end ! connection = @klass.connection cache = @klass.find_by_statement_cache ! s = cache[key] || cache.synchronize { cache[key] ||= begin bvs = bind_values.map { |bv| bv = bv.dup bv[1] = StatementCache::Substitute.new bv } StatementCache.create_with_binds(connection, arel, bvs) end } s.execute bind_values.map(&:last), @klass, connection end Key Calculation Query Execution
185. Over 11 variables impact the key
186. This code only handles two types of queries
187. One more experiment
188. Finder Comparison Benchmark.ips do |x| x.report('Person.where') { Person.where(name: 'Aaron').first } x.report('Person.find_by') { Person.find_by_name('Aaron') } end
189. Finder Comparison Calls Per Second 0 3500 7000 10500 14000 Finder .where() .find_by_name
190. find_by_name is 3x faster
191. Finder Comparison Benchmark.ips do |x| x.report('Person.where') { Person.where(name: 'Aaron').first } x.report('Person.find_by') { Person.find_by_name('Aaron') } end New AR::Relation Allocation
192. Should we cache all relations?
193. I’m not sure.
194. I want a new API.
195. New API class Person < ActiveRecord::Base cached_query(:find_all_by_name) do |name| where(name: name) end end ! Person.find_all_by_name("foo") Person.find_all_by_name("bar") Build a! cached relation Execute it
196. Cache Key is Easy
197. Relation API maintained
198. –Steve Jobs “One other more thing.”
199. Let’s merge Adequate Record
201. AdequateRecord has been merged.
202. Rails 4.2 Will Be the Fastest Ever.
203. This is just the beginning.
204. Thank you!