Yak shaving is best shaving

Madison Ruby 2013

Transcript

1. Yak shaving is best shaving
2. Hello World!
3. Aaron Patterson @tenderlove
7. Rails Core Ruby Core
8. AT&T, AT&T logo and all AT&T related marks are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies.
9. ap433d@att.com
10. Madison!!!!!
11. #madruby13
13. I am very HHonored to be here
20. collagehumor.com
26. REDACTED
28. 2 problems.
29. It goes all the way up
30. Black plastic seat
31. What is Yak Shaving?
32. Oh No! A Bug! User.select(“10 as n”).n # => “10”
33. Let’s fix it!!
34. Run time / Security
35. AR Methods require 'active_record' ActiveRecord::Base.establish_connection 'sqlite3:///:memory:' class User < ActiveRecord::Base connection.create_table :users do |t| t.string :name end end
36. AR Methods begin User.instance_method(:name) rescue NameError puts "no method found!" end User.create! :name => 'foo' p User.instance_method :name
37. Output $ ruby methods.rb no method found! #<UnboundMethod: User(#<Module: 0x007fe9db9e20d8>)#name(__temp__e616d656)> $
38. source_location def define_method_attribute(name) safe_name = name.unpack('h*').first generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__ + 1 def __temp__#{safe_name} read_attribute(:'#{name.inspect}') { |n| missing_attribute(n, caller) } end alias_method #{name.inspect}, :__temp__#{safe_name} undef_method :__temp__#{safe_name} STR end
39. source_location def define_method_attribute(name) safe_name = name.unpack('h*').first generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__ + 1 def __temp__#{safe_name} read_attribute(:'#{name.inspect}') { |n| missing_attribute(n, caller) } end alias_method #{name.inspect}, :__temp__#{safe_name} undef_method :__temp__#{safe_name} STR end original name
40. source_location def define_method_attribute(name) safe_name = name.unpack('h*').first generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__ + 1 def __temp__#{safe_name} read_attribute(:'#{name.inspect}') { |n| missing_attribute(n, caller) } end alias_method #{name.inspect}, :__temp__#{safe_name} undef_method :__temp__#{safe_name} STR end real name
41. Simplify read_attribute(:'#{name.inspect}')
42. Be CAREFUL loop { :"#{SecureRandom.hex}" }
43. Dangerous Access user = User.find user[params[:some_value]]
44. Use a string read_attribute('#{name.inspect}')
45. Super Simplify read_attribute('name')
46. Under the hood >> insns = RubyVM::InstructionSequence.new "read_attribute(:name)" => <RubyVM::InstructionSequence:<compiled>@<compiled>> >> puts insns.disasm; nil == disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>========== 0000 trace 1 ( 1) 0002 putself 0003 putobject :name 0005 opt_send_simple <callinfo!mid:read_attribute, argc:1, FCALL|ARGS_SKIP> 0007 leave SYM B O L
47. Under the hood >> insns = RubyVM::InstructionSequence.new "read_attribute('name')" => <RubyVM::InstructionSequence:<compiled>@<compiled>> >> puts insns.disasm; nil == disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>========== 0000 trace 1 ( 1) 0002 putself 0003 putstring "name" 0005 opt_send_simple <callinfo!mid:read_attribute, argc:1, FCALL|ARGS_SKIP> 0007 leave STR IN G
48. Difference 0003 putobject :name 0003 putstring "name"
49. Instruction diff >> a = RubyVM::InstructionSequence.new( ?> 'read_attribute("name")') => <RubyVM::InstructionSequence:<compiled>@<compiled>> >> b = RubyVM::InstructionSequence.new( ?> 'read_attribute(:"name")') => <RubyVM::InstructionSequence:<compiled>@<compiled>> >> a.to_a.last - b.to_a.last => [[:putstring, "name"]] >> b.to_a.last - a.to_a.last => [[:putobject, :name]]
50. putobject putobject (VALUE val) () (VALUE val) { /* */ } insns.def
51. putstring putstring (VALUE str) () (VALUE val) { val = rb_str_resurrect(str); } insns.def
52. rb_str_resurrect VALUE rb_str_resurrect(VALUE str) { if (RUBY_DTRACE_STRING_CREATE_ENABLED()) { RUBY_DTRACE_STRING_CREATE(RSTRING_LEN(str), rb_sourcefile(), rb_sourceline()); } return str_replace(str_alloc(rb_cString), str); } string.c
53. rb_str_resurrect VALUE rb_str_resurrect(VALUE str) { if (RUBY_DTRACE_STRING_CREATE_ENABLED()) { RUBY_DTRACE_STRING_CREATE(RSTRING_LEN(str), rb_sourcefile(), rb_sourceline()); } return str_replace(str_alloc(rb_cString), str); } string.c
54. >> 10.times { ?> puts "foo".object_id >> } 70351655497760 70351655497540 70351655497360 70351655497300 70351655497120 70351655496900 70351655496760 70351655496640 70351655496380 70351655496320
55. In Ruby, strings are mutable
56. Best of both worlds?
57. Use a constant! NAME = "name" loop do read_attribute NAME end
58. Under the hood >> ins = RubyVM::InstructionSequence.new 'read_attribute(CONSTANT)' => <RubyVM::InstructionSequence:<compiled>@<compiled>> >> puts ins.disasm; nil == disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>========== 0000 trace 1 ( 1) 0002 putself 0003 getinlinecache 10, <ic:0> 0006 getconstant :CONSTANT 0008 setinlinecache <ic:0> 0010 opt_send_simple <callinfo!mid:read_attribute, argc:1, FCALL|ARGS_SKIP> 0012 leave C O N STA N T
59. getconstant getconstant (ID id) (VALUE klass) (VALUE val) { val = vm_get_ev_const(th, GET_ISEQ(), klass, id, 0); } insns.def
60. rb_const_get_0 st_lookup(RCLASS_CONST_TBL(tmp), (st_data_t)id, &data)) variable.c
61. No new allocations >> NAME = "foo" => "foo" >> 10.times { puts NAME.object_id } 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800
62. Benchmark
63. class Thing def string read_attribute "name" end def symbol read_attribute :name end NAME = "name" def constant read_attribute NAME end private def read_attribute n; end end
64. Benchmark.ips do |x| t = Thing.new x.report("constant") { t.constant } x.report("symbol") { t.symbol } x.report("string") { t.string } end
65. Calculating ------------------------------------- constant 86936 i/100ms symbol 89678 i/100ms string 80625 i/100ms ------------------------------------------------- constant 4635426.5 (±5.9%) i/s - 23124976 in 5.011890s symbol 4617404.5 (±5.7%) i/s - 23047246 in 5.011275s string 3529063.8 (±7.2%) i/s - 17576250 in 5.016706s Result
66. Change Implementation
67. Today def define_method_attribute(name) generated_methods::AttrNames.const_set "ATTR_#{name}", name generated_methods.module_eval <<-STR def #{name} read_attribute(AttrNames::ATTR_#{name}) end STR end
68. Speed of symbols. Safety of strings.
69. Memory Consumption (and boot time)
70. We want to generate the methods via module_eval rather than define_method, because define_method is slower on dispatch and uses more memory (because it creates a closure).
71. Benchmark class Thing define_method(:defnd) { } module_eval <<-eos def evald; end eos end Benchmark.ips do |x| t = Thing.new x.report("defnd") { t.defnd } x.report("evald") { t.evald } end
72. Results Calculating ------------------------------------- defnd 83910 i/100ms evald 89965 i/100ms ------------------------------------------------- defnd 4704667.3 (±6.8%) i/s - 23410890 in 5.008203s evald 6548961.6 (±6.8%) i/s - 32567330 in 5.004078s
73. Documentation === Implementation from Module ------------------------------------------------------------------------------ define_method(symbol, method) -> new_method define_method(symbol) { block } -> proc ------------------------------------------------------------------------------
74. Benchmark class Thing def foo; end define_method(:defnd) { } define_method(:defnd2, instance_method(:foo)) module_eval <<-eos def evald; end eos end Benchmark.ips do |x| t = Thing.new x.report("defnd") { t.defnd } x.report("defnd2") { t.defnd2 } x.report("evald") { t.evald } end
75. Results Calculating ------------------------------------- defnd 84511 i/100ms defnd2 91485 i/100ms evald 91643 i/100ms ------------------------------------------------- defnd 4034153.8 (±6.2%) i/s - 20113618 in 5.009757s defnd2 6549218.1 (±5.6%) i/s - 32660145 in 5.007169s evald 6522551.2 (±6.7%) i/s - 32441622 in 5.003291s
76. Is module_eval faster than define_method?
77. Is module_eval faster than define_method? IT DEPENDS
78. Refactor our comment
79. We want to generate the methods via module_eval rather than define_method, because define_method is slower on dispatch and uses more memory (because it creates a closure).
80. We want to generate the methods via module_eval rather than define_method, because define_method uses more memory (because it creates a closure).
81. We want to generate the methods via module_eval rather than define_method, because define_method uses more memory (because it creates a closure).
82. Closure reference class Thing def self.make_method thing define_method(:defnd) { } end end Thing.make_method nil Thing.make_method("X" * 90000)
83. Memory test class Thing N = 100_000 if ENV['USE_EVAL'] N.times { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end puts "DONE!" sleep
85. Which uses more memory?
86. Which uses more memory? IT DEPENDS
87. Why so much? require 'objspace' class Thing N = 100_000 if ENV['USE_EVAL'] N.times { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end p ObjectSpace.memsize_of_all(RubyVM::InstructionSequence)
88. Results $ ruby omg.rb 721128 $ USE_EVAL=1 ruby omg.rb 45981336
89. Refactor our comment
90. We want to generate the methods via module_eval rather than define_method, because define_method uses more memory (because it creates a closure).
91. We want to generate the methods via module_eval rather than define_method, because.
92. We want to generate the methods via module_eval rather than define_method, because.
93. Boot time
94. Benchmark class Thing N = 100_000 if ENV['USE_EVAL'] N.times { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end
95. Result $ time USE_EVAL=1 ruby omg.rb real 0m3.629s user 0m3.530s sys 0m0.091s $ time ruby omg.rb real 0m0.576s user 0m0.521s sys 0m0.047s 3.6s vs 0.6s!
96. Memory define_method < module_eval
97. Boot Time define_method < module_eval
98. Run Time define_method { } < module_eval
99. But Aaron...
100. define_method class Thing def foo; end define_method(:defnd2, instance_method(:foo)) end
101. Attribute Def def define_method_attribute(name) attribute_methods.module_eval <<-STR def #{name} read_attribute(ATTR_#{name}) end STR end
102. Method Transplanting
103. Transplant class A def a; end end class B define_method :a, A.instance_method(:a) end
104. Error $ ruby omg.rb omg.rb:6:in `define_method': bind argument must be a subclass of A (TypeError) from omg.rb:6:in `<class:B>' from omg.rb:5:in `<main>'
105. Module Transplant module AllMethods def name; end def created_at; end # .. etc .. end
106. Module Transplant class B methods = Module.new do define_method :name, AllMethods.instance_method(:name) end include methods end
107. Define all accessors, then transplant
108. create_model = lambda { # Simulate many different models, but just share the same table # so we don't need to make many different tables Class.new(ActiveRecord::Base) { self.table_name = 'users' } } user_model = create_model.call user = user_model.create!(name: 'Aaron') x = [] growth = [10, 90, 900].map { |i| i.times { model = create_model.call x << model model.find(user.id).name } [x.size, ObjectSpace.memsize_of_all(RubyVM::InstructionSequence) - current] }
109. 100000 1000000 10000000 100000000 10 100 1000 ISeq Bytes Number of Models Rails 4.0 Rails Master
110. Rails Master: 18kb /m Rails 4: 31kb /m
111. Definition Speed Benchmark.ips do |x| id = user.id x.report("create model") do model = create_model.call model.find(id).name end end
112. Results [aaron@higgins rails (4-0-stable)]$ bundle exec ruby omg.rb Calculating ------------------------------------- create model 8 i/100ms ------------------------------------------------- create model 85.9 (±11.6%) i/s - 424 in 5.006028s [aaron@higgins rails (master)]$ bundle exec ruby omg.rb Calculating ------------------------------------- create model 17 i/100ms ------------------------------------------------- create model 165.5 (±13.3%) i/s - 816 in 5.020841s
113. Remaining Problems:
114. 100% cache hit, but memory still grows.
115. Only works on Ruby 2.0
116. Recap
117. Querying
118. Querying Fixing a Bug Security Performance Instructions Ruby Source
119. Fixing bugs.
121. *don’t worry, the bug was fixed.
122. Don’t believe everything you read.
123. Yak shaving can be fun and educational.
124. Just don’t get TOO lost.
125. Thank you! <3<3
128. ͜Μʹͪ͸ɺ Θ͕ͨࣾ͠௕Ͱ͢ɻ *Hello, I’m the boss
130. Gorbachev Puff Puff Thuderhorse The Third
137. basket Top View basket
139. basket Top View basket basket basket
140. basket Top View basket basket basket
142. WHY???