Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Yak shaving is best shaving

Yak shaving is best shaving

Madison Ruby 2013

F29327647a9cff5c69618bae420792ea?s=128

Aaron Patterson

August 25, 2013
Tweet

Transcript

  1. Yak shaving is best shaving

  2. Hello World!

  3. Aaron Patterson @tenderlove

  4. None
  5. None
  6. None
  7. Rails Core Ruby Core

  8. AT&T, AT&T logo and all AT&T related marks are trademarks

    of AT&T Intellectual Property and/or AT&T affiliated companies.
  9. ap433d@att.com

  10. Madison!!!!!

  11. #madruby13

  12. None
  13. I am very HHonored to be here

  14. None
  15. None
  16. None
  17. None
  18. None
  19. None
  20. collagehumor.com

  21. None
  22. None
  23. None
  24. None
  25. None
  26. REDACTED

  27. None
  28. 2 problems.

  29. It goes all the way up

  30. Black plastic seat

  31. What is Yak Shaving?

  32. Oh No! A Bug! User.select(“10 as n”).n # => “10”

  33. Let’s fix it!!

  34. Run time / Security

  35. AR Methods require 'active_record' ActiveRecord::Base.establish_connection 'sqlite3:///:memory:' class User < ActiveRecord::Base

    connection.create_table :users do |t| t.string :name end end
  36. AR Methods begin User.instance_method(:name) rescue NameError puts "no method found!"

    end User.create! :name => 'foo' p User.instance_method :name
  37. Output $ ruby methods.rb no method found! #<UnboundMethod: User(#<Module: 0x007fe9db9e20d8>)#name(__temp__e616d656)>

    $
  38. source_location def define_method_attribute(name) safe_name = name.unpack('h*').first generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__

    + 1 def __temp__#{safe_name} read_attribute(:'#{name.inspect}') { |n| missing_attribute(n, caller) } end alias_method #{name.inspect}, :__temp__#{safe_name} undef_method :__temp__#{safe_name} STR end
  39. source_location def define_method_attribute(name) safe_name = name.unpack('h*').first generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__

    + 1 def __temp__#{safe_name} read_attribute(:'#{name.inspect}') { |n| missing_attribute(n, caller) } end alias_method #{name.inspect}, :__temp__#{safe_name} undef_method :__temp__#{safe_name} STR end original name
  40. source_location def define_method_attribute(name) safe_name = name.unpack('h*').first generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__

    + 1 def __temp__#{safe_name} read_attribute(:'#{name.inspect}') { |n| missing_attribute(n, caller) } end alias_method #{name.inspect}, :__temp__#{safe_name} undef_method :__temp__#{safe_name} STR end real name
  41. Simplify read_attribute(:'#{name.inspect}')

  42. Be CAREFUL loop { :"#{SecureRandom.hex}" }

  43. Dangerous Access user = User.find user[params[:some_value]]

  44. Use a string read_attribute('#{name.inspect}')

  45. Super Simplify read_attribute('name')

  46. Under the hood >> insns = RubyVM::InstructionSequence.new "read_attribute(:name)" => <RubyVM::InstructionSequence:<compiled>@<compiled>>

    >> puts insns.disasm; nil == disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>========== 0000 trace 1 ( 1) 0002 putself 0003 putobject :name 0005 opt_send_simple <callinfo!mid:read_attribute, argc:1, FCALL|ARGS_SKIP> 0007 leave SYM B O L
  47. Under the hood >> insns = RubyVM::InstructionSequence.new "read_attribute('name')" => <RubyVM::InstructionSequence:<compiled>@<compiled>>

    >> puts insns.disasm; nil == disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>========== 0000 trace 1 ( 1) 0002 putself 0003 putstring "name" 0005 opt_send_simple <callinfo!mid:read_attribute, argc:1, FCALL|ARGS_SKIP> 0007 leave STR IN G
  48. Difference 0003 putobject :name 0003 putstring "name"

  49. Instruction diff >> a = RubyVM::InstructionSequence.new( ?> 'read_attribute("name")') => <RubyVM::InstructionSequence:<compiled>@<compiled>>

    >> b = RubyVM::InstructionSequence.new( ?> 'read_attribute(:"name")') => <RubyVM::InstructionSequence:<compiled>@<compiled>> >> a.to_a.last - b.to_a.last => [[:putstring, "name"]] >> b.to_a.last - a.to_a.last => [[:putobject, :name]]
  50. putobject putobject (VALUE val) () (VALUE val) { /* */

    } insns.def
  51. putstring putstring (VALUE str) () (VALUE val) { val =

    rb_str_resurrect(str); } insns.def
  52. rb_str_resurrect VALUE rb_str_resurrect(VALUE str) { if (RUBY_DTRACE_STRING_CREATE_ENABLED()) { RUBY_DTRACE_STRING_CREATE(RSTRING_LEN(str), rb_sourcefile(),

    rb_sourceline()); } return str_replace(str_alloc(rb_cString), str); } string.c
  53. rb_str_resurrect VALUE rb_str_resurrect(VALUE str) { if (RUBY_DTRACE_STRING_CREATE_ENABLED()) { RUBY_DTRACE_STRING_CREATE(RSTRING_LEN(str), rb_sourcefile(),

    rb_sourceline()); } return str_replace(str_alloc(rb_cString), str); } string.c
  54. >> 10.times { ?> puts "foo".object_id >> } 70351655497760 70351655497540

    70351655497360 70351655497300 70351655497120 70351655496900 70351655496760 70351655496640 70351655496380 70351655496320
  55. In Ruby, strings are mutable

  56. Best of both worlds?

  57. Use a constant! NAME = "name" loop do read_attribute NAME

    end
  58. Under the hood >> ins = RubyVM::InstructionSequence.new 'read_attribute(CONSTANT)' => <RubyVM::InstructionSequence:<compiled>@<compiled>>

    >> puts ins.disasm; nil == disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>========== 0000 trace 1 ( 1) 0002 putself 0003 getinlinecache 10, <ic:0> 0006 getconstant :CONSTANT 0008 setinlinecache <ic:0> 0010 opt_send_simple <callinfo!mid:read_attribute, argc:1, FCALL|ARGS_SKIP> 0012 leave C O N STA N T
  59. getconstant getconstant (ID id) (VALUE klass) (VALUE val) { val

    = vm_get_ev_const(th, GET_ISEQ(), klass, id, 0); } insns.def
  60. rb_const_get_0 st_lookup(RCLASS_CONST_TBL(tmp), (st_data_t)id, &data)) variable.c

  61. No new allocations >> NAME = "foo" => "foo" >>

    10.times { puts NAME.object_id } 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800
  62. Benchmark

  63. class Thing def string read_attribute "name" end def symbol read_attribute

    :name end NAME = "name" def constant read_attribute NAME end private def read_attribute n; end end
  64. Benchmark.ips do |x| t = Thing.new x.report("constant") { t.constant }

    x.report("symbol") { t.symbol } x.report("string") { t.string } end
  65. Calculating ------------------------------------- constant 86936 i/100ms symbol 89678 i/100ms string 80625

    i/100ms ------------------------------------------------- constant 4635426.5 (±5.9%) i/s - 23124976 in 5.011890s symbol 4617404.5 (±5.7%) i/s - 23047246 in 5.011275s string 3529063.8 (±7.2%) i/s - 17576250 in 5.016706s Result
  66. Change Implementation

  67. Today def define_method_attribute(name) generated_methods::AttrNames.const_set "ATTR_#{name}", name generated_methods.module_eval <<-STR def #{name}

    read_attribute(AttrNames::ATTR_#{name}) end STR end
  68. Speed of symbols. Safety of strings.

  69. Memory Consumption (and boot time)

  70. We want to generate the methods via module_eval rather than

    define_method, because define_method is slower on dispatch and uses more memory (because it creates a closure).
  71. Benchmark class Thing define_method(:defnd) { } module_eval <<-eos def evald;

    end eos end Benchmark.ips do |x| t = Thing.new x.report("defnd") { t.defnd } x.report("evald") { t.evald } end
  72. Results Calculating ------------------------------------- defnd 83910 i/100ms evald 89965 i/100ms -------------------------------------------------

    defnd 4704667.3 (±6.8%) i/s - 23410890 in 5.008203s evald 6548961.6 (±6.8%) i/s - 32567330 in 5.004078s
  73. Documentation === Implementation from Module ------------------------------------------------------------------------------ define_method(symbol, method) -> new_method

    define_method(symbol) { block } -> proc ------------------------------------------------------------------------------
  74. Benchmark class Thing def foo; end define_method(:defnd) { } define_method(:defnd2,

    instance_method(:foo)) module_eval <<-eos def evald; end eos end Benchmark.ips do |x| t = Thing.new x.report("defnd") { t.defnd } x.report("defnd2") { t.defnd2 } x.report("evald") { t.evald } end
  75. Results Calculating ------------------------------------- defnd 84511 i/100ms defnd2 91485 i/100ms evald

    91643 i/100ms ------------------------------------------------- defnd 4034153.8 (±6.2%) i/s - 20113618 in 5.009757s defnd2 6549218.1 (±5.6%) i/s - 32660145 in 5.007169s evald 6522551.2 (±6.7%) i/s - 32441622 in 5.003291s
  76. Is module_eval faster than define_method?

  77. Is module_eval faster than define_method? IT DEPENDS

  78. Refactor our comment

  79. We want to generate the methods via module_eval rather than

    define_method, because define_method is slower on dispatch and uses more memory (because it creates a closure).
  80. We want to generate the methods via module_eval rather than

    define_method, because define_method uses more memory (because it creates a closure).
  81. We want to generate the methods via module_eval rather than

    define_method, because define_method uses more memory (because it creates a closure).
  82. Closure reference class Thing def self.make_method thing define_method(:defnd) { }

    end end Thing.make_method nil Thing.make_method("X" * 90000)
  83. Memory test class Thing N = 100_000 if ENV['USE_EVAL'] N.times

    { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end puts "DONE!" sleep
  84. None
  85. Which uses more memory?

  86. Which uses more memory? IT DEPENDS

  87. Why so much? require 'objspace' class Thing N = 100_000

    if ENV['USE_EVAL'] N.times { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end p ObjectSpace.memsize_of_all(RubyVM::InstructionSequence)
  88. Results $ ruby omg.rb 721128 $ USE_EVAL=1 ruby omg.rb 45981336

  89. Refactor our comment

  90. We want to generate the methods via module_eval rather than

    define_method, because define_method uses more memory (because it creates a closure).
  91. We want to generate the methods via module_eval rather than

    define_method, because.
  92. We want to generate the methods via module_eval rather than

    define_method, because.
  93. Boot time

  94. Benchmark class Thing N = 100_000 if ENV['USE_EVAL'] N.times {

    |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end
  95. Result $ time USE_EVAL=1 ruby omg.rb real 0m3.629s user 0m3.530s

    sys 0m0.091s $ time ruby omg.rb real 0m0.576s user 0m0.521s sys 0m0.047s 3.6s vs 0.6s!
  96. Memory define_method < module_eval

  97. Boot Time define_method < module_eval

  98. Run Time define_method { } < module_eval

  99. But Aaron...

  100. define_method class Thing def foo; end define_method(:defnd2, instance_method(:foo)) end

  101. Attribute Def def define_method_attribute(name) attribute_methods.module_eval <<-STR def #{name} read_attribute(ATTR_#{name}) end

    STR end
  102. Method Transplanting

  103. Transplant class A def a; end end class B define_method

    :a, A.instance_method(:a) end
  104. Error $ ruby omg.rb omg.rb:6:in `define_method': bind argument must be

    a subclass of A (TypeError) from omg.rb:6:in `<class:B>' from omg.rb:5:in `<main>'
  105. Module Transplant module AllMethods def name; end def created_at; end

    # .. etc .. end
  106. Module Transplant class B methods = Module.new do define_method :name,

    AllMethods.instance_method(:name) end include methods end
  107. Define all accessors, then transplant

  108. create_model = lambda { # Simulate many different models, but

    just share the same table # so we don't need to make many different tables Class.new(ActiveRecord::Base) { self.table_name = 'users' } } user_model = create_model.call user = user_model.create!(name: 'Aaron') x = [] growth = [10, 90, 900].map { |i| i.times { model = create_model.call x << model model.find(user.id).name } [x.size, ObjectSpace.memsize_of_all(RubyVM::InstructionSequence) - current] }
  109. 100000 1000000 10000000 100000000 10 100 1000 ISeq Bytes Number

    of Models Rails 4.0 Rails Master
  110. Rails Master: 18kb /m Rails 4: 31kb /m

  111. Definition Speed Benchmark.ips do |x| id = user.id x.report("create model")

    do model = create_model.call model.find(id).name end end
  112. Results [aaron@higgins rails (4-0-stable)]$ bundle exec ruby omg.rb Calculating -------------------------------------

    create model 8 i/100ms ------------------------------------------------- create model 85.9 (±11.6%) i/s - 424 in 5.006028s [aaron@higgins rails (master)]$ bundle exec ruby omg.rb Calculating ------------------------------------- create model 17 i/100ms ------------------------------------------------- create model 165.5 (±13.3%) i/s - 816 in 5.020841s
  113. Remaining Problems:

  114. 100% cache hit, but memory still grows.

  115. Only works on Ruby 2.0

  116. Recap

  117. Querying

  118. Querying Fixing a Bug Security Performance Instructions Ruby Source

  119. Fixing bugs.

  120. None
  121. *don’t worry, the bug was fixed.

  122. Don’t believe everything you read.

  123. Yak shaving can be fun and educational.

  124. Just don’t get TOO lost.

  125. Thank you! <3<3

  126. None
  127. None
  128. ͜Μʹͪ͸ɺ Θ͕ͨࣾ͠௕Ͱ͢ɻ *Hello, I’m the boss

  129. None
  130. Gorbachev Puff Puff Thuderhorse The Third

  131. None
  132. None
  133. None
  134. None
  135. None
  136. None
  137. basket Top View basket

  138. None
  139. basket Top View basket basket basket

  140. basket Top View basket basket basket

  141. None
  142. WHY???