Yak shaving is best shaving

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Hello World!

Slide 3

Slide 3 text

Aaron Patterson @tenderlove

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Rails Core Ruby Core

Slide 8

Slide 8 text

AT&T, AT&T logo and all AT&T related marks are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies.

Slide 9

Slide 9 text

[email protected]

Slide 10

Slide 10 text

Madison!!!!!

Slide 11

Slide 11 text

#madruby13

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

I am very HHonored to be here

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

collagehumor.com

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

REDACTED

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

2 problems.

Slide 29

Slide 29 text

It goes all the way up

Slide 30

Slide 30 text

Black plastic seat

Slide 31

Slide 31 text

What is Yak Shaving?

Slide 32

Slide 32 text

Oh No! A Bug! User.select(“10 as n”).n # => “10”

Slide 33

Slide 33 text

Let’s fix it!!

Slide 34

Slide 34 text

Run time / Security

Slide 35

Slide 35 text

AR Methods require 'active_record' ActiveRecord::Base.establish_connection 'sqlite3:///:memory:' class User < ActiveRecord::Base connection.create_table :users do |t| t.string :name end end

Slide 36

Slide 36 text

AR Methods begin User.instance_method(:name) rescue NameError puts "no method found!" end User.create! :name => 'foo' p User.instance_method :name

Slide 37

Slide 37 text

Output $ ruby methods.rb no method found! #)#name(__temp__e616d656)> $

Slide 38

Slide 38 text

source_location def define_method_attribute(name) safe_name = name.unpack('h*').first generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__ + 1 def __temp__#{safe_name} read_attribute(:'#{name.inspect}') { |n| missing_attribute(n, caller) } end alias_method #{name.inspect}, :__temp__#{safe_name} undef_method :__temp__#{safe_name} STR end

Slide 39

Slide 39 text

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Simplify read_attribute(:'#{name.inspect}')

Slide 42

Slide 42 text

Be CAREFUL loop { :"#{SecureRandom.hex}" }

Slide 43

Slide 43 text

Dangerous Access user = User.find user[params[:some_value]]

Slide 44

Slide 44 text

Use a string read_attribute('#{name.inspect}')

Slide 45

Slide 45 text

Super Simplify read_attribute('name')

Slide 46

Slide 46 text

Under the hood >> insns = RubyVM::InstructionSequence.new "read_attribute(:name)" => @> >> puts insns.disasm; nil == disasm: @>========== 0000 trace 1 ( 1) 0002 putself 0003 putobject :name 0005 opt_send_simple 0007 leave SYM B O L

Slide 47

Slide 47 text

Under the hood >> insns = RubyVM::InstructionSequence.new "read_attribute('name')" => @> >> puts insns.disasm; nil == disasm: @>========== 0000 trace 1 ( 1) 0002 putself 0003 putstring "name" 0005 opt_send_simple 0007 leave STR IN G

Slide 48

Slide 48 text

Difference 0003 putobject :name 0003 putstring "name"

Slide 49

Slide 49 text

Instruction diff >> a = RubyVM::InstructionSequence.new( ?> 'read_attribute("name")') => @> >> b = RubyVM::InstructionSequence.new( ?> 'read_attribute(:"name")') => @> >> a.to_a.last - b.to_a.last => [[:putstring, "name"]] >> b.to_a.last - a.to_a.last => [[:putobject, :name]]

Slide 50

Slide 50 text

putobject putobject (VALUE val) () (VALUE val) { /* */ } insns.def

Slide 51

Slide 51 text

putstring putstring (VALUE str) () (VALUE val) { val = rb_str_resurrect(str); } insns.def

Slide 52

Slide 52 text

rb_str_resurrect VALUE rb_str_resurrect(VALUE str) { if (RUBY_DTRACE_STRING_CREATE_ENABLED()) { RUBY_DTRACE_STRING_CREATE(RSTRING_LEN(str), rb_sourcefile(), rb_sourceline()); } return str_replace(str_alloc(rb_cString), str); } string.c

Slide 53

Slide 53 text

Slide 54

Slide 54 text

>> 10.times { ?> puts "foo".object_id >> } 70351655497760 70351655497540 70351655497360 70351655497300 70351655497120 70351655496900 70351655496760 70351655496640 70351655496380 70351655496320

Slide 55

Slide 55 text

In Ruby, strings are mutable

Slide 56

Slide 56 text

Best of both worlds?

Slide 57

Slide 57 text

Use a constant! NAME = "name" loop do read_attribute NAME end

Slide 58

Slide 58 text

Under the hood >> ins = RubyVM::InstructionSequence.new 'read_attribute(CONSTANT)' => @> >> puts ins.disasm; nil == disasm: @>========== 0000 trace 1 ( 1) 0002 putself 0003 getinlinecache 10, 0006 getconstant :CONSTANT 0008 setinlinecache 0010 opt_send_simple 0012 leave C O N STA N T

Slide 59

Slide 59 text

getconstant getconstant (ID id) (VALUE klass) (VALUE val) { val = vm_get_ev_const(th, GET_ISEQ(), klass, id, 0); } insns.def

Slide 60

Slide 60 text

rb_const_get_0 st_lookup(RCLASS_CONST_TBL(tmp), (st_data_t)id, &data)) variable.c

Slide 61

Slide 61 text

No new allocations >> NAME = "foo" => "foo" >> 10.times { puts NAME.object_id } 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800 70351655353800

Slide 62

Slide 62 text

Benchmark

Slide 63

Slide 63 text

class Thing def string read_attribute "name" end def symbol read_attribute :name end NAME = "name" def constant read_attribute NAME end private def read_attribute n; end end

Slide 64

Slide 64 text

Benchmark.ips do |x| t = Thing.new x.report("constant") { t.constant } x.report("symbol") { t.symbol } x.report("string") { t.string } end

Slide 65

Slide 65 text

Calculating ------------------------------------- constant 86936 i/100ms symbol 89678 i/100ms string 80625 i/100ms ------------------------------------------------- constant 4635426.5 (±5.9%) i/s - 23124976 in 5.011890s symbol 4617404.5 (±5.7%) i/s - 23047246 in 5.011275s string 3529063.8 (±7.2%) i/s - 17576250 in 5.016706s Result

Slide 66

Slide 66 text

Change Implementation

Slide 67

Slide 67 text

Today def define_method_attribute(name) generated_methods::AttrNames.const_set "ATTR_#{name}", name generated_methods.module_eval <<-STR def #{name} read_attribute(AttrNames::ATTR_#{name}) end STR end

Slide 68

Slide 68 text

Speed of symbols. Safety of strings.

Slide 69

Slide 69 text

Memory Consumption (and boot time)

Slide 70

Slide 70 text

We want to generate the methods via module_eval rather than deﬁne_method, because deﬁne_method is slower on dispatch and uses more memory (because it creates a closure).

Slide 71

Slide 71 text

Benchmark class Thing define_method(:defnd) { } module_eval <<-eos def evald; end eos end Benchmark.ips do |x| t = Thing.new x.report("defnd") { t.defnd } x.report("evald") { t.evald } end

Slide 72

Slide 72 text

Results Calculating ------------------------------------- defnd 83910 i/100ms evald 89965 i/100ms ------------------------------------------------- defnd 4704667.3 (±6.8%) i/s - 23410890 in 5.008203s evald 6548961.6 (±6.8%) i/s - 32567330 in 5.004078s

Slide 73

Slide 73 text

Documentation === Implementation from Module ------------------------------------------------------------------------------ define_method(symbol, method) -> new_method define_method(symbol) { block } -> proc ------------------------------------------------------------------------------

Slide 74

Slide 74 text

Benchmark class Thing def foo; end define_method(:defnd) { } define_method(:defnd2, instance_method(:foo)) module_eval <<-eos def evald; end eos end Benchmark.ips do |x| t = Thing.new x.report("defnd") { t.defnd } x.report("defnd2") { t.defnd2 } x.report("evald") { t.evald } end

Slide 75

Slide 75 text

Results Calculating ------------------------------------- defnd 84511 i/100ms defnd2 91485 i/100ms evald 91643 i/100ms ------------------------------------------------- defnd 4034153.8 (±6.2%) i/s - 20113618 in 5.009757s defnd2 6549218.1 (±5.6%) i/s - 32660145 in 5.007169s evald 6522551.2 (±6.7%) i/s - 32441622 in 5.003291s

Slide 76

Slide 76 text

Is module_eval faster than define_method?

Slide 77

Slide 77 text

Is module_eval faster than define_method? IT DEPENDS

Slide 78

Slide 78 text

Refactor our comment

Slide 79

Slide 79 text

We want to generate the methods via module_eval rather than deﬁne_method, because deﬁne_method is slower on dispatch and uses more memory (because it creates a closure).

Slide 80

Slide 80 text

We want to generate the methods via module_eval rather than deﬁne_method, because deﬁne_method uses more memory (because it creates a closure).

Slide 81

Slide 81 text

We want to generate the methods via module_eval rather than deﬁne_method, because deﬁne_method uses more memory (because it creates a closure).

Slide 82

Slide 82 text

Closure reference class Thing def self.make_method thing define_method(:defnd) { } end end Thing.make_method nil Thing.make_method("X" * 90000)

Slide 83

Slide 83 text

Memory test class Thing N = 100_000 if ENV['USE_EVAL'] N.times { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end puts "DONE!" sleep

Slide 84

Slide 84 text

No content

Slide 85

Slide 85 text

Which uses more memory?

Slide 86

Slide 86 text

Which uses more memory? IT DEPENDS

Slide 87

Slide 87 text

Why so much? require 'objspace' class Thing N = 100_000 if ENV['USE_EVAL'] N.times { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end p ObjectSpace.memsize_of_all(RubyVM::InstructionSequence)

Slide 88

Slide 88 text

Results $ ruby omg.rb 721128 $ USE_EVAL=1 ruby omg.rb 45981336

Slide 89

Slide 89 text

Refactor our comment

Slide 90

Slide 90 text

We want to generate the methods via module_eval rather than deﬁne_method, because deﬁne_method uses more memory (because it creates a closure).

Slide 91

Slide 91 text

We want to generate the methods via module_eval rather than deﬁne_method, because.

Slide 92

Slide 92 text

We want to generate the methods via module_eval rather than deﬁne_method, because.

Slide 93

Slide 93 text

Boot time

Slide 94

Slide 94 text

Benchmark class Thing N = 100_000 if ENV['USE_EVAL'] N.times { |i| module_eval "def foo_#{i}; end" } else N.times { |i| define_method("foo_#{i}") { } } end end

Slide 95

Slide 95 text

Result $ time USE_EVAL=1 ruby omg.rb real 0m3.629s user 0m3.530s sys 0m0.091s $ time ruby omg.rb real 0m0.576s user 0m0.521s sys 0m0.047s 3.6s vs 0.6s!

Slide 96

Slide 96 text

Memory deﬁne_method < module_eval

Slide 97

Slide 97 text

Boot Time deﬁne_method < module_eval

Slide 98

Slide 98 text

Run Time deﬁne_method { } < module_eval

Slide 99

Slide 99 text

But Aaron...

Slide 100

Slide 100 text

define_method class Thing def foo; end define_method(:defnd2, instance_method(:foo)) end

Slide 101

Slide 101 text

Attribute Def def define_method_attribute(name) attribute_methods.module_eval <<-STR def #{name} read_attribute(ATTR_#{name}) end STR end

Slide 102

Slide 102 text

Method Transplanting

Slide 103

Slide 103 text

Transplant class A def a; end end class B define_method :a, A.instance_method(:a) end

Slide 104

Slide 104 text

Error $ ruby omg.rb omg.rb:6:in `define_method': bind argument must be a subclass of A (TypeError) from omg.rb:6:in `' from omg.rb:5:in `'

Slide 105

Slide 105 text

Module Transplant module AllMethods def name; end def created_at; end # .. etc .. end

Slide 106

Slide 106 text

Module Transplant class B methods = Module.new do define_method :name, AllMethods.instance_method(:name) end include methods end

Slide 107

Slide 107 text

Define all accessors, then transplant

Slide 108

Slide 108 text

create_model = lambda { # Simulate many different models, but just share the same table # so we don't need to make many different tables Class.new(ActiveRecord::Base) { self.table_name = 'users' } } user_model = create_model.call user = user_model.create!(name: 'Aaron') x = [] growth = [10, 90, 900].map { |i| i.times { model = create_model.call x << model model.find(user.id).name } [x.size, ObjectSpace.memsize_of_all(RubyVM::InstructionSequence) - current] }

Slide 109

Slide 109 text

100000 1000000 10000000 100000000 10 100 1000 ISeq Bytes Number of Models Rails 4.0 Rails Master

Slide 110

Slide 110 text

Rails Master: 18kb /m Rails 4: 31kb /m

Slide 111

Slide 111 text

Definition Speed Benchmark.ips do |x| id = user.id x.report("create model") do model = create_model.call model.find(id).name end end

Slide 112

Slide 112 text

Results [aaron@higgins rails (4-0-stable)]$ bundle exec ruby omg.rb Calculating ------------------------------------- create model 8 i/100ms ------------------------------------------------- create model 85.9 (±11.6%) i/s - 424 in 5.006028s [aaron@higgins rails (master)]$ bundle exec ruby omg.rb Calculating ------------------------------------- create model 17 i/100ms ------------------------------------------------- create model 165.5 (±13.3%) i/s - 816 in 5.020841s