Code Charcuterie - Speaker Deck

Slide 1

Slide 1 text

ZOMG HAPPY FRIDAY!

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

Rails Barry

Slide 4

Slide 4 text

Do you have a Mac? Do you have a Wii?

Slide 5

Slide 5 text

DarwiinRemote

Slide 6

Slide 6 text

Aaron Patterson

Slide 7

Slide 7 text

@tenderlove

Slide 8

Slide 8 text

@tenderlove TWEET TO ME!

Slide 9

Slide 9 text

[email protected]

Slide 10

Slide 10 text

[email protected] EMAIL TO ME!

Slide 11

Slide 11 text

AT&T, AT&T logo and all AT&T related marks are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies.

Slide 12

Slide 12 text

Señor Software Engineer

Slide 13

Slide 13 text

ruby-core rails-core

Slide 14

Slide 14 text

WWFMD?

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

AWESOME LOGOS

Slide 25

Slide 25 text

STOP IT

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

NOTHING

Slide 29

Slide 29 text

Code Charcuterie

Slide 30

Slide 30 text

Amateur Charcuterier (sp?)

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

Tight Coupling?

Slide 36

Slide 36 text

Setting Expectations

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Instrumentation Concurrency Attributes

Slide 39

Slide 39 text

Number of Doubts Chart of Doubts

Slide 40

Slide 40 text

Number of Doubts Chart of Doubts Few Doubts

Slide 41

Slide 41 text

Number of Doubts Chart of Doubts Many Doubts

Slide 42

Slide 42 text

Instrumentation

Slide 43

Slide 43 text

Publishing ActiveSupport::Notifications.instrument("render") do render :text => "Foo" end

Slide 44

Slide 44 text

Subscribing events = [] ActiveSupport::Notifications.subscribe("render") do |*e| events << e end

Slide 45

Slide 45 text

def instrument(name, payload={}) started = Time.now begin yield ensure @notifier.publish(name, started, Time.now, @id, payload) end end

Slide 46

Slide 46 text

def instrument(name, payload={}) started = Time.now begin yield ensure @notifier.publish(name, started, Time.now, @id, payload) end end Number of Doubts

Slide 47

Slide 47 text

my time + children

Slide 48

Slide 48 text

-> sql executed -> view rendered Events Received

Slide 49

Slide 49 text

-> sql executed -> view rendered Events Received Number of Doubts

Slide 50

Slide 50 text

Missing Information

Slide 51

Slide 51 text

-> start rendering -> start execute SQL <- finish execute SQL -> start execute SQL <- finish execute SQL <- finish rendering

Slide 52

Slide 52 text

Rendering SQL SQL

Slide 53

Slide 53 text

class Listener def start(event) end def finish(event) end end listener = Listener.new AS::Notifications.subscribe("render", listener)

Slide 54

Slide 54 text

Evented Publishing def instrument(name, payload={}) @notifier.start(name, @id, payload) begin yield ensure @notifier.finish(name, @id, payload) end end

Slide 55

Slide 55 text

Make it Quack class Timed def initialize(pattern, delegate) @delegate = delegate @timestack = Hash.new { |h,id| h[id] = Hash.new { |ids,name| ids[name] = [] } } end def start(name, id, payload) @timestack[id][name].push Time.now end def finish(name, id, payload) started = @timestack[id][name].pop @delegate.call(name, started, Time.now, id, payload) end end

Slide 56

Slide 56 text

Tiny Factory def subscribe(pattern = nil, listener = Proc.new) if listener.respond_to?(:call) subscriber = Timed.new pattern, listener else subscriber = listener end @subscribers << subscriber end

Slide 57

Slide 57 text

Our Reward

Slide 58

Slide 58 text

class Indented def initialize @indentation = 0 end def start name, id, payload print indent puts "-> #{name} -- #{payload[:virtual_path]}" @indentation += 1 end def finish name, id, payload @indentation -= 1 print indent puts "<- #{name} -- #{payload[:virtual_path]}" end private def indent " " * @indentation end end ActiveSupport::Notifications.notifier.subscribe(nil, Indented.new)

Slide 59

Slide 59 text

-> process_action.action_controller -- -> render_template.action_view -- -> !render_template.action_view -- sessions/index <- !render_template.action_view -- sessions/index <- render_template.action_view -- -> !render_template.action_view -- layouts/application -> render_partial.action_view -- -> !render_template.action_view -- layouts/_navigation <- !render_template.action_view -- layouts/_navigation <- render_partial.action_view -- -> render_partial.action_view -- -> !render_template.action_view -- sessions/_slim_login <- !render_template.action_view -- sessions/_slim_login <- render_partial.action_view -- <- !render_template.action_view -- layouts/application <- process_action.action_controller --

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

ml !render_template.action_view (4.564 ms) virtual_path layouts/application render_partial.action_view (0.346 ms) identiﬁer /app/views/layouts/_navigation render_partial.action_view (0.133 ms) identiﬁer /app/views/sessions/_logout.ht !render_template.action_view (0.291 ms) virtual_path layouts/_navigation !render_template.action_view (0.087 ms) virtual_path sessions/_logout _l

Slide 62

Slide 62 text

tion_controller (0.021 ms) MessagesController index GET /messages process_action.action_controller (95.463 ms) controller MessagesController action index method GET path /messages sql.active_record (5.671 ms) sql SELECT "messages"."id" AS t0_r0, "messages"."address_id" AS t0_r1, "messages"."from" AS t0_r2, "messages"."to" AS t0_r3, "messages"."disposable" AS t0_r4, "messages"."subject" AS t0_r5, "messages"."body" AS t0_r6, "messages"."plain" AS t0_r7, "messages"."html" AS t0_r8, "messages"."created_at" AS t0_r9, "messages"."updated_at" AS t0_r10, "messages"."parsed_message_id" AS t0_r11, "addresses"."id" AS t1_r0, "addresses"."name" AS t1_r1, "addresses"."person_id" AS t1_r2, "addresses"."created_at" AS t1_r3, "addresses"."updated_at" AS t1_r4 FROM "messages" LEFT render_template.action_view (63.632 ms) identiﬁer /app/views/messages/index.html layout layouts/application

Slide 63

Slide 63 text

Concurrency

Slide 64

Slide 64 text

Amdahl's Law

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

laptop = Laptop.new 1 10.times { laptop.execute { sleep(1) } }

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

= 1

Slide 69

Slide 69 text

$ time ruby test.rb real 0m10.046s user 0m0.023s sys 0m0.008s $

Slide 70

Slide 70 text

laptop = Laptop.new ∞ 10.times { laptop.execute { sleep(1) } }

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

= 10

Slide 73

Slide 73 text

$ time ruby test.rb real 0m1.033s user 0m0.023s sys 0m0.008s $

Slide 74

Slide 74 text

Modeling Our Machine

Slide 75

Slide 75 text

class Sleeper def run; sleep(1); end end q = Queue.new 10.times { q.push Sleeper.new } 10.times { q.push nil } # 10 cores! wow! 10.times.map { Thread.new { while obj = q.pop obj.run end } }.each(&:join)

Slide 76

Slide 76 text

What is my P?

Slide 77

Slide 77 text

class Sleeper def run; sleep(1); end end q = SlowQueue.new # make `pop` cost more 10.times { q.push Sleeper.new } 10.times { q.push nil } # 10 cores! wow! 10.times.map { Thread.new { while obj = q.pop obj.run end } }.each(&:join)

Slide 78

Slide 78 text

$ time ruby test.rb real 0m2.153s user 0m0.027s sys 0m0.012s

Slide 79

Slide 79 text

Hidden Sequential Code

Slide 80

Slide 80 text

Starvation

Slide 81

Slide 81 text

Scarce Resources

Slide 82

Slide 82 text

No content

Slide 83

Slide 83 text

ME EBI

Slide 84

Slide 84 text

ME EBI

Slide 85

Slide 85 text

Connection Pool

Slide 86

Slide 86 text

Connections Are Not Thread Safe

Slide 87

Slide 87 text

One to One

Slide 88

Slide 88 text

One to One Number of Doubts

Slide 89

Slide 89 text

pool.connection pool = ActiveRecord::Base.connection_pool p pool.connection.object_id # => 1234 p pool.connection.object_id # => 1234

Slide 90

Slide 90 text

Starvation 5.times { Thread.new { pool.connection } } Thread.new { pool.connection }

Slide 91

Slide 91 text

Starvation 5.times { Thread.new { pool.connection } } Thread.new { pool.connection }

Slide 92

Slide 92 text

Queueing

Slide 93

Slide 93 text

Splitting Work cpu_queue = Queue.new io_queue = Queue.new 5.times { Thread.new { io_queue.pop.run } } 10.times { Thread.new { cpu_queue.pop.run } }

Slide 94

Slide 94 text

Consumer class Saver def initialize(record) @record = record end def run @record.save! end end

Slide 95

Slide 95 text

Producer class CreatePost def initialize(q) @q = q end def run post = Post.new(:name => 'aaron') @q.push SavePost.new(post) end end

Slide 96

Slide 96 text

No content

Slide 97

Slide 97 text

Parallel CPU def fib n if n < 3 1 else fib(n-1) + fib(n-2) end end 2.times.map { fib(34) }

Slide 98

Slide 98 text

$ time ruby test.rb real 0m3.315s user 0m3.264s sys 0m0.010s

Slide 99

Slide 99 text

2.times.map { Thread.new { fib(34) } }.each(&:join) Parallelize

Slide 100

Slide 100 text

Caught in the GIL $ time ruby test.rb real 0m4.825s user 0m3.823s sys 0m1.978s

Slide 101

Slide 101 text

Slow I/O Server server = GenericServer.new('127.0.0.1') server.start do |socket| sleep 0.5 socket.print "HELLO" socket.close end

Slide 102

Slide 102 text

TCP Client require 'socket' 5.times.map { sock = TCPSocket.new '127.0.0.1', 28561 puts sock.read }

Slide 103

Slide 103 text

$ time ruby client.rb HELLO HELLO HELLO HELLO HELLO real 0m2.546s user 0m0.026s sys 0m0.009s

Slide 104

Slide 104 text

require 'socket' 5.times.map { Thread.new { sock = TCPSocket.new '127.0.0.1', 28561 puts sock.read } }.each(&:join) Parallelize

Slide 105

Slide 105 text

$ time ruby client.rb HELLO HELLO HELLO HELLO HELLO real 0m0.552s user 0m0.028s sys 0m0.009s

Slide 106

Slide 106 text

Sending Email!

Slide 107

Slide 107 text

class UserController < ApplicationController def create user = User.create!(params[:user]) UserMailer.deliver_welcome_email(user) end end

Slide 108

Slide 108 text

class UserController < ApplicationController def create user = User.create!(params[:user]) UserMailer.deliver_welcome_email(user) end end

Slide 109

Slide 109 text

class UserController < ApplicationController def create user = User.create!(params[:user]) UserMailer.deliver_welcome_email(user) end end Number of Doubts

Slide 110

Slide 110 text

class MailSender def initialize(id) @id = id end def run user = User.find(@id) UserMailer.deliver_welcome_email(user) end end

Slide 111

Slide 111 text

class UserController < ApplicationController def create user = User.create!(params[:user]) mail_queue.push MailSender.new(user.id) end end

Slide 112

Slide 112 text

Thread.new { loop do mail_queue.pop.run } }

Slide 113

Slide 113 text

No content

Slide 114

Slide 114 text

Outside the loop More Parallel

Slide 115

Slide 115 text

You are ENCOURAGED to use threads

Slide 116

Slide 116 text

I/O can be parallelized CPU cannot * in MRI

Slide 117

Slide 117 text

Attribute Accessors

Slide 118

Slide 118 text

Attribute Access post = Post.new(:title => 'hello!') post.title # => 'hello!' post.respond_to?(:title) # => true

Slide 119

Slide 119 text

Lazily Deﬁned begin p Post.instance_method(:name) rescue NameError puts "oh no!" end Post.new(:name => 'aaron') p Post.instance_method(:name)

Slide 120

Slide 120 text

Output $ ruby -I lib:test test/cases/argh.rb oh no! #)#__temp__>

Slide 121

Slide 121 text

Deﬁnition Time def method_missing(method, *args, &block) unless self.class.attribute_methods_generated? self.class.define_attribute_methods ... end end def respond_to?(name, include_private = false) unless self.class.attribute_methods_generated? self.class.define_attribute_methods end super end

Slide 122

Slide 122 text

Who called respond_to? {:method=>"name="} ########################################################################################## lib/active_record/attribute_assignment.rb:81:in `block in assign_attributes' lib/active_record/attribute_assignment.rb:78:in èach' lib/active_record/attribute_assignment.rb:78:in àssign_attributes' lib/active_record/base.rb:498:in ìnitialize' test/cases/argh.rb:16:in `new' test/cases/argh.rb:16:in `' ########################################################################################## {:method=>"_run__1290908416815974468__initialize__2045902499019596289__callbacks"} ########################################################################################## lib/active_support/callbacks.rb:398:in `__run_callback' lib/active_support/callbacks.rb:385:in `_run_initialize_callbacks' lib/active_support/callbacks.rb:81:in `run_callbacks' lib/active_record/base.rb:501:in ìnitialize' test/cases/argh.rb:16:in `new' test/cases/argh.rb:16:in `' ########################################################################################## Number of Doubts

Slide 123

Slide 123 text

after_initialize class Subject < ActiveRecord::Base after_initialize :set_email_address protected def set_email_address ... end end

Slide 124

Slide 124 text

use super class Subject < ActiveRecord::Base def initialize(*args) super set_email_address # after_init end protected def set_email_address ... end end Number of Doubts

Slide 125

Slide 125 text

DO IT AGAIN! 10.times { Post.new(:name => 'aaron') }

Slide 126

Slide 126 text

What method? {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} {:method=>"name="} {:method=>"_run__1048083580144971540__initialize__3082064574212577540__callbacks"} Number of Doubts

Slide 127

Slide 127 text

Deﬁne Methods def define_attribute_methods .... @attribute_methods_mutex.synchronize do return if attribute_methods_generated? superclass.define_attribute_methods unless self == base_class super(column_names) column_names.each { |name| define_external_attribute_method(name) } @attribute_methods_generated = true end end

Slide 128

Slide 128 text

Slide 129

Slide 129 text

RAWWWRRRR class A def hi(stuff) end end class B < A def hi super(%w{ problem? }) end end

Slide 130

Slide 130 text

:'( def calls_hi(obj) obj.hi(%w{ how are you? }) end calls_hi(A.new) # => OK! calls_hi(B.new) # => Boom!

Slide 131

Slide 131 text

IF THEN B is an A B can be used as an A

Slide 132

Slide 132 text

Number of Doubts

Slide 133

Slide 133 text

deﬁne_attribute_methods

Slide 134

Slide 134 text

ActiveModel

Slide 135

Slide 135 text

deﬁne_attribute_methods def define_attribute_methods(attr_names) attr_names.each { |attr_name| define_attribute_method(attr_name) } end

Slide 136

Slide 136 text

deﬁne_attribute_methods def define_attribute_methods(attr_names) attr_names.each { |attr_name| define_attribute_method(attr_name) } end THANKS ACTIVE MODEL

Slide 137

Slide 137 text

deﬁne_attribute_method def define_attribute_method(attr_name) ... unless instance_method_already_implemented?(method_name) generate_method = "define_method_#{matcher.method_missing_target}" if respond_to?(generate_method, true) send(generate_method, attr_name) else define_optimized_call generated_attribute_methods, method_name, matcher.method_missing_target, attr_name.to_s end end end

Slide 138

Slide 138 text

Slide 139

Slide 139 text

Slide 140

Slide 140 text

Slide 141

Slide 141 text

Slide 142

Slide 142 text

WHY AREN'T THEY ALL OPTIMIZED????

Slide 143

Slide 143 text

define_method_attribute define_method_attribute= define_method_attribute_before_type_cast define_method_attribute? define_method_attribute_changed? define_method_attribute_change define_method_attribute_will_change! define_method_attribute_was define_method_reset_attribute! define_method__attribute define_method_attribute define_method_attribute= define_method_attribute_before_type_cast define_method_attribute? define_method_attribute_changed? define_method_attribute_change define_method_attribute_will_change! define_method_attribute_was define_method_reset_attribute! define_method__attribute

Slide 144

Slide 144 text

Slide 145

Slide 145 text

Slide 146

Slide 146 text

Slide 147

Slide 147 text

Number of Doubts

Slide 148

Slide 148 text

Side Note: Does anyone USE this?

Slide 149

Slide 149 text

Side Note: Should you use this?

Slide 150

Slide 150 text

Where were we?

Slide 151

Slide 151 text

Post.new respond_to? AR define_attr_methods AM define_attr_methods AM define_attr_method AR define_method_attr

Slide 152

Slide 152 text

In VIM •Ctrl-] •Ctrl-o •:help ctags

Slide 153

Slide 153 text

def define_method_attribute(attr_name) generated_attribute_methods.module_eval <<-STR, __FILE__, __LINE__ + 1 def __temp__ #{internal_attribute_access_code(attr_name, attribute_cast_code(attr_name))} end alias_method '#{attr_name}', :__temp__ undef_method :__temp__ STR end

Slide 154

Slide 154 text

method(:name).source

Slide 155

Slide 155 text

Type Cast Value Lookup post.method(:title).source

Slide 156

Slide 156 text

Post#name def name attr_name = 'name' unless @attributes.has_key?(attr_name) missing_attribute(attr_name, caller) end v = @attributes[attr_name] && v end

Slide 157

Slide 157 text

Post#name def name attr_name = 'name' unless @attributes.has_key?(attr_name) missing_attribute(attr_name, caller) end v = @attributes[attr_name] && v end SEEMS LEGIT

Slide 158

Slide 158 text

Post#created_at def created_at attr_name = 'created_at' unless @attributes.has_key?(attr_name) @attributes_cache[attr_name] ||= (missing_attribute(attr_name, caller) end (v=@attributes[attr_name]) && ActiveRecord::ConnectionAdapters::SQLiteColumn.string_to_time(v)) end

Slide 159

Slide 159 text

Slide 160

Slide 160 text

attribute_cast_code def attribute_cast_code(attr_name) columns_hash[attr_name].type_cast_code('v') end

Slide 161

Slide 161 text

type_cast_code def type_cast_code(var_name) klass = self.class.name case type when :string, :text then var_name when :integer then "(#{var_name}.to_i rescue #{var_name} ? 1 : 0)" when :float then "#{var_name}.to_f" when :decimal then "#{klass}.value_to_decimal(#{var_name})" when :datetime, :timestamp then "#{klass}.string_to_time(#{var_name})" when :time then "#{klass}.string_to_dummy_time(#{var_name})" when :date then "#{klass}.string_to_date(#{var_name})" when :binary then "#{klass}.binary_to_string(#{var_name})" when :boolean then "#{klass}.value_to_boolean(#{var_name})" else var_name end end

Slide 162

Slide 162 text

type_cast def type_cast(value) return nil if value.nil? return coder.load(value) if encoded? klass = self.class case type when :string, :text then value when :integer then value.to_i rescue value ? 1 : 0 when :float then value.to_f when :decimal then klass.value_to_decimal(value) when :datetime, :timestamp then klass.string_to_time(value) when :time then klass.string_to_dummy_time(value) when :date then klass.string_to_date(value) when :binary then klass.binary_to_string(value) when :boolean then klass.value_to_boolean(value) else value end end

Slide 163

Slide 163 text

Number of Doubts

Slide 164

Slide 164 text

Number of Doubts TOO MANY!

Slide 165

Slide 165 text

HOWTO FIX?

Slide 166

Slide 166 text

What do we want? •Less dynamic code •Consistent method deﬁnitions •Matching speed (even on read_attribute)

Slide 167

Slide 167 text

What does it do? •Attribute lookup •Column lookup •Value typecasting •Typecast value caching

Slide 168

Slide 168 text

def read_attribute(attr_name) # If it's cached, just return it @attributes_cache.fetch(attr_name.to_s) { |name| column = @columns_hash[name] value = @attributes.fetch(name) { return attribute_missing(name, caller) } # Cache if we're supposed to if self.class.cache_attribute?(name) @attributes_cache[name] = column.type_cast(value) else column.type_cast value end } end