Extending Gems (OS Bridge 2014)

Slide 1

Slide 1 text

Extending Gems Jason Clark @jasonrclark Ruby Agent Engineer Patterns and Anti-Patterns of Pluggable Gems Thursday, June 26, 14

Slide 2

Slide 2 text

Storytime! Thursday, June 26, 14

Slide 3

Slide 3 text

newrelic_rpm module NewRelic::Agent::BrowserMonitoring def browser_monitoring_queue_time queue_time = current_transaction.queue_time millis = queue_time.to_f * 1000.0 clamp_to_positive(millis.round) end def footer_js_string(config) "...#{browser_monitoring_queue_time}" end end Thursday, June 26, 14

Slide 4

Slide 4 text

module NewRelic::Agent::BrowserMonitoring def current_timings TransactionTimings.new end def footer_js_string(config) "...#{current_timings.queue_time_in_millis}" end end newrelic_rpm Thursday, June 26, 14

Slide 5

Slide 5 text

Meanwhile... Elsewhere At New Relic http://ﬂic.kr/p/9xEUug Thursday, June 26, 14

Slide 6

Slide 6 text

module NewRelic::Agent::BrowserMonitoring # MONKEY PATCH! def footer_js_string(config) "...#{browser_monitoring_queue_time}" end end rpm_site Thursday, June 26, 14

Slide 7

Slide 7 text

http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Thursday, June 26, 14

Slide 8

Slide 8 text

Thursday, June 26, 14

Slide 9

Slide 9 text

Reskiq It forks then it threads! Thursday, June 26, 14

Slide 10

Slide 10 text

gem install my_awesome_code Thursday, June 26, 14

Slide 11

Slide 11 text

Where We're Going • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 12

Slide 12 text

Pass It On Thursday, June 26, 14

Slide 13

Slide 13 text

Loggers Thursday, June 26, 14

Slide 14

Slide 14 text

activerecord ActiveRecord::Base.logger = ::Logger.new("my_special.log") Thursday, June 26, 14

Slide 15

Slide 15 text

http://ﬂic.kr/p/7GDRue Dependency Injection Thursday, June 26, 14

Slide 16

Slide 16 text

http://ﬂic.kr/p/gzxjKu Duck Typing Thursday, June 26, 14

Slide 17

Slide 17 text

Instrumentors Thursday, June 26, 14

Slide 18

Slide 18 text

excon connection = Excon.new( 'http://nerd.jasonrclark.com', :instrumentor => SimpleInstrumentor ) Thursday, June 26, 14

Slide 19

Slide 19 text

excon class SimpleInstrumentor class << self def instrument(name, params = {}, &blk) puts "#{name} just happened." yield if block_given? end end end Thursday, June 26, 14

Slide 20

Slide 20 text

http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Thursday, June 26, 14

Slide 21

Slide 21 text

Backends Thursday, June 26, 14

Slide 22

Slide 22 text

Delayed::Job # without delayed_job @user.activate!(@device) # with delayed_job @user.delay.activate!(@device) Thursday, June 26, 14

Slide 23

Slide 23 text

Delayed::Job Thursday, June 26, 14

Slide 24

Slide 24 text

Delayed::Job ::Delayed.const_set(:Job, backend) Thursday, June 26, 14

Slide 25

Slide 25 text

newrelic_rpm def failed_jobs Delayed::Job.count( :conditions => 'failed_at is not NULL') end Thursday, June 26, 14

Slide 26

Slide 26 text

Gem-Speciﬁc Thursday, June 26, 14

Slide 27

Slide 27 text

Resque class SendMessageJob def self.perform(id, text) user = User.find(id) user.send_message(text) end end Resque.enqueue(SendMessageJob, 42, "Yo") Thursday, June 26, 14

Slide 28

Slide 28 text

Resque https://github.com/resque/resque/blob/master/docs/HOOKS.md class SendMessageJob def self.before_perform_log_it(*args) logger.debug("Messaging #{args}") end ... end Thursday, June 26, 14

Slide 29

Slide 29 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 30

Slide 30 text

Events Thursday, June 26, 14

Slide 31

Slide 31 text

http://www.confreaks.com/videos/3327-railsconf- make-an-event-of-it Thursday, June 26, 14

Slide 32

Slide 32 text

Resque Resque.after_fork do |job| ActiveRecord::Base.establish_connection end Thursday, June 26, 14

Slide 33

Slide 33 text

http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Thursday, June 26, 14

Slide 34

Slide 34 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 35

Slide 35 text

Middleware Thursday, June 26, 14

Slide 36

Slide 36 text

Rack::Cache ActionDispatch::Cookies YourApplication Web Request Thursday, June 26, 14

Slide 37

Slide 37 text

# [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Thursday, June 26, 14

Slide 38

Slide 38 text

# [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Thursday, June 26, 14

Slide 39

Slide 39 text

http://ﬂic.kr/p/SSGmF ActionDispatch::Static Rack::Lock Rack::Runtime Rack::MethodOverride ActionDispatch::RequestId Rails::Rack::Logger ActionDispatch::ShowExceptions ActionDispatch::DebugExceptions ActionDispatch::RemoteIp ActionDispatch::Reloader ActionDispatch::Callbacks ActiveRecord::ConnectionAdapters ActiveRecord::QueryCache ActionDispatch::Cookies ActionDispatch::Session::CookieStore ActionDispatch::Flash ActionDispatch::ParamsParser Rack::Head Rack::ConditionalGet Rack::ETag AgentSnoop::Middleware RpmTestApp::Application.routes Thursday, June 26, 14

Slide 40

Slide 40 text

Who's There? Thursday, June 26, 14

Slide 41

Slide 41 text

# [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Thursday, June 26, 14

Slide 42

Slide 42 text

class MyMiddleware def initialize(options={}) # options == { :foo => 1 } end def call(worker, msg, queue) yield end end Thursday, June 26, 14

Slide 43

Slide 43 text

Sidekiq.configure_server do |config| config.server_middleware do |chain| chain.add MyMiddleware, :foo => 1 end end Thursday, June 26, 14

Slide 44

Slide 44 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 45

Slide 45 text

Lifecycle Thursday, June 26, 14

Slide 46

Slide 46 text

Forks and Daemons http://ﬂic.kr/p/79tE5Q Thursday, June 26, 14

Slide 47

Slide 47 text

Forking fork() state Thursday, June 26, 14

Slide 48

Slide 48 text

fork() state fork() Forking state Thursday, June 26, 14

Slide 49

Slide 49 text

before_fork after_fork Thursday, June 26, 14

Slide 50

Slide 50 text

Add Threads fork() Thursday, June 26, 14

Slide 51

Slide 51 text

fork() Add Threads Thursday, June 26, 14

Slide 52

Slide 52 text

Enter the Lock fork() @lock Thursday, June 26, 14

Slide 53

Slide 53 text

@loick fork() Enter the Lock @lock Thursday, June 26, 14

Slide 54

Slide 54 text

https://github.com/resque/resque/issues/1101 Forks + Threads + == Locks Resque Thursday, June 26, 14

Slide 55

Slide 55 text

Load Up! Thursday, June 26, 14

Slide 56

Slide 56 text

unicorn ♥ὑὑ~/myapp:unicorn_rails -c ./unicorn.rb unicorn_rails master unicorn_rails worker[0] unicorn_rails worker[1] unicorn_rails worker[2] unicorn_rails worker[3] Thursday, June 26, 14

Slide 57

Slide 57 text

loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Thursday, June 26, 14

Slide 58

Slide 58 text

loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Thursday, June 26, 14

Slide 59

Slide 59 text

loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Thursday, June 26, 14

Slide 60

Slide 60 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 61

Slide 61 text

Names and Paths Thursday, June 26, 14

Slide 62

Slide 62 text

Forced Thursday, June 26, 14

Slide 63

Slide 63 text

Sequel.plugin :newrelic_instrumentation Thursday, June 26, 14

Slide 64

Slide 64 text

Sequel.plugin :newrelic_instrumentation => require 'sequel/plugins/newrelic_instrumentation' Thursday, June 26, 14

Slide 65

Slide 65 text

~/source/newrelic/ruby_agent/lib: ls -la drwxr-xr-x Sep 19 14:46 . drwxr-xr-x Oct 10 11:19 .. drwxr-xr-x Oct 10 11:20 new_relic -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb drwxr-xr-x Sep 19 14:46 sequel drwxr-xr-x Oct 7 08:53 tasks Thursday, June 26, 14

Slide 66

Slide 66 text

Slide 67

Slide 67 text

Sequel.plugin NewRelic::Sequel::Instrumentation Thursday, June 26, 14

Slide 68

Slide 68 text

Generic Thursday, June 26, 14

Slide 69

Slide 69 text

Thursday, June 26, 14

Slide 70

Slide 70 text

ActiveRecord Thursday, June 26, 14

Slide 71

Slide 71 text

ActiveRecord DataMapper Thursday, June 26, 14

Slide 72

Slide 72 text

ActiveRecord DataMapper HTTPClient Thursday, June 26, 14

Slide 73

Slide 73 text

Qualify Yourself Thursday, June 26, 14

Slide 74

Slide 74 text

newrelic_rpm module NewRelic module Agent class Thread < ::Thread ... end end end Thursday, June 26, 14

Slide 75

Slide 75 text

elsewhere... class Thread # Re-opens NewRelic::Agent::Thread # D'oh! end Thursday, June 26, 14

Slide 76

Slide 76 text

newrelic_plugin module NewRelic class Logger ... end end Thursday, June 26, 14

Slide 77

Slide 77 text

newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => Logger::DEBUG, ... } end end end Thursday, June 26, 14

Slide 78

Slide 78 text

newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => Logger::DEBUG, ... } end end end Thursday, June 26, 14

Slide 79

Slide 79 text

newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => ::Logger::DEBUG, ... } end end end Thursday, June 26, 14

Slide 80

Slide 80 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 81

Slide 81 text

Conﬁg Thursday, June 26, 14

Slide 82

Slide 82 text

yml Thursday, June 26, 14

Slide 83

Slide 83 text

yml + ERB Thursday, June 26, 14

Slide 84

Slide 84 text

yml + ERB file = File.read("./config/awesome_gem.yml") # Locals available in ERB default_awesome_key = "YEAH!" erb = ERB.new(file).result(binding) config = YAML.load(erb) Thursday, June 26, 14

Slide 85

Slide 85 text

yml + ERB ~/config/awesome_gem.yml development: name: <%= default_awesome_key %> license_key: <%= ENV["LICENSE_KEY"] %> Thursday, June 26, 14

Slide 86

Slide 86 text

Ruby Thursday, June 26, 14

Slide 87

Slide 87 text

unicorn worker_processes 5 preload_app true timeout 30 after_fork do |server, worker| defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection end Thursday, June 26, 14

Slide 88

Slide 88 text

on_worker_boot do ActiveSupport.on_load(:active_record) do ActiveRecord::Base.establish_connection end end Thursday, June 26, 14

Slide 89

Slide 89 text

Great for Apps? What about Gems? Thursday, June 26, 14

Slide 90

Slide 90 text

newrelic_rpm if defined?(::Puma) && ::Puma.respond_to?(:cli_config) config = ::Puma.cli_config config.options[:worker_boot] << Proc.new do NewRelic::Agent.after_fork(...) end end Thursday, June 26, 14

Slide 91

Slide 91 text

Slide 92

Slide 92 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 93

Slide 93 text

Docs Thursday, June 26, 14

Slide 94

Slide 94 text

README Thursday, June 26, 14

Slide 95

Slide 95 text

sinatra Thursday, June 26, 14

Slide 96

Slide 96 text

Thursday, June 26, 14

Slide 97

Slide 97 text

Thursday, June 26, 14

Slide 98

Slide 98 text

Thursday, June 26, 14

Slide 99

Slide 99 text

Expected Extensions? Thursday, June 26, 14

Slide 100

Slide 100 text

Thursday, June 26, 14

Slide 101

Slide 101 text

Don't Bury It Thursday, June 26, 14

Slide 102

Slide 102 text

Thursday, June 26, 14

Slide 103

Slide 103 text

Thursday, June 26, 14

Slide 104

Slide 104 text

Versioning Thursday, June 26, 14

Slide 105

Slide 105 text

Thursday, June 26, 14

Slide 106

Slide 106 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 107

Slide 107 text

Where We've Been Thursday, June 26, 14

Slide 108

Slide 108 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Thursday, June 26, 14

Slide 109

Slide 109 text

Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Conﬁg • Docs Jason Clark @jasonrclark Ruby Agent Engineer Thursday, June 26, 14