Slide 1

Slide 1 text

Extending Gems Patterns and Anti-Patterns of Making Your Gem Pluggable Jason Clark @jasonrclark Ruby Agent Engineer Monday, November 11, 13

Slide 2

Slide 2 text

2 Storytime! Monday, November 11, 13

Slide 3

Slide 3 text

3 newrelic_rpm module NewRelic::Agent::BrowserMonitoring def browser_monitoring_queue_time queue_time = current_transaction.queue_time millis = queue_time.to_f * 1000.0 clamp_to_positive(millis.round) end def footer_js_string(config) "... #{browser_monitoring_queue_time}" end end Monday, November 11, 13

Slide 4

Slide 4 text

4 module NewRelic::Agent::BrowserMonitoring def current_timings TransactionTimings.new end def footer_js_string(config) "... #{current_timings.queue_time_in_millis}" end end newrelic_rpm Monday, November 11, 13

Slide 5

Slide 5 text

5 Meanwhile... Elsewhere At New Relic http://flic.kr/p/9xEUug Monday, November 11, 13

Slide 6

Slide 6 text

6 module NewRelic::Agent::BrowserMonitoring # MONKEY PATCHED! def footer_js_string(config) "... #{browser_monitoring_queue_time}" end end rpm_site Monday, November 11, 13

Slide 7

Slide 7 text

7 http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Monday, November 11, 13

Slide 8

Slide 8 text

8 Monday, November 11, 13

Slide 9

Slide 9 text

9 Reskiq It forks then it threads! Monday, November 11, 13

Slide 10

Slide 10 text

10 gem install my_awesome_code Monday, November 11, 13

Slide 11

Slide 11 text

Where We're Going 11 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 12

Slide 12 text

Pass It On 12 Monday, November 11, 13

Slide 13

Slide 13 text

Loggers 13 Monday, November 11, 13

Slide 14

Slide 14 text

activerecord 14 ActiveRecord::Base.logger = ::Logger.new("my_special.log") Monday, November 11, 13

Slide 15

Slide 15 text

15 http://flic.kr/p/7GDRue Dependency Injection Monday, November 11, 13

Slide 16

Slide 16 text

16 http://flic.kr/p/gzxjKu Duck Typing Monday, November 11, 13

Slide 17

Slide 17 text

Instrumentors 17 Monday, November 11, 13

Slide 18

Slide 18 text

excon 18 connection = Excon.new( 'http://nerd.jasonrclark.com', :instrumentor => SimpleInstrumentor ) Monday, November 11, 13

Slide 19

Slide 19 text

excon 19 class SimpleInstrumentor class << self attr_accessor :events def instrument(name, params = {}, &blk) puts "#{name} just happened." yield if block_given? end end end Monday, November 11, 13

Slide 20

Slide 20 text

20 http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Monday, November 11, 13

Slide 21

Slide 21 text

Backends 21 Monday, November 11, 13

Slide 22

Slide 22 text

Delayed::Job 22 # without delayed_job @user.activate!(@device) # with delayed_job @user.delay.activate!(@device) Monday, November 11, 13

Slide 23

Slide 23 text

Delayed::Job 23 Monday, November 11, 13

Slide 24

Slide 24 text

Delayed::Job 24 ::Delayed.const_set(:Job, backend) Monday, November 11, 13

Slide 25

Slide 25 text

25 newrelic_rpm def failed_jobs Delayed::Job.count( :conditions => 'failed_at is not NULL') end Monday, November 11, 13

Slide 26

Slide 26 text

Gem-Specific 26 Monday, November 11, 13

Slide 27

Slide 27 text

Resque 27 class SendMessageJob def self.perform(id, text) user = User.find(id) user.send_message(text) end end Resque.enqueue(SendMessageJob, 42, "Yo") Monday, November 11, 13

Slide 28

Slide 28 text

28 Resque https://github.com/resque/resque/blob/master/docs/HOOKS.md class SendMessageJob def self.before_perform_log_it(*args) logger.debug("Messaging #{args}") end ... end Monday, November 11, 13

Slide 29

Slide 29 text

Where We've Been 29 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 30

Slide 30 text

Events 30 Monday, November 11, 13

Slide 31

Slide 31 text

What's Important? 31 Monday, November 11, 13

Slide 32

Slide 32 text

32 Monday, November 11, 13

Slide 33

Slide 33 text

Surfacing Events 33 Monday, November 11, 13

Slide 34

Slide 34 text

34 newrelic_rpm events = NewRelic::Agent.instance.events events.subscribe(:start_transaction) do set_transaction_custom_parameters end Monday, November 11, 13

Slide 35

Slide 35 text

35 Resque Resque.after_fork do |job| ActiveRecord::Base.establish_connection end Monday, November 11, 13

Slide 36

Slide 36 text

36 http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Monday, November 11, 13

Slide 37

Slide 37 text

Ordering? 37 Monday, November 11, 13

Slide 38

Slide 38 text

ActiveSupport::Notifications 38 activesupport Monday, November 11, 13

Slide 39

Slide 39 text

Where We've Been 39 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 40

Slide 40 text

Middleware 40 Monday, November 11, 13

Slide 41

Slide 41 text

41 Rack::Cache ActionDispatch::Cookies YourApplication Web Request Monday, November 11, 13

Slide 42

Slide 42 text

42 # [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Monday, November 11, 13

Slide 43

Slide 43 text

43 # [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Monday, November 11, 13

Slide 44

Slide 44 text

http://flic.kr/p/SSGmF ActionDispatch::Static Rack::Lock Rack::Runtime Rack::MethodOverride ActionDispatch::RequestId Rails::Rack::Logger ActionDispatch::ShowExceptions ActionDispatch::DebugExceptions ActionDispatch::RemoteIp ActionDispatch::Reloader ActionDispatch::Callbacks ActiveRecord::ConnectionAdapters ActiveRecord::QueryCache ActionDispatch::Cookies ActionDispatch::Session::CookieStore ActionDispatch::Flash ActionDispatch::ParamsParser Rack::Head Rack::ConditionalGet Rack::ETag AgentSnoop::Middleware RpmTestApp::Application.routes Monday, November 11, 13

Slide 45

Slide 45 text

Who's There? 45 Monday, November 11, 13

Slide 46

Slide 46 text

46 # [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Monday, November 11, 13

Slide 47

Slide 47 text

47 class MyMiddleware def initialize(options=nil) # options == { :foo => 1 } end def call(worker, msg, queue) yield end end Monday, November 11, 13

Slide 48

Slide 48 text

48 Sidekiq.configure_server do |config| config.server_middleware do |chain| chain.add MyMiddleware, :foo => 1 end end Monday, November 11, 13

Slide 49

Slide 49 text

Where We've Been 49 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 50

Slide 50 text

Lifecycle 50 Monday, November 11, 13

Slide 51

Slide 51 text

Forks and Daemons 51 http://flic.kr/p/79tE5Q Monday, November 11, 13

Slide 52

Slide 52 text

Forking 52 fork() state Monday, November 11, 13

Slide 53

Slide 53 text

fork() state fork() Forking 53 state Monday, November 11, 13

Slide 54

Slide 54 text

before_fork after_fork 54 Monday, November 11, 13

Slide 55

Slide 55 text

Add Threads 55 fork() Monday, November 11, 13

Slide 56

Slide 56 text

fork() Add Threads 56 Monday, November 11, 13

Slide 57

Slide 57 text

Enter the Lock 57 fork() @lock Monday, November 11, 13

Slide 58

Slide 58 text

@loick fork() Enter the Lock 58 @lock Monday, November 11, 13

Slide 59

Slide 59 text

59 https://github.com/resque/resque/issues/1101 Forks + Threads + == Locks Resque Monday, November 11, 13

Slide 60

Slide 60 text

Load Up! 60 Monday, November 11, 13

Slide 61

Slide 61 text

unicorn 61 ♥ὑὑ~/myapp:unicorn_rails -c ./unicorn.rb unicorn_rails master unicorn_rails worker[0] unicorn_rails worker[1] unicorn_rails worker[2] unicorn_rails worker[3] Monday, November 11, 13

Slide 62

Slide 62 text

62 loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Monday, November 11, 13

Slide 63

Slide 63 text

63 loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Monday, November 11, 13

Slide 64

Slide 64 text

Where We've Been 64 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 65

Slide 65 text

Names and Paths 65 Monday, November 11, 13

Slide 66

Slide 66 text

Forced 66 Monday, November 11, 13

Slide 67

Slide 67 text

67 Sequel.plugin :newrelic_instrumentation Monday, November 11, 13

Slide 68

Slide 68 text

68 Sequel.plugin :newrelic_instrumentation => require 'sequel/plugins/newrelic_instrumentation' Monday, November 11, 13

Slide 69

Slide 69 text

69 ~/source/newrelic/ruby_agent/lib: ls -la drwxr-xr-x Sep 19 14:46 . drwxr-xr-x Oct 10 11:19 .. drwxr-xr-x Oct 10 11:20 new_relic -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb drwxr-xr-x Sep 19 14:46 sequel drwxr-xr-x Oct 7 08:53 tasks Monday, November 11, 13

Slide 70

Slide 70 text

70 ~/source/newrelic/ruby_agent/lib: ls -la drwxr-xr-x Sep 19 14:46 . drwxr-xr-x Oct 10 11:19 .. drwxr-xr-x Oct 10 11:20 new_relic -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb drwxr-xr-x Sep 19 14:46 sequel drwxr-xr-x Oct 7 08:53 tasks Monday, November 11, 13

Slide 71

Slide 71 text

71 Sequel.plugin NewRelic::Sequel::Instrumentation Monday, November 11, 13

Slide 72

Slide 72 text

Generic 72 Monday, November 11, 13

Slide 73

Slide 73 text

73 Monday, November 11, 13

Slide 74

Slide 74 text

73 ActiveRecord Monday, November 11, 13

Slide 75

Slide 75 text

73 ActiveRecord DataMapper Monday, November 11, 13

Slide 76

Slide 76 text

73 ActiveRecord DataMapper HTTPClient Monday, November 11, 13

Slide 77

Slide 77 text

Qualify Yourself 74 Monday, November 11, 13

Slide 78

Slide 78 text

75 newrelic_rpm module NewRelic module Agent class Thread < ::Thread ... end end end Monday, November 11, 13

Slide 79

Slide 79 text

elsewhere... 76 class Thread # Re-opened NewRelic::Agent::Thread # D'oh! end Monday, November 11, 13

Slide 80

Slide 80 text

77 newrelic_plugin module NewRelic class Logger ... end end Monday, November 11, 13

Slide 81

Slide 81 text

78 newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => Logger::DEBUG, ... } end end end Monday, November 11, 13

Slide 82

Slide 82 text

79 newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => Logger::DEBUG, ... } end end end Monday, November 11, 13

Slide 83

Slide 83 text

80 newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => ::Logger::DEBUG, ... } end end end Monday, November 11, 13

Slide 84

Slide 84 text

Where We've Been 81 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 85

Slide 85 text

Config 82 Monday, November 11, 13

Slide 86

Slide 86 text

yml 83 Monday, November 11, 13

Slide 87

Slide 87 text

yml + ERB 84 Monday, November 11, 13

Slide 88

Slide 88 text

yml + ERB 85 file = File.read("~/config/awesome_gem.yml") # Locals available in ERB default_awesome_key = "YEAH!" erb = ERB.new(file).result(binding) config = YAML.load(erb) Monday, November 11, 13

Slide 89

Slide 89 text

yml + ERB 86 ~/config/awesome_gem.yml development: name: <%= default_awesome_key %> license_key: <%= ENV["LICENSE_KEY"] %> Monday, November 11, 13

Slide 90

Slide 90 text

Ruby 87 Monday, November 11, 13

Slide 91

Slide 91 text

88 unicorn worker_processes 5 preload_app true timeout 30 after_fork do |server, worker| defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection end Monday, November 11, 13

Slide 92

Slide 92 text

89 on_worker_boot do ActiveSupport.on_load(:active_record) do ActiveRecord::Base.establish_connection end end Monday, November 11, 13

Slide 93

Slide 93 text

Great for Apps? What about Gems? 90 Monday, November 11, 13

Slide 94

Slide 94 text

91 newrelic_rpm if defined?(::Puma) && ::Puma.respond_to?(:cli_config) config = ::Puma.cli_config config.options[:worker_boot] << Proc.new do NewRelic::Agent.after_fork(...) end end Monday, November 11, 13

Slide 95

Slide 95 text

92 newrelic_rpm if defined?(::Puma) && ::Puma.respond_to?(:cli_config) config = ::Puma.cli_config config.options[:worker_boot] << Proc.new do NewRelic::Agent.after_fork(...) end end Monday, November 11, 13

Slide 96

Slide 96 text

Where We've Been 93 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 97

Slide 97 text

Docs 94 Monday, November 11, 13

Slide 98

Slide 98 text

README 95 Monday, November 11, 13

Slide 99

Slide 99 text

sinatra 96 Monday, November 11, 13

Slide 100

Slide 100 text

97 Monday, November 11, 13

Slide 101

Slide 101 text

98 Monday, November 11, 13

Slide 102

Slide 102 text

99 Monday, November 11, 13

Slide 103

Slide 103 text

Expected Extensions? 100 Monday, November 11, 13

Slide 104

Slide 104 text

101 Monday, November 11, 13

Slide 105

Slide 105 text

Don't Bury It 102 Monday, November 11, 13

Slide 106

Slide 106 text

103 Monday, November 11, 13

Slide 107

Slide 107 text

104 Monday, November 11, 13

Slide 108

Slide 108 text

Versioning 105 Monday, November 11, 13

Slide 109

Slide 109 text

106 Monday, November 11, 13

Slide 110

Slide 110 text

Where We've Been 107 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 111

Slide 111 text

Where We've Been 108 Monday, November 11, 13

Slide 112

Slide 112 text

Where We've Been 108 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Monday, November 11, 13

Slide 113

Slide 113 text

Where We've Been 108 • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Jason Clark @jasonrclark Ruby Agent Engineer Monday, November 11, 13