Extending Gems (OS Bridge 2014)

One strength of the Ruby community is the simplicity of sharing code via gems. When a gem is popular enough, it can even develop an ecosystem of additional gems that build on it.

But extending a gem that wasn’t built with that flexibility in mind isn’t always easy. This talk highlights tips and techniques for making your gem simpler to plug into, and avoid mistakes that will have other devs cursing under their breath.

Based on experience working on New Relic’s Ruby agent (aka the newrelic_rpm gem), we’ll cover the highs and lows of interacting with others gems, from configuration to documentation and everywhere in between.

Transcript

1. Extending Gems Jason Clark @jasonrclark Ruby Agent Engineer Patterns and Anti-Patterns of Pluggable Gems Thursday, June 26, 14
2. Storytime! Thursday, June 26, 14
3. newrelic_rpm module NewRelic::Agent::BrowserMonitoring def browser_monitoring_queue_time queue_time = current_transaction.queue_time millis = queue_time.to_f * 1000.0 clamp_to_positive(millis.round) end def footer_js_string(config) "<script>...#{browser_monitoring_queue_time}" end end Thursday, June 26, 14
4. module NewRelic::Agent::BrowserMonitoring def current_timings TransactionTimings.new end def footer_js_string(config) "<script>...#{current_timings.queue_time_in_millis}" end end newrelic_rpm Thursday, June 26, 14
5. Meanwhile... Elsewhere At New Relic http://flic.kr/p/9xEUug Thursday, June 26, 14
6. module NewRelic::Agent::BrowserMonitoring # MONKEY PATCH! def footer_js_string(config) "<script>...#{browser_monitoring_queue_time}" end end rpm_site Thursday, June 26, 14
7. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Thursday, June 26, 14
8. Thursday, June 26, 14
9. Reskiq It forks then it threads! Thursday, June 26, 14
10. gem install my_awesome_code Thursday, June 26, 14
11. Where We're Going • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
12. Pass It On Thursday, June 26, 14
13. Loggers Thursday, June 26, 14
14. activerecord ActiveRecord::Base.logger = ::Logger.new("my_special.log") Thursday, June 26, 14
15. http://flic.kr/p/7GDRue Dependency Injection Thursday, June 26, 14
16. http://flic.kr/p/gzxjKu Duck Typing Thursday, June 26, 14
17. Instrumentors Thursday, June 26, 14
18. excon connection = Excon.new( 'http://nerd.jasonrclark.com', :instrumentor => SimpleInstrumentor ) Thursday, June 26, 14
19. excon class SimpleInstrumentor class << self def instrument(name, params = {}, &blk) puts "#{name} just happened." yield if block_given? end end end Thursday, June 26, 14
20. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Thursday, June 26, 14
21. Backends Thursday, June 26, 14
22. Delayed::Job # without delayed_job @user.activate!(@device) # with delayed_job @user.delay.activate!(@device) Thursday, June 26, 14
23. Delayed::Job Thursday, June 26, 14
24. Delayed::Job ::Delayed.const_set(:Job, backend) Thursday, June 26, 14
25. newrelic_rpm def failed_jobs Delayed::Job.count( :conditions => 'failed_at is not NULL') end Thursday, June 26, 14
26. Gem-Specific Thursday, June 26, 14
27. Resque class SendMessageJob def self.perform(id, text) user = User.find(id) user.send_message(text) end end Resque.enqueue(SendMessageJob, 42, "Yo") Thursday, June 26, 14
28. Resque https://github.com/resque/resque/blob/master/docs/HOOKS.md class SendMessageJob def self.before_perform_log_it(*args) logger.debug("Messaging #{args}") end ... end Thursday, June 26, 14
29. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
30. Events Thursday, June 26, 14
31. http://www.confreaks.com/videos/3327-railsconf- make-an-event-of-it Thursday, June 26, 14
32. Resque Resque.after_fork do |job| ActiveRecord::Base.establish_connection end Thursday, June 26, 14
33. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/ Thursday, June 26, 14
34. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
35. Middleware Thursday, June 26, 14
36. Rack::Cache ActionDispatch::Cookies YourApplication Web Request Thursday, June 26, 14
37. # [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Thursday, June 26, 14
38. # [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Thursday, June 26, 14
39. http://flic.kr/p/SSGmF ActionDispatch::Static Rack::Lock <ActiveSupport::Cache::Strategy> Rack::Runtime Rack::MethodOverride ActionDispatch::RequestId Rails::Rack::Logger ActionDispatch::ShowExceptions ActionDispatch::DebugExceptions ActionDispatch::RemoteIp ActionDispatch::Reloader ActionDispatch::Callbacks ActiveRecord::ConnectionAdapters ActiveRecord::QueryCache ActionDispatch::Cookies ActionDispatch::Session::CookieStore ActionDispatch::Flash ActionDispatch::ParamsParser Rack::Head Rack::ConditionalGet Rack::ETag AgentSnoop::Middleware RpmTestApp::Application.routes Thursday, June 26, 14
40. Who's There? Thursday, June 26, 14
41. # [status, headers, response] def call(env) # ... before result = @app.call(env) # ... after result end Thursday, June 26, 14
42. class MyMiddleware def initialize(options={}) # options == { :foo => 1 } end def call(worker, msg, queue) yield end end Thursday, June 26, 14
43. Sidekiq.configure_server do |config| config.server_middleware do |chain| chain.add MyMiddleware, :foo => 1 end end Thursday, June 26, 14
44. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
45. Lifecycle Thursday, June 26, 14
46. Forks and Daemons http://flic.kr/p/79tE5Q Thursday, June 26, 14
47. Forking fork() state Thursday, June 26, 14
48. fork() state fork() Forking state Thursday, June 26, 14
49. before_fork after_fork Thursday, June 26, 14
50. Add Threads fork() Thursday, June 26, 14
51. fork() Add Threads Thursday, June 26, 14
52. Enter the Lock fork() @lock Thursday, June 26, 14
53. @loick fork() Enter the Lock @lock Thursday, June 26, 14
54. https://github.com/resque/resque/issues/1101 Forks + Threads + == Locks Resque Thursday, June 26, 14
55. Load Up! Thursday, June 26, 14
56. unicorn ♥ὑὑ~/myapp:unicorn_rails -c ./unicorn.rb unicorn_rails master unicorn_rails worker[0] unicorn_rails worker[1] unicorn_rails worker[2] unicorn_rails worker[3] Thursday, June 26, 14
57. loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Thursday, June 26, 14
58. loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Thursday, June 26, 14
59. loads gems forks after_fork starts loop forks after_fork starts loop loads gems preload_app true preload_app false unicorn Thursday, June 26, 14
60. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
61. Names and Paths Thursday, June 26, 14
62. Forced Thursday, June 26, 14
63. Sequel.plugin :newrelic_instrumentation Thursday, June 26, 14
64. Sequel.plugin :newrelic_instrumentation => require 'sequel/plugins/newrelic_instrumentation' Thursday, June 26, 14
65. ~/source/newrelic/ruby_agent/lib: ls -la drwxr-xr-x Sep 19 14:46 . drwxr-xr-x Oct 10 11:19 .. drwxr-xr-x Oct 10 11:20 new_relic -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb drwxr-xr-x Sep 19 14:46 sequel drwxr-xr-x Oct 7 08:53 tasks Thursday, June 26, 14
66. ~/source/newrelic/ruby_agent/lib: ls -la drwxr-xr-x Sep 19 14:46 . drwxr-xr-x Oct 10 11:19 .. drwxr-xr-x Oct 10 11:20 new_relic -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb drwxr-xr-x Sep 19 14:46 sequel drwxr-xr-x Oct 7 08:53 tasks Thursday, June 26, 14
67. Sequel.plugin NewRelic::Sequel::Instrumentation Thursday, June 26, 14
68. Generic Thursday, June 26, 14
69. Thursday, June 26, 14
70. ActiveRecord Thursday, June 26, 14
71. ActiveRecord DataMapper Thursday, June 26, 14
72. ActiveRecord DataMapper HTTPClient Thursday, June 26, 14
73. Qualify Yourself Thursday, June 26, 14
74. newrelic_rpm module NewRelic module Agent class Thread < ::Thread ... end end end Thursday, June 26, 14
75. elsewhere... class Thread # Re-opens NewRelic::Agent::Thread # D'oh! end Thursday, June 26, 14
76. newrelic_plugin module NewRelic class Logger ... end end Thursday, June 26, 14
77. newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => Logger::DEBUG, ... } end end end Thursday, June 26, 14
78. newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => Logger::DEBUG, ... } end end end Thursday, June 26, 14
79. newrelic_rpm module NewRelic module Agent class AgentLogger LOG_LEVELS = { "debug" => ::Logger::DEBUG, ... } end end end Thursday, June 26, 14
80. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
81. Config Thursday, June 26, 14
82. yml Thursday, June 26, 14
83. yml + ERB Thursday, June 26, 14
84. yml + ERB file = File.read("./config/awesome_gem.yml") # Locals available in ERB default_awesome_key = "YEAH!" erb = ERB.new(file).result(binding) config = YAML.load(erb) Thursday, June 26, 14
85. yml + ERB ~/config/awesome_gem.yml development: name: <%= default_awesome_key %> license_key: <%= ENV["LICENSE_KEY"] %> Thursday, June 26, 14
86. Ruby Thursday, June 26, 14
87. unicorn worker_processes 5 preload_app true timeout 30 after_fork do |server, worker| defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection end Thursday, June 26, 14
88. on_worker_boot do ActiveSupport.on_load(:active_record) do ActiveRecord::Base.establish_connection end end Thursday, June 26, 14
89. Great for Apps? What about Gems? Thursday, June 26, 14
90. newrelic_rpm if defined?(::Puma) && ::Puma.respond_to?(:cli_config) config = ::Puma.cli_config config.options[:worker_boot] << Proc.new do NewRelic::Agent.after_fork(...) end end Thursday, June 26, 14
91. newrelic_rpm if defined?(::Puma) && ::Puma.respond_to?(:cli_config) config = ::Puma.cli_config config.options[:worker_boot] << Proc.new do NewRelic::Agent.after_fork(...) end end Thursday, June 26, 14
92. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
93. Docs Thursday, June 26, 14
94. README Thursday, June 26, 14
95. sinatra Thursday, June 26, 14
96. Thursday, June 26, 14
97. Thursday, June 26, 14
98. Thursday, June 26, 14
99. Expected Extensions? Thursday, June 26, 14
100. Thursday, June 26, 14
101. Don't Bury It Thursday, June 26, 14
102. Thursday, June 26, 14
103. Thursday, June 26, 14
104. Versioning Thursday, June 26, 14
105. Thursday, June 26, 14
106. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
107. Where We've Been Thursday, June 26, 14
108. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Thursday, June 26, 14
109. Where We've Been • Pass It On • Events • Middleware • Lifecycle • Names and Paths • Config • Docs Jason Clark @jasonrclark Ruby Agent Engineer Thursday, June 26, 14