Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Extending Gems (OS Bridge 2014)

Extending Gems (OS Bridge 2014)

One strength of the Ruby community is the simplicity of sharing code via gems. When a gem is popular enough, it can even develop an ecosystem of additional gems that build on it.

But extending a gem that wasn’t built with that flexibility in mind isn’t always easy. This talk highlights tips and techniques for making your gem simpler to plug into, and avoid mistakes that will have other devs cursing under their breath.

Based on experience working on New Relic’s Ruby agent (aka the newrelic_rpm gem), we’ll cover the highs and lows of interacting with others gems, from configuration to documentation and everywhere in between.

Jason R Clark

June 26, 2014
Tweet

More Decks by Jason R Clark

Other Decks in Technology

Transcript

  1. Extending Gems
    Jason Clark
    @jasonrclark
    Ruby Agent Engineer
    Patterns and Anti-Patterns of
    Pluggable Gems
    Thursday, June 26, 14

    View Slide

  2. Storytime!
    Thursday, June 26, 14

    View Slide

  3. newrelic_rpm
    module NewRelic::Agent::BrowserMonitoring
    def browser_monitoring_queue_time
    queue_time = current_transaction.queue_time
    millis = queue_time.to_f * 1000.0
    clamp_to_positive(millis.round)
    end
    def footer_js_string(config)
    "...#{browser_monitoring_queue_time}"<br/>end<br/>end<br/>Thursday, June 26, 14<br/>

    View Slide

  4. module NewRelic::Agent::BrowserMonitoring
    def current_timings
    TransactionTimings.new
    end
    def footer_js_string(config)
    "...#{current_timings.queue_time_in_millis}"<br/>end<br/>end<br/>newrelic_rpm<br/>Thursday, June 26, 14<br/>

    View Slide

  5. Meanwhile...
    Elsewhere At New Relic
    http://flic.kr/p/9xEUug
    Thursday, June 26, 14

    View Slide

  6. module NewRelic::Agent::BrowserMonitoring
    # MONKEY PATCH!
    def footer_js_string(config)
    "...#{browser_monitoring_queue_time}"<br/>end<br/>end<br/>rpm_site<br/>Thursday, June 26, 14<br/>

    View Slide

  7. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/
    Thursday, June 26, 14

    View Slide

  8. Thursday, June 26, 14

    View Slide

  9. Reskiq
    It forks then it threads!
    Thursday, June 26, 14

    View Slide

  10. gem install
    my_awesome_code
    Thursday, June 26, 14

    View Slide

  11. Where We're Going
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  12. Pass It On
    Thursday, June 26, 14

    View Slide

  13. Loggers
    Thursday, June 26, 14

    View Slide

  14. activerecord
    ActiveRecord::Base.logger =
    ::Logger.new("my_special.log")
    Thursday, June 26, 14

    View Slide

  15. http://flic.kr/p/7GDRue
    Dependency Injection
    Thursday, June 26, 14

    View Slide

  16. http://flic.kr/p/gzxjKu
    Duck Typing
    Thursday, June 26, 14

    View Slide

  17. Instrumentors
    Thursday, June 26, 14

    View Slide

  18. excon
    connection = Excon.new(
    'http://nerd.jasonrclark.com',
    :instrumentor => SimpleInstrumentor
    )
    Thursday, June 26, 14

    View Slide

  19. excon
    class SimpleInstrumentor
    class << self
    def instrument(name, params = {}, &blk)
    puts "#{name} just happened."
    yield if block_given?
    end
    end
    end
    Thursday, June 26, 14

    View Slide

  20. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/
    Thursday, June 26, 14

    View Slide

  21. Backends
    Thursday, June 26, 14

    View Slide

  22. Delayed::Job
    # without delayed_job
    @user.activate!(@device)
    # with delayed_job
    @user.delay.activate!(@device)
    Thursday, June 26, 14

    View Slide

  23. Delayed::Job
    Thursday, June 26, 14

    View Slide

  24. Delayed::Job
    ::Delayed.const_set(:Job, backend)
    Thursday, June 26, 14

    View Slide

  25. newrelic_rpm
    def failed_jobs
    Delayed::Job.count(
    :conditions => 'failed_at is not NULL')
    end
    Thursday, June 26, 14

    View Slide

  26. Gem-Specific
    Thursday, June 26, 14

    View Slide

  27. Resque
    class SendMessageJob
    def self.perform(id, text)
    user = User.find(id)
    user.send_message(text)
    end
    end
    Resque.enqueue(SendMessageJob, 42, "Yo")
    Thursday, June 26, 14

    View Slide

  28. Resque
    https://github.com/resque/resque/blob/master/docs/HOOKS.md
    class SendMessageJob
    def self.before_perform_log_it(*args)
    logger.debug("Messaging #{args}")
    end
    ...
    end
    Thursday, June 26, 14

    View Slide

  29. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  30. Events
    Thursday, June 26, 14

    View Slide

  31. http://www.confreaks.com/videos/3327-railsconf-
    make-an-event-of-it
    Thursday, June 26, 14

    View Slide

  32. Resque
    Resque.after_fork do |job|
    ActiveRecord::Base.establish_connection
    end
    Thursday, June 26, 14

    View Slide

  33. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/
    Thursday, June 26, 14

    View Slide

  34. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  35. Middleware
    Thursday, June 26, 14

    View Slide

  36. Rack::Cache
    ActionDispatch::Cookies
    YourApplication
    Web Request
    Thursday, June 26, 14

    View Slide

  37. # [status, headers, response]
    def call(env)
    # ... before
    result = @app.call(env)
    # ... after
    result
    end
    Thursday, June 26, 14

    View Slide

  38. # [status, headers, response]
    def call(env)
    # ... before
    result = @app.call(env)
    # ... after
    result
    end
    Thursday, June 26, 14

    View Slide

  39. http://flic.kr/p/SSGmF
    ActionDispatch::Static
    Rack::Lock

    Rack::Runtime
    Rack::MethodOverride
    ActionDispatch::RequestId
    Rails::Rack::Logger
    ActionDispatch::ShowExceptions
    ActionDispatch::DebugExceptions
    ActionDispatch::RemoteIp
    ActionDispatch::Reloader
    ActionDispatch::Callbacks
    ActiveRecord::ConnectionAdapters
    ActiveRecord::QueryCache
    ActionDispatch::Cookies
    ActionDispatch::Session::CookieStore
    ActionDispatch::Flash
    ActionDispatch::ParamsParser
    Rack::Head
    Rack::ConditionalGet
    Rack::ETag
    AgentSnoop::Middleware
    RpmTestApp::Application.routes
    Thursday, June 26, 14

    View Slide

  40. Who's There?
    Thursday, June 26, 14

    View Slide

  41. # [status, headers, response]
    def call(env)
    # ... before
    result = @app.call(env)
    # ... after
    result
    end
    Thursday, June 26, 14

    View Slide

  42. class MyMiddleware
    def initialize(options={})
    # options == { :foo => 1 }
    end
    def call(worker, msg, queue)
    yield
    end
    end
    Thursday, June 26, 14

    View Slide

  43. Sidekiq.configure_server do |config|
    config.server_middleware do |chain|
    chain.add MyMiddleware, :foo => 1
    end
    end
    Thursday, June 26, 14

    View Slide

  44. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  45. Lifecycle
    Thursday, June 26, 14

    View Slide

  46. Forks and Daemons
    http://flic.kr/p/79tE5Q
    Thursday, June 26, 14

    View Slide

  47. Forking
    fork()
    state
    Thursday, June 26, 14

    View Slide

  48. fork()
    state
    fork()
    Forking
    state
    Thursday, June 26, 14

    View Slide

  49. before_fork
    after_fork
    Thursday, June 26, 14

    View Slide

  50. Add Threads
    fork()
    Thursday, June 26, 14

    View Slide

  51. fork()
    Add Threads
    Thursday, June 26, 14

    View Slide

  52. Enter the Lock
    fork()
    @lock
    Thursday, June 26, 14

    View Slide

  53. @loick
    fork()
    Enter the Lock
    @lock
    Thursday, June 26, 14

    View Slide

  54. https://github.com/resque/resque/issues/1101
    Forks +
    Threads + ==
    Locks
    Resque
    Thursday, June 26, 14

    View Slide

  55. Load Up!
    Thursday, June 26, 14

    View Slide

  56. unicorn
    ♥ὑὑ~/myapp:unicorn_rails -c ./unicorn.rb
    unicorn_rails
    master
    unicorn_rails
    worker[0]
    unicorn_rails
    worker[1]
    unicorn_rails
    worker[2]
    unicorn_rails
    worker[3]
    Thursday, June 26, 14

    View Slide

  57. loads gems
    forks
    after_fork
    starts loop
    forks
    after_fork
    starts loop
    loads gems
    preload_app true preload_app false
    unicorn
    Thursday, June 26, 14

    View Slide

  58. loads gems
    forks
    after_fork
    starts loop
    forks
    after_fork
    starts loop
    loads gems
    preload_app true preload_app false
    unicorn
    Thursday, June 26, 14

    View Slide

  59. loads gems
    forks
    after_fork
    starts loop
    forks
    after_fork
    starts loop
    loads gems
    preload_app true preload_app false
    unicorn
    Thursday, June 26, 14

    View Slide

  60. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  61. Names and
    Paths
    Thursday, June 26, 14

    View Slide

  62. Forced
    Thursday, June 26, 14

    View Slide

  63. Sequel.plugin :newrelic_instrumentation
    Thursday, June 26, 14

    View Slide

  64. Sequel.plugin :newrelic_instrumentation
    =>
    require 'sequel/plugins/newrelic_instrumentation'
    Thursday, June 26, 14

    View Slide

  65. ~/source/newrelic/ruby_agent/lib: ls -la
    drwxr-xr-x Sep 19 14:46 .
    drwxr-xr-x Oct 10 11:19 ..
    drwxr-xr-x Oct 10 11:20 new_relic
    -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb
    drwxr-xr-x Sep 19 14:46 sequel
    drwxr-xr-x Oct 7 08:53 tasks
    Thursday, June 26, 14

    View Slide

  66. ~/source/newrelic/ruby_agent/lib: ls -la
    drwxr-xr-x Sep 19 14:46 .
    drwxr-xr-x Oct 10 11:19 ..
    drwxr-xr-x Oct 10 11:20 new_relic
    -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb
    drwxr-xr-x Sep 19 14:46 sequel
    drwxr-xr-x Oct 7 08:53 tasks
    Thursday, June 26, 14

    View Slide

  67. Sequel.plugin NewRelic::Sequel::Instrumentation
    Thursday, June 26, 14

    View Slide

  68. Generic
    Thursday, June 26, 14

    View Slide

  69. Thursday, June 26, 14

    View Slide

  70. ActiveRecord
    Thursday, June 26, 14

    View Slide

  71. ActiveRecord
    DataMapper
    Thursday, June 26, 14

    View Slide

  72. ActiveRecord
    DataMapper
    HTTPClient
    Thursday, June 26, 14

    View Slide

  73. Qualify Yourself
    Thursday, June 26, 14

    View Slide

  74. newrelic_rpm
    module NewRelic
    module Agent
    class Thread < ::Thread
    ...
    end
    end
    end
    Thursday, June 26, 14

    View Slide

  75. elsewhere...
    class Thread
    # Re-opens NewRelic::Agent::Thread
    # D'oh!
    end
    Thursday, June 26, 14

    View Slide

  76. newrelic_plugin
    module NewRelic
    class Logger
    ...
    end
    end
    Thursday, June 26, 14

    View Slide

  77. newrelic_rpm
    module NewRelic
    module Agent
    class AgentLogger
    LOG_LEVELS = {
    "debug" => Logger::DEBUG,
    ...
    }
    end
    end
    end
    Thursday, June 26, 14

    View Slide

  78. newrelic_rpm
    module NewRelic
    module Agent
    class AgentLogger
    LOG_LEVELS = {
    "debug" => Logger::DEBUG,
    ...
    }
    end
    end
    end
    Thursday, June 26, 14

    View Slide

  79. newrelic_rpm
    module NewRelic
    module Agent
    class AgentLogger
    LOG_LEVELS = {
    "debug" => ::Logger::DEBUG,
    ...
    }
    end
    end
    end
    Thursday, June 26, 14

    View Slide

  80. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  81. Config
    Thursday, June 26, 14

    View Slide

  82. yml
    Thursday, June 26, 14

    View Slide

  83. yml + ERB
    Thursday, June 26, 14

    View Slide

  84. yml + ERB
    file = File.read("./config/awesome_gem.yml")
    # Locals available in ERB
    default_awesome_key = "YEAH!"
    erb = ERB.new(file).result(binding)
    config = YAML.load(erb)
    Thursday, June 26, 14

    View Slide

  85. yml + ERB
    ~/config/awesome_gem.yml
    development:
    name: <%= default_awesome_key %>
    license_key: <%= ENV["LICENSE_KEY"] %>
    Thursday, June 26, 14

    View Slide

  86. Ruby
    Thursday, June 26, 14

    View Slide

  87. unicorn
    worker_processes 5
    preload_app true
    timeout 30
    after_fork do |server, worker|
    defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection
    end
    Thursday, June 26, 14

    View Slide

  88. on_worker_boot do
    ActiveSupport.on_load(:active_record) do
    ActiveRecord::Base.establish_connection
    end
    end
    Thursday, June 26, 14

    View Slide

  89. Great for Apps?
    What about Gems?
    Thursday, June 26, 14

    View Slide

  90. newrelic_rpm
    if defined?(::Puma) &&
    ::Puma.respond_to?(:cli_config)
    config = ::Puma.cli_config
    config.options[:worker_boot] << Proc.new do
    NewRelic::Agent.after_fork(...)
    end
    end
    Thursday, June 26, 14

    View Slide

  91. newrelic_rpm
    if defined?(::Puma) &&
    ::Puma.respond_to?(:cli_config)
    config = ::Puma.cli_config
    config.options[:worker_boot] << Proc.new do
    NewRelic::Agent.after_fork(...)
    end
    end
    Thursday, June 26, 14

    View Slide

  92. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  93. Docs
    Thursday, June 26, 14

    View Slide

  94. README
    Thursday, June 26, 14

    View Slide

  95. sinatra
    Thursday, June 26, 14

    View Slide

  96. Thursday, June 26, 14

    View Slide

  97. Thursday, June 26, 14

    View Slide

  98. Thursday, June 26, 14

    View Slide

  99. Expected
    Extensions?
    Thursday, June 26, 14

    View Slide

  100. Thursday, June 26, 14

    View Slide

  101. Don't Bury It
    Thursday, June 26, 14

    View Slide

  102. Thursday, June 26, 14

    View Slide

  103. Thursday, June 26, 14

    View Slide

  104. Versioning
    Thursday, June 26, 14

    View Slide

  105. Thursday, June 26, 14

    View Slide

  106. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  107. Where We've Been
    Thursday, June 26, 14

    View Slide

  108. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Thursday, June 26, 14

    View Slide

  109. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Jason Clark
    @jasonrclark
    Ruby Agent Engineer
    Thursday, June 26, 14

    View Slide