Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Patterns of Pluggable Gems (ConFoo 2014)

Patterns of Pluggable Gems (ConFoo 2014)

One strength of the Ruby community is the simplicity of sharing code via gems. Popular gems even develop their own ecosystem of plugins around them.

But extending a gem that wasn't built with flexibility in mind isn't easy. This talk covers the highs and lows of interacting with other gems. We'll cover how to make your gem easy to plug into, from patterns and events, to configuration and documentation.

Jason R Clark

February 26, 2014
Tweet

More Decks by Jason R Clark

Other Decks in Technology

Transcript

  1. Patterns of
    Pluggable Gems
    Jason Clark
    @jasonrclark
    Ruby Agent Engineer
    Wednesday, February 26, 14

    View Slide

  2. Storytime!
    Wednesday, February 26, 14

    View Slide

  3. newrelic_rpm
    module NewRelic::Agent::BrowserMonitoring
    def browser_monitoring_queue_time
    queue_time = current_transaction.queue_time
    millis = queue_time.to_f * 1000.0
    clamp_to_positive(millis.round)
    end
    def footer_js_string(config)
    "...#{browser_monitoring_queue_time}"<br/>end<br/>end<br/>Wednesday, February 26, 14<br/>

    View Slide

  4. module NewRelic::Agent::BrowserMonitoring
    def current_timings
    TransactionTimings.new
    end
    def footer_js_string(config)
    "...#{current_timings.queue_time_in_millis}"<br/>end<br/>end<br/>newrelic_rpm<br/>Wednesday, February 26, 14<br/>

    View Slide

  5. Meanwhile...
    Elsewhere At New Relic
    http://flic.kr/p/9xEUug
    Wednesday, February 26, 14

    View Slide

  6. module NewRelic::Agent::BrowserMonitoring
    # MONKEY PATCH!
    def footer_js_string(config)
    "...#{browser_monitoring_queue_time}"<br/>end<br/>end<br/>rpm_site<br/>Wednesday, February 26, 14<br/>

    View Slide

  7. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/
    Wednesday, February 26, 14

    View Slide

  8. Wednesday, February 26, 14

    View Slide

  9. Reskiq
    It forks then it threads!
    Wednesday, February 26, 14

    View Slide

  10. gem install
    my_awesome_code
    Wednesday, February 26, 14

    View Slide

  11. Where We're Going
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  12. Pass It On
    Wednesday, February 26, 14

    View Slide

  13. Loggers
    Wednesday, February 26, 14

    View Slide

  14. activerecord
    ActiveRecord::Base.logger =
    ::Logger.new("my_special.log")
    Wednesday, February 26, 14

    View Slide

  15. http://flic.kr/p/7GDRue
    Dependency Injection
    Wednesday, February 26, 14

    View Slide

  16. http://flic.kr/p/gzxjKu
    Duck Typing
    Wednesday, February 26, 14

    View Slide

  17. Instrumentors
    Wednesday, February 26, 14

    View Slide

  18. excon
    connection = Excon.new(
    'http://nerd.jasonrclark.com',
    :instrumentor => SimpleInstrumentor
    )
    Wednesday, February 26, 14

    View Slide

  19. excon
    class SimpleInstrumentor
    class << self
    def instrument(name, params = {}, &blk)
    puts "#{name} just happened."
    yield if block_given?
    end
    end
    end
    Wednesday, February 26, 14

    View Slide

  20. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/
    Wednesday, February 26, 14

    View Slide

  21. Backends
    Wednesday, February 26, 14

    View Slide

  22. Delayed::Job
    # without delayed_job
    @user.activate!(@device)
    # with delayed_job
    @user.delay.activate!(@device)
    Wednesday, February 26, 14

    View Slide

  23. Delayed::Job
    Wednesday, February 26, 14

    View Slide

  24. Delayed::Job
    ::Delayed.const_set(:Job, backend)
    Wednesday, February 26, 14

    View Slide

  25. newrelic_rpm
    def failed_jobs
    Delayed::Job.count(
    :conditions => 'failed_at is not NULL')
    end
    Wednesday, February 26, 14

    View Slide

  26. Gem-Specific
    Wednesday, February 26, 14

    View Slide

  27. Resque
    class SendMessageJob
    def self.perform(id, text)
    user = User.find(id)
    user.send_message(text)
    end
    end
    Resque.enqueue(SendMessageJob, 42, "Yo")
    Wednesday, February 26, 14

    View Slide

  28. Resque
    https://github.com/resque/resque/blob/master/docs/HOOKS.md
    class SendMessageJob
    def self.before_perform_log_it(*args)
    logger.debug("Messaging #{args}")
    end
    ...
    end
    Wednesday, February 26, 14

    View Slide

  29. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  30. Events
    Wednesday, February 26, 14

    View Slide

  31. See my talk
    tomorrow at 3!
    Wednesday, February 26, 14

    View Slide

  32. Resque
    Resque.after_fork do |job|
    ActiveRecord::Base.establish_connection
    end
    Wednesday, February 26, 14

    View Slide

  33. http://pixabay.com/en/elephant-foot-ten-foot-elephant-52536/
    Wednesday, February 26, 14

    View Slide

  34. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  35. Middleware
    Wednesday, February 26, 14

    View Slide

  36. Rack::Cache
    ActionDispatch::Cookies
    YourApplication
    Web Request
    Wednesday, February 26, 14

    View Slide

  37. # [status, headers, response]
    def call(env)
    # ... before
    result = @app.call(env)
    # ... after
    result
    end
    Wednesday, February 26, 14

    View Slide

  38. # [status, headers, response]
    def call(env)
    # ... before
    result = @app.call(env)
    # ... after
    result
    end
    Wednesday, February 26, 14

    View Slide

  39. http://flic.kr/p/SSGmF
    ActionDispatch::Static
    Rack::Lock

    Rack::Runtime
    Rack::MethodOverride
    ActionDispatch::RequestId
    Rails::Rack::Logger
    ActionDispatch::ShowExceptions
    ActionDispatch::DebugExceptions
    ActionDispatch::RemoteIp
    ActionDispatch::Reloader
    ActionDispatch::Callbacks
    ActiveRecord::ConnectionAdapters
    ActiveRecord::QueryCache
    ActionDispatch::Cookies
    ActionDispatch::Session::CookieStore
    ActionDispatch::Flash
    ActionDispatch::ParamsParser
    Rack::Head
    Rack::ConditionalGet
    Rack::ETag
    AgentSnoop::Middleware
    RpmTestApp::Application.routes
    Wednesday, February 26, 14

    View Slide

  40. Who's There?
    Wednesday, February 26, 14

    View Slide

  41. # [status, headers, response]
    def call(env)
    # ... before
    result = @app.call(env)
    # ... after
    result
    end
    Wednesday, February 26, 14

    View Slide

  42. class MyMiddleware
    def initialize(options={})
    # options == { :foo => 1 }
    end
    def call(worker, msg, queue)
    yield
    end
    end
    Wednesday, February 26, 14

    View Slide

  43. Sidekiq.configure_server do |config|
    config.server_middleware do |chain|
    chain.add MyMiddleware, :foo => 1
    end
    end
    Wednesday, February 26, 14

    View Slide

  44. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  45. Lifecycle
    Wednesday, February 26, 14

    View Slide

  46. Forks and Daemons
    http://flic.kr/p/79tE5Q
    Wednesday, February 26, 14

    View Slide

  47. Forking
    fork()
    state
    Wednesday, February 26, 14

    View Slide

  48. fork()
    state
    fork()
    Forking
    state
    Wednesday, February 26, 14

    View Slide

  49. before_fork
    after_fork
    Wednesday, February 26, 14

    View Slide

  50. Add Threads
    fork()
    Wednesday, February 26, 14

    View Slide

  51. fork()
    Add Threads
    Wednesday, February 26, 14

    View Slide

  52. Enter the Lock
    fork()
    @lock
    Wednesday, February 26, 14

    View Slide

  53. @loick
    fork()
    Enter the Lock
    @lock
    Wednesday, February 26, 14

    View Slide

  54. https://github.com/resque/resque/issues/1101
    Forks +
    Threads + ==
    Locks
    Resque
    Wednesday, February 26, 14

    View Slide

  55. Load Up!
    Wednesday, February 26, 14

    View Slide

  56. unicorn
    ♥ὑὑ~/myapp:unicorn_rails -c ./unicorn.rb
    unicorn_rails
    master
    unicorn_rails
    worker[0]
    unicorn_rails
    worker[1]
    unicorn_rails
    worker[2]
    unicorn_rails
    worker[3]
    Wednesday, February 26, 14

    View Slide

  57. loads gems
    forks
    after_fork
    starts loop
    forks
    after_fork
    starts loop
    loads gems
    preload_app true preload_app false
    unicorn
    Wednesday, February 26, 14

    View Slide

  58. loads gems
    forks
    after_fork
    starts loop
    forks
    after_fork
    starts loop
    loads gems
    preload_app true preload_app false
    unicorn
    Wednesday, February 26, 14

    View Slide

  59. loads gems
    forks
    after_fork
    starts loop
    forks
    after_fork
    starts loop
    loads gems
    preload_app true preload_app false
    unicorn
    Wednesday, February 26, 14

    View Slide

  60. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  61. Names and
    Paths
    Wednesday, February 26, 14

    View Slide

  62. Forced
    Wednesday, February 26, 14

    View Slide

  63. Sequel.plugin :newrelic_instrumentation
    Wednesday, February 26, 14

    View Slide

  64. Sequel.plugin :newrelic_instrumentation
    =>
    require 'sequel/plugins/newrelic_instrumentation'
    Wednesday, February 26, 14

    View Slide

  65. ~/source/newrelic/ruby_agent/lib: ls -la
    drwxr-xr-x Sep 19 14:46 .
    drwxr-xr-x Oct 10 11:19 ..
    drwxr-xr-x Oct 10 11:20 new_relic
    -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb
    drwxr-xr-x Sep 19 14:46 sequel
    drwxr-xr-x Oct 7 08:53 tasks
    Wednesday, February 26, 14

    View Slide

  66. ~/source/newrelic/ruby_agent/lib: ls -la
    drwxr-xr-x Sep 19 14:46 .
    drwxr-xr-x Oct 10 11:19 ..
    drwxr-xr-x Oct 10 11:20 new_relic
    -rw-r--r-- Sep 19 14:46 newrelic_rpm.rb
    drwxr-xr-x Sep 19 14:46 sequel
    drwxr-xr-x Oct 7 08:53 tasks
    Wednesday, February 26, 14

    View Slide

  67. Sequel.plugin NewRelic::Sequel::Instrumentation
    Wednesday, February 26, 14

    View Slide

  68. Generic
    Wednesday, February 26, 14

    View Slide

  69. Wednesday, February 26, 14

    View Slide

  70. ActiveRecord
    Wednesday, February 26, 14

    View Slide

  71. ActiveRecord
    DataMapper
    Wednesday, February 26, 14

    View Slide

  72. ActiveRecord
    DataMapper
    HTTPClient
    Wednesday, February 26, 14

    View Slide

  73. Qualify Yourself
    Wednesday, February 26, 14

    View Slide

  74. newrelic_rpm
    module NewRelic
    module Agent
    class Thread < ::Thread
    ...
    end
    end
    end
    Wednesday, February 26, 14

    View Slide

  75. elsewhere...
    class Thread
    # Re-opens NewRelic::Agent::Thread
    # D'oh!
    end
    Wednesday, February 26, 14

    View Slide

  76. newrelic_plugin
    module NewRelic
    class Logger
    ...
    end
    end
    Wednesday, February 26, 14

    View Slide

  77. newrelic_rpm
    module NewRelic
    module Agent
    class AgentLogger
    LOG_LEVELS = {
    "debug" => Logger::DEBUG,
    ...
    }
    end
    end
    end
    Wednesday, February 26, 14

    View Slide

  78. newrelic_rpm
    module NewRelic
    module Agent
    class AgentLogger
    LOG_LEVELS = {
    "debug" => Logger::DEBUG,
    ...
    }
    end
    end
    end
    Wednesday, February 26, 14

    View Slide

  79. newrelic_rpm
    module NewRelic
    module Agent
    class AgentLogger
    LOG_LEVELS = {
    "debug" => ::Logger::DEBUG,
    ...
    }
    end
    end
    end
    Wednesday, February 26, 14

    View Slide

  80. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  81. Config
    Wednesday, February 26, 14

    View Slide

  82. yml
    Wednesday, February 26, 14

    View Slide

  83. yml + ERB
    Wednesday, February 26, 14

    View Slide

  84. yml + ERB
    file = File.read("./config/awesome_gem.yml")
    # Locals available in ERB
    default_awesome_key = "YEAH!"
    erb = ERB.new(file).result(binding)
    config = YAML.load(erb)
    Wednesday, February 26, 14

    View Slide

  85. yml + ERB
    ~/config/awesome_gem.yml
    development:
    name: <%= default_awesome_key %>
    license_key: <%= ENV["LICENSE_KEY"] %>
    Wednesday, February 26, 14

    View Slide

  86. Ruby
    Wednesday, February 26, 14

    View Slide

  87. unicorn
    worker_processes 5
    preload_app true
    timeout 30
    after_fork do |server, worker|
    defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection
    end
    Wednesday, February 26, 14

    View Slide

  88. on_worker_boot do
    ActiveSupport.on_load(:active_record) do
    ActiveRecord::Base.establish_connection
    end
    end
    Wednesday, February 26, 14

    View Slide

  89. Great for Apps?
    What about Gems?
    Wednesday, February 26, 14

    View Slide

  90. newrelic_rpm
    if defined?(::Puma) &&
    ::Puma.respond_to?(:cli_config)
    config = ::Puma.cli_config
    config.options[:worker_boot] << Proc.new do
    NewRelic::Agent.after_fork(...)
    end
    end
    Wednesday, February 26, 14

    View Slide

  91. newrelic_rpm
    if defined?(::Puma) &&
    ::Puma.respond_to?(:cli_config)
    config = ::Puma.cli_config
    config.options[:worker_boot] << Proc.new do
    NewRelic::Agent.after_fork(...)
    end
    end
    Wednesday, February 26, 14

    View Slide

  92. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  93. Docs
    Wednesday, February 26, 14

    View Slide

  94. README
    Wednesday, February 26, 14

    View Slide

  95. sinatra
    Wednesday, February 26, 14

    View Slide

  96. Wednesday, February 26, 14

    View Slide

  97. Wednesday, February 26, 14

    View Slide

  98. Wednesday, February 26, 14

    View Slide

  99. Expected
    Extensions?
    Wednesday, February 26, 14

    View Slide

  100. Wednesday, February 26, 14

    View Slide

  101. Don't Bury It
    Wednesday, February 26, 14

    View Slide

  102. Wednesday, February 26, 14

    View Slide

  103. Wednesday, February 26, 14

    View Slide

  104. Versioning
    Wednesday, February 26, 14

    View Slide

  105. Wednesday, February 26, 14

    View Slide

  106. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  107. Where We've Been
    Wednesday, February 26, 14

    View Slide

  108. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Wednesday, February 26, 14

    View Slide

  109. Where We've Been
    • Pass It On
    • Events
    • Middleware
    • Lifecycle
    • Names and Paths
    • Config
    • Docs
    Jason Clark
    @jasonrclark
    Ruby Agent Engineer
    Wednesday, February 26, 14

    View Slide