Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ruby Can Too Scale: Highly Performant Microservices in Ruby (OSCON 2016)

Ruby Can Too Scale: Highly Performant Microservices in Ruby (OSCON 2016)

Ruby developers have known for years that our beloved language is one of the most joyful to use. And yet, “Ruby Can’t Scale” seems to be a daily article on Hacker News. Tim Krajcar presents some tips and tricks that he’s learned at New Relic writing Ruby services that scale to hundreds of thousands of requests per minute.

Topics include:

* Building performance into your architecture from your first commit
* Where to invest precious development hours to realize the most performance value
* Which frameworks to use, and which to avoid, when working at scale
* Designing to avoid cascading failures when building highly interdependent services

Tim Krajcar

May 18, 2016
Tweet

More Decks by Tim Krajcar

Other Decks in Technology

Transcript

  1. Ruby Can Too Scale:
    Highly Performant Microservices in Ruby
    Tim Krajcar

    View full-size slide

  2. TimKrajcar
    This document and the information herein (including any information that may be incorporated by reference) is provided for
    informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc.
    (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby
    is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.
    Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a
    historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future
    events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the
    statement will include words such as “believes,” “anticipates,” “expects” or words of similar import.
    Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date
    hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties
    transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement
    or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations,
    and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual
    results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further
    information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time
    to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the
    SEC’s website at www.sec.gov.
    New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New
    Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.

    View full-size slide

  3. TimKrajcar
    why this talk?

    View full-size slide

  4. TimKrajcar
    "ruby can't scale"

    View full-size slide

  5. –Johnny Appleseed

    View full-size slide

  6. –Johnny Appleseed

    View full-size slide

  7. –Johnny Appleseed

    View full-size slide

  8. TimKrajcar
    "ruby can't scale"

    View full-size slide

  9. TimKrajcar
    "rails can't scale"

    View full-size slide

  10. TimKrajcar
    second system
    effect

    View full-size slide

  11. –Johnny Appleseed

    View full-size slide

  12. –Johnny Appleseed
    and, even more importantly, stack

    View full-size slide

  13. TimKrajcar
    why ruby?

    View full-size slide

  14. TimKrajcar
    expressive programming language
    =
    fun to write code
    =
    I want to write more of it

    View full-size slide

  15. TimKrajcar
    expressive programming language
    =
    fun to write code
    =
    I want to write more of it
    productivity!!

    View full-size slide

  16. TimKrajcar
    things not covered

    View full-size slide

  17. TimKrajcar
    1. database tuning

    View full-size slide

  18. TimKrajcar
    1. database tuning
    2. caching

    View full-size slide

  19. TimKrajcar
    1. database tuning
    2. caching
    3. network optimization

    View full-size slide

  20. TimKrajcar
    1. database tuning
    2. caching
    3. network optimization
    4. browser-y things

    View full-size slide

  21. TimKrajcar
    microservices at New
    Relic

    View full-size slide

  22. TimKrajcar
    ~60 total services

    View full-size slide

  23. TimKrajcar
    some not so micro

    View full-size slide

  24. TimKrajcar
    Ruby:
    common for prototypes

    View full-size slide

  25. TimKrajcar
    templates and auto-
    generation

    View full-size slide

  26. –Johnny Appleseed

    View full-size slide

  27. TimKrajcar
    stereotypical example:
    customer permissions service

    View full-size slide

  28. –Johnny Appleseed

    View full-size slide

  29. –Johnny Appleseed

    View full-size slide

  30. PERFORMANCE FROM THE
    FIRST COMMIT

    View full-size slide

  31. –Johnny Appleseed
    Structured programming with 'go to' statements
    Donald E. Knuth
    COMPUTING SURVEYS, DECEMBER 1974

    View full-size slide

  32. –Johnny Appleseed
    We should forget about small
    efficiencies, say about 97% of the
    time: premature optimization is the
    root of all evil.

    View full-size slide

  33. –Johnny Appleseed
    Yet we should not pass up our opportunities in
    that critical 3%. A good programmer will not be
    lulled into complacency by such reasoning,
    [and] will be wise to look carefully at the critical
    code.

    View full-size slide

  34. TimKrajcar
    benchmarking
    methodologies matter

    View full-size slide

  35. TimKrajcar
    app server time
    vs
    request queuing

    View full-size slide

  36. –Johnny Appleseed

    View full-size slide

  37. –Johnny Appleseed

    View full-size slide

  38. TimKrajcar
    X-Request-Start
    HTTP header

    View full-size slide

  39. TimKrajcar
    test suite tools

    View full-size slide

  40. TimKrajcar
    minitest-perf
    https://github.com/mrsimo/minitest-perf

    View full-size slide

  41. > bundle exec minitest-perf
    Slowest individual tests
    1210.00ms | NewRelic::Agent::AgentTest#test_after_fork_should_
    1190.78ms | LicenseTest#test_for_scary_license_terms
    1034.58ms | NewRelic::Agent::Agent::ConnectTest#test_connect_g
    1034.20ms | NewRelic::Agent::Agent::ConnectTest#test_logging_c
    1032.60ms | NewRelic::Agent::Agent::ConnectTest#test_environme
    1031.61ms | NewRelic::Agent::Agent::ConnectTest#test_connect_s
    1030.49ms | NewRelic::Agent::Agent::ConnectTest#test_connect_s
    730.88ms | OrphanedConfigTest#test_all_default_source_config_
    455.05ms | NewRelic::Agent::MetricStatsTest#test_record_scope
    434.18ms | NewRelicServiceTest#test_json_marshaller_should_ha

    View full-size slide

  42. Slowest test suites
    638.87ms | 2 | LicenseTest
    293.86ms | 3 | OrphanedConfigTest
    193.41ms | 27 | NewRelic::Agent::Agent::ConnectTest
    191.36ms | 1 | NewRelic::Agent::Threading::BacktraceServic
    130.64ms | 14 | NewRelic::Agent::PipeChannelManagerTest
    118.76ms | 14 | PipeServiceTest
    111.14ms | 4 | DispatcherTest
    32.82ms | 63 | NewRelic::Agent::AgentTest
    25.50ms | 20 | NewRelic::Agent::MetricStatsTest
    19.37ms | 12 | NewRelic::Agent::UtilizationDataTest

    View full-size slide

  43. TimKrajcar
    framework choices

    View full-size slide

  44. TimKrajcar
    framework choices
    reductions

    View full-size slide

  45. –Johnny Appleseed

    View full-size slide

  46. TimKrajcar
    example!!

    View full-size slide

  47. TimKrajcar
    a typical Ruby microservice:
    REST DB-to-JSON API
    https://github.com/tkrajcar/ruby-can-too-scale

    View full-size slide

  48. TimKrajcar
    a typical Ruby microservice:
    REST DB-to-JSON API
    https://github.com/tkrajcar/ruby-can-too-scale
    !! warning: YOLOCODE !!

    View full-size slide

  49. TimKrajcar
    •Rails 4.2.6 straight outta rails new + Puma +
    nginx
    •JSON only, using scaffolded controller & model
    •testing #index (~25kb) and #show (<1 kb)
    •read-only
    •Dedicated EC2 c3.large instances (2 vCPUs)
    •Postgres on its own dedicated RDS instance
    •30 users as fast as possible (separate instance)

    View full-size slide

  50. TimKrajcar
    best metric:
    client requests per minute

    View full-size slide

  51. TimKrajcar
    #index
    /articles.json
    #show
    /articles/1.json
    Stock Rails scaffolding 4,211 18,011

    View full-size slide

  52. TimKrajcar
    #index
    /articles.json
    #show
    /articles/1.json
    Stock Rails scaffolding 4,211 18,011
    No jbuilder templates
    4,561
    +8.3%
    18,909
    +4.9%

    View full-size slide

  53. TimKrajcar
    #index
    /articles.json
    #show
    /articles/1.json
    Stock Rails scaffolding 4,211 18,011
    No jbuilder templates
    4,561
    +8.3%
    18,909
    +4.9%
    oj gem
    5,548
    +21.6%
    19,714
    +9.4%

    View full-size slide

  54. TimKrajcar
    #index
    /articles.json
    #show
    /articles/1.json
    Stock Rails scaffolding 4,211 18,011
    No jbuilder templates
    4,561
    +8.3%
    18,909
    +4.9%
    oj gem
    5,548
    +21.6%
    19,714
    +9.4%
    Sequel instead of
    ActiveRecord
    7,035
    +26.8%
    22,378
    +13.5%

    View full-size slide

  55. TimKrajcar
    #index
    /articles.json
    #show
    /articles/1.json
    Stock Rails scaffolding 4,211 18,011
    No jbuilder templates
    4,561
    +8.3%
    18,909
    +4.9%
    oj gem
    5,548
    +21.6%
    19,714
    +9.4%
    Sequel instead of
    ActiveRecord
    7,035
    +26.8%
    22,378
    +13.5%
    Sinatra
    (with Sequel and oj)
    13,245 85,017

    View full-size slide

  56. TimKrajcar
    #index
    /articles.json
    #show
    /articles/1.json
    Stock Rails scaffolding 4,211 18,011
    No jbuilder templates
    4,561
    +8.3%
    18,909
    +4.9%
    oj gem
    5,548
    +21.6%
    19,714
    +9.4%
    Sequel instead of
    ActiveRecord
    7,035
    +26.8%
    22,378
    +13.5%
    Sinatra
    (with Sequel and oj)
    13,245 85,017


    View full-size slide

  57. TimKrajcar
    tl;dr:
    use Sinatra and Sequel

    View full-size slide

  58. DIVING DEEPER

    View full-size slide

  59. TimKrajcar
    Derailed Benchmarks
    https://github.com/schneems/derailed_benchmarks

    View full-size slide

  60. > bundle exec derailed bundle:mem
    TOP: 46.5625 MiB
    activeadmin: 17.5156 MiB
    active_admin: 17.5039 MiB
    meta_search: 6.2773 MiB
    meta_search/searches/active_record: 1.707 MiB
    meta_search/builder: 1.5938 MiB
    meta_search/where: 1.0977 MiB
    polyamorous: 0.4688 MiB
    active_admin/comments: 3.6055 MiB
    active_admin/comments/views: 1.375 MiB
    active_admin/views: 1.2305 MiB (Also required by:
    active_admin/comments/views/active_admin_comments)
    /Users/tkrajcar/.rbenv/versions/2.1.7/lib/ruby/gems/
    2.1.0/gems/activeadmin-0.6.3/lib/active_admin/views/components/
    paginated_collection.rb: 0.3125 MiB
    bourbon: 3.3438 MiB

    View full-size slide

  61. > bundle exec derailed bundle:objects
    Measuring objects created by gems in groups [:default, "production"]
    Total allocated: 72414462 bytes (425465 objects)
    Total retained: 24736698 bytes (44790 objects)
    allocated memory by gem
    -----------------------------------
    27407703 activesupport-3.2.22.2
    9240545 erubis-2.7.0
    5722758 actionpack-3.2.22.2
    3287857 activerecord-3.2.22.2
    3176375 haml-4.0.5
    1728840 meta_search-1.1.3
    1300638 devise-1.5.4
    1216006 2.1.7/lib
    1016914 formtastic-2.2.1
    993225 activeadmin-0.6.3

    View full-size slide

  62. TimKrajcar
    Stackprof
    https://github.com/tmm1/stackprof

    View full-size slide

  63. –Johnny Appleseed

    View full-size slide

  64. TimKrajcar
    other interpreters

    View full-size slide

  65. DESIGNING TO AVOID
    CASCADING FAILURES

    View full-size slide

  66. TimKrajcar
    timeouts

    View full-size slide

  67. –Johnny Appleseed

    View full-size slide

  68. TimKrajcar
    Timeout

    View full-size slide

  69. require 'timeout'
    result = begin
    Timeout::timeout(0.5) { do_a_possibly_slow_thing }
    rescue Timeout::Error
    # log/notify about the timeout
    0
    end

    View full-size slide

  70. TimKrajcar
    circuit breaker pattern

    View full-size slide

  71. –Johnny Appleseed
    http://martinfowler.com/bliki/CircuitBreaker.html

    View full-size slide

  72. TimKrajcar
    cb2
    https://github.com/pedro/cb2

    View full-size slide

  73. require 'cb2'
    breaker = CB2::Breaker.new(
    service: "cashwings", # identify each circuit breaker individually
    duration: 60, # in this amount of seconds...
    threshold: 5, # ...this % of errors will open the breaker
    reenable_after: 600, # once opened, how long until we try again
    redis: Redis.new) # uses Redis to keep track by service name

    View full-size slide

  74. require 'cb2'
    breaker = CB2::Breaker.new(
    service: "cashwings", # identify each circuit breaker individually
    duration: 60, # in this amount of seconds...
    threshold: 5, # ...this % of errors will open the breaker
    reenable_after: 600, # once opened, how long until we try again
    redis: Redis.new) # uses Redis to keep track by service name

    View full-size slide

  75. begin
    breaker.run do
    some_api_request()
    end
    rescue CB2::BreakerOpen
    # log/notify about the breaker opening!
    alternate_response()
    end

    View full-size slide

  76. TimKrajcar
    parallelize

    View full-size slide

  77. TimKrajcar
    parallelize everything
    especially HTTP

    View full-size slide

  78. TimKrajcar
    Typhoeus::Hydra
    https://github.com/typhoeus/typhoeus

    View full-size slide

  79. require 'typhoeus'
    request1 = Typhoeus::Request.new("http://example.com/1.json")
    request2 = Typhoeus::Request.new("http://example.com/2.json")
    hydra = Typhoeus::Hydra.new
    hydra.queue(request1)
    hydra.queue(request2)
    hydra.run # blocks here until queue finishes
    puts "Request 1 status #{request1.response.code}"
    puts "Request 2 status #{request2.response.code}"

    View full-size slide

  80. TimKrajcar
    background everything*
    *unless you can't

    View full-size slide

  81. TimKrajcar
    https://github.com/mperham/sidekiq

    View full-size slide

  82. # app/workers/welcome_emailer.rb
    class WelcomeEmailer
    include Sidekiq::Worker
    def perform(email)
    send_email_to(email, "Welcome!")
    end
    end
    # app/controllers/welcome_controller.rb
    def welcome
    # blah blah create a @user
    WelcomeEmailer.perform_async(@user.email)
    end

    View full-size slide

  83. # app/workers/welcome_emailer.rb
    class WelcomeEmailer
    include Sidekiq::Worker
    def perform(email)
    send_email_to(email, "Welcome!")
    end
    end
    # app/controllers/welcome_controller.rb
    def welcome
    # blah blah create a @user
    WelcomeEmailer.perform_async(@user.email)
    end
    doesn't
    block

    View full-size slide

  84. TimKrajcar
    rate limit by client

    View full-size slide

  85. TimKrajcar
    Rack::Throttle
    https://github.com/bendiken/rack-throttle

    View full-size slide

  86. # config/application.rb
    require 'rack/throttle'
    class Application < Rails::Application
    config.middleware.use Rack::Throttle::Hourly, max: 100
    end

    View full-size slide

  87. # config/application.rb
    require 'rack/throttle'
    class Application < Rails::Application
    config.middleware.use Rack::Throttle::Hourly, max: 100
    end
    client = by IP
    or you can define custom client IDs

    View full-size slide

  88. TimKrajcar
    monitoring
    &
    alerting

    View full-size slide

  89. TimKrajcar
    low traffic?
    drive synthetic traffic

    View full-size slide

  90. TimKrajcar
    synthetic-monitor
    https://github.com/johnboyes/synthetic-monitor

    View full-size slide

  91. TimKrajcar
    RECAP!!

    View full-size slide

  92. TimKrajcar
    1. use Ruby to write services, because it's fun!
    2. monitor both request queuing time and app server time
    3. make informed framework decisions (use sinatra)
    4. set real timeouts and use circuit breakers
    5. parallelize everything
    6. background everything
    7. rate limit by client, even for internal clients
    8. drive synthetic traffic to help catch problems right away

    View full-size slide

  93. TimKrajcar
    references

    View full-size slide

  94. TimKrajcar
    Nate Berkopec: nateberkopec.com
    High Scalability: highscalability.com
    Improved Production Stability With Circuit
    Breakers: blog.heroku.com

    View full-size slide

  95. TimKrajcar
    thanks!

    View full-size slide