Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ruby Can Too Scale: Highly Performant Microservices in Ruby (Confoo Vancouver 2016)

2fad1f0f6ab9b2a4ed6699e2c96c9a45?s=47 Tim Krajcar
December 05, 2016

Ruby Can Too Scale: Highly Performant Microservices in Ruby (Confoo Vancouver 2016)

Ruby developers have known for years that our beloved language is one of the most joyful to use. And yet, “Ruby Can’t Scale” seems to be a daily article on Hacker News. Tim Krajcar presents some tips and tricks that he’s learned at New Relic writing Ruby services that scale to hundreds of thousands of requests per minute.

Topics include:

* Building performance into your architecture from your first commit
* Where to invest precious development hours to realize the most performance value
* Which frameworks to use, and which to avoid, when working at scale
* Designing to avoid cascading failures when building highly interdependent services

2fad1f0f6ab9b2a4ed6699e2c96c9a45?s=128

Tim Krajcar

December 05, 2016
Tweet

Transcript

  1. Ruby Can Too Scale: Highly Performant Microservices in Ruby Tim

    Krajcar
  2. April 2015

  3. TimKrajcar \

  4. TimKrajcar This document and the information herein (including any information

    that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,” “expects” or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov. New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
  5. TimKrajcar

  6. TimKrajcar why this talk?

  7. TimKrajcar "ruby can't scale"

  8. –Johnny Appleseed

  9. –Johnny Appleseed

  10. –Johnny Appleseed

  11. TimKrajcar "ruby can't scale"

  12. TimKrajcar "rails can't scale"

  13. TimKrajcar second system effect

  14. –Johnny Appleseed

  15. –Johnny Appleseed and, even more importantly, stack

  16. TimKrajcar why ruby?

  17. TimKrajcar expressive programming language = fun to write code =

    I want to write more of it
  18. TimKrajcar expressive programming language = fun to write code =

    I want to write more of it productivity!!
  19. TimKrajcar things not covered

  20. TimKrajcar 1. database tuning

  21. TimKrajcar 1. database tuning 2. caching

  22. TimKrajcar 1. database tuning 2. caching 3. network optimization

  23. TimKrajcar 1. database tuning 2. caching 3. network optimization 4.

    browser-y things
  24. TimKrajcar microservices at New Relic

  25. TimKrajcar ~110 total services

  26. TimKrajcar some not so micro

  27. TimKrajcar Ruby: common for prototypes

  28. TimKrajcar service templates and app auto-generation

  29. –Johnny Appleseed

  30. TimKrajcar stereotypical example: customer permissions service

  31. –Johnny Appleseed

  32. –Johnny Appleseed

  33. PERFORMANCE FROM THE FIRST COMMIT

  34. –Johnny Appleseed Structured programming with 'go to' statements Donald E.

    Knuth COMPUTING SURVEYS, DECEMBER 1974
  35. –Johnny Appleseed We should forget about small efficiencies, say about

    97% of the time: premature optimization is the root of all evil.
  36. –Johnny Appleseed Yet we should not pass up our opportunities

    in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, [and] will be wise to look carefully at the critical code.
  37. TimKrajcar benchmarking methodologies matter

  38. TimKrajcar app server time vs request queuing

  39. –Johnny Appleseed

  40. –Johnny Appleseed

  41. TimKrajcar X-Request-Start HTTP header

  42. TimKrajcar X-Amzn-Trace-Id HTTP header

  43. TimKrajcar test suite tools

  44. TimKrajcar minitest-perf https://github.com/mrsimo/minitest-perf

  45. > bundle exec minitest-perf Slowest individual tests 1210.00ms | NewRelic::Agent::AgentTest#test_after_fork_should_

    1190.78ms | LicenseTest#test_for_scary_license_terms 1034.58ms | NewRelic::Agent::Agent::ConnectTest#test_connect_g 1034.20ms | NewRelic::Agent::Agent::ConnectTest#test_logging_c 1032.60ms | NewRelic::Agent::Agent::ConnectTest#test_environme 1031.61ms | NewRelic::Agent::Agent::ConnectTest#test_connect_s 1030.49ms | NewRelic::Agent::Agent::ConnectTest#test_connect_s 730.88ms | OrphanedConfigTest#test_all_default_source_config_ 455.05ms | NewRelic::Agent::MetricStatsTest#test_record_scope 434.18ms | NewRelicServiceTest#test_json_marshaller_should_ha
  46. Slowest test suites 638.87ms | 2 | LicenseTest 293.86ms |

    3 | OrphanedConfigTest 193.41ms | 27 | NewRelic::Agent::Agent::ConnectTest 191.36ms | 1 | NewRelic::Agent::Threading::BacktraceServic 130.64ms | 14 | NewRelic::Agent::PipeChannelManagerTest 118.76ms | 14 | PipeServiceTest 111.14ms | 4 | DispatcherTest 32.82ms | 63 | NewRelic::Agent::AgentTest 25.50ms | 20 | NewRelic::Agent::MetricStatsTest 19.37ms | 12 | NewRelic::Agent::UtilizationDataTest
  47. TimKrajcar framework choices

  48. TimKrajcar framework choices reductions

  49. –Johnny Appleseed

  50. TimKrajcar example!!

  51. TimKrajcar a typical Ruby microservice: REST DB-to-JSON API https://github.com/tkrajcar/ruby-can-too-scale

  52. TimKrajcar a typical Ruby microservice: REST DB-to-JSON API https://github.com/tkrajcar/ruby-can-too-scale !!

    warning: YOLOCODE !!
  53. TimKrajcar •Rails 4.2.6 straight outta rails new + Puma +

    nginx •JSON only, using scaffolded controller & model •testing #index (~25kb) and #show (<1 kb) •read-only •Dedicated EC2 c3.large instances (2 vCPUs) •Postgres on its own dedicated RDS instance •30 users as fast as possible (separate instance)
  54. TimKrajcar best metric: client requests per minute

  55. TimKrajcar #index /articles.json #show /articles/1.json Stock Rails scaffolding 4,211 18,011

  56. TimKrajcar #index /articles.json #show /articles/1.json Stock Rails scaffolding 4,211 18,011

    No jbuilder templates 4,561 +8.3% 18,909 +4.9%
  57. TimKrajcar #index /articles.json #show /articles/1.json Stock Rails scaffolding 4,211 18,011

    No jbuilder templates 4,561 +8.3% 18,909 +4.9% oj gem 5,548 +21.6% 19,714 +9.4%
  58. TimKrajcar #index /articles.json #show /articles/1.json Stock Rails scaffolding 4,211 18,011

    No jbuilder templates 4,561 +8.3% 18,909 +4.9% oj gem 5,548 +21.6% 19,714 +9.4% Sequel instead of ActiveRecord 7,035 +26.8% 22,378 +13.5%
  59. TimKrajcar #index /articles.json #show /articles/1.json Stock Rails scaffolding 4,211 18,011

    No jbuilder templates 4,561 +8.3% 18,909 +4.9% oj gem 5,548 +21.6% 19,714 +9.4% Sequel instead of ActiveRecord 7,035 +26.8% 22,378 +13.5% Sinatra (with Sequel and oj) 13,245 85,017
  60. TimKrajcar #index /articles.json #show /articles/1.json Stock Rails scaffolding 4,211 18,011

    No jbuilder templates 4,561 +8.3% 18,909 +4.9% oj gem 5,548 +21.6% 19,714 +9.4% Sequel instead of ActiveRecord 7,035 +26.8% 22,378 +13.5% Sinatra (with Sequel and oj) 13,245 85,017
  61. TimKrajcar tl;dr: use Sinatra and Sequel

  62. DIVING DEEPER

  63. TimKrajcar Derailed Benchmarks https://github.com/schneems/derailed_benchmarks

  64. > bundle exec derailed bundle:mem TOP: 46.5625 MiB activeadmin: 17.5156

    MiB active_admin: 17.5039 MiB meta_search: 6.2773 MiB meta_search/searches/active_record: 1.707 MiB meta_search/builder: 1.5938 MiB meta_search/where: 1.0977 MiB polyamorous: 0.4688 MiB active_admin/comments: 3.6055 MiB active_admin/comments/views: 1.375 MiB active_admin/views: 1.2305 MiB (Also required by: active_admin/comments/views/active_admin_comments) /Users/tkrajcar/.rbenv/versions/2.1.7/lib/ruby/gems/ 2.1.0/gems/activeadmin-0.6.3/lib/active_admin/views/components/ paginated_collection.rb: 0.3125 MiB bourbon: 3.3438 MiB
  65. > bundle exec derailed bundle:objects Measuring objects created by gems

    in groups [:default, "production"] Total allocated: 72414462 bytes (425465 objects) Total retained: 24736698 bytes (44790 objects) allocated memory by gem ----------------------------------- 27407703 activesupport-3.2.22.2 9240545 erubis-2.7.0 5722758 actionpack-3.2.22.2 3287857 activerecord-3.2.22.2 3176375 haml-4.0.5 1728840 meta_search-1.1.3 1300638 devise-1.5.4 1216006 2.1.7/lib 1016914 formtastic-2.2.1 993225 activeadmin-0.6.3
  66. TimKrajcar Stackprof https://github.com/tmm1/stackprof

  67. –Johnny Appleseed

  68. TimKrajcar other interpreters

  69. DESIGNING TO AVOID CASCADING FAILURES

  70. TimKrajcar timeouts

  71. –Johnny Appleseed

  72. None
  73. TimKrajcar Timeout

  74. require 'timeout' result = begin Timeout::timeout(0.5) { do_a_possibly_slow_thing } rescue

    Timeout::Error # log/notify about the timeout 0 end
  75. None
  76. None
  77. None
  78. TimKrajcar circuit breaker pattern

  79. –Johnny Appleseed http://martinfowler.com/bliki/CircuitBreaker.html

  80. TimKrajcar cb2 https://github.com/pedro/cb2

  81. require 'cb2' breaker = CB2::Breaker.new( service: "cashwings", # identify each

    circuit breaker individually duration: 60, # in this amount of seconds... threshold: 5, # ...this % of errors will open the breaker reenable_after: 600, # once opened, how long until we try again redis: Redis.new) # uses Redis to keep track by service name
  82. require 'cb2' breaker = CB2::Breaker.new( service: "cashwings", # identify each

    circuit breaker individually duration: 60, # in this amount of seconds... threshold: 5, # ...this % of errors will open the breaker reenable_after: 600, # once opened, how long until we try again redis: Redis.new) # uses Redis to keep track by service name
  83. begin breaker.run do some_api_request() end rescue CB2::BreakerOpen # log/notify about

    the breaker opening! alternate_response() end
  84. TimKrajcar parallelize

  85. TimKrajcar parallelize everything especially HTTP

  86. TimKrajcar Typhoeus::Hydra https://github.com/typhoeus/typhoeus

  87. require 'typhoeus' request1 = Typhoeus::Request.new("http://example.com/1.json") request2 = Typhoeus::Request.new("http://example.com/2.json") hydra =

    Typhoeus::Hydra.new hydra.queue(request1) hydra.queue(request2) hydra.run # blocks here until queue finishes puts "Request 1 status #{request1.response.code}" puts "Request 2 status #{request2.response.code}"
  88. TimKrajcar background everything* *unless you can't

  89. TimKrajcar https://github.com/mperham/sidekiq

  90. # app/workers/welcome_emailer.rb class WelcomeEmailer include Sidekiq::Worker def perform(email) send_email_to(email, "Welcome!")

    end end # app/controllers/welcome_controller.rb def welcome # blah blah create a @user WelcomeEmailer.perform_async(@user.email) end
  91. # app/workers/welcome_emailer.rb class WelcomeEmailer include Sidekiq::Worker def perform(email) send_email_to(email, "Welcome!")

    end end # app/controllers/welcome_controller.rb def welcome # blah blah create a @user WelcomeEmailer.perform_async(@user.email) end doesn't block
  92. TimKrajcar rate limit by client

  93. TimKrajcar Rack::Throttle https://github.com/bendiken/rack-throttle

  94. # config/application.rb require 'rack/throttle' class Application < Rails::Application config.middleware.use Rack::Throttle::Hourly,

    max: 100 end
  95. # config/application.rb require 'rack/throttle' class Application < Rails::Application config.middleware.use Rack::Throttle::Hourly,

    max: 100 end client = by IP or you can define custom client IDs
  96. TimKrajcar monitoring & alerting

  97. TimKrajcar low traffic? drive synthetic traffic

  98. TimKrajcar synthetic-monitor https://github.com/johnboyes/synthetic-monitor

  99. None
  100. TimKrajcar RECAP!!

  101. TimKrajcar 1. use Ruby to write services, because it's fun!

    2. monitor both request queuing time and app server time 3. make informed framework decisions (use sinatra) 4. set real timeouts and use circuit breakers 5. parallelize everything 6. background everything 7. rate limit by client, even for internal clients 8. drive synthetic traffic to help catch problems right away
  102. TimKrajcar references

  103. TimKrajcar Nate Berkopec: nateberkopec.com High Scalability: highscalability.com Improved Production Stability

    With Circuit Breakers: blog.heroku.com
  104. TimKrajcar thanks!