Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Asynchronous APIs

Building Asynchronous APIs

13c60e2a5806406b537daa542d3495da?s=128

flyerhzm

June 07, 2013
Tweet

Transcript

  1. Building Asynchronous APIs Richard Huang flyerhzm at gmail dot com

    github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  2. I’m ... • Currently Ruby/Rails Freelancer • Previously worked for

    OpenFeint (GREE) • Author of bullet and rails_best_practices gems flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  3. Hardware is Cheap, Programmers are Expensive flyerhzm at gmail dot

    com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  4. App Server DB Server Web Server Single Server flyerhzm at

    gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  5. but... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  6. App Server DB Server Web Server Load Balancer App Server

    DB Server Web Server App Server Web Server App Server Web Server Memcached ... ... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm ... Friday, June 7, 13
  7. Hardwares get Expensive, so... flyerhzm at gmail dot com github.com/flyerhzm

    twitter.com/flyerhzm Friday, June 7, 13
  8. http://highscalability.com/blog/2012/10/4/linkedin-moved-from-rails-to-node-27-servers-cut-and-up-to-2.html flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  9. http://highscalability.com/blog/2013/3/13/ironio-moved-from-ruby-to-go-28-servers-cut-and-colossal-clu.html flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  10. Is Ruby/Rails so slow? flyerhzm at gmail dot com github.com/flyerhzm

    twitter.com/flyerhzm Friday, June 7, 13
  11. From my experience, no! flyerhzm at gmail dot com github.com/flyerhzm

    twitter.com/flyerhzm Friday, June 7, 13
  12. • Rails REE 1.8.7 Mysql 50 ms 60k rpm 13

    machines x 6 passenger instances • Rails JRuby 1.7.0 Mysql 30 ms 60k rpm 10 machines x torquebox (5 threads) • Goliath Ruby 1.9.3 Redis 4 ms 240k rpm 4 machines x 4 goliath instances flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  13. Migrating from ruby to ruby, 48 servers cut and10x faster

    flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  14. Sync vs Async Blocking vs Nonblocking flyerhzm at gmail dot

    com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  15. Blocking IO flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm DB

    HTTP IO CPU Friday, June 7, 13
  16. Multi Processes DB HTTP IO CPU DB HTTP IO CPU

    Process A Process B flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  17. Multi Processes • Parallel on a multi cores cpu •

    Easy to manage • Process switching is expensive • Consume many memory flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  18. Multi Threads DB HTTP IO CPU DB HTTP IO CPU

    Thread A Thread B flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  19. Multi Threads • Thread switching is cheap • Consume less

    memory • Thread safety • GIL flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  20. Evented flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm DB HTTP

    IO CPU Friday, June 7, 13
  21. Evented (Eventmachine) • No blocking io • No context switching

    • Least memory usage • Callback! Callback! Callback! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  22. eventmachine Friday, June 7, 13

  23. Evented (em-synchrony) • EM + Fiber = No callback! flyerhzm

    at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  24. em-synchrony Friday, June 7, 13

  25. Concurrency Parallelism Friday, June 7, 13

  26. Multi Processes + Evented flyerhzm at gmail dot com github.com/flyerhzm

    twitter.com/flyerhzm Friday, June 7, 13
  27. How? flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm https://github.com/flyerhzm/apis-bench Friday,

    June 7, 13
  28. View Controller Model Skinny controller Fat model Render json response

    Rails Router Dispatch request flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  29. Model flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  30. Controller / View flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm

    Friday, June 7, 13
  31. Router flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  32. Rails => Grape Decrease response time flyerhzm at gmail dot

    com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  33. Grape Model Router / Controller / View Fat model Grape

    flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  34. Fat model No need to do any change, nice! flyerhzm

    at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  35. Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  36. Grape => Grape + Goliath Increase throughput flyerhzm at gmail

    dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  37. Grpae Model Router / Controller / View Fat model Grape

    + Goliath Goliath Non-blocking web server framework flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  38. Goliath with Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm

    Friday, June 7, 13
  39. Goliath with Grape mysql2 => em_mysql2 net/http => em-http-request mongo

    => em-mongo ... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  40. Rails => Sinatra Decrease response time flyerhzm at gmail dot

    com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  41. Sinatra Model Router / Controller / View Fat model Sinatra

    flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  42. Sinatra flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  43. Sinatra => Sinatra-synchrony+Thin flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm

    Friday, June 7, 13
  44. Sinatra / Sinatra-synchrony Model Router / Controller / View Fat

    model Sinatra-synchrony+Thin Thin Ruby web server based on EM flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  45. Sinatra-synchrony flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June

    7, 13
  46. Sinatra-synchrony em-* libraries flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm

    Friday, June 7, 13
  47. Benchmark flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm https://github.com/flyerhzm/apis-bench Friday,

    June 7, 13
  48. CPU bound measured with ab flyerhzm at gmail dot com

    github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  49. 0 5 10 15 20 n1000 c10 n1000 c50 n1000

    c100 n1000 c200 n1000 c500 n2000 c1000 rails sinatra grape sinatra threads grape threads sinatra-synchrony + thin grape + goliath rails, sinatra, grape, sinatra threads and grape threads are timed out with n 1000 c 200 flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  50. IO bound measured with NewRelic flyerhzm at gmail dot com

    github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  51. leaderboard rails 1200rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm

    Friday, June 7, 13
  52. leaderboard-sinatra 1200 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm

    Friday, June 7, 13
  53. leaderboard-grape 1200 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm

    Friday, June 7, 13
  54. leaderboard-sinatra threads 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm

    twitter.com/flyerhzm Friday, June 7, 13
  55. leaderboard-grape threads 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm

    twitter.com/flyerhzm Friday, June 7, 13
  56. leaderboard grape goliath 3000 rpm flyerhzm at gmail dot com

    github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
  57. flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Known issues •

    Failed to add newrelic to sinatra-synchrony • Failed to add fiber_pool to goliath Friday, June 7, 13
  58. Conclusion • Rails is good • Sinatra/Grape decreases response time

    • Sinatra-synchrony/Goliath increases throughput • Async I/O is awesome! Try Friday, June 7, 13
  59. Thank you flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Questions?

    Friday, June 7, 13