$30 off During Our Annual Pro Sale. View Details »

Building Asynchronous APIs

Building Asynchronous APIs

flyerhzm

June 07, 2013
Tweet

More Decks by flyerhzm

Other Decks in Technology

Transcript

  1. Building Asynchronous APIs
    Richard Huang
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  2. I’m ...
    • Currently Ruby/Rails Freelancer
    • Previously worked for OpenFeint (GREE)
    • Author of bullet and rails_best_practices gems
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  3. Hardware is Cheap,
    Programmers are Expensive
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  4. App Server
    DB Server
    Web Server
    Single Server
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  5. but...
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  6. App Server
    DB Server
    Web Server
    Load Balancer
    App Server
    DB Server
    Web Server
    App Server
    Web Server
    App Server
    Web Server
    Memcached
    ...
    ...
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    ...
    Friday, June 7, 13

    View Slide

  7. Hardwares get Expensive,
    so...
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  8. http://highscalability.com/blog/2012/10/4/linkedin-moved-from-rails-to-node-27-servers-cut-and-up-to-2.html
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  9. http://highscalability.com/blog/2013/3/13/ironio-moved-from-ruby-to-go-28-servers-cut-and-colossal-clu.html
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  10. Is Ruby/Rails so slow?
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  11. From my experience, no!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  12. • Rails REE 1.8.7 Mysql 50 ms 60k rpm
    13 machines x 6 passenger instances
    • Rails JRuby 1.7.0 Mysql 30 ms 60k rpm
    10 machines x torquebox (5 threads)
    • Goliath Ruby 1.9.3 Redis 4 ms 240k rpm
    4 machines x 4 goliath instances
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  13. Migrating from ruby to ruby,
    48 servers cut and10x faster
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  14. Sync vs Async
    Blocking vs Nonblocking
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  15. Blocking IO
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    DB HTTP
    IO
    CPU
    Friday, June 7, 13

    View Slide

  16. Multi Processes
    DB HTTP
    IO
    CPU
    DB HTTP
    IO
    CPU
    Process A
    Process B
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  17. Multi Processes
    • Parallel on a multi cores cpu
    • Easy to manage
    • Process switching is expensive
    • Consume many memory
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  18. Multi Threads
    DB HTTP
    IO
    CPU
    DB HTTP
    IO
    CPU
    Thread A
    Thread B
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  19. Multi Threads
    • Thread switching is cheap
    • Consume less memory
    • Thread safety
    • GIL
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  20. Evented
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    DB HTTP
    IO
    CPU
    Friday, June 7, 13

    View Slide

  21. Evented (Eventmachine)
    • No blocking io
    • No context switching
    • Least memory usage
    • Callback! Callback! Callback!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  22. eventmachine
    Friday, June 7, 13

    View Slide

  23. Evented (em-synchrony)
    • EM + Fiber = No callback!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  24. em-synchrony
    Friday, June 7, 13

    View Slide

  25. Concurrency
    Parallelism
    Friday, June 7, 13

    View Slide

  26. Multi Processes + Evented
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  27. How?
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    https://github.com/flyerhzm/apis-bench
    Friday, June 7, 13

    View Slide

  28. View
    Controller
    Model
    Skinny controller
    Fat model
    Render json response
    Rails
    Router Dispatch request
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  29. Model
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  30. Controller / View
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  31. Router
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  32. Rails => Grape
    Decrease response time
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  33. Grape
    Model
    Router / Controller / View
    Fat model
    Grape
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  34. Fat model
    No need to do any change, nice!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  35. Grape
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  36. Grape =>
    Grape + Goliath
    Increase throughput
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  37. Grpae
    Model
    Router / Controller / View
    Fat model
    Grape + Goliath
    Goliath Non-blocking web server framework
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  38. Goliath with Grape
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  39. Goliath with Grape
    mysql2 => em_mysql2
    net/http => em-http-request
    mongo => em-mongo
    ...
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  40. Rails => Sinatra
    Decrease response time
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  41. Sinatra
    Model
    Router / Controller / View
    Fat model
    Sinatra
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  42. Sinatra
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  43. Sinatra =>
    Sinatra-synchrony+Thin
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  44. Sinatra / Sinatra-synchrony
    Model
    Router / Controller / View
    Fat model
    Sinatra-synchrony+Thin
    Thin Ruby web server based on EM
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  45. Sinatra-synchrony
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  46. Sinatra-synchrony
    em-* libraries
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  47. Benchmark
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    https://github.com/flyerhzm/apis-bench
    Friday, June 7, 13

    View Slide

  48. CPU bound
    measured with ab
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  49. 0
    5
    10
    15
    20
    n1000 c10 n1000 c50 n1000 c100 n1000 c200 n1000 c500 n2000 c1000
    rails sinatra
    grape sinatra threads
    grape threads sinatra-synchrony + thin
    grape + goliath
    rails, sinatra, grape, sinatra threads and grape threads are timed out with n 1000 c 200
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  50. IO bound
    measured with NewRelic
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  51. leaderboard rails 1200rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  52. leaderboard-sinatra 1200 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  53. leaderboard-grape 1200 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  54. leaderboard-sinatra threads 3000 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  55. leaderboard-grape threads 3000 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  56. leaderboard grape goliath 3000 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13

    View Slide

  57. flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Known issues
    • Failed to add newrelic to sinatra-synchrony
    • Failed to add fiber_pool to goliath
    Friday, June 7, 13

    View Slide

  58. Conclusion
    • Rails is good
    • Sinatra/Grape decreases response time
    • Sinatra-synchrony/Goliath increases throughput
    • Async I/O is awesome! Try
    Friday, June 7, 13

    View Slide

  59. Thank you
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Questions?
    Friday, June 7, 13

    View Slide