Building Asynchronous APIs

Transcript

1. Building Asynchronous APIs
   Richard Huang
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

2. I’m ...
   • Currently Ruby/Rails Freelancer
   • Previously worked for OpenFeint (GREE)
   • Author of bullet and rails_best_practices gems
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

3. Hardware is Cheap,
   Programmers are Expensive
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

4. App Server
   DB Server
   Web Server
   Single Server
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

5. but...
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

6. App Server
   DB Server
   Web Server
   Load Balancer
   App Server
   DB Server
   Web Server
   App Server
   Web Server
   App Server
   Web Server
   Memcached
   ...
   ...
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   ...
   Friday, June 7, 13
   

   View Slide

7. Hardwares get Expensive,
   so...
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

8. http://highscalability.com/blog/2012/10/4/linkedin-moved-from-rails-to-node-27-servers-cut-and-up-to-2.html
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

9. http://highscalability.com/blog/2013/3/13/ironio-moved-from-ruby-to-go-28-servers-cut-and-colossal-clu.html
   flyerhzm at gmail dot com
   github.com/flyerhzm twitter.com/flyerhzm
   Friday, June 7, 13
   

   View Slide

10. Is Ruby/Rails so slow?
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

11. From my experience, no!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

12. • Rails REE 1.8.7 Mysql 50 ms 60k rpm
    13 machines x 6 passenger instances
    • Rails JRuby 1.7.0 Mysql 30 ms 60k rpm
    10 machines x torquebox (5 threads)
    • Goliath Ruby 1.9.3 Redis 4 ms 240k rpm
    4 machines x 4 goliath instances
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

13. Migrating from ruby to ruby,
    48 servers cut and10x faster
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

14. Sync vs Async
    Blocking vs Nonblocking
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

15. Blocking IO
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    DB HTTP
    IO
    CPU
    Friday, June 7, 13
    

    View Slide

16. Multi Processes
    DB HTTP
    IO
    CPU
    DB HTTP
    IO
    CPU
    Process A
    Process B
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

17. Multi Processes
    • Parallel on a multi cores cpu
    • Easy to manage
    • Process switching is expensive
    • Consume many memory
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

18. Multi Threads
    DB HTTP
    IO
    CPU
    DB HTTP
    IO
    CPU
    Thread A
    Thread B
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

19. Multi Threads
    • Thread switching is cheap
    • Consume less memory
    • Thread safety
    • GIL
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

20. Evented
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    DB HTTP
    IO
    CPU
    Friday, June 7, 13
    

    View Slide

21. Evented (Eventmachine)
    • No blocking io
    • No context switching
    • Least memory usage
    • Callback! Callback! Callback!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

22. eventmachine
    Friday, June 7, 13
    

    View Slide

23. Evented (em-synchrony)
    • EM + Fiber = No callback!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

24. em-synchrony
    Friday, June 7, 13
    

    View Slide

25. Concurrency
    Parallelism
    Friday, June 7, 13
    

    View Slide

26. Multi Processes + Evented
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

27. How?
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    https://github.com/flyerhzm/apis-bench
    Friday, June 7, 13
    

    View Slide

28. View
    Controller
    Model
    Skinny controller
    Fat model
    Render json response
    Rails
    Router Dispatch request
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

29. Model
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

30. Controller / View
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

31. Router
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

32. Rails => Grape
    Decrease response time
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

33. Grape
    Model
    Router / Controller / View
    Fat model
    Grape
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

34. Fat model
    No need to do any change, nice!
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

35. Grape
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

36. Grape =>
    Grape + Goliath
    Increase throughput
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

37. Grpae
    Model
    Router / Controller / View
    Fat model
    Grape + Goliath
    Goliath Non-blocking web server framework
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

38. Goliath with Grape
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

39. Goliath with Grape
    mysql2 => em_mysql2
    net/http => em-http-request
    mongo => em-mongo
    ...
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

40. Rails => Sinatra
    Decrease response time
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

41. Sinatra
    Model
    Router / Controller / View
    Fat model
    Sinatra
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

42. Sinatra
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

43. Sinatra =>
    Sinatra-synchrony+Thin
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

44. Sinatra / Sinatra-synchrony
    Model
    Router / Controller / View
    Fat model
    Sinatra-synchrony+Thin
    Thin Ruby web server based on EM
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

45. Sinatra-synchrony
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

46. Sinatra-synchrony
    em-* libraries
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

47. Benchmark
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    https://github.com/flyerhzm/apis-bench
    Friday, June 7, 13
    

    View Slide

48. CPU bound
    measured with ab
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

49. 0
    5
    10
    15
    20
    n1000 c10 n1000 c50 n1000 c100 n1000 c200 n1000 c500 n2000 c1000
    rails sinatra
    grape sinatra threads
    grape threads sinatra-synchrony + thin
    grape + goliath
    rails, sinatra, grape, sinatra threads and grape threads are timed out with n 1000 c 200
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

50. IO bound
    measured with NewRelic
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

51. leaderboard rails 1200rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

52. leaderboard-sinatra 1200 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

53. leaderboard-grape 1200 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

54. leaderboard-sinatra threads 3000 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

55. leaderboard-grape threads 3000 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

56. leaderboard grape goliath 3000 rpm
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Friday, June 7, 13
    

    View Slide

57. flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Known issues
    • Failed to add newrelic to sinatra-synchrony
    • Failed to add fiber_pool to goliath
    Friday, June 7, 13
    

    View Slide

58. Conclusion
    • Rails is good
    • Sinatra/Grape decreases response time
    • Sinatra-synchrony/Goliath increases throughput
    • Async I/O is awesome! Try
    Friday, June 7, 13
    

    View Slide

59. Thank you
    flyerhzm at gmail dot com
    github.com/flyerhzm twitter.com/flyerhzm
    Questions?
    Friday, June 7, 13
    

    View Slide