Building Asynchronous APIs

Transcript

1. Building Asynchronous APIs Richard Huang flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
2. I’m ... • Currently Ruby/Rails Freelancer • Previously worked for OpenFeint (GREE) • Author of bullet and rails_best_practices gems flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
3. Hardware is Cheap, Programmers are Expensive flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
4. App Server DB Server Web Server Single Server flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
5. but... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
6. App Server DB Server Web Server Load Balancer App Server DB Server Web Server App Server Web Server App Server Web Server Memcached ... ... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm ... Friday, June 7, 13
7. Hardwares get Expensive, so... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
8. http://highscalability.com/blog/2012/10/4/linkedin-moved-from-rails-to-node-27-servers-cut-and-up-to-2.html flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
9. http://highscalability.com/blog/2013/3/13/ironio-moved-from-ruby-to-go-28-servers-cut-and-colossal-clu.html flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
10. Is Ruby/Rails so slow? flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
11. From my experience, no! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
12. • Rails REE 1.8.7 Mysql 50 ms 60k rpm 13 machines x 6 passenger instances • Rails JRuby 1.7.0 Mysql 30 ms 60k rpm 10 machines x torquebox (5 threads) • Goliath Ruby 1.9.3 Redis 4 ms 240k rpm 4 machines x 4 goliath instances flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
13. Migrating from ruby to ruby, 48 servers cut and10x faster flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
14. Sync vs Async Blocking vs Nonblocking flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
15. Blocking IO flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm DB HTTP IO CPU Friday, June 7, 13
16. Multi Processes DB HTTP IO CPU DB HTTP IO CPU Process A Process B flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
17. Multi Processes • Parallel on a multi cores cpu • Easy to manage • Process switching is expensive • Consume many memory flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
18. Multi Threads DB HTTP IO CPU DB HTTP IO CPU Thread A Thread B flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
19. Multi Threads • Thread switching is cheap • Consume less memory • Thread safety • GIL flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
20. Evented flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm DB HTTP IO CPU Friday, June 7, 13
21. Evented (Eventmachine) • No blocking io • No context switching • Least memory usage • Callback! Callback! Callback! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
22. eventmachine Friday, June 7, 13
23. Evented (em-synchrony) • EM + Fiber = No callback! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
24. em-synchrony Friday, June 7, 13
25. Concurrency Parallelism Friday, June 7, 13
26. Multi Processes + Evented flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
27. How? flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm https://github.com/flyerhzm/apis-bench Friday, June 7, 13
28. View Controller Model Skinny controller Fat model Render json response Rails Router Dispatch request flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
29. Model flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
30. Controller / View flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
31. Router flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
32. Rails => Grape Decrease response time flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
33. Grape Model Router / Controller / View Fat model Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
34. Fat model No need to do any change, nice! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
35. Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
36. Grape => Grape + Goliath Increase throughput flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
37. Grpae Model Router / Controller / View Fat model Grape + Goliath Goliath Non-blocking web server framework flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
38. Goliath with Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
39. Goliath with Grape mysql2 => em_mysql2 net/http => em-http-request mongo => em-mongo ... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
40. Rails => Sinatra Decrease response time flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
41. Sinatra Model Router / Controller / View Fat model Sinatra flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
42. Sinatra flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
43. Sinatra => Sinatra-synchrony+Thin flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
44. Sinatra / Sinatra-synchrony Model Router / Controller / View Fat model Sinatra-synchrony+Thin Thin Ruby web server based on EM flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
45. Sinatra-synchrony flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
46. Sinatra-synchrony em-* libraries flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
47. Benchmark flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm https://github.com/flyerhzm/apis-bench Friday, June 7, 13
48. CPU bound measured with ab flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
49. 0 5 10 15 20 n1000 c10 n1000 c50 n1000 c100 n1000 c200 n1000 c500 n2000 c1000 rails sinatra grape sinatra threads grape threads sinatra-synchrony + thin grape + goliath rails, sinatra, grape, sinatra threads and grape threads are timed out with n 1000 c 200 flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
50. IO bound measured with NewRelic flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
51. leaderboard rails 1200rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
52. leaderboard-sinatra 1200 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
53. leaderboard-grape 1200 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
54. leaderboard-sinatra threads 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
55. leaderboard-grape threads 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
56. leaderboard grape goliath 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13
57. flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Known issues • Failed to add newrelic to sinatra-synchrony • Failed to add fiber_pool to goliath Friday, June 7, 13
58. Conclusion • Rails is good • Sinatra/Grape decreases response time • Sinatra-synchrony/Goliath increases throughput • Async I/O is awesome! Try Friday, June 7, 13
59. Thank you flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Questions? Friday, June 7, 13