Building Asynchronous APIs

Slide 1

Slide 1 text

Building Asynchronous APIs Richard Huang flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 2

Slide 2 text

I’m ... • Currently Ruby/Rails Freelancer • Previously worked for OpenFeint (GREE) • Author of bullet and rails_best_practices gems flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 3

Slide 3 text

Hardware is Cheap, Programmers are Expensive flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 4

Slide 4 text

App Server DB Server Web Server Single Server flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 5

Slide 5 text

but... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 6

Slide 6 text

App Server DB Server Web Server Load Balancer App Server DB Server Web Server App Server Web Server App Server Web Server Memcached ... ... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm ... Friday, June 7, 13

Slide 7

Slide 7 text

Hardwares get Expensive, so... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 8

Slide 8 text

http://highscalability.com/blog/2012/10/4/linkedin-moved-from-rails-to-node-27-servers-cut-and-up-to-2.html flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 9

Slide 9 text

http://highscalability.com/blog/2013/3/13/ironio-moved-from-ruby-to-go-28-servers-cut-and-colossal-clu.html flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 10

Slide 10 text

Is Ruby/Rails so slow? flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 11

Slide 11 text

From my experience, no! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 12

Slide 12 text

• Rails REE 1.8.7 Mysql 50 ms 60k rpm 13 machines x 6 passenger instances • Rails JRuby 1.7.0 Mysql 30 ms 60k rpm 10 machines x torquebox (5 threads) • Goliath Ruby 1.9.3 Redis 4 ms 240k rpm 4 machines x 4 goliath instances flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 13

Slide 13 text

Migrating from ruby to ruby, 48 servers cut and10x faster flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 14

Slide 14 text

Sync vs Async Blocking vs Nonblocking flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 15

Slide 15 text

Blocking IO flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm DB HTTP IO CPU Friday, June 7, 13

Slide 16

Slide 16 text

Multi Processes DB HTTP IO CPU DB HTTP IO CPU Process A Process B flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 17

Slide 17 text

Multi Processes • Parallel on a multi cores cpu • Easy to manage • Process switching is expensive • Consume many memory flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 18

Slide 18 text

Multi Threads DB HTTP IO CPU DB HTTP IO CPU Thread A Thread B flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 19

Slide 19 text

Multi Threads • Thread switching is cheap • Consume less memory • Thread safety • GIL flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 20

Slide 20 text

Evented flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm DB HTTP IO CPU Friday, June 7, 13

Slide 21

Slide 21 text

Evented (Eventmachine) • No blocking io • No context switching • Least memory usage • Callback! Callback! Callback! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 22

Slide 22 text

eventmachine Friday, June 7, 13

Slide 23

Slide 23 text

Evented (em-synchrony) • EM + Fiber = No callback! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 24

Slide 24 text

em-synchrony Friday, June 7, 13

Slide 25

Slide 25 text

Concurrency Parallelism Friday, June 7, 13

Slide 26

Slide 26 text

Multi Processes + Evented flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 27

Slide 27 text

How? flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm https://github.com/flyerhzm/apis-bench Friday, June 7, 13

Slide 28

Slide 28 text

View Controller Model Skinny controller Fat model Render json response Rails Router Dispatch request flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 29

Slide 29 text

Model flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 30

Slide 30 text

Controller / View flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 31

Slide 31 text

Router flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 32

Slide 32 text

Rails => Grape Decrease response time flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 33

Slide 33 text

Grape Model Router / Controller / View Fat model Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 34

Slide 34 text

Fat model No need to do any change, nice! flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 35

Slide 35 text

Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 36

Slide 36 text

Grape => Grape + Goliath Increase throughput flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 37

Slide 37 text

Grpae Model Router / Controller / View Fat model Grape + Goliath Goliath Non-blocking web server framework flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 38

Slide 38 text

Goliath with Grape flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 39

Slide 39 text

Goliath with Grape mysql2 => em_mysql2 net/http => em-http-request mongo => em-mongo ... flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 40

Slide 40 text

Rails => Sinatra Decrease response time flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 41

Slide 41 text

Sinatra Model Router / Controller / View Fat model Sinatra flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 42

Slide 42 text

Sinatra flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 43

Slide 43 text

Sinatra => Sinatra-synchrony+Thin flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 44

Slide 44 text

Sinatra / Sinatra-synchrony Model Router / Controller / View Fat model Sinatra-synchrony+Thin Thin Ruby web server based on EM flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 45

Slide 45 text

Sinatra-synchrony flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 46

Slide 46 text

Sinatra-synchrony em-* libraries flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 47

Slide 47 text

Benchmark flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm https://github.com/flyerhzm/apis-bench Friday, June 7, 13

Slide 48

Slide 48 text

CPU bound measured with ab flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 49

Slide 49 text

0 5 10 15 20 n1000 c10 n1000 c50 n1000 c100 n1000 c200 n1000 c500 n2000 c1000 rails sinatra grape sinatra threads grape threads sinatra-synchrony + thin grape + goliath rails, sinatra, grape, sinatra threads and grape threads are timed out with n 1000 c 200 flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 50

Slide 50 text

IO bound measured with NewRelic flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 51

Slide 51 text

leaderboard rails 1200rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 52

Slide 52 text

leaderboard-sinatra 1200 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 53

Slide 53 text

leaderboard-grape 1200 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 54

Slide 54 text

leaderboard-sinatra threads 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 55

Slide 55 text

leaderboard-grape threads 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 56

Slide 56 text

leaderboard grape goliath 3000 rpm flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Friday, June 7, 13

Slide 57

Slide 57 text

flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Known issues • Failed to add newrelic to sinatra-synchrony • Failed to add fiber_pool to goliath Friday, June 7, 13

Slide 58

Slide 58 text

Conclusion • Rails is good • Sinatra/Grape decreases response time • Sinatra-synchrony/Goliath increases throughput • Async I/O is awesome! Try Friday, June 7, 13

Slide 59

Slide 59 text

Thank you flyerhzm at gmail dot com github.com/flyerhzm twitter.com/flyerhzm Questions? Friday, June 7, 13