Building an Async Server with Fiber

#rubyconfchina2013

Transcript

1. Async Server And Fiber c_zlui@groupon.com
2. Me github/luikore ruby-china/luikore
3. Motivation
4. Challenges To Ruby
5. Ruby Advantages Unicode done right Builtin continuation - Fiber And... "" . l e n g t h
6. Fast
7. How To Build A High Performance Server
8. How To Build An Async Server
9. How To Build An Async Server Yet Providing Good API
10. Study The Background
12. The Bottle Neck Client Side Of View IO bound - network is slow CPU bound - complex page rendering Server load - wait for other requests to finish
13. The Bottle Neck Server Side Of View IO bound - visiting database / oauth is slow CPU bound - context switch
14. Scaling Increase App Throughput Reduce server side IO-waiting is the most effective way Create less threads, to avoid context switching
15. Learn From Existing Solutions
16. Chances To Ruby
17. Limited Server Objects
18. Many Crazy Clients
19. Multiplexing
20. Single Threaded One Client, One Process, Prefork unicorn -- multiplex with accept thin -- multiplex with load balancer
21. Multithreaded One Client, One Thread, No Prefork puma, recommended for rails4 A simple way to implement is to use POSIX functions with callbacks: aio_read(), aio_write()
22. Single Threaded And Event Loop One Client, One Event Context, Prefork nginx, redis, nodejs, twisted, eventmachine
23. Thread Pool On Event Loop One Client, One Event Context, No Prefork erlang, golang, scala actors
24. Hello World
25. TCPSocket In 5 Minutes client server receive send T C P S o c k e t . n e w h o s t , p o r t T C P S e r v e r . n e w h o s t , p o r t s o c k e t . r e a d s o c k e t . r e a d _ n o n b l o c k s o c k e t . w r i t e s o c k e t . w r i t e _ n o n b l o c k
26. Hello World HTTP Server s e r v e r = T C P S e r v e r . n e w ' l o c a l h o s t ' , 4 0 0 0 l o o p d o c = s e r v e r . a c c e p t c < < " H T T P / 1 . 1 2 0 0 O K C o n t e n t - L e n g t h : 1 2 H e l l o w o r l d ! " c . c l o s e e n d
27. Make It Fast
28. Non­Blocking Our Loop l o o p d o c l i e n t = s e r v e r . a c c e p t T h r e a d . n e w d o . . . e n d e n d
29. Threads Are Outdated Lets Use System Events
30. Multiplexing Functions select poll I O . s e l e c t r , w , e r r , t i m e o u t I O . p o l l f d s , t i m e o u t
31. System Events Linux For Example poll - iterate check all fds epoll - is more effective and provides more options
32. Epoll In 5 Minutes s u d o a p t - g e t i n s t a l l m a n p a g e s - d e v m a n e p o l l
33. Epoll In 5 Minutes register event: retrieve event: e p o l l _ c t l ( ) e p o l l ( )
34. Cons Ruby does not ship with epoll API. You need to wrap it with C-ext.
35. Loop With Epoll l o o p d o e v e n t = e p o l l ( t i m e o u t ) i f f r o m _ s e r v e r ? ( e v e n t ) c o n n e c t i o n = s e r v e r . a c c e p t e l s e c o n n e c t i o n = f i n d _ c o n n _ b y ( e v e n t ) e n d . . . # r e s u m e p r o c e s s i n g e n d
36. How To Make It Resumable?
37. Eventmachine async web server
38. clients are multiplexed to you define callbacks for readable events: E M : : C o n n e c t i o n d e f r e c e i v e _ d a t a ( d a t a )
39. "Resume Processing" c o n n e c t i o n . r e c e i v e _ d a t a r e a d _ n o n b l o c k
40. Make It Good
41. How EM Send_data Works write to socket, if the socket can’t take any more, append the rest to a buffer chain when the socket becomes writable again (IO event), try consume the buffer chain
42. Pitfalls Callbacks Are Not Sufficient avoid loss of chained buffers avoid infinite recursion when registering events in callback: c l o s e _ c o n n e c t i o n _ a f t e r _ w r i t i n g E M . n e x t _ t i c k { . . . }
43. Resume Processing With Fiber Fiber was the “Thread” in 1.8 Like thread, but lighter, pausable, and no need to lock
44. Example f i b e r = F i b e r . n e w d o i = 0 l o o p d o F i b e r . y i e l d i + = 1 e n d e n d f i b e r . r e s u m e # = > 1 f i b e r . r e s u m e # = > 2
45. Read With Fiber d e f r e a d _ w i t h _ f i b e r r e s u l t = ' ' w h i l e b u f f e r = r e a d _ n o n b l o c k r e s u l t < < b u f f e r i f n o t h i n g t o r e a d F i b e r . y i e l d : r e a d i n g ! e n d e n d r e s u l t e n d
46. Write With Fiber d e f w r i t e _ w i t h _ f i b e r c o n t e n t l o o p d o w r i t e _ n o n b l o c k c o n t e n t c o n t e n t = u n w r i t t e n p a r t o f c o n t e n t b r e a k i f c o n t e n t . e m p t y ? F i b e r . y i e l d : w r i t i n g ! e n d e n d
47. Recall Our Loop l o o p d o e v e n t = e p o l l ( t i m e o u t ) i f f r o m _ s e r v e r ? ( e v e n t ) c o n n e c t i o n = s e r v e r . a c c e p t e l s e c o n n e c t i o n = f i n d _ c o n n _ b y ( e v e n t ) e n d . . . # r e s u m e p r o c e s s i n g e n d
48. l o o p d o e v e n t = e p o l l ( t i m e o u t ) i f f r o m _ s e r v e r ? ( e v e n t ) c o n n e c t i o n = s e r v e r . a c c e p t f i b e r = F i b e r . n e w { . . . r e a d _ w i t h _ f i b e r . . . w r i t e _ w i t h _ f i b e r } e l s e f i b e r = f i n d _ f i b e r _ b y ( e v e n t ) e n d f i b e r . r e s u m e e n d
49. Summary
50. Network latency and C10k leads to async solutions. With Fiber it can be callback-free.
51. In concept it is simple. In practice it is complex: protocols, IO exceptions, ...
52. Product github.com/luikore/nyara faster than sinatra/expressjs in hello world benchmarks