Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building an Async Server with Fiber

00e4a880b1262a125b5e342e4b536765?s=47 Zete
October 26, 2013

Building an Async Server with Fiber

#rubyconfchina2013

00e4a880b1262a125b5e342e4b536765?s=128

Zete

October 26, 2013
Tweet

More Decks by Zete

Other Decks in Programming

Transcript

  1. Async Server And Fiber c_zlui@groupon.com

  2. Me github/luikore ruby-china/luikore

  3. Motivation

  4. Challenges To Ruby

  5. Ruby Advantages Unicode done right Builtin continuation - Fiber And...

    "" . l e n g t h
  6. Fast

  7. How To Build A High Performance Server

  8. How To Build An Async Server

  9. How To Build An Async Server Yet Providing Good API

  10. Study The Background

  11. None
  12. The Bottle Neck Client Side Of View IO bound -

    network is slow CPU bound - complex page rendering Server load - wait for other requests to finish
  13. The Bottle Neck Server Side Of View IO bound -

    visiting database / oauth is slow CPU bound - context switch
  14. Scaling Increase App Throughput Reduce server side IO-waiting is the

    most effective way Create less threads, to avoid context switching
  15. Learn From Existing Solutions

  16. Chances To Ruby

  17. Limited Server Objects

  18. Many Crazy Clients

  19. Multiplexing

  20. Single Threaded One Client, One Process, Prefork unicorn -- multiplex

    with accept thin -- multiplex with load balancer
  21. Multithreaded One Client, One Thread, No Prefork puma, recommended for

    rails4 A simple way to implement is to use POSIX functions with callbacks: aio_read(), aio_write()
  22. Single Threaded And Event Loop One Client, One Event Context,

    Prefork nginx, redis, nodejs, twisted, eventmachine
  23. Thread Pool On Event Loop One Client, One Event Context,

    No Prefork erlang, golang, scala actors
  24. Hello World

  25. TCPSocket In 5 Minutes client server receive send T C

    P S o c k e t . n e w h o s t , p o r t T C P S e r v e r . n e w h o s t , p o r t s o c k e t . r e a d s o c k e t . r e a d _ n o n b l o c k s o c k e t . w r i t e s o c k e t . w r i t e _ n o n b l o c k
  26. Hello World HTTP Server s e r v e r

    = T C P S e r v e r . n e w ' l o c a l h o s t ' , 4 0 0 0 l o o p d o c = s e r v e r . a c c e p t c < < " H T T P / 1 . 1 2 0 0 O K C o n t e n t - L e n g t h : 1 2 H e l l o w o r l d ! " c . c l o s e e n d
  27. Make It Fast

  28. Non­Blocking Our Loop l o o p d o c

    l i e n t = s e r v e r . a c c e p t T h r e a d . n e w d o . . . e n d e n d
  29. Threads Are Outdated Lets Use System Events

  30. Multiplexing Functions select poll I O . s e l

    e c t r , w , e r r , t i m e o u t I O . p o l l f d s , t i m e o u t
  31. System Events Linux For Example poll - iterate check all

    fds epoll - is more effective and provides more options
  32. Epoll In 5 Minutes s u d o a p

    t - g e t i n s t a l l m a n p a g e s - d e v m a n e p o l l
  33. Epoll In 5 Minutes register event: retrieve event: e p

    o l l _ c t l ( ) e p o l l ( )
  34. Cons Ruby does not ship with epoll API. You need

    to wrap it with C-ext.
  35. Loop With Epoll l o o p d o e

    v e n t = e p o l l ( t i m e o u t ) i f f r o m _ s e r v e r ? ( e v e n t ) c o n n e c t i o n = s e r v e r . a c c e p t e l s e c o n n e c t i o n = f i n d _ c o n n _ b y ( e v e n t ) e n d . . . # r e s u m e p r o c e s s i n g e n d
  36. How To Make It Resumable?

  37. Eventmachine async web server

  38. clients are multiplexed to you define callbacks for readable events:

    E M : : C o n n e c t i o n d e f r e c e i v e _ d a t a ( d a t a )
  39. "Resume Processing" c o n n e c t i

    o n . r e c e i v e _ d a t a r e a d _ n o n b l o c k
  40. Make It Good

  41. How EM Send_data Works write to socket, if the socket

    can’t take any more, append the rest to a buffer chain when the socket becomes writable again (IO event), try consume the buffer chain
  42. Pitfalls Callbacks Are Not Sufficient avoid loss of chained buffers

    avoid infinite recursion when registering events in callback: c l o s e _ c o n n e c t i o n _ a f t e r _ w r i t i n g E M . n e x t _ t i c k { . . . }
  43. Resume Processing With Fiber Fiber was the “Thread” in 1.8

    Like thread, but lighter, pausable, and no need to lock
  44. Example f i b e r = F i b

    e r . n e w d o i = 0 l o o p d o F i b e r . y i e l d i + = 1 e n d e n d f i b e r . r e s u m e # = > 1 f i b e r . r e s u m e # = > 2
  45. Read With Fiber d e f r e a d

    _ w i t h _ f i b e r r e s u l t = ' ' w h i l e b u f f e r = r e a d _ n o n b l o c k r e s u l t < < b u f f e r i f n o t h i n g t o r e a d F i b e r . y i e l d : r e a d i n g ! e n d e n d r e s u l t e n d
  46. Write With Fiber d e f w r i t

    e _ w i t h _ f i b e r c o n t e n t l o o p d o w r i t e _ n o n b l o c k c o n t e n t c o n t e n t = u n w r i t t e n p a r t o f c o n t e n t b r e a k i f c o n t e n t . e m p t y ? F i b e r . y i e l d : w r i t i n g ! e n d e n d
  47. Recall Our Loop l o o p d o e

    v e n t = e p o l l ( t i m e o u t ) i f f r o m _ s e r v e r ? ( e v e n t ) c o n n e c t i o n = s e r v e r . a c c e p t e l s e c o n n e c t i o n = f i n d _ c o n n _ b y ( e v e n t ) e n d . . . # r e s u m e p r o c e s s i n g e n d
  48. l o o p d o e v e n

    t = e p o l l ( t i m e o u t ) i f f r o m _ s e r v e r ? ( e v e n t ) c o n n e c t i o n = s e r v e r . a c c e p t f i b e r = F i b e r . n e w { . . . r e a d _ w i t h _ f i b e r . . . w r i t e _ w i t h _ f i b e r } e l s e f i b e r = f i n d _ f i b e r _ b y ( e v e n t ) e n d f i b e r . r e s u m e e n d
  49. Summary

  50. Network latency and C10k leads to async solutions. With Fiber

    it can be callback-free.
  51. In concept it is simple. In practice it is complex:

    protocols, IO exceptions, ...
  52. Product github.com/luikore/nyara faster than sinatra/expressjs in hello world benchmarks