Design pattern for embedding mruby into middleware

Design pattern for embedding mruby into middleware

A design and implementation to connect middleware
supporting Internet service with mruby

Ryosuke Matsumoto / Pepabo R&D Institute, GMO Pepabo, Inc.
2018.06.02 RubyKaigi2018

2b692bd83f4418103142a053ecf5ff59?s=128

MATSUMOTO Ryosuke

June 02, 2018
Tweet

Transcript

  1. A design and implementation to connect middleware supporting Internet service

    with mruby Ryosuke Matsumoto / Pepabo R&D Institute, GMO Pepabo, Inc. 2018.06.02 RubyKaigi2018 Design pattern for embedding mruby into middleware
  2. • Chief engineer at GMO Pepabo, inc. • Chief researcher

    at Pepabo R&D Institute • Ph.D of Informatics, Kyoto University • my interests: Operation technology, Internet foundation technology, System programming, Highly integrated multi- tenant architecture, Hosting service, Cloud platform, C, mruby, Rust • Engineering => Technology <= Research 2 Ryosuke Matsumoto @matsumotory
  3. 3

  4. 4

  5. 1. Introduction of embedding mruby into middleware 2. Design pattern

    for middleware 1. init / exit 2. each request 3. non-blocking 3. Conclusion 5 Table of Contents
  6. 1. Introduction of embedding mruby into middleware

  7. • System management for network, OS and middleware • Hosting

    service, Cloud platform service • operation technology for middleware written by C • Apache, nginx, dovecot, postfix, sendmail, bind, proftp… • want to write operation tools for middleware using perl • want to write extension module for middleware • each middleware dependency • Apache module, nginx module, dovecot plugin…. 7 I’m a former web operation engineer
  8. met mruby in April 2012

  9. mruby / CRuby for middleware 9 NJEEMFXBSFJO$ NSVCZ NJEEMFXBSFNFUIPE JONSVCZ

    NJEEMFXBSFGVODUJPOTJO$ NJEEMFXBSFJO$3VCZ NJEEMFXBSFNFUIPE JO$3VCZ NSVCZGPSNJEEMFXBSF $3VCZGPSNJEEMFXBSF
  10. middleware configuration as code

  11. basically embed mruby into middleware 11 NJEEMFXBSFQSPDFTT NSC@TUBUFNSVCZ NSVCZCZUFDPEF NJEEMFXBSFGVODUJPOTJO$

    NSVCZ4DSJQUT lNJEEMFXBSFzXBTJNQMFNFOUFECZ$JOUIJTQSFTFOUBUJPO NBTUFS TFSWBOU
  12. • mod_mruby / ngx_mruby (http server extension) • dovecot-mruby-plugin (imap

    server extension) • pmilter (milter server) • Trusterd HTTP/2 Web Server • h2o_mruby (Initial implementation with v1.4) • other system tools: rcon, capcon, pfds, virtualing, ab-mruby • developed the greatest number of mrbgem in mgem-list • also received Ph.D of informatics 12 my developed middleware with mruby
  13. • middleware behavior scripting with mruby • middleware configuration as

    code • limit Ruby methods by mruby • dynamically control middleware by session parameters • cooperate with microservices • record a session data and utilize the data • implement Service Mesh functions • HTTP(S)ɾTCP/UDP stream 13 Why embed mruby into middleware
  14. • consider several viewpoints • memory management (allocate/free/leak) • performance

    • usability, operation / development technology • strategy for multi process / multi threading • ʮmruby = mrb_stateʯ in this presentation 14 How embed mruby into middleware
  15. Basic multi threading strategy

  16. Concurrency with native thread

  17. multi-threading strategy for mruby 17 QSPDFTT UISFBE NSC@TUBUF CZUFDPEF UISFBE

    NSC@TUBUF CZUFDPEF QSPDFTT NSC@TUBUF UISFBE CZUFDPEF UISFBE CZUFDPEF QSPDFTT NSC@TUBUF CZUFDPEF UISFBE UISFBE
  18. multi-threading strategy for mruby 18 QSPDFTT UISFBE NSC@TUBUF CZUFDPEF UISFBE

    NSC@TUBUF CZUFDPEF QSPDFTT NSC@TUBUF UISFBE CZUFDPEF UISFBE CZUFDPEF QSPDFTT NSC@TUBUF CZUFDPEF UISFBE UISFBE MPDL MPDL
  19. • The basic strategy is embedding mruby into each thread

    • lock between threads in single mrb_state is complicated • A cost of creating mrb_state is much higher than creating thread • The cost becomes bottleneck each request to middleware • We want to create mrb_state on init phase of middleware • control only bytecode each request 19 considerations
  20. • talk about design patterns for embedding mruby into middleware

    • try and error over and over again for 6 years • consider several viewpoints • memory management • performance • usability, operation / development technology 20 In this presentation
  21. 2. Design pattern for middleware

  22. 1. init / exit

  23. init: independent mrb_state 23 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    HFOFSBUFDPEFBOESVOPONBTUFSJOJU GPSL DSFBUFNSC@TUBUFPONBTUFSJOJU NSC@TUBUF CZUFDPEF HFOFSBUFDPEFBOESVOPOXPSLFSJOJU DSFBUFNSC@TUBUFPOXPSLFSJOJU NVMUJQSPDFTTNPEFM
  24. init: independent mrb_state 24 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    GSFFDPEFPONBTUFSJOJU GPSL DMPTFNSC@TUBUFPONBTUFSJOJU NSC@TUBUF CZUFDPEF GSFFDPEFPOXPSLFSJOJU DMPTFNSC@TUBUFPOXPSLFSJOJU NVMUJQSPDFTTNPEFM GSFF GSFF GSFF OPUIJOHUPEPPOXPSLFSBOENBTUFSFYJU
  25. • Advantages • mruby scripts are executed completely independently •

    can save memory • Disadvantages • can’t pass objects to a request phase • can’t share objects with each other script 25 init: independent mrb_state
  26. init: share mrb_state 26 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    CZUFDPEF HFOFSBUFDPEFBOESVOPONBTUFSJOJU POMZHFOFSBUFDPEFPOXPSLFSJOJU GPSL DSFBUFNSC@TUBUFPONBTUFSJOJU NVMUJQSPDFTTNPEFM
  27. init: share mrb_state 27 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    CZUFDPEF GSFFDPEFPONBTUFSJOJU GPSL DMPTFNSC@TUBUFPONBTUFSJOJU NVMUJQSPDFTTNPEFM GSFFDPEFPOXPSLFSJOJU DMPTFNSC@TUBUFPOXPSLFSJOJU OPUIJOHUPEPPOXPSLFSBOENBTUFSFYJU
  28. • Advantages • decrease a cost of creating mrb_state each

    script • can save memory • can share objects with each other script on init phase • Disadvantages • can’t pass objects to a request phase 28 init: share mrb_state
  29. init: share mrb_state with request phase 29 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF

    NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF HFOFSBUFDPEFBOESVOPONBTUFSJOJU POMZHFOFSBUFDPEFPOXPSLFSJOJU SFVTFUIFNSC@TUBUFFBDISFRVFTU GPSL DSFBUFNSC@TUBUFPONBTUFSJOJU NVMUJQSPDFTTNPEFM
  30. exit: share mrb_state with request phase 30 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF

    NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF GSFFDPEFPONBTUFSFYJU GPSL DMPTFNSC@TUBUFPONBTUFSFYJU NVMUJQSPDFTTNPEFM GSFFDPEFPOXPSLFSFYJU DMPTFNSC@TUBUFPOXPSLFSFYJU
  31. • Advantages • decrease a cost of creating mrb_state each

    script • can share objects with each other script on init phase • can pass objects to a request phase • use DB object which connect the DB at init phase • Disadvantages • use more memory than other init pattern • complicated memory management each request 31 init: share mrb_state with request phase
  32. 2. each request

  33. multi process model

  34. XPSLFSQSPDFTT request: independent mrb_state 34 XPSLFSQSPDFTT NSC@TUBUF CZUFDPEF NSC@TUBUF CZUFDPEF

    3FRVFTU XPSLFSQSPDFTT 3FRVFTU 3FTQPOTF 3FTQPOTF NSC@TUBUF CZUFDPEF 3FRVFTU NSC@TUBUF CZUFDPEF NSC@TUBUF CZUFDPEF GSFF GSFF DSFBUFNSC@TUBUFBOEHFOFSBUFDPEF BOESVOCZUFDPEFFBDISFRVFTU *OJU 3FRVFTU 3FTQPOTF
  35. • Advantages • mruby scripts are executed completely independently •

    can manage memory easily and correctly • change script each request • Disadvantages • low performance because of creating mrb_state each request • can’t share objects with each other script each request 35 request: independent mrb_state
  36. request: shared mrb_state 36 XPSLFSQSPDFTT XPSLFSQSPDFTT 3FRVFTU XPSLFSQSPDFTT 3FRVFTU 3FTQPOTF

    3FTQPOTF *OJU 3FRVFTU 3FTQPOTF NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF 3FRVFTU BSFOB FYD BSFOB FYD BSFOB FYD DMFBOVQBSFOBBOEFYD PCKFDUTFBDISFRVFTU GSFF GSFF QBSTFTDSJQUBOEHFOFSBUFBOE SVOCZUFDPEFFBDISFRVFTU DSFBUFNSC@TUBUF POJOJUJBMJ[BUJPO
  37. • Advantages • can share objects with each other script

    each request and init phase • change script each request • Disadvantages • slightly low performance because of compiling • memory management is very complicated because of compiling and freeing bytecode each request while sharing mrb_state 37 request: independent mrb_state
  38. request: shared mrb_state and bytecode 38 XPSLFSQSPDFTT XPSLFSQSPDFTT 3FRVFTU XPSLFSQSPDFTT

    3FTQPOTF *OJU 3FRVFTU 3FTQPOTF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF 3FRVFTU BSFOB FYD BSFOB FYD BSFOB FYD DMFBOVQBSFOBBOEFYDPCKFDUT FBDISFRVFTU GSFFDPEFDPOUFYU FBDISFRVFTU DSFBUFNSC@TUBUF QBSTFTDSJQUBOEHFOFSBUF CZUFDPEFPOJOJUJBMJ[BUJPO SVOCZUFDPEFFBDISFRVFTU
  39. • Advantages • high performance • can share objects with

    each other script each request and init phase • Disadvantages • can’t change script each request • memory management is a bit complicated 39 request: independent mrb_state
  40. Performance for request pattern 40

  41. multi threading model

  42. multi-threading strategy for mruby 42 QSPDFTT UISFBE NSC@TUBUF CZUFDPEF UISFBE

    NSC@TUBUF CZUFDPEF QSPDFTT NSC@TUBUF UISFBE CZUFDPEF UISFBE CZUFDPEF QSPDFTT NSC@TUBUF CZUFDPEF UISFBE UISFBE
  43. XPSLFSQSPDFTT request: independent mrb_state 43 XPSLFSQSPDFTT XPSLFSQSPDFTT DSFBUFNSC@TUBUFBOEHFOFSBUFCZUFDPEF BOESVOCZUFDPEFFBDISFRVFTU XPSLFSUISFBE

    XPSLFSUISFBE XPSLFSUISFBE NSC@TUBUF CZUFDPEF XPSLFSUISFBE XPSLFSUISFBE NSC@TUBUF CZUFDPEF XPSLFSUISFBE 3FRVFTU 3FTQPOTF NSC@TUBUF CZUFDPEF 3FRVFTU GSFF *OJU 3FRVFTU 3FTQPOTF NSC@TUBUF CZUFDPEF 3FRVFTU
  44. request: share mrb_state 44 XPSLFSQSPDFTT *OJU 3FRVFTU 3FTQPOTF XPSLFSQSPDFTT XPSLFSQSPDFTT

    XPSLFSUISFBE NSC@TUBUF BSFOB FYD XPSLFSUISFBE NSC@TUBUF BSFOB FYD XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD XPSLFSUISFBE NSC@TUBUF BSFOB FYD XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD 3FTQPOTF 3FRVFTU 3FRVFTU DMFBOVQ BSFOBBOEFYD PCKFDUTFBDI SFRVFTU GSFF QBSTFTDSJQU BOEHFOFSBUF CZUFDPEFBOE SVOCZUFDPEF FBDISFRVFTU CZUFDPEF 3FRVFTU DSFBUFNSC@TUBUFPOJOJUJBMJ[BUJPO
  45. request: share mrb_state and bytecode 45 XPSLFSQSPDFTT *OJU 3FRVFTU 3FTQPOTF

    XPSLFSQSPDFTT XPSLFSQSPDFTT XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF 3FTQPOTF 3FRVFTU 3FRVFTU DMFBOVQ BSFOBBOEFYD PCKFDUTFBDI SFRVFTU 3FRVFTU GSFFDPEFDPOUFYU FBDISFRVFTU DSFBUFNSC@TUBUF QBSTFTDSJQUBOEHFOFSBUFCZUFDPEFPOJOJUJBMJ[BUJPO SVOCZUFDPEF FBDISFRVFTU
  46. • multi process model • share mrb_state and bytecode •

    mod_mruby, ngx_mruby, postfix-mruby, dovecot- mruby, trusterd HTTP/2 server • multi threading model • share mrb_state and bytecode • H2O, nghttpd • independent mrb_state • pmilter 46 Summary: each request pattern
  47. 3. non-blocking

  48. • want to accept as many requests simultaneously as possible

    in single process/thread (C10k) • want to use multi cpu core sufficiently • process blocking operation in non-blocking mode • File I/O, Network I/O, sleep… • monitor I/O state and notify status change like epoll() • generally implement suitable event loop for middleware 48 non-blocking middleware
  49. • mruby blocks middleware processing generally • bottleneck of performance

    when accepting multiple requests simultaneously • Other responses are delayed in proportion to the time of processing of mruby blocking 49 non-blocking middleware with mruby
  50. blocking each request with mruby 50 SFRVFTUQSPDFTTJOH NSVCZ NSVCZ SFRVFTUQSPDFTTJOH

    SFRVFTUQSPDFTTJOH NSVCZ DSFBUFSFTQPOTF DSFBUFSFTQPOTF DSFBUFSFTQPOTF TFOESFTQPOTF OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT SFDWSFRVFTU BUUIFTBNFUJNF
  51. blocking each request with mruby 51 SFRVFTU NSVCZ NSVCZ SFTQPOTF

    SFRVFTU SFRVFTU SFTQPOTF SFTQPOTF NSVCZ TFOESFTQPOTF SFDWSFRVFTU BUUIFTBNFUJNF Other responses are delayed in proportion to the time of processing of mruby blocking OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT
  52. 52

  53. non-blocking each request with mruby 53 SFRVFTU SFTQPOTF SFRVFTU SFRVFTU

    SFTQPOTF SFTQPOTF TFOESFTQPOTF SFDWSFRVFTU BUUIFTBNFUJNF CMPDLJOH PQFSBJUPO NSVCZ CMPDLJOH PQFSBJUPO NSVCZ NSVCZ CMPDLJOH PQFSBJUPO OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT
  54. 54

  55. • change a blocking method of mruby to non-blocking •

    suspend mruby processing until the blocking method is completed • return to event loop of middleware from mruby • resume mruby processing by callback func in event loop 55 non-blocking middleware with mruby
  56. 3VCZ4DSJQUʢ3VCZXPSMEʣ 1SPD 'JCFSSFTVNF non-blocking each request with mruby 56 NJEEMFXBSF

    $XPSME NSC@TUBUF 'JCFS 3VCZCZUFDPEF JODMVEFECMPDLJOH NFUIPE DGVODPGCMPDLJOHNFUIPE SVOCMPDLJOHPQFSBUJPOXJUIOPOCMPDLJOHNPEF SVONSC@pCFS@ZJFME GSPN$BOETFUDBMMCBDL DGVODUPFWFOUMPPQPGNJEEMFXBSF DGVODPGNJEEMFXBSFPOFWFOUMPPQ QSPDFTTPUIFSSFRVFTUPOFWFOUMPPQ DBMMCBDLDGVOD SVOSFTVNFCZSVOOJOHQSPDPCKFDUGSPN$ DGVODPGNJEEMFXBSFXJUINSC@TUBUF SVOQSPDPCKFDUGSPN$      $VSSFOUNSVCZOPOCMPDLJOHNPEFM 3FRVFTU
  57. My homework of RubyKaigi 2018 57 Show me your ideal

    design of non-blocking model using Fiber for middleware embedding mruby.
  58. 3VCZ4DSJQUʢ3VCZXPSMEʣ non-blocking each request with mruby 58 NJEEMFXBSF $XPSME NSC@TUBUF

    3VCZCZUFDPEFJODMVEFE CMPDLJOHNFUIPEBOE'JCFS DGVODPGCMPDLJOHNFUIPE SVOCMPDLJOHPQFSBUJPOXJUIOPOCMPDLJOHNPEF SVONSC@pCFS@ZJFME GSPN$BOETFUDBMMCBDL DGVODUPFWFOUMPPQPGNJEEMFXBSF DGVODPGNJEEMFXBSFPOFWFOUMPPQ QSPDFTTPUIFSSFRVFTUPOFWFOUMPPQ DBMMCBDLDGVOD SVOSFTVNFCZNSC@pCFS@SFTVNF GSPN$ SFTFU'JCFSDPOUFYUUP  DGVODPGNJEEMFXBSFXJUINSC@TUBUF SVOpCFSPCKFDU CZNSC@pCFS@SFTVNF GSPN$      NZ*EFBMNSVCZOPOCMPDLJOHNPEFM 3FRVFTU $VSSFOUNSVCZDBUDIFTTFHGBVMU
  59. non-blocking sleep example of ngx_mruby v2

  60. non-blocking sleep sample script 60

  61. non-blocking sleep sample script 61

  62. wrapped Ruby script by Fiber 62

  63. wrapped Ruby script by fiber 63

  64. run fiber object 64 SFUVSOUPFWFOUMPPQXIFOpCFSJTBMJWJOH

  65. blocking method (Nginx::Async.sleep) 65 QSPDDFTTPUIFSSFRVFTUT VOUJMDBMMCBDLGVODUJPO

  66. callback handler from event loop 66

  67. now supports non-blocking http[s] client of ngx_mruby v2 in RubyKaigi

    2018 !!
  68. 68

  69. Special thanks for v2 developers 69 !QZBNB !TZV@DSFBN

  70. 3. Conclusion

  71. • Why / How embed mruby into middleware • connect

    services dynamically and programmably • design pattern for embedding mruby into middleware • init /exit • each request • blocking / non-blocking • We will release ngx_mruby v2 in the near future • Happy hacking mruby! 71 Conclusion