Design pattern for embedding mruby into middleware

Design pattern for embedding mruby into middleware

A design and implementation to connect middleware
supporting Internet service with mruby

Ryosuke Matsumoto / Pepabo R&D Institute, GMO Pepabo, Inc.
2018.06.02 RubyKaigi2018

2b692bd83f4418103142a053ecf5ff59?s=128

MATSUMOTO Ryosuke

June 02, 2018
Tweet

Transcript

  1. 1.

    A design and implementation to connect middleware supporting Internet service

    with mruby Ryosuke Matsumoto / Pepabo R&D Institute, GMO Pepabo, Inc. 2018.06.02 RubyKaigi2018 Design pattern for embedding mruby into middleware
  2. 2.

    • Chief engineer at GMO Pepabo, inc. • Chief researcher

    at Pepabo R&D Institute • Ph.D of Informatics, Kyoto University • my interests: Operation technology, Internet foundation technology, System programming, Highly integrated multi- tenant architecture, Hosting service, Cloud platform, C, mruby, Rust • Engineering => Technology <= Research 2 Ryosuke Matsumoto @matsumotory
  3. 3.

    3

  4. 4.

    4

  5. 5.

    1. Introduction of embedding mruby into middleware 2. Design pattern

    for middleware 1. init / exit 2. each request 3. non-blocking 3. Conclusion 5 Table of Contents
  6. 7.

    • System management for network, OS and middleware • Hosting

    service, Cloud platform service • operation technology for middleware written by C • Apache, nginx, dovecot, postfix, sendmail, bind, proftp… • want to write operation tools for middleware using perl • want to write extension module for middleware • each middleware dependency • Apache module, nginx module, dovecot plugin…. 7 I’m a former web operation engineer
  7. 9.

    mruby / CRuby for middleware 9 NJEEMFXBSFJO$ NSVCZ NJEEMFXBSFNFUIPE JONSVCZ

    NJEEMFXBSFGVODUJPOTJO$ NJEEMFXBSFJO$3VCZ NJEEMFXBSFNFUIPE JO$3VCZ NSVCZGPSNJEEMFXBSF $3VCZGPSNJEEMFXBSF
  8. 11.

    basically embed mruby into middleware 11 NJEEMFXBSFQSPDFTT NSC@TUBUFNSVCZ NSVCZCZUFDPEF NJEEMFXBSFGVODUJPOTJO$

    NSVCZ4DSJQUT lNJEEMFXBSFzXBTJNQMFNFOUFECZ$JOUIJTQSFTFOUBUJPO NBTUFS TFSWBOU
  9. 12.

    • mod_mruby / ngx_mruby (http server extension) • dovecot-mruby-plugin (imap

    server extension) • pmilter (milter server) • Trusterd HTTP/2 Web Server • h2o_mruby (Initial implementation with v1.4) • other system tools: rcon, capcon, pfds, virtualing, ab-mruby • developed the greatest number of mrbgem in mgem-list • also received Ph.D of informatics 12 my developed middleware with mruby
  10. 13.

    • middleware behavior scripting with mruby • middleware configuration as

    code • limit Ruby methods by mruby • dynamically control middleware by session parameters • cooperate with microservices • record a session data and utilize the data • implement Service Mesh functions • HTTP(S)ɾTCP/UDP stream 13 Why embed mruby into middleware
  11. 14.

    • consider several viewpoints • memory management (allocate/free/leak) • performance

    • usability, operation / development technology • strategy for multi process / multi threading • ʮmruby = mrb_stateʯ in this presentation 14 How embed mruby into middleware
  12. 17.

    multi-threading strategy for mruby 17 QSPDFTT UISFBE NSC@TUBUF CZUFDPEF UISFBE

    NSC@TUBUF CZUFDPEF QSPDFTT NSC@TUBUF UISFBE CZUFDPEF UISFBE CZUFDPEF QSPDFTT NSC@TUBUF CZUFDPEF UISFBE UISFBE
  13. 18.

    multi-threading strategy for mruby 18 QSPDFTT UISFBE NSC@TUBUF CZUFDPEF UISFBE

    NSC@TUBUF CZUFDPEF QSPDFTT NSC@TUBUF UISFBE CZUFDPEF UISFBE CZUFDPEF QSPDFTT NSC@TUBUF CZUFDPEF UISFBE UISFBE MPDL MPDL
  14. 19.

    • The basic strategy is embedding mruby into each thread

    • lock between threads in single mrb_state is complicated • A cost of creating mrb_state is much higher than creating thread • The cost becomes bottleneck each request to middleware • We want to create mrb_state on init phase of middleware • control only bytecode each request 19 considerations
  15. 20.

    • talk about design patterns for embedding mruby into middleware

    • try and error over and over again for 6 years • consider several viewpoints • memory management • performance • usability, operation / development technology 20 In this presentation
  16. 23.

    init: independent mrb_state 23 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    HFOFSBUFDPEFBOESVOPONBTUFSJOJU GPSL DSFBUFNSC@TUBUFPONBTUFSJOJU NSC@TUBUF CZUFDPEF HFOFSBUFDPEFBOESVOPOXPSLFSJOJU DSFBUFNSC@TUBUFPOXPSLFSJOJU NVMUJQSPDFTTNPEFM
  17. 24.

    init: independent mrb_state 24 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    GSFFDPEFPONBTUFSJOJU GPSL DMPTFNSC@TUBUFPONBTUFSJOJU NSC@TUBUF CZUFDPEF GSFFDPEFPOXPSLFSJOJU DMPTFNSC@TUBUFPOXPSLFSJOJU NVMUJQSPDFTTNPEFM GSFF GSFF GSFF OPUIJOHUPEPPOXPSLFSBOENBTUFSFYJU
  18. 25.

    • Advantages • mruby scripts are executed completely independently •

    can save memory • Disadvantages • can’t pass objects to a request phase • can’t share objects with each other script 25 init: independent mrb_state
  19. 26.

    init: share mrb_state 26 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    CZUFDPEF HFOFSBUFDPEFBOESVOPONBTUFSJOJU POMZHFOFSBUFDPEFPOXPSLFSJOJU GPSL DSFBUFNSC@TUBUFPONBTUFSJOJU NVMUJQSPDFTTNPEFM
  20. 27.

    init: share mrb_state 27 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF

    CZUFDPEF GSFFDPEFPONBTUFSJOJU GPSL DMPTFNSC@TUBUFPONBTUFSJOJU NVMUJQSPDFTTNPEFM GSFFDPEFPOXPSLFSJOJU DMPTFNSC@TUBUFPOXPSLFSJOJU OPUIJOHUPEPPOXPSLFSBOENBTUFSFYJU
  21. 28.

    • Advantages • decrease a cost of creating mrb_state each

    script • can save memory • can share objects with each other script on init phase • Disadvantages • can’t pass objects to a request phase 28 init: share mrb_state
  22. 29.

    init: share mrb_state with request phase 29 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF

    NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF HFOFSBUFDPEFBOESVOPONBTUFSJOJU POMZHFOFSBUFDPEFPOXPSLFSJOJU SFVTFUIFNSC@TUBUFFBDISFRVFTU GPSL DSFBUFNSC@TUBUFPONBTUFSJOJU NVMUJQSPDFTTNPEFM
  23. 30.

    exit: share mrb_state with request phase 30 NBTUFSQSPDFTT XPSLFSQSPDFTT NSC@TUBUF

    NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF GSFFDPEFPONBTUFSFYJU GPSL DMPTFNSC@TUBUFPONBTUFSFYJU NVMUJQSPDFTTNPEFM GSFFDPEFPOXPSLFSFYJU DMPTFNSC@TUBUFPOXPSLFSFYJU
  24. 31.

    • Advantages • decrease a cost of creating mrb_state each

    script • can share objects with each other script on init phase • can pass objects to a request phase • use DB object which connect the DB at init phase • Disadvantages • use more memory than other init pattern • complicated memory management each request 31 init: share mrb_state with request phase
  25. 34.

    XPSLFSQSPDFTT request: independent mrb_state 34 XPSLFSQSPDFTT NSC@TUBUF CZUFDPEF NSC@TUBUF CZUFDPEF

    3FRVFTU XPSLFSQSPDFTT 3FRVFTU 3FTQPOTF 3FTQPOTF NSC@TUBUF CZUFDPEF 3FRVFTU NSC@TUBUF CZUFDPEF NSC@TUBUF CZUFDPEF GSFF GSFF DSFBUFNSC@TUBUFBOEHFOFSBUFDPEF BOESVOCZUFDPEFFBDISFRVFTU *OJU 3FRVFTU 3FTQPOTF
  26. 35.

    • Advantages • mruby scripts are executed completely independently •

    can manage memory easily and correctly • change script each request • Disadvantages • low performance because of creating mrb_state each request • can’t share objects with each other script each request 35 request: independent mrb_state
  27. 36.

    request: shared mrb_state 36 XPSLFSQSPDFTT XPSLFSQSPDFTT 3FRVFTU XPSLFSQSPDFTT 3FRVFTU 3FTQPOTF

    3FTQPOTF *OJU 3FRVFTU 3FTQPOTF NSC@TUBUF NSC@TUBUF CZUFDPEF CZUFDPEF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF 3FRVFTU BSFOB FYD BSFOB FYD BSFOB FYD DMFBOVQBSFOBBOEFYD PCKFDUTFBDISFRVFTU GSFF GSFF QBSTFTDSJQUBOEHFOFSBUFBOE SVOCZUFDPEFFBDISFRVFTU DSFBUFNSC@TUBUF POJOJUJBMJ[BUJPO
  28. 37.

    • Advantages • can share objects with each other script

    each request and init phase • change script each request • Disadvantages • slightly low performance because of compiling • memory management is very complicated because of compiling and freeing bytecode each request while sharing mrb_state 37 request: independent mrb_state
  29. 38.

    request: shared mrb_state and bytecode 38 XPSLFSQSPDFTT XPSLFSQSPDFTT 3FRVFTU XPSLFSQSPDFTT

    3FTQPOTF *OJU 3FRVFTU 3FTQPOTF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF NSC@TUBUF CZUFDPEF CZUFDPEF CZUFDPEF 3FRVFTU BSFOB FYD BSFOB FYD BSFOB FYD DMFBOVQBSFOBBOEFYDPCKFDUT FBDISFRVFTU GSFFDPEFDPOUFYU FBDISFRVFTU DSFBUFNSC@TUBUF QBSTFTDSJQUBOEHFOFSBUF CZUFDPEFPOJOJUJBMJ[BUJPO SVOCZUFDPEFFBDISFRVFTU
  30. 39.

    • Advantages • high performance • can share objects with

    each other script each request and init phase • Disadvantages • can’t change script each request • memory management is a bit complicated 39 request: independent mrb_state
  31. 42.

    multi-threading strategy for mruby 42 QSPDFTT UISFBE NSC@TUBUF CZUFDPEF UISFBE

    NSC@TUBUF CZUFDPEF QSPDFTT NSC@TUBUF UISFBE CZUFDPEF UISFBE CZUFDPEF QSPDFTT NSC@TUBUF CZUFDPEF UISFBE UISFBE
  32. 43.

    XPSLFSQSPDFTT request: independent mrb_state 43 XPSLFSQSPDFTT XPSLFSQSPDFTT DSFBUFNSC@TUBUFBOEHFOFSBUFCZUFDPEF BOESVOCZUFDPEFFBDISFRVFTU XPSLFSUISFBE

    XPSLFSUISFBE XPSLFSUISFBE NSC@TUBUF CZUFDPEF XPSLFSUISFBE XPSLFSUISFBE NSC@TUBUF CZUFDPEF XPSLFSUISFBE 3FRVFTU 3FTQPOTF NSC@TUBUF CZUFDPEF 3FRVFTU GSFF *OJU 3FRVFTU 3FTQPOTF NSC@TUBUF CZUFDPEF 3FRVFTU
  33. 44.

    request: share mrb_state 44 XPSLFSQSPDFTT *OJU 3FRVFTU 3FTQPOTF XPSLFSQSPDFTT XPSLFSQSPDFTT

    XPSLFSUISFBE NSC@TUBUF BSFOB FYD XPSLFSUISFBE NSC@TUBUF BSFOB FYD XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD XPSLFSUISFBE NSC@TUBUF BSFOB FYD XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD 3FTQPOTF 3FRVFTU 3FRVFTU DMFBOVQ BSFOBBOEFYD PCKFDUTFBDI SFRVFTU GSFF QBSTFTDSJQU BOEHFOFSBUF CZUFDPEFBOE SVOCZUFDPEF FBDISFRVFTU CZUFDPEF 3FRVFTU DSFBUFNSC@TUBUFPOJOJUJBMJ[BUJPO
  34. 45.

    request: share mrb_state and bytecode 45 XPSLFSQSPDFTT *OJU 3FRVFTU 3FTQPOTF

    XPSLFSQSPDFTT XPSLFSQSPDFTT XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF XPSLFSUISFBE NSC@TUBUF CZUFDPEF BSFOB FYD CZUFDPEF 3FTQPOTF 3FRVFTU 3FRVFTU DMFBOVQ BSFOBBOEFYD PCKFDUTFBDI SFRVFTU 3FRVFTU GSFFDPEFDPOUFYU FBDISFRVFTU DSFBUFNSC@TUBUF QBSTFTDSJQUBOEHFOFSBUFCZUFDPEFPOJOJUJBMJ[BUJPO SVOCZUFDPEF FBDISFRVFTU
  35. 46.

    • multi process model • share mrb_state and bytecode •

    mod_mruby, ngx_mruby, postfix-mruby, dovecot- mruby, trusterd HTTP/2 server • multi threading model • share mrb_state and bytecode • H2O, nghttpd • independent mrb_state • pmilter 46 Summary: each request pattern
  36. 48.

    • want to accept as many requests simultaneously as possible

    in single process/thread (C10k) • want to use multi cpu core sufficiently • process blocking operation in non-blocking mode • File I/O, Network I/O, sleep… • monitor I/O state and notify status change like epoll() • generally implement suitable event loop for middleware 48 non-blocking middleware
  37. 49.

    • mruby blocks middleware processing generally • bottleneck of performance

    when accepting multiple requests simultaneously • Other responses are delayed in proportion to the time of processing of mruby blocking 49 non-blocking middleware with mruby
  38. 50.

    blocking each request with mruby 50 SFRVFTUQSPDFTTJOH NSVCZ NSVCZ SFRVFTUQSPDFTTJOH

    SFRVFTUQSPDFTTJOH NSVCZ DSFBUFSFTQPOTF DSFBUFSFTQPOTF DSFBUFSFTQPOTF TFOESFTQPOTF OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT SFDWSFRVFTU BUUIFTBNFUJNF
  39. 51.

    blocking each request with mruby 51 SFRVFTU NSVCZ NSVCZ SFTQPOTF

    SFRVFTU SFRVFTU SFTQPOTF SFTQPOTF NSVCZ TFOESFTQPOTF SFDWSFRVFTU BUUIFTBNFUJNF Other responses are delayed in proportion to the time of processing of mruby blocking OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT
  40. 52.

    52

  41. 53.

    non-blocking each request with mruby 53 SFRVFTU SFTQPOTF SFRVFTU SFRVFTU

    SFTQPOTF SFTQPOTF TFOESFTQPOTF SFDWSFRVFTU BUUIFTBNFUJNF CMPDLJOH PQFSBJUPO NSVCZ CMPDLJOH PQFSBJUPO NSVCZ NSVCZ CMPDLJOH PQFSBJUPO OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT
  42. 54.

    54

  43. 55.

    • change a blocking method of mruby to non-blocking •

    suspend mruby processing until the blocking method is completed • return to event loop of middleware from mruby • resume mruby processing by callback func in event loop 55 non-blocking middleware with mruby
  44. 56.

    3VCZ4DSJQUʢ3VCZXPSMEʣ 1SPD 'JCFSSFTVNF non-blocking each request with mruby 56 NJEEMFXBSF

    $XPSME NSC@TUBUF 'JCFS 3VCZCZUFDPEF JODMVEFECMPDLJOH NFUIPE DGVODPGCMPDLJOHNFUIPE SVOCMPDLJOHPQFSBUJPOXJUIOPOCMPDLJOHNPEF SVONSC@pCFS@ZJFME GSPN$BOETFUDBMMCBDL DGVODUPFWFOUMPPQPGNJEEMFXBSF DGVODPGNJEEMFXBSFPOFWFOUMPPQ QSPDFTTPUIFSSFRVFTUPOFWFOUMPPQ DBMMCBDLDGVOD SVOSFTVNFCZSVOOJOHQSPDPCKFDUGSPN$ DGVODPGNJEEMFXBSFXJUINSC@TUBUF SVOQSPDPCKFDUGSPN$      $VSSFOUNSVCZOPOCMPDLJOHNPEFM 3FRVFTU
  45. 57.

    My homework of RubyKaigi 2018 57 Show me your ideal

    design of non-blocking model using Fiber for middleware embedding mruby.
  46. 58.

    3VCZ4DSJQUʢ3VCZXPSMEʣ non-blocking each request with mruby 58 NJEEMFXBSF $XPSME NSC@TUBUF

    3VCZCZUFDPEFJODMVEFE CMPDLJOHNFUIPEBOE'JCFS DGVODPGCMPDLJOHNFUIPE SVOCMPDLJOHPQFSBUJPOXJUIOPOCMPDLJOHNPEF SVONSC@pCFS@ZJFME GSPN$BOETFUDBMMCBDL DGVODUPFWFOUMPPQPGNJEEMFXBSF DGVODPGNJEEMFXBSFPOFWFOUMPPQ QSPDFTTPUIFSSFRVFTUPOFWFOUMPPQ DBMMCBDLDGVOD SVOSFTVNFCZNSC@pCFS@SFTVNF GSPN$ SFTFU'JCFSDPOUFYUUP  DGVODPGNJEEMFXBSFXJUINSC@TUBUF SVOpCFSPCKFDU CZNSC@pCFS@SFTVNF GSPN$      NZ*EFBMNSVCZOPOCMPDLJOHNPEFM 3FRVFTU $VSSFOUNSVCZDBUDIFTTFHGBVMU
  47. 68.

    68

  48. 71.

    • Why / How embed mruby into middleware • connect

    services dynamically and programmably • design pattern for embedding mruby into middleware • init /exit • each request • blocking / non-blocking • We will release ngx_mruby v2 in the near future • Happy hacking mruby! 71 Conclusion