SAKURA internet Inc. (C) Copyright 1996-2019 SAKURA Internet Inc SAKURA Research Center mruby meets nginx 2019/06/29 Ryosuke Matsumoto Hanabi Hack: The first Russian-Japanese hackathon in Moscow

• Senior Researcher at SAKURA internet Inc. • Forkwell Technical Advisor at Grooves Inc. • Visiting Researcher at GMO Pepabo, Inc. • Ph.D of Informatics, Kyoto University • My interests: Operation technology, Internet foundation technology, System programming, Highly integrated multi-tenant architecture, Hosting service, Cloud platform, C, mruby, Rust • Engineering => Technology <= Research 2 Ryosuke Matsumoto @matsumotory

3 GitHub: matsumotory

4 write a paper for academic conferences

1. Introduction of embedding mruby into nginx 2. Blocking vs Non-blocking with nginx + mruby 3. Conclusions 5 Table of Contents

1. Introduction of embedding mruby into middleware

• System management for network, OS and middleware • Hosting service, Cloud platform service • Operation technology for middleware written by C • Apache, nginx, dovecot, postfix, sendmail, bind, proftp… • want to write operation tools for middleware using perl • want to write extension module for middleware • each middleware dependency • Apache module, nginx module, dovecot plugin…. 7 I’m a former web operation engineer

I met mruby in April 2012

9 mruby / CRuby for middleware

Middleware configuration as code

Basically embed mruby into middleware 11 lNJEEMFXBSFzXBTJNQMFNFOUFECZ$JOUIJTQSFTFOUBUJPO

mruby in Japan meets nginx in Russia!

• ngx_mruby/mod_mruby(http server extension) • dovecot-mruby-plugin (imap server extension) • pmilter (milter server) • Trusterd HTTP/2 Web Server • h2o_mruby (Initial implementation with v1.4) • Other system tools: rcon, capcon, pfds, virtualing, ab-mruby • developed the greatest number of mrbgem in mgem-list • also received Ph.D. of informatics 13 My developed middleware with mruby

• Middleware behavior scripting with mruby • Middleware configuration as code • limit Ruby methods by mruby • Dynamically control middleware by session parameters • cooperate with microservices • record a session data and utilize the data • implement Service Mesh functions • HTTP(S)ɾTCP/UDP stream 14 Why embed mruby into middleware

• consider several viewpoints • memory management (allocate/free/leak) • performance (non-blocking operation) • usability, operation/development technology • Strategy for multi-process/multi-threading • focus on performance like non-blocking in this presentation 15 How embed mruby into middleware

2. Blocking vs Non-blocking with nginx + mruby

• want to accept as many requests simultaneously as possible in single process/thread (C10k) • want to use multi cpu core sufficiently • process blocking operation in non-blocking mode • File I/O, Network I/O, sleep… • monitor I/O state and notify status change like epoll() • generally implement suitable event loop for middleware 17 Non-blocking middleware

• Non-blocking middleware like nginx • mruby blocks middleware processing generally • Bottleneck of performance when accepting multiple requests simultaneously • Other responses are delayed in proportion to the time of processing of mruby blocking 18 Non-blocking middleware with mruby

Blocking each request with mruby 19 TFOESFTQPOTF OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT SFDWSFRVFTUBUUIFTBNFUJNF

Blocking each request with mruby 20 TFOESFTQPOTF OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT SFDWSFRVFTUBUUIFTBNFUJNF

Blocking example 21

Non-blocking each request with mruby 22 TFOESFTQPOTF OPOCMPDLJOHNJEEMFXBSFMJLFOHJOYJOTJOHMFQSPDFTT SFDWSFRVFTUBUUIFTBNFUJNF

Non-blocking example 23

1.change type a blocking method of mruby to non-blocking 2.suspend mruby processing until the blocking method is completed • using Fiber of mruby 3.return to event loop of nginx from mruby VM • monitor non-blocking function on nginx 4.nginx process other requests…. 5.resume mruby processing by callback func in event loop 24 Non-blocking middleware nginx with mruby

• We perform the following two experiments. 1. While changing the response time on a app server from 1 ms to 50 ms, benchmark the number of simultaneous connections with 100 and measure request/s 2. Fix the response time on the app server to 10 ms, change the number of simultaneous connections from 1 to 100, and measure request/s • We compared the performance of blocking http client and non- blocking http client under these conditions. 25 ngx_mruby blocking vs non-blocking

App server configuration 26

Proxy configuration with blocking operation 27

Proxy configuration with non-blocking 28

Change response time of the App server 29

Change the number of concurrent requests 30

3. Conclusions

• mruby in Japan meets nginx in Russia • I would be very happy if Japanese and Russian engineers could communicate more • Why / How embed mruby into middleware like nginx • connect services dynamically and programmably • blocking/non-blocking • Happy hacking nginx + mruby (ngx_mruby) ! • https://github.com/matsumotory/ngx_mruby 32 Conclusions