Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical nginx module development〜C and Lua〜

Practical nginx module development〜C and Lua〜

Tatsuhiko Kubo

August 22, 2015
Tweet

More Decks by Tatsuhiko Kubo

Other Decks in Technology

Transcript

  1. Practical nginx module development ʙC and Luaʙ Tatsuhiko Kubo@cubicdaiya YAPC::Asia

    TOKYO 2015
  2. @cubicdaiya / Tatsuhiko Kubo • Software Engineer @ Mercari, Inc.

    • Infrastructure Engineering • Skills • C, Go, Lua, nginx, … • OSS developer • ngx_small_light, ngx_dynamic_upstream, nginx-build, slackboard, cachectl, gaurun, …
  3. Activities on Github

  4. Other Activities IUUQNP[BJDGNQPTUOHJOY

  5. None
  6. None
  7. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  8. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  9. nginx • Open Source Software • HTTP server for various

    purpose • Static content delivery • Reverse proxy ( HTTP / FastCGI / uWSGI / … ) • Load balancer • Cache • TLS Termination / SPDY Gateway • etc… • Mail proxy ( SMTP / POP3 / IMAP ) • TCP proxy
  10. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  11. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  12. nginx is module-oriented • most features are designed as module

    • ngx_http_core_module • ngx_http_access_module • ngx_http_log_module • ngx_http_proxy_module • ngx_http_upstream_module • etc…
  13. 3rd party module for nginx • nginx-user can develop an

    original module • It is not required to modify source code of nginx • embed original module with ‘./configure’ $ cd nginx-1.9.x $ ./configure —-add-module=${ngx_module_path}
  14. Examples

  15. Example • ngx_http_empty_gif_module (Official) • return 1x1 GIF location /gif

    { empty_gif; } $ curl -I http://127.0.0.1/gif HTTP/1.1 200 OK Content-Length: 43 Content-Type: image/gif ... $
  16. Example • ngx_small_light • Dynamic Image Transformation for nginx small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; }
  17. ngx_small_light • Dynamic Image Transformation for nginx • https://github.com/cubicdaiya/ngx_small_light •

    Provides various • Image Manipulations ( Resize / Crop / Composite / …) • Image Processing Libraries (Image Magick / Imlib2 / GD) • Image Formats (JPEG / GIF / PNG / WEBP)
  18. Transform image ▪URI for resize /small_light(dw=300,dh=200)/img/image.jpg →

  19. Transform image on S3 proxy from nginx to S3 small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location /img { proxy_pass http://<s3_bucket_url>; }
  20. Adoption case https://github.com/wantedly/nginx-image-server

  21. Adoption case https://speakerdeck.com/yano3/dynamic-image-transformation-server-okara-beta

  22. Example • ngx_dynamic_upstream • Dynamic Upstream for nginx location /

    { allow xxx.xxx.xxx.xxx deny all; dynamic_upstream; }
  23. ngx_dynamic_upstream • Dynamic Upstream for nginx • https://github.com/cubicdaiya/ngx_dynamic_upstream • Provides

    HTTP APIs for manipulating upstream for nginx • It is useful for • zero-down-time deployment • blue green deployment
  24. Zero-down-time deployment on

  25. ɾɾɾ ɾɾɾ "1* "QBDIF NPE@QIQ OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY 

    OHY@EZOBNJD@VQTUSFBN HTTPS or SPDY OHJOY  OHY@EZOBNJD@VQTUSFBN HTTP Mercari frontend "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ ɾɾɾ ɾɾɾ
  26. "1* "QBDIF NPE@QIQ Previous deployment "1* "QBDIF NPE@QIQ "1* "QBDIF

    NPE@QIQ ChatOps with Slack yes EFQMPZCPU rsync rsync rsync
  27. Pros / Cons • Pros • Very Very Very simple

    • Cons • many 500 error happens at deployment!!!!!
  28. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ
  29. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  30. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  31. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ
  32. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  33. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ
  34. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  35. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  36. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  37. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  38. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  39. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  40. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  41. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  42. Hello, World!

  43. Hello, World! • ngx_http_hello_world • https://github.com/cubicdaiya/ngx_http_hello_world location /hello { hello_world;

    } $ curl http://127.0.0.1/hello Hello, World! $
  44. Layout of ngx_http_hello_world ngx_http_hello_world |———... |———config |———ngx_http_hello_world_module.c └──-...

  45. config • module configuration file ngx_addon_name=ngx_http_hello_world_module HTTP_MODULES="$HTTP_MODULES \ ngx_http_hello_world_module” NGX_ADDON_SRCS="$NGX_ADDON_SRCS

    \ $ngx_addon_dir/ngx_http_hello_world_module.c”
  46. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  47. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  48. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; # original directive } $ curl http://127.0.0.1/hello Hello, World! $
  49. Directive declaration struct ngx_command_s { ngx_str_t name; ngx_uint_t type; char

    *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_uint_t conf; ngx_uint_t offset; void *post; };
  50. Directive definition(hello_world) static ngx_command_t ngx_http_hello_world_commands[] = { { ngx_string("hello_world"), NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS,

    ngx_http_hello_world, 0, 0, NULL }, ngx_null_command };
  51. Module context typedef struct { ngx_int_t (*preconfiguration)(ngx_conf_t *cf); ngx_int_t (*postconfiguration)(ngx_conf_t

    *cf); void *(*create_main_conf)(ngx_conf_t *cf); char *(*init_main_conf)(ngx_conf_t *cf, void *conf); void *(*create_srv_conf)(ngx_conf_t *cf); char *(*merge_srv_conf)(ngx_conf_t *cf, void *prev, void *conf); void *(*create_loc_conf)(ngx_conf_t *cf); char *(*merge_loc_conf)(ngx_conf_t *cf, void *prev, void *conf); } ngx_http_module_t;
  52. Module context static ngx_http_module_t ngx_http_hello_world_module_ctx = { NULL, /* preconfiguration

    */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; In this case, all is NULL.
  53. Module definition ngx_module_t ngx_http_hello_world_module = { NGX_MODULE_V1, &ngx_http_hello_world_module_ctx, /* module

    context */ ngx_http_hello_world_commands, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING };
  54. Hook original handler static char *ngx_http_hello_world(ngx_conf_t *cf, ngx_command_t *cmd, void

    *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_hello_world_handler; return NGX_CONF_OK; }
  55. ngx_http_hello_world_handler() #define NGX_HTTP_HELLO_WORLD “Hello, World!” static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r) {

    ngx_int_t rc; ngx_chain_t out; ngx_buf_t *b; ngx_str_t body = ngx_string(NGX_HTTP_HELLO_WORLD); if (r->method != NGX_HTTP_GET && r->method != NGX_HTTP_HEAD) { return NGX_HTTP_NOT_ALLOWED; } if (r->headers_in.if_modified_since) { return NGX_HTTP_NOT_MODIFIED; } b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } b->pos = body.data; b->last = b->pos + body.len; b->memory = 1; b->last_buf = 1; out.buf = b; out.next = NULL; ngx_str_set(&r->headers_out.content_type, "text/plain"); r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = body.len; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; }
  56. Embed ngx_http_hello_world into nginx $ ./confgiure \ —-add-module=../ngx_http_hello_world $ make

    $ sudo make install
  57. Hook details…

  58. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location
  59. Phase definitions typedef enum { NGX_HTTP_POST_READ_PHASE = 0, NGX_HTTP_SERVER_REWRITE_PHASE, NGX_HTTP_FIND_CONFIG_PHASE,

    NGX_HTTP_REWRITE_PHASE, NGX_HTTP_POST_REWRITE_PHASE, NGX_HTTP_PREACCESS_PHASE, NGX_HTTP_ACCESS_PHASE, NGX_HTTP_POST_ACCESS_PHASE, NGX_HTTP_TRY_FILES_PHASE, NGX_HTTP_CONTENT_PHASE, NGX_HTTP_LOG_PHASE } ngx_http_phases;
  60. Hook handler to specified phases ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cmcf; cmcf

    = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cmcf->phases[NGX_HTTP_ACCESS_PHASE].handlers); if (h == NULL) { return NGX_ERROR; } *h = ngx_http_xxx_handler;
  61. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  62. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  63. nginx API • memory-pool • string • array • hash-table

    • time • list • regular-expression • temporary-file • shared-memory • etc…
  64. essence of nginx API programming • understand nginx architecture •

    event-driven • non-blocking • asynchronous • single-threaded • multi-process
  65. essence of nginx API programming • memory management strategies •

    memory-pool • slab ( for shared-memory) • string with length • string is not always NULL-terminated
  66. essence of nginx API programming • structs for HTTP •

    ngx_http_request_t • ngx_http_headers_in • ngx_http_headers_out • etc… ( See nginx/src/http/ngx_http_request.h )
  67. Move to terminal…

  68. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  69. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  70. Test::Nginx https://github.com/openresty/test-nginx

  71. Test::Nginx • Testing framework for nginx • https://github.com/openresty/test-nginx • We

    are able to test nginx’s behavior with CLI • Install from CPAN $ cpanm Test::Nginx
  72. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200
  73. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests
  74. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests define behavior ɾnginx.conf ɾHTTP request ɾHTTP response
  75. By the way, • C programming is difficult and complicated

    • productivity is not high If scripting language is available on nginx, productivity would improve dramatically.
  76. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  77. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  78. ngx_lua https://github.com/openresty/lua-nginx-moduleʙ

  79. ngx_lua • embed Lua/LuaJIT into nginx • Lua scripting in

    nginx.conf • Hooking request processing phase with Lua • wrap nginx API with Lua • non-blocking APIs (e.g. ngx.location.capture())
  80. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file Hook directives for request processing phases
  81. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-string Hook directives for request processing phases
  82. Hook directives for request processing phases • set_by_lua / set_by_lua_file

    • rewrite_by_lua / rewrite_by_lua_file • access_by_lua / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-file
  83. content_by_lua <lua-script-string> location /hello { content_by_lua “ngx.say(‘Hello, World!’)”; } ▪

    nginx.conf $ curl -s http://127.0.0.1/hello Hello, World! $ ▪ run
  84. content_by_lua_file <lua-script-path> location /hello { content_by_lua_file /etc/nginx/lua/hello.lua; } ▪ nginx.conf

    ▪ /etc/nginx/lua/hello.lua ngx.say(“Hello, World!”)
  85. set_by_lua $result <lua-script-string> location /yapc { set $conference YAPC::Asia; set

    $location Tokyo; set $year 2015; set_by_lua $result ‘return ngx.var.conference .. “ ” .. ngx.var.location .. “ ” .. ngx.var.year ‘; content_by_lua ‘ngx.say(ngx.var.result)’; } ▪ nginx.conf $ curl -s http://127.0.0.1/hello YAPC::Asia Tokyo 2015 $ ▪ run
  86. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location set_by_lua*
 rewrite_by_lua* access_by_lua* content_by_lua* log_by_lua*
  87. Other hook directives and phases IPPLOBNF QIBTF JOJU@CZ@MVB JOJU@CZ@MVB@pMF MPBEJOHOHJOYDPOG

    JOJU@XPSLFS@CZ@MVB JOJU@XPSLFS@CZ@MVB@pMF TUBSUJOHXPSLFSQSPDFTT IFBEFS@pMUFS@CZ@MVB IFBEFS@pMUFS@CZ@MVB@pMF pMUFSJOHIFBEFS CPEZ@pMUFS@CZ@MVB CPEZ@pMUFS@CZ@MVB@pMF pMUFSJOHCPEZ
  88. ngx_lua APIs • ngx.say() / ngx.exit() / ngx.log / …

    • ngx.var.VARIABLE • ngx.shared.DICT • ngx.header.HEADER • etc…
  89. ngx.say() ngx.say(“Hello, World!”) • output string to response body

  90. ngx.exit() ngx.exit(ngx.HTTP_OK) • return HTTP status code

  91. HTTP status code in ngx_lua • ngx.HTTP_OK (200) • ngx.HTTP_CREATED

    (201) • ngx.HTTP_MOVED_PERMANENTLY (301) • ngx.HTTP_MOVED_TEMPORARILY (302) • ngx.HTTP_NOT_MODIFIED (304) • ngx.HTTP_BAD_REQUEST (400) • ngx.HTTP_INTERNAL_SERVER_ERROR (500) • etc…
  92. ngx.log() ngx.exit(ngx.ERR, “some error”) • output message to error.log •

    $arg1: log-level, $arg2: message
  93. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG
  94. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG log-level High Low
  95. ngx.var.VARIABLE • read / overwrite nginx variables location / {

    set $fn “Tatsuhiko Kubo”; set $hn “@cubicdaiya”; content_by_lua ‘ local body = ngx.var.fn .. ngx.var.hn ngx.say(body) —- Tatsuhiko Kubo@cubicdaiya ‘; }
  96. ngx.var.VARIABLE • creating new variable is not allowed location /

    { content_by_lua ‘ ngx.var.fn = “Tatsuhiko Kubo” ngx.var.hn = “@cubicdaiya” ‘; } runtime error: … variable “fn” not found for writing; maybe it is a built-in variable that is not changeable or … ɾ error.log
  97. ngx.shared.DICT • dictionary with shared memory http { lua_shared_dict yapc

    1m; init_by_lua ‘ ngx.shared.yapc:set(“presentator”, “@cubicdaiya”) ’; … server { … location / { content_by_lua ‘ ngx.say(ngx.shared.yapc:get(“presentator”)) ’; } } }
  98. ngx.header.HEADER • response header reference and manipulation location / {

    content_by_lua ‘ local body = “{ \\”presentator\\” : \\”@cubicdaiya\\” }” —- assign application/json to Content-Type ngx.header.content_type = “application/json” ngx.say(body) ‘; }
  99. Header reference and manipulation • ngx.req.get_headers() • ngx.req.set_header() • ngx.req.clear_header()

    • … • ngx.resp.get_headers() • ngx.header.HEADER • … ▪ For request-header ▪ For response-header
  100. ngx.time() • return current timestamp local timestamp = ngx.time()

  101. ngx_lua time APIs • ngx.time() • ngx.today() • ngx.now() •

    ngx.update_time() • ngx.parse_http_time() • ngx.localtime() • ngx.utctime() • ngx.cookie_time() • ngx.http_time()
  102. Use ngx.time() instead of os.time() • os.time() always calls gettimeofday()

    • ngx.time() uses timer-cache of nginx • avoid system-call’s overhead • ngx.time() is more efficient than os.time()
  103. ngx.re.match() • Regular Expression matching • powered by PCRE local

    matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”) -— matches[0] -> image/webp -— matches[1] -> image -— matches[2] -> webp
  104. ngx.re.* (Regular Expression API) • ngx.re.find() • ngx.re.match() • ngx.re.gmatch()

    • ngx.re.sub() • ngx.re.gsub()
  105. • Use PCRE options(e.g. o, j) • o -> enable

    compile-once • j -> enable PCRE-JIT Optimize ngx.re.* local matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”, “oj”)
  106. • ngx.re is based PCRE • Powerful & Fat •

    Lua builtin-re(regular expression) • Simple & Light ngx.re.* VS Lua builtin-re If ‘re’ is simple, Lua builtin-re is faster.
  107. ngx.location.capture() • send non-blocking sub-request to other location location /external

    { internal; proxy_pass http://external; } location / { content_by_lua ‘ —— non-blocking behavior local res = ngx.location.capture(“/external”) … ‘; }
  108. By the way, • ngx_lua is very powerful • But

    setup and deployment is a little painful • Lua / LuaJIT instalation • Lua packages management
  109. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  110. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  111. OpenResty http://openresty.org/

  112. OpenResty • Web application framework bundled • nginx and 3rd

    party modules (e.g. ngx_lua) • Lua, LuaJIT • Resty modules(lua-resty-xxx…) • All-in-one superset of nginx
  113. 3rd party modules • lua-nginx-module (ngx_lua) • echo-nginx-module • headers-more-nginx-module

    • etc…
  114. Resty modules • lua-resty-core • resty-cli • lua-resty-mysql / redis

    / memcached • etc…
  115. lua-resty-core • pure Lua partial-re-implementation of ngx_lua APIs • accelerate

    ngx_lua APIs with LuaJIT • because ngx_lua APIs with C is not JIT compiled http { # accelerate ngx_lua APIs init_by_lua ‘require resty.core’; … }
  116. lua-resty-cli • command-line utility for OpenResty • run OpenResty with

    CLI • all APIs are not always enabled $ resty -e ‘ngx.say(“Hello, World!”)’ Hello, World! $
  117. lua-resty-mysql / redis / memcached • client drivers for mysql

    / redis / memcached • non-blocking behavior • Powered by ngx.socket
  118. OpenResty on

  119. PascalʙMercari app-event analysis baseʙ 0QFO3FTUZ 0QFO3FTUZ 0QFO3FTUZ (PPHMF#JH2VFSZ Developer Data

    Sientist Analyze by SQL send events send events send events Powered by cookpad/puree-(ios|android) utilize events transfer utilize events utilize events transfer
  120. OpenResty’s role on Pascal • High performance event logger •

    receive events as JSON from devices • parse JSON • utilize events for BigQuery • log events into file every event-channel
  121. References • The Architecture of Open Source Applications (Volume 2):

    nginx • http://www.aosabook.org/en/nginx.html • Nginx Internals • http://www.slideshare.net/joshzhu/nginx-internals • Emiller’s Guide To Nginx Module Development • http://www.evanmiller.org/nginx-modules-guide.html
  122. References • openresty/lua-nginx-module • https://github.com/openresty/lua-nginx-module • Using ngx_lua / lua-nginx-module

    in pixiv • http://www.slideshare.net/harukayon/ngx-lua-public • ngx_mruby and ngx_lua • http://qiita.com/cubicdaiya/items/3913d3dddf6999984c5b
  123. Thanks!