Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical nginx module development〜C and Lua〜

Practical nginx module development〜C and Lua〜

5d74d743eabd2bf7d4d2f68b9d3c727d?s=128

Tatsuhiko Kubo

August 22, 2015
Tweet

Transcript

  1. Practical nginx module development ʙC and Luaʙ Tatsuhiko Kubo@cubicdaiya YAPC::Asia

    TOKYO 2015
  2. @cubicdaiya / Tatsuhiko Kubo • Software Engineer @ Mercari, Inc.

    • Infrastructure Engineering • Skills • C, Go, Lua, nginx, … • OSS developer • ngx_small_light, ngx_dynamic_upstream, nginx-build, slackboard, cachectl, gaurun, …
  3. Activities on Github

  4. Other Activities IUUQNP[BJDGNQPTUOHJOY

  5. None
  6. None
  7. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  8. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  9. nginx • Open Source Software • HTTP server for various

    purpose • Static content delivery • Reverse proxy ( HTTP / FastCGI / uWSGI / … ) • Load balancer • Cache • TLS Termination / SPDY Gateway • etc… • Mail proxy ( SMTP / POP3 / IMAP ) • TCP proxy
  10. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  11. nginx’s architecture • event-driven • non-blocking • asynchronous • single-threaded

    ( multi-threaded partially) • multi-processes ( master / wokers / …) • modular
  12. nginx is module-oriented • most features are designed as module

    • ngx_http_core_module • ngx_http_access_module • ngx_http_log_module • ngx_http_proxy_module • ngx_http_upstream_module • etc…
  13. 3rd party module for nginx • nginx-user can develop an

    original module • It is not required to modify source code of nginx • embed original module with ‘./configure’ $ cd nginx-1.9.x $ ./configure —-add-module=${ngx_module_path}
  14. Examples

  15. Example • ngx_http_empty_gif_module (Official) • return 1x1 GIF location /gif

    { empty_gif; } $ curl -I http://127.0.0.1/gif HTTP/1.1 200 OK Content-Length: 43 Content-Type: image/gif ... $
  16. Example • ngx_small_light • Dynamic Image Transformation for nginx small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; }
  17. ngx_small_light • Dynamic Image Transformation for nginx • https://github.com/cubicdaiya/ngx_small_light •

    Provides various • Image Manipulations ( Resize / Crop / Composite / …) • Image Processing Libraries (Image Magick / Imlib2 / GD) • Image Formats (JPEG / GIF / PNG / WEBP)
  18. Transform image ▪URI for resize /small_light(dw=300,dh=200)/img/image.jpg →

  19. Transform image on S3 proxy from nginx to S3 small_light

    on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location /img { proxy_pass http://<s3_bucket_url>; }
  20. Adoption case https://github.com/wantedly/nginx-image-server

  21. Adoption case https://speakerdeck.com/yano3/dynamic-image-transformation-server-okara-beta

  22. Example • ngx_dynamic_upstream • Dynamic Upstream for nginx location /

    { allow xxx.xxx.xxx.xxx deny all; dynamic_upstream; }
  23. ngx_dynamic_upstream • Dynamic Upstream for nginx • https://github.com/cubicdaiya/ngx_dynamic_upstream • Provides

    HTTP APIs for manipulating upstream for nginx • It is useful for • zero-down-time deployment • blue green deployment
  24. Zero-down-time deployment on

  25. ɾɾɾ ɾɾɾ "1* "QBDIF NPE@QIQ OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY 

    OHY@EZOBNJD@VQTUSFBN HTTPS or SPDY OHJOY  OHY@EZOBNJD@VQTUSFBN HTTP Mercari frontend "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ ɾɾɾ ɾɾɾ
  26. "1* "QBDIF NPE@QIQ Previous deployment "1* "QBDIF NPE@QIQ "1* "QBDIF

    NPE@QIQ ChatOps with Slack yes EFQMPZCPU rsync rsync rsync
  27. Pros / Cons • Pros • Very Very Very simple

    • Cons • many 500 error happens at deployment!!!!!
  28. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ
  29. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  30. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  31. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ
  32. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  33. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ
  34. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  35. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  36. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream
  37. Current deployment "1* "QBDIF NPE@QIQ ChatOps with Slack yes EFQMPZCPU

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  38. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ ※rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  39. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream down "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  40. "1* "QBDIF NPE@QIQ Current deployment "1* "QBDIF NPE@QIQ ChatOps with

    Slack yes OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN "1* "QBDIF NPE@QIQ rsync ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream EFQMPZCPU
  41. Current deployment ChatOps with Slack yes EFQMPZCPU OHJOY  OHY@EZOBNJD@VQTUSFBN

    OHJOY  OHY@EZOBNJD@VQTUSFBN OHJOY  OHY@EZOBNJD@VQTUSFBN ※ rsync ——rsync-path=mercari_app_rsync(↓) #!/bin/sh mercari_app_ctl down # deactivate server on nginx upstream rsync $* # deploy mercari_app_ctl up # activate server on nginx upstream up "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ "1* "QBDIF NPE@QIQ
  42. Hello, World!

  43. Hello, World! • ngx_http_hello_world • https://github.com/cubicdaiya/ngx_http_hello_world location /hello { hello_world;

    } $ curl http://127.0.0.1/hello Hello, World! $
  44. Layout of ngx_http_hello_world ngx_http_hello_world |———... |———config |———ngx_http_hello_world_module.c └──-...

  45. config • module configuration file ngx_addon_name=ngx_http_hello_world_module HTTP_MODULES="$HTTP_MODULES \ ngx_http_hello_world_module” NGX_ADDON_SRCS="$NGX_ADDON_SRCS

    \ $ngx_addon_dir/ngx_http_hello_world_module.c”
  46. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  47. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; } $ curl http://127.0.0.1/hello Hello, World! $
  48. Hello, World! • ngx_http_hello_world • return “Hello, World!” location /hello

    { hello_world; # original directive } $ curl http://127.0.0.1/hello Hello, World! $
  49. Directive declaration struct ngx_command_s { ngx_str_t name; ngx_uint_t type; char

    *(*set)(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_uint_t conf; ngx_uint_t offset; void *post; };
  50. Directive definition(hello_world) static ngx_command_t ngx_http_hello_world_commands[] = { { ngx_string("hello_world"), NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS,

    ngx_http_hello_world, 0, 0, NULL }, ngx_null_command };
  51. Module context typedef struct { ngx_int_t (*preconfiguration)(ngx_conf_t *cf); ngx_int_t (*postconfiguration)(ngx_conf_t

    *cf); void *(*create_main_conf)(ngx_conf_t *cf); char *(*init_main_conf)(ngx_conf_t *cf, void *conf); void *(*create_srv_conf)(ngx_conf_t *cf); char *(*merge_srv_conf)(ngx_conf_t *cf, void *prev, void *conf); void *(*create_loc_conf)(ngx_conf_t *cf); char *(*merge_loc_conf)(ngx_conf_t *cf, void *prev, void *conf); } ngx_http_module_t;
  52. Module context static ngx_http_module_t ngx_http_hello_world_module_ctx = { NULL, /* preconfiguration

    */ NULL, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ NULL, /* create location configuration */ NULL /* merge location configuration */ }; In this case, all is NULL.
  53. Module definition ngx_module_t ngx_http_hello_world_module = { NGX_MODULE_V1, &ngx_http_hello_world_module_ctx, /* module

    context */ ngx_http_hello_world_commands, /* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING };
  54. Hook original handler static char *ngx_http_hello_world(ngx_conf_t *cf, ngx_command_t *cmd, void

    *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_hello_world_handler; return NGX_CONF_OK; }
  55. ngx_http_hello_world_handler() #define NGX_HTTP_HELLO_WORLD “Hello, World!” static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r) {

    ngx_int_t rc; ngx_chain_t out; ngx_buf_t *b; ngx_str_t body = ngx_string(NGX_HTTP_HELLO_WORLD); if (r->method != NGX_HTTP_GET && r->method != NGX_HTTP_HEAD) { return NGX_HTTP_NOT_ALLOWED; } if (r->headers_in.if_modified_since) { return NGX_HTTP_NOT_MODIFIED; } b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } b->pos = body.data; b->last = b->pos + body.len; b->memory = 1; b->last_buf = 1; out.buf = b; out.next = NULL; ngx_str_set(&r->headers_out.content_type, "text/plain"); r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = body.len; rc = ngx_http_send_header(r); if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) { return rc; }
  56. Embed ngx_http_hello_world into nginx $ ./confgiure \ —-add-module=../ngx_http_hello_world $ make

    $ sudo make install
  57. Hook details…

  58. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location
  59. Phase definitions typedef enum { NGX_HTTP_POST_READ_PHASE = 0, NGX_HTTP_SERVER_REWRITE_PHASE, NGX_HTTP_FIND_CONFIG_PHASE,

    NGX_HTTP_REWRITE_PHASE, NGX_HTTP_POST_REWRITE_PHASE, NGX_HTTP_PREACCESS_PHASE, NGX_HTTP_ACCESS_PHASE, NGX_HTTP_POST_ACCESS_PHASE, NGX_HTTP_TRY_FILES_PHASE, NGX_HTTP_CONTENT_PHASE, NGX_HTTP_LOG_PHASE } ngx_http_phases;
  60. Hook handler to specified phases ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cmcf; cmcf

    = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cmcf->phases[NGX_HTTP_ACCESS_PHASE].handlers); if (h == NULL) { return NGX_ERROR; } *h = ngx_http_xxx_handler;
  61. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  62. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  63. nginx API • memory-pool • string • array • hash-table

    • time • list • regular-expression • temporary-file • shared-memory • etc…
  64. essence of nginx API programming • understand nginx architecture •

    event-driven • non-blocking • asynchronous • single-threaded • multi-process
  65. essence of nginx API programming • memory management strategies •

    memory-pool • slab ( for shared-memory) • string with length • string is not always NULL-terminated
  66. essence of nginx API programming • structs for HTTP •

    ngx_http_request_t • ngx_http_headers_in • ngx_http_headers_out • etc… ( See nginx/src/http/ngx_http_request.h )
  67. Move to terminal…

  68. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  69. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  70. Test::Nginx https://github.com/openresty/test-nginx

  71. Test::Nginx • Testing framework for nginx • https://github.com/openresty/test-nginx • We

    are able to test nginx’s behavior with CLI • Install from CPAN $ cpanm Test::Nginx
  72. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200
  73. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests
  74. test nginx’s behavior with Test::Nginx use Test::Nginx::Socket; plan tests =>

    repeat_each() * 2 * blocks(); run_tests(); __DATA__ === TEST 1: hello world --- config location /hello { return 200 "Hello, World!\n"; } --- request GET /hello --- response_body Hello, World! --- error_code: 200 run tests define behavior ɾnginx.conf ɾHTTP request ɾHTTP response
  75. By the way, • C programming is difficult and complicated

    • productivity is not high If scripting language is available on nginx, productivity would improve dramatically.
  76. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  77. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  78. ngx_lua https://github.com/openresty/lua-nginx-moduleʙ

  79. ngx_lua • embed Lua/LuaJIT into nginx • Lua scripting in

    nginx.conf • Hooking request processing phase with Lua • wrap nginx API with Lua • non-blocking APIs (e.g. ngx.location.capture())
  80. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file Hook directives for request processing phases
  81. • set_by_lua / set_by_lua_file • rewrite_by_lua / rewrite_by_lua_file • access_by_lua

    / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-string Hook directives for request processing phases
  82. Hook directives for request processing phases • set_by_lua / set_by_lua_file

    • rewrite_by_lua / rewrite_by_lua_file • access_by_lua / access_by_lua_file • content_by_lua / content_by_lua_file • log_by_lua / log_by_lua_file run Lua script-file
  83. content_by_lua <lua-script-string> location /hello { content_by_lua “ngx.say(‘Hello, World!’)”; } ▪

    nginx.conf $ curl -s http://127.0.0.1/hello Hello, World! $ ▪ run
  84. content_by_lua_file <lua-script-path> location /hello { content_by_lua_file /etc/nginx/lua/hello.lua; } ▪ nginx.conf

    ▪ /etc/nginx/lua/hello.lua ngx.say(“Hello, World!”)
  85. set_by_lua $result <lua-script-string> location /yapc { set $conference YAPC::Asia; set

    $location Tokyo; set $year 2015; set_by_lua $result ‘return ngx.var.conference .. “ ” .. ngx.var.location .. “ ” .. ngx.var.year ‘; content_by_lua ‘ngx.say(ngx.var.result)’; } ▪ nginx.conf $ curl -s http://127.0.0.1/hello YAPC::Asia Tokyo 2015 $ ▪ run
  86. Request processing phases in nginx TFSWFSSFXSJUF QPTUBDDFTT MPH pOEDPOpH SFXSJUF

    QPTUSFXSJUF QSFBDDFTT USZpMFT DPOUFOU Request BDDFTT try_files processing access-control response body generation logging rewrite in server find location rewrite in location set_by_lua*
 rewrite_by_lua* access_by_lua* content_by_lua* log_by_lua*
  87. Other hook directives and phases IPPLOBNF QIBTF JOJU@CZ@MVB JOJU@CZ@MVB@pMF MPBEJOHOHJOYDPOG

    JOJU@XPSLFS@CZ@MVB JOJU@XPSLFS@CZ@MVB@pMF TUBSUJOHXPSLFSQSPDFTT IFBEFS@pMUFS@CZ@MVB IFBEFS@pMUFS@CZ@MVB@pMF pMUFSJOHIFBEFS CPEZ@pMUFS@CZ@MVB CPEZ@pMUFS@CZ@MVB@pMF pMUFSJOHCPEZ
  88. ngx_lua APIs • ngx.say() / ngx.exit() / ngx.log / …

    • ngx.var.VARIABLE • ngx.shared.DICT • ngx.header.HEADER • etc…
  89. ngx.say() ngx.say(“Hello, World!”) • output string to response body

  90. ngx.exit() ngx.exit(ngx.HTTP_OK) • return HTTP status code

  91. HTTP status code in ngx_lua • ngx.HTTP_OK (200) • ngx.HTTP_CREATED

    (201) • ngx.HTTP_MOVED_PERMANENTLY (301) • ngx.HTTP_MOVED_TEMPORARILY (302) • ngx.HTTP_NOT_MODIFIED (304) • ngx.HTTP_BAD_REQUEST (400) • ngx.HTTP_INTERNAL_SERVER_ERROR (500) • etc…
  92. ngx.log() ngx.exit(ngx.ERR, “some error”) • output message to error.log •

    $arg1: log-level, $arg2: message
  93. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG
  94. log-level in ngx_lua • ngx.EMERG • ngx.ALERT • ngx.CRIT •

    ngx.ERR • ngx.WARN • ngx.NOTICE • ngx.INFO • ngx.DEBUG log-level High Low
  95. ngx.var.VARIABLE • read / overwrite nginx variables location / {

    set $fn “Tatsuhiko Kubo”; set $hn “@cubicdaiya”; content_by_lua ‘ local body = ngx.var.fn .. ngx.var.hn ngx.say(body) —- Tatsuhiko Kubo@cubicdaiya ‘; }
  96. ngx.var.VARIABLE • creating new variable is not allowed location /

    { content_by_lua ‘ ngx.var.fn = “Tatsuhiko Kubo” ngx.var.hn = “@cubicdaiya” ‘; } runtime error: … variable “fn” not found for writing; maybe it is a built-in variable that is not changeable or … ɾ error.log
  97. ngx.shared.DICT • dictionary with shared memory http { lua_shared_dict yapc

    1m; init_by_lua ‘ ngx.shared.yapc:set(“presentator”, “@cubicdaiya”) ’; … server { … location / { content_by_lua ‘ ngx.say(ngx.shared.yapc:get(“presentator”)) ’; } } }
  98. ngx.header.HEADER • response header reference and manipulation location / {

    content_by_lua ‘ local body = “{ \\”presentator\\” : \\”@cubicdaiya\\” }” —- assign application/json to Content-Type ngx.header.content_type = “application/json” ngx.say(body) ‘; }
  99. Header reference and manipulation • ngx.req.get_headers() • ngx.req.set_header() • ngx.req.clear_header()

    • … • ngx.resp.get_headers() • ngx.header.HEADER • … ▪ For request-header ▪ For response-header
  100. ngx.time() • return current timestamp local timestamp = ngx.time()

  101. ngx_lua time APIs • ngx.time() • ngx.today() • ngx.now() •

    ngx.update_time() • ngx.parse_http_time() • ngx.localtime() • ngx.utctime() • ngx.cookie_time() • ngx.http_time()
  102. Use ngx.time() instead of os.time() • os.time() always calls gettimeofday()

    • ngx.time() uses timer-cache of nginx • avoid system-call’s overhead • ngx.time() is more efficient than os.time()
  103. ngx.re.match() • Regular Expression matching • powered by PCRE local

    matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”) -— matches[0] -> image/webp -— matches[1] -> image -— matches[2] -> webp
  104. ngx.re.* (Regular Expression API) • ngx.re.find() • ngx.re.match() • ngx.re.gmatch()

    • ngx.re.sub() • ngx.re.gsub()
  105. • Use PCRE options(e.g. o, j) • o -> enable

    compile-once • j -> enable PCRE-JIT Optimize ngx.re.* local matches, err = ngx.re.match(“image/webp”, “([^/]+)\/(.+)”, “oj”)
  106. • ngx.re is based PCRE • Powerful & Fat •

    Lua builtin-re(regular expression) • Simple & Light ngx.re.* VS Lua builtin-re If ‘re’ is simple, Lua builtin-re is faster.
  107. ngx.location.capture() • send non-blocking sub-request to other location location /external

    { internal; proxy_pass http://external; } location / { content_by_lua ‘ —— non-blocking behavior local res = ngx.location.capture(“/external”) … ‘; }
  108. By the way, • ngx_lua is very powerful • But

    setup and deployment is a little painful • Lua / LuaJIT instalation • Lua packages management
  109. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  110. Agenda • nginx http module development with C • nginx

    API programming • Test::Nginx • ngx_lua • OpenResty
  111. OpenResty http://openresty.org/

  112. OpenResty • Web application framework bundled • nginx and 3rd

    party modules (e.g. ngx_lua) • Lua, LuaJIT • Resty modules(lua-resty-xxx…) • All-in-one superset of nginx
  113. 3rd party modules • lua-nginx-module (ngx_lua) • echo-nginx-module • headers-more-nginx-module

    • etc…
  114. Resty modules • lua-resty-core • resty-cli • lua-resty-mysql / redis

    / memcached • etc…
  115. lua-resty-core • pure Lua partial-re-implementation of ngx_lua APIs • accelerate

    ngx_lua APIs with LuaJIT • because ngx_lua APIs with C is not JIT compiled http { # accelerate ngx_lua APIs init_by_lua ‘require resty.core’; … }
  116. lua-resty-cli • command-line utility for OpenResty • run OpenResty with

    CLI • all APIs are not always enabled $ resty -e ‘ngx.say(“Hello, World!”)’ Hello, World! $
  117. lua-resty-mysql / redis / memcached • client drivers for mysql

    / redis / memcached • non-blocking behavior • Powered by ngx.socket
  118. OpenResty on

  119. PascalʙMercari app-event analysis baseʙ 0QFO3FTUZ 0QFO3FTUZ 0QFO3FTUZ (PPHMF#JH2VFSZ Developer Data

    Sientist Analyze by SQL send events send events send events Powered by cookpad/puree-(ios|android) utilize events transfer utilize events utilize events transfer
  120. OpenResty’s role on Pascal • High performance event logger •

    receive events as JSON from devices • parse JSON • utilize events for BigQuery • log events into file every event-channel
  121. References • The Architecture of Open Source Applications (Volume 2):

    nginx • http://www.aosabook.org/en/nginx.html • Nginx Internals • http://www.slideshare.net/joshzhu/nginx-internals • Emiller’s Guide To Nginx Module Development • http://www.evanmiller.org/nginx-modules-guide.html
  122. References • openresty/lua-nginx-module • https://github.com/openresty/lua-nginx-module • Using ngx_lua / lua-nginx-module

    in pixiv • http://www.slideshare.net/harukayon/ngx-lua-public • ngx_mruby and ngx_lua • http://qiita.com/cubicdaiya/items/3913d3dddf6999984c5b
  123. Thanks!