Practical nginx lua in KAYAC nginx Tech Talks @fujiwara

@fujiwara github.com/fujiwara sfujiwara.hatenablog.com ٕज़෦

Game & Community

nginx(OpenResty) ࢖ͬͯ·͢

ࣄྫ1: ૝ఆ͍ͯ͠ͳ͍vhostʹେྔΞΫηε

૝ఆ͍ͯ͠ͳ͍vhostʹେྔΞΫηε # ͳ͔ͥxxxͰ͸ͳ͘yyyʹϦΫΤετ&ϦτϥΠ͞ΕΔݱ৅͕֬ೝ͞Ε͍ͯΔͷͰ # ͻͱ·ͣϨεϙϯεΛ஗Ԇͤͯ͞ແବͳϦΫΤετΛݮΒ͢ location ~ ^/foo { default_type "application/json; charset=utf-8"; add_header "Cache-Control" "no-cache"; content_by_lua ' ngx.sleep(60) ngx.print("Not Found") ngx.exit(404) '; }

ࣄྫ2: AngularJS αΠτ ͷ SEO

ܦҢ - AngularJS αΠτ ͷ SEO AngularJS ͰϦϦʔε SEOରԠͱͯ͠Prerender.io Λಋೖ → Prerenderͷϓϩηε͕ࢮʹ·͘ΔͷͰఘΊΔ ݕࡧྲྀೖ͕ফ͑Δ Ϋϩʔϥʔઐ༻ͷHTMLΛग़ྗ͢Δʁ → ΫϩʔΩϯάѻ͍͞ΕΔڪΕ͕…

ઃܭ - AngularJS αΠτͷ SEO PerlͷWebApp΁ϦΫΤετ Job queueʹjobੜ੒ job worker ͕ PhantomJS ͰϨϯμϦϯά → memcached΁ X-Accel-Redirect Λ෇͚ͯଈ nginx ʹϨεϙϯεΛฦ͢ X-Accel-Redirect: /wait_memcached X-Memcached-Key: $key

No content

location /wait_memcached { internal; default_type "text/html; charset=utf-8"; if ( $upstream_http_x_memcached_key ) { set $memcached_key $upstream_http_x_memcached_key; } access_by_lua ' while true do ngx.sleep(2); local res = ngx.location.capture("/has_memcached", { share_all_vars = true } ); if res.status == 200 then return; end if 30 < ngx.now() - ngx.req.start_time() then ngx.header.Retry_After = 60; ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE); return; end end '; memcached_pass 127.0.0.1:11211; } location /has_memcached { internal; memcached_pass 127.0.0.1:11211; }

ࣄྫ3: Fluentd ΁ϩάΛૹ৴͢Δ

ࣄྫ3: Fluentd ΁ϩάΛૹ৴͢Δ Google Analytics Έ͍ͨͳ΋ͷΛ࡞Γ͍ͨ ϖʔδΛ։͍ͨΒJavaScriptͰϩάૹ৴ URL Ҿ਺Λ Fluentd ʹૹΕ͹͋ͱ͸ूܭΛؤுΔ (͚ͩ… Fluentd → S3 → Redshift

Fluentd ΁ϩάΛૹ৴͢Δ Perl ͷ WebApp Ͱ ! ָ " Φʔόʔϔου͕ͪΐͬͱେ͖͍ (ʙ10ms) Go Ͱઐ༻αʔόΛॻ͘ ! ଎͍ " daemon͕૿͑Δͱӡ༻ͱ؂ࢹͷख͕ؒ

Fluentd ΁ϩάΛૹ৴͢Δ nginx lua Ͱ ! े෼଎͍ ! daemon͸૿͑ͳ͍ " Մ༻ੑʹएׯ೉(࢓༷ͰΧόʔ) → ࠾༻

Fluentd ΁ϩάΛૹ৴͢Δ طଘϥΠϒϥϦ github.com/united-adstir/fluent-logger-lua → nginxͷΠϕϯτϧʔϓʹ৐Βͳ͍ ngx.socket.tcp ͰࣗલͰૹΔ github.com/fperrad/lua-MessagePack [tag, time, event] ܗࣜͷMessagePackΛTCPͰྲྀ͚ͩ͢

msgpack = require("msgpack") local sock = ngx.socket.tcp() sock:settimeout(5000) -- URLҾ਺Λऔಘ local args, err = ngx.req.get_uri_args() if not args then return ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) end

-- 127.0.0.1:24224 ͷ fluentd ʹ઀ଓ local ok, err = sock:connect("127.0.0.1", 24224) if not ok then return ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE) end -- forward ܗࣜͷϝοηʔδΛ૊Έཱͯͯ౤͛Δ local bytes, err = sock:send( msgpack.pack({"access", ngx.time(), args}) ) if err then return ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE) end sock:setkeepalive(10000) -- keepalive 10 sec. -- ϨεϙϯεΛฦ͢ ngx.header.content_type = "text/plain" ngx.say("ok")

Benchmark $ wrk -c 50 -d 10 -t 4 "http://localhost:8000/?foo=bar&bar=baz" Running 10s test @ http://localhost:8000/?foo=bar&bar=baz 4 threads and 50 connections Thread Stats Avg Stdev Max +/- Stdev Latency 5.20ms 13.79ms 247.88ms 92.33% Req/Sec 7.97k 2.16k 14.03k 67.94% 270322 requests in 10.08s, 44.84MB read Requests/sec: 26809.95 Transfer/sec: 4.45MB Requests/sec: 26809.95 ! Macbook Pro 13 (Early 2015) fluentd-0.12.16 ruby 2.2.2p95 openresty/1.9.3.1 wrk 4.0.0