Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Practical nginx lua in KAYAC

Practical nginx lua in KAYAC

#nginxbook

Ca6281fff64797dc419b78f51f25c0a5?s=128

FUJIWARA Shunichiro

February 08, 2016
Tweet

More Decks by FUJIWARA Shunichiro

Other Decks in Technology

Transcript

  1. Practical nginx lua in KAYAC nginx Tech Talks @fujiwara

  2. @fujiwara github.com/fujiwara sfujiwara.hatenablog.com ٕज़෦

  3. Game & Community

  4. nginx(OpenResty) ࢖ͬͯ·͢

  5. ࣄྫ1: ૝ఆ͍ͯ͠ͳ͍vhostʹେྔΞΫηε

  6. ૝ఆ͍ͯ͠ͳ͍vhostʹେྔΞΫηε # ͳ͔ͥxxxͰ͸ͳ͘yyyʹϦΫΤετ&ϦτϥΠ͞ΕΔݱ৅͕֬ೝ͞Ε͍ͯΔͷͰ # ͻͱ·ͣϨεϙϯεΛ஗Ԇͤͯ͞ແବͳϦΫΤετΛݮΒ͢ location ~ ^/foo { default_type

    "application/json; charset=utf-8"; add_header "Cache-Control" "no-cache"; content_by_lua ' ngx.sleep(60) ngx.print("Not Found") ngx.exit(404) '; }
  7. ࣄྫ2: AngularJS αΠτ ͷ SEO

  8. ܦҢ - AngularJS αΠτ ͷ SEO AngularJS ͰϦϦʔε SEOରԠͱͯ͠Prerender.io Λಋೖ

    → Prerenderͷϓϩηε͕ࢮʹ·͘ΔͷͰఘΊΔ ݕࡧྲྀೖ͕ফ͑Δ Ϋϩʔϥʔઐ༻ͷHTMLΛग़ྗ͢Δʁ → ΫϩʔΩϯάѻ͍͞ΕΔڪΕ͕…
  9. ઃܭ - AngularJS αΠτͷ SEO PerlͷWebApp΁ϦΫΤετ Job queueʹjobੜ੒ job worker

    ͕ PhantomJS ͰϨϯμϦϯά → memcached΁ X-Accel-Redirect Λ෇͚ͯଈ nginx ʹϨεϙϯεΛฦ͢ X-Accel-Redirect: /wait_memcached X-Memcached-Key: $key
  10. None
  11. location /wait_memcached { internal; default_type "text/html; charset=utf-8"; if ( $upstream_http_x_memcached_key

    ) { set $memcached_key $upstream_http_x_memcached_key; } access_by_lua ' while true do ngx.sleep(2); local res = ngx.location.capture("/has_memcached", { share_all_vars = true } ); if res.status == 200 then return; end if 30 < ngx.now() - ngx.req.start_time() then ngx.header.Retry_After = 60; ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE); return; end end '; memcached_pass 127.0.0.1:11211; } location /has_memcached { internal; memcached_pass 127.0.0.1:11211; }
  12. ࣄྫ3: Fluentd ΁ϩάΛૹ৴͢Δ

  13. ࣄྫ3: Fluentd ΁ϩάΛૹ৴͢Δ Google Analytics Έ͍ͨͳ΋ͷΛ࡞Γ͍ͨ ϖʔδΛ։͍ͨΒJavaScriptͰϩάૹ৴ URL Ҿ਺Λ Fluentd

    ʹૹΕ͹͋ͱ͸ूܭΛؤுΔ (͚ͩ… Fluentd → S3 → Redshift
  14. Fluentd ΁ϩάΛૹ৴͢Δ Perl ͷ WebApp Ͱ ! ָ " Φʔόʔϔου͕ͪΐͬͱେ͖͍

    (ʙ10ms) Go Ͱઐ༻αʔόΛॻ͘ ! ଎͍ " daemon͕૿͑Δͱӡ༻ͱ؂ࢹͷख͕ؒ
  15. Fluentd ΁ϩάΛૹ৴͢Δ nginx lua Ͱ ! े෼଎͍ ! daemon͸૿͑ͳ͍ "

    Մ༻ੑʹएׯ೉(࢓༷ͰΧόʔ) → ࠾༻
  16. Fluentd ΁ϩάΛૹ৴͢Δ طଘϥΠϒϥϦ github.com/united-adstir/fluent-logger-lua → nginxͷΠϕϯτϧʔϓʹ৐Βͳ͍ ngx.socket.tcp ͰࣗલͰૹΔ github.com/fperrad/lua-MessagePack [tag,

    time, event] ܗࣜͷMessagePackΛTCPͰྲྀ͚ͩ͢
  17. msgpack = require("msgpack") local sock = ngx.socket.tcp() sock:settimeout(5000) -- URLҾ਺Λऔಘ

    local args, err = ngx.req.get_uri_args() if not args then return ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR) end
  18. -- 127.0.0.1:24224 ͷ fluentd ʹ઀ଓ local ok, err = sock:connect("127.0.0.1",

    24224) if not ok then return ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE) end -- forward ܗࣜͷϝοηʔδΛ૊Έཱͯͯ౤͛Δ local bytes, err = sock:send( msgpack.pack({"access", ngx.time(), args}) ) if err then return ngx.exit(ngx.HTTP_SERVICE_UNAVAILABLE) end sock:setkeepalive(10000) -- keepalive 10 sec. -- ϨεϙϯεΛฦ͢ ngx.header.content_type = "text/plain" ngx.say("ok")
  19. Benchmark $ wrk -c 50 -d 10 -t 4 "http://localhost:8000/?foo=bar&bar=baz"

    Running 10s test @ http://localhost:8000/?foo=bar&bar=baz 4 threads and 50 connections Thread Stats Avg Stdev Max +/- Stdev Latency 5.20ms 13.79ms 247.88ms 92.33% Req/Sec 7.97k 2.16k 14.03k 67.94% 270322 requests in 10.08s, 44.84MB read Requests/sec: 26809.95 Transfer/sec: 4.45MB Requests/sec: 26809.95 ! Macbook Pro 13 (Early 2015) fluentd-0.12.16 ruby 2.2.2p95 openresty/1.9.3.1 wrk 4.0.0