実践ngx_small_light入門

at 画像変換Night

Transcript

1. ࣮ફngx_small_lightೖ໳ Tatsuhiko Kubo@cubicdaiya ը૾ม׵Night@2015/02/18
2. ࣗݾ঺հ • ٱอୡ඙(Tatsuhiko Kubo) • bokko@cubicdaiya • Software Engineer in Infrastructure Engineering • Mercari, Inc. (ࡢ೥ͷࠓࠒ͸·ͩpixiv, Inc.Ͱͨ͠) • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…
3. ngx_small_light
4. ngx_small_light • nginxͰಈతͳαϜωΠϧੜ੒͕Ͱ͖Δ֦ுϞδϡʔϧ • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ • ImageMagick / Imlib2 / GD • JPEG / GIF / PNG / WebP • https://github.com/cubicdaiya/ngx_small_light
5. ngx_small_lightͰͰ͖Δ͜ͱ • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹੒ / ճస • ϑΥʔϚοτม׵(e.g. PNG -> JPEG) • ϒϥʔ / (Ξϯ)γϟʔϓ౳ͷՃ޻ॲཧ • etc…
6. Setup cd ngx_small_light # enable ImageMagick ./setup # enable ImageMagick and Imlib2 ./setup —-with-imlib2 # enable ImageMagick and GD ./setup —-with-gd # enable ImageMagick and Imlib2 and GD ./setup —-with-imlib2 —with—gd
7. Install cd ${ngx_small_light_dir} ./setup cd ${nginx_dir} ./configure ¥ —-add-moduole=${ngx_small_light_dir} make sudo make install
8. Quick Start # server context in nginx.conf small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } ݩը૾ͷ63* JNHJNBHFKQH ม׵༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH
9. σΟϨΫςΟϒҰཡ σΟϨΫςΟϒ໊ ղઆ small_light small_lightͷon/off small_light_pattern_define αϜωΠϧੜ੒ύλʔϯΛఆٛ small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜ੒ύλʔϯΛఆٛ small_light_material_dir ߹੒ૉࡐͷσΟϨΫτϦΛࢦఆ small_light_imlib2_temp_dir imlib2͕ੜ੒͢ΔςϯϙϥϦϑΝΠϧͷύε small_light_buffer Content-Length͕ۭͷ৔߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ
10. ngx_small_lightΛ༗ޮʹ͢Δ small_light on; serverίϯςΩετͰͷΈར༻ՄೳͰ͢
11. αϜωΠϧੜ੒ύλʔϯΛఆٛ # in server context small_light_pattern_define medium dw=600,dh=400; small_light_pattern_define small dw=300,dh=200; /small_light(dw=300,dh=200)/img/image.jpg /small_light(p=small)/img/image.jpg → ▪ઃఆ ▪URI
12. جຊతͳύϥϝʔλ ύϥϝʔλ໊ ղઆ dw, dh ੜ੒͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200) of ੜ੒͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg / gif / png / webp) q ը࣭(e.g. 95) p small_light_pattern_defineͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛ୅ೖ e ը૾ม׵ϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)
13. small_light_getparam_mode small_light_getparam_mode on; σϑΥϧτ͸offͰ͢ /small_light(dw=300,dh=200)/img/image.jpg /img/image.jpg?dw=300&dh=200 ▪ઃఆ ▪URI
14. small_light_material_dir # not trailing slash small_light_material_dir /var/materials; # there is the icon.jpg in /var/materials. small_light_pattern_define embed dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15; /small_light(p=embed)/img/image.jpg + = ▪URI ▪ઃఆ
15. small_light_imlib2_temp_dir # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ • Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε • Imlib2͸ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍ • ઃఆͷॻࣜ͸proxy_temp_pathͱҰॹ
16. small_light_buffer # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ • ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ • ௨ৗ͸ར༻͞Εͳ͍(Content-Length෼֬อ͢Δ) • ྫ͑͹ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ Content-Lengthؚ͕·Ε͍ͯͳ͍৔߹ʹར༻͢Δ # default setting small_light_buffer 1m;
17. ֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ ngx_small_lightͰϦαΠζ # server context in nginx.conf small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location ~ /img { proxy_pass http://external_host; } /small_light(dw=300,dh=200)/img/image.jpg ▪ઃఆ ▪URI
18. υΩϡϝϯτ͸ͪ͜Β ↓ https://github.com/cubicdaiya/ ngx_small_light/wiki/Configuration
19. ը૾ϥΠϒϥϦͷൺֱ ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭ ImageMagick ˕ ˚ ˕ Imlib2 ˚ ˕ ʓ GD ʓ ʓ ʓ
20. ͦͷଞͷը૾ม׵Ϟδϡʔϧ • mod_small_light • ngx_http_image_filter • go-thumber
21. mod_small_light • ݩ૆SmalllightͷApacheϞδϡʔϧ • https://github.com/yamac/smalllight • ngx_small_light͸͜ͷϞδϡʔϧͷnginx޲͚ ϙʔςΟϯάͱͯ͠ελʔτ • ͔ͭͯpixivͰ΋ຐվ଄ͯ͠ར༻͞Ε͍ͯ·ͨ͠
22. Quick Start <LocationMatch ^/small_light[^/]*/> SetOutputFilter SMALL_LIGHT </LocationMatch> ݩը૾ͷ63* JNHJNBHFKQH ม׵༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH
23. ngx_small_lightͱͷҧ͍ • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ • ngx_small_lightͰ͸of͕ͳ͍৔߹ɺ ೖྗ/ग़ྗϑΥʔϚοτ͸ಉ͡ʹͳΔ • ར༻Մೳͳػೳ΍ύϥϝʔλ͕एׯগͳΊ
24. ngx_http_image_filter • ΋͏Ұͭͷnginxͷը૾ม׵Ϟδϡʔϧ • ͳΜͱඪ४ϞδϡʔϧͰ͢ • nginx/src/http/modules/ngx_http_image_filter_module.c
25. Quick Start location ~ \.(png|gif|jpe?g)$ { image_filter resize 150 150; } (mod|ngx)_small_lightͷΑ͏ͳ ม׵༻URIͷ֓೦͕ͳ͍
26. ngx_small_lightͱͷҧ͍ • ػೳগͳΊ • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only • image_filter_buffer͕ৗʹར༻͞ΕΔ • σϑΥϧτ஋͕1mͱখ͍͞ͷͰ஫ҙ • ࣮૷Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ
27. go-thumber • pixivۘ੡ը૾ม׵ϓϩΩγαʔό • ੍໿ͱੑೳ • JPEG & Resize only • High Speed & High Quality • ಛఆͷ৚݅ԼͰઈେͳޮՌΛൃش͢Δ
28. Image Convert Optimization
29. worker_(processes|connections)
30. worker_(proccesses|connections) • worker_processes͸ଟΊʹ • worker_connections͸গͳΊʹ • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱ͸๨ΕΔ
31. JPEG Hinting
32. JPEG Hinting • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴ଎Խ͢Δ࢓૊Έ • ৄ͘͠͸ʮຊ౰͸଎͍ImageMagickʯͰάάΖ͏ • ImageMagickͱImlib2Ͱར༻Մೳ • jpeghint=yΛࢦఆ͢Δ͚ͩ • ਺ഒʙ10ഒ଎͘ͳΔέʔε΋͋Δ
33. JPEG Hinting with ngx_small_light small_light_pattern_define thumb dw=600,dh=400,jpeghint=y; JPEGҎ֎ͷը૾Λॲཧ͢Δࡍ͸ແࢹ͞Ε·͢
34. libjpeg-turbo
35. libjpeg-turbo • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg • SIMD໋ྩ(MMX, SSE2)Λར༻ • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ • e.g. LD_LIBRARY_PATH
36. libjpegͱlibjpeg-turbo
37. OpenMP
38. OpenMP • ImageMagick͸σϑΥϧτͰOpenMP͕༗ޮ • Ϛϧνϓϩηε؀ڥͩͱܶతʹεϩʔμ΢ϯ͢Δ • OpenMPΛແޮʹ͢Δํ๏ • —disable-openmp෇͖Ͱ࠶ίϯύΠϧ • OMP_NUM_THREADS=1 • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ
39. OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ
40. OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ
41. OMP_NUM_THREADS=1 with nginx env OMP_NUM_THREADS=1; nginx͸ىಈ࣌ʹTZΛআ͘͢΂ͯͷ؀ڥม਺Λ ࡟আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏
42. WebP
43. WebP • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ • ωοτϫʔΫଳҬ࡟ݮͱදࣔ଎౓ͷ୹ॖ͕໨త • JPEG΍PNG౳ͱൺ΂͔ͯͳΓαΠζ͕খ͍͞ • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ୺຤͕গͳ͍ͷ͕೉఺
44. ngx_small_lightͷWebPαϙʔτ • ImageMagickɺGDͰར༻Մೳ • ImageMagickͳΒlibwebp • GDͳΒlibvpx͕ඞཁ • of=webpͬͯॻ͚ͩ͘
45. ngx_small_light & WebP࠾༻ࣄྫ ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢
46. Cache
47. ngx_small_lightͷϨεϙϯεΛ nginxͰΩϟογϡ͢Δ؆୯ͳྫ proxy_cache_path /tmp/cache levels=1:2 key_zone=images:5m inactive=1h max_size=10G; server { server_name origin_server; … location ~ / { proxy_cache images; proxy_cache_valid 200 1h; proxy_cache_key “$scheme://$host$request_uri$is_args$args”; proxy_pass http://small_light_server; } } server { server_name small_light_server; … small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } }
48. ·ͱΊ • ngx_small_light͸ಈతͳαϜωΠϧੜ੒͕Ͱ ͖Δػೳ๛෋ͳnginxϞδϡʔϧ • ࠷ۙ͸࠾༻ࣄྫΛฉ͘͜ͱ΋ͪ´ • ը૾ม׵͸ॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά ΍ΩϟογϡΛߦ͍·͠ΐ͏