実践ngx_small_light入門

Transcript

1. ࣮ફngx_small_lightೖ໳ Tatsuhiko Kubo@cubicdaiya ը૾ม׵Night@2015/02/18

2. ࣗݾ঺հ • ٱอୡ඙(Tatsuhiko Kubo) • bokko@cubicdaiya • Software Engineer in

   Infrastructure Engineering • Mercari, Inc. (ࡢ೥ͷࠓࠒ͸·ͩpixiv, Inc.Ͱͨ͠) • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…

3. ngx_small_light

4. ngx_small_light • nginxͰಈతͳαϜωΠϧੜ੒͕Ͱ͖Δ֦ுϞδϡʔϧ • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ • ImageMagick / Imlib2 /

   GD • JPEG / GIF / PNG / WebP • https://github.com/cubicdaiya/ngx_small_light

5. ngx_small_lightͰͰ͖Δ͜ͱ • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹੒ / ճస •

   ϑΥʔϚοτม׵(e.g. PNG -> JPEG) • ϒϥʔ / (Ξϯ)γϟʔϓ౳ͷՃ޻ॲཧ • etc…

6. Setup cd ngx_small_light # enable ImageMagick ./setup # enable ImageMagick

   and Imlib2 ./setup —-with-imlib2 # enable ImageMagick and GD ./setup —-with-gd # enable ImageMagick and Imlib2 and GD ./setup —-with-imlib2 —with—gd

7. Install cd ${ngx_small_light_dir} ./setup cd ${nginx_dir} ./configure ¥ —-add-moduole=${ngx_small_light_dir} make

   sudo make install

8. Quick Start # server context in nginx.conf small_light on; location

   ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } ݩը૾ͷ63* JNHJNBHFKQH ม׵༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH

9. σΟϨΫςΟϒҰཡ σΟϨΫςΟϒ໊ ղઆ small_light small_lightͷon/off small_light_pattern_define αϜωΠϧੜ੒ύλʔϯΛఆٛ small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜ੒ύλʔϯΛఆٛ small_light_material_dir

   ߹੒ૉࡐͷσΟϨΫτϦΛࢦఆ small_light_imlib2_temp_dir imlib2͕ੜ੒͢ΔςϯϙϥϦϑΝΠϧͷύε small_light_buffer Content-Length͕ۭͷ৔߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ

10. ngx_small_lightΛ༗ޮʹ͢Δ small_light on; serverίϯςΩετͰͷΈར༻ՄೳͰ͢

11. αϜωΠϧੜ੒ύλʔϯΛఆٛ # in server context small_light_pattern_define medium dw=600,dh=400; small_light_pattern_define small

    dw=300,dh=200; /small_light(dw=300,dh=200)/img/image.jpg /small_light(p=small)/img/image.jpg → ▪ઃఆ ▪URI

12. جຊతͳύϥϝʔλ ύϥϝʔλ໊ ղઆ dw, dh ੜ੒͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200) of ੜ੒͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg

    / gif / png / webp) q ը࣭(e.g. 95) p small_light_pattern_defineͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛ୅ೖ e ը૾ม׵ϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)

13. small_light_getparam_mode small_light_getparam_mode on; σϑΥϧτ͸offͰ͢ /small_light(dw=300,dh=200)/img/image.jpg /img/image.jpg?dw=300&dh=200 ▪ઃఆ ▪URI

14. small_light_material_dir # not trailing slash small_light_material_dir /var/materials; # there is

    the icon.jpg in /var/materials. small_light_pattern_define embed dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15; /small_light(p=embed)/img/image.jpg + = ▪URI ▪ઃఆ

15. small_light_imlib2_temp_dir # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ

    • Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε • Imlib2͸ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍ • ઃఆͷॻࣜ͸proxy_temp_pathͱҰॹ

16. small_light_buffer # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ

    • ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ • ௨ৗ͸ར༻͞Εͳ͍(Content-Length෼֬อ͢Δ) • ྫ͑͹ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ Content-Lengthؚ͕·Ε͍ͯͳ͍৔߹ʹར༻͢Δ # default setting small_light_buffer 1m;

17. ֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ ngx_small_lightͰϦαΠζ # server context in nginx.conf small_light on;

    location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location ~ /img { proxy_pass http://external_host; } /small_light(dw=300,dh=200)/img/image.jpg ▪ઃఆ ▪URI

18. υΩϡϝϯτ͸ͪ͜Β ↓ https://github.com/cubicdaiya/ ngx_small_light/wiki/Configuration

19. ը૾ϥΠϒϥϦͷൺֱ ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭ ImageMagick ˕ ˚ ˕ Imlib2

    ˚ ˕ ʓ GD ʓ ʓ ʓ

20. ͦͷଞͷը૾ม׵Ϟδϡʔϧ • mod_small_light • ngx_http_image_filter • go-thumber

21. mod_small_light • ݩ૆SmalllightͷApacheϞδϡʔϧ • https://github.com/yamac/smalllight • ngx_small_light͸͜ͷϞδϡʔϧͷnginx޲͚ ϙʔςΟϯάͱͯ͠ελʔτ • ͔ͭͯpixivͰ΋ຐվ଄ͯ͠ར༻͞Ε͍ͯ·ͨ͠

22. Quick Start <LocationMatch ^/small_light[^/]*/> SetOutputFilter SMALL_LIGHT </LocationMatch> ݩը૾ͷ63* JNHJNBHFKQH ม׵༻ͷ63*

    TNBMM@MJHIU EX EI JNHJNBHFKQH

23. ngx_small_lightͱͷҧ͍ • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ • ngx_small_lightͰ͸of͕ͳ͍৔߹ɺ ೖྗ/ग़ྗϑΥʔϚοτ͸ಉ͡ʹͳΔ • ར༻Մೳͳػೳ΍ύϥϝʔλ͕एׯগͳΊ

24. ngx_http_image_filter • ΋͏Ұͭͷnginxͷը૾ม׵Ϟδϡʔϧ • ͳΜͱඪ४ϞδϡʔϧͰ͢ • nginx/src/http/modules/ngx_http_image_filter_module.c

25. Quick Start location ~ \.(png|gif|jpe?g)$ { image_filter resize 150 150;

    } (mod|ngx)_small_lightͷΑ͏ͳ ม׵༻URIͷ֓೦͕ͳ͍

26. ngx_small_lightͱͷҧ͍ • ػೳগͳΊ • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only • image_filter_buffer͕ৗʹར༻͞ΕΔ • σϑΥϧτ஋͕1mͱখ͍͞ͷͰ஫ҙ

    • ࣮૷Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ

27. go-thumber • pixivۘ੡ը૾ม׵ϓϩΩγαʔό • ੍໿ͱੑೳ • JPEG & Resize only

    • High Speed & High Quality • ಛఆͷ৚݅ԼͰઈେͳޮՌΛൃش͢Δ

28. Image Convert Optimization

29. worker_(processes|connections)

30. worker_(proccesses|connections) • worker_processes͸ଟΊʹ • worker_connections͸গͳΊʹ • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱ͸๨ΕΔ

31. JPEG Hinting

32. JPEG Hinting • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴ଎Խ͢Δ࢓૊Έ • ৄ͘͠͸ʮຊ౰͸଎͍ImageMagickʯͰάάΖ͏ • ImageMagickͱImlib2Ͱར༻Մೳ • jpeghint=yΛࢦఆ͢Δ͚ͩ

    • ਺ഒʙ10ഒ଎͘ͳΔέʔε΋͋Δ

33. JPEG Hinting with ngx_small_light small_light_pattern_define thumb dw=600,dh=400,jpeghint=y; JPEGҎ֎ͷը૾Λॲཧ͢Δࡍ͸ແࢹ͞Ε·͢

34. libjpeg-turbo

35. libjpeg-turbo • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg • SIMD໋ྩ(MMX, SSE2)Λར༻ • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ • e.g.

    LD_LIBRARY_PATH

36. libjpegͱlibjpeg-turbo

37. OpenMP

38. OpenMP • ImageMagick͸σϑΥϧτͰOpenMP͕༗ޮ • Ϛϧνϓϩηε؀ڥͩͱܶతʹεϩʔμ΢ϯ͢Δ • OpenMPΛແޮʹ͢Δํ๏ • —disable-openmp෇͖Ͱ࠶ίϯύΠϧ •

    OMP_NUM_THREADS=1 • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ

39. OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ

40. OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ

41. OMP_NUM_THREADS=1 with nginx env OMP_NUM_THREADS=1; nginx͸ىಈ࣌ʹTZΛআ͘͢΂ͯͷ؀ڥม਺Λ ࡟আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏

42. WebP

43. WebP • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ • ωοτϫʔΫଳҬ࡟ݮͱදࣔ଎౓ͷ୹ॖ͕໨త • JPEG΍PNG౳ͱൺ΂͔ͯͳΓαΠζ͕খ͍͞ • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ୺຤͕গͳ͍ͷ͕೉఺

44. ngx_small_lightͷWebPαϙʔτ • ImageMagickɺGDͰར༻Մೳ • ImageMagickͳΒlibwebp • GDͳΒlibvpx͕ඞཁ • of=webpͬͯॻ͚ͩ͘

45. ngx_small_light & WebP࠾༻ࣄྫ ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢

46. Cache

47. ngx_small_lightͷϨεϙϯεΛ nginxͰΩϟογϡ͢Δ؆୯ͳྫ proxy_cache_path /tmp/cache levels=1:2 key_zone=images:5m inactive=1h max_size=10G; server {

    server_name origin_server; … location ~ / { proxy_cache images; proxy_cache_valid 200 1h; proxy_cache_key “$scheme://$host$request_uri$is_args$args”; proxy_pass http://small_light_server; } } server { server_name small_light_server; … small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } }

48. ·ͱΊ • ngx_small_light͸ಈతͳαϜωΠϧੜ੒͕Ͱ ͖Δػೳ๛෋ͳnginxϞδϡʔϧ • ࠷ۙ͸࠾༻ࣄྫΛฉ͘͜ͱ΋ͪ´ • ը૾ม׵͸ॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά ΍ΩϟογϡΛߦ͍·͠ΐ͏