実践ngx_small_light入門

at 画像変換Night

Transcript

1. ࣮ફngx_small_lightೖ໳
   Tatsuhiko Kubo@cubicdaiya
   ը૾ม׵Night@2015/02/18
   

   View Slide

2. ࣗݾ঺հ
   • ٱอୡ඙(Tatsuhiko Kubo)
   • bokko@cubicdaiya
   • Software Engineer in Infrastructure Engineering
   • Mercari, Inc. (ࡢ೥ͷࠓࠒ͸·ͩpixiv, Inc.Ͱͨ͠)
   • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…
   

   View Slide

3. ngx_small_light
   

   View Slide

4. ngx_small_light
   • nginxͰಈతͳαϜωΠϧੜ੒͕Ͱ͖Δ֦ுϞδϡʔϧ
   • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ
   • ImageMagick / Imlib2 / GD
   • JPEG / GIF / PNG / WebP
   • https://github.com/cubicdaiya/ngx_small_light
   

   View Slide

5. ngx_small_lightͰͰ͖Δ͜ͱ
   • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹੒ / ճస
   • ϑΥʔϚοτม׵(e.g. PNG -> JPEG)
   • ϒϥʔ / (Ξϯ)γϟʔϓ౳ͷՃ޻ॲཧ
   • etc…
   

   View Slide

6. Setup
   cd ngx_small_light
   # enable ImageMagick
   ./setup
   # enable ImageMagick and Imlib2
   ./setup —-with-imlib2
   # enable ImageMagick and GD
   ./setup —-with-gd
   # enable ImageMagick and Imlib2 and GD
   ./setup —-with-imlib2 —with—gd
   

   View Slide

7. Install
   cd ${ngx_small_light_dir}
   ./setup
   cd ${nginx_dir}
   ./configure ¥
   —-add-moduole=${ngx_small_light_dir}
   make
   sudo make install
   

   View Slide

8. Quick Start
   # server context in nginx.conf
   small_light on;
   location ~ small_light[^/]*/(.+)$ {
   set $file $1;
   rewrite ^ /$file;
   }
   ݩը૾ͷ63* JNHJNBHFKQH
   ม׵༻ͷ63* TNBMM@MJHIU EX EI
   JNHJNBHFKQH
   

   View Slide

9. σΟϨΫςΟϒҰཡ
   σΟϨΫςΟϒ໊ ղઆ
   small_light small_lightͷon/off
   small_light_pattern_define αϜωΠϧੜ੒ύλʔϯΛఆٛ
   small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜ੒ύλʔϯΛఆٛ
   small_light_material_dir ߹੒ૉࡐͷσΟϨΫτϦΛࢦఆ
   small_light_imlib2_temp_dir imlib2͕ੜ੒͢ΔςϯϙϥϦϑΝΠϧͷύε
   small_light_buffer Content-Length͕ۭͷ৔߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ
   

   View Slide

10. ngx_small_lightΛ༗ޮʹ͢Δ
    small_light on;
    serverίϯςΩετͰͷΈར༻ՄೳͰ͢
    

    View Slide

11. αϜωΠϧੜ੒ύλʔϯΛఆٛ
    # in server context
    small_light_pattern_define medium dw=600,dh=400;
    small_light_pattern_define small dw=300,dh=200;
    /small_light(dw=300,dh=200)/img/image.jpg
    /small_light(p=small)/img/image.jpg
    →
    ■ઃఆ
    ■URI
    

    View Slide

12. جຊతͳύϥϝʔλ
    ύϥϝʔλ໊ ղઆ
    dw, dh ੜ੒͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200)
    of ੜ੒͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg / gif / png / webp)
    q ը࣭(e.g. 95)
    p small_light_pattern_defineͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛ୅ೖ
    e ը૾ม׵ϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)
    

    View Slide

13. small_light_getparam_mode
    small_light_getparam_mode on;
    σϑΥϧτ͸offͰ͢
    /small_light(dw=300,dh=200)/img/image.jpg
    /img/image.jpg?dw=300&dh=200
    ■ઃఆ
    ■URI
    

    View Slide

14. small_light_material_dir
    # not trailing slash
    small_light_material_dir /var/materials;
    # there is the icon.jpg in /var/materials.
    small_light_pattern_define embed
    dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15;
    /small_light(p=embed)/img/image.jpg
    + =
    ■URI
    ■ઃఆ
    

    View Slide

15. small_light_imlib2_temp_dir
    # default setting
    small_light_imlib2_temp_dir /tmp 1 2 0;
    ■ઃఆ
    • Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε
    • Imlib2͸ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍
    • ઃఆͷॻࣜ͸proxy_temp_pathͱҰॹ
    

    View Slide

16. small_light_buffer
    # default setting
    small_light_imlib2_temp_dir /tmp 1 2 0;
    ■ઃఆ
    • ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ
    • ௨ৗ͸ར༻͞Εͳ͍(Content-Length෼֬อ͢Δ)
    • ྫ͑͹ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ
    Content-Lengthؚ͕·Ε͍ͯͳ͍৔߹ʹར༻͢Δ
    # default setting
    small_light_buffer 1m;
    

    View Slide

17. ֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ
    ngx_small_lightͰϦαΠζ
    # server context in nginx.conf
    small_light on;
    location ~ small_light[^/]*/(.+)$ {
    set $file $1;
    rewrite ^ /$file;
    }
    location ~ /img {
    proxy_pass http://external_host;
    }
    /small_light(dw=300,dh=200)/img/image.jpg
    ■ઃఆ
    ■URI
    

    View Slide

18. υΩϡϝϯτ͸ͪ͜Β
    ↓
    https://github.com/cubicdaiya/
    ngx_small_light/wiki/Configuration
    

    View Slide

19. ը૾ϥΠϒϥϦͷൺֱ
    ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭
    ImageMagick ˕ ˚ ˕
    Imlib2 ˚ ˕ ʓ
    GD ʓ ʓ ʓ
    

    View Slide

20. ͦͷଞͷը૾ม׵Ϟδϡʔϧ
    • mod_small_light
    • ngx_http_image_filter
    • go-thumber
    

    View Slide

21. mod_small_light
    • ݩ૆SmalllightͷApacheϞδϡʔϧ
    • https://github.com/yamac/smalllight
    • ngx_small_light͸͜ͷϞδϡʔϧͷnginx޲͚
    ϙʔςΟϯάͱͯ͠ελʔτ
    • ͔ͭͯpixivͰ΋ຐվ଄ͯ͠ར༻͞Ε͍ͯ·ͨ͠
    

    View Slide

22. Quick Start
    
    SetOutputFilter SMALL_LIGHT
    
    ݩը૾ͷ63* JNHJNBHFKQH
    ม׵༻ͷ63* TNBMM@MJHIU EX EI
    JNHJNBHFKQH
    

    View Slide

23. ngx_small_lightͱͷҧ͍
    • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ
    • ngx_small_lightͰ͸of͕ͳ͍৔߹ɺ
    ೖྗ/ग़ྗϑΥʔϚοτ͸ಉ͡ʹͳΔ
    • ར༻Մೳͳػೳ΍ύϥϝʔλ͕एׯগͳΊ
    

    View Slide

24. ngx_http_image_filter
    • ΋͏Ұͭͷnginxͷը૾ม׵Ϟδϡʔϧ
    • ͳΜͱඪ४ϞδϡʔϧͰ͢
    • nginx/src/http/modules/ngx_http_image_filter_module.c
    

    View Slide

25. Quick Start
    location ~ \.(png|gif|jpe?g)$ {
    image_filter resize 150 150;
    }
    (mod|ngx)_small_lightͷΑ͏ͳ
    ม׵༻URIͷ֓೦͕ͳ͍
    

    View Slide

26. ngx_small_lightͱͷҧ͍
    • ػೳগͳΊ
    • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only
    • image_filter_buffer͕ৗʹར༻͞ΕΔ
    • σϑΥϧτ஋͕1mͱখ͍͞ͷͰ஫ҙ
    • ࣮૷Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ
    

    View Slide

27. go-thumber
    • pixivۘ੡ը૾ม׵ϓϩΩγαʔό
    • ੍໿ͱੑೳ
    • JPEG & Resize only
    • High Speed & High Quality
    • ಛఆͷ৚݅ԼͰઈେͳޮՌΛൃش͢Δ
    

    View Slide

28. Image Convert
    Optimization
    

    View Slide

29. worker_(processes|connections)
    

    View Slide

30. worker_(proccesses|connections)
    • worker_processes͸ଟΊʹ
    • worker_connections͸গͳΊʹ
    • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ
    • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱ͸๨ΕΔ
    

    View Slide

31. JPEG Hinting
    

    View Slide

32. JPEG Hinting
    • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴ଎Խ͢Δ࢓૊Έ
    • ৄ͘͠͸ʮຊ౰͸଎͍ImageMagickʯͰάάΖ͏
    • ImageMagickͱImlib2Ͱར༻Մೳ
    • jpeghint=yΛࢦఆ͢Δ͚ͩ
    • ਺ഒʙ10ഒ଎͘ͳΔέʔε΋͋Δ
    

    View Slide

33. JPEG Hinting
    with ngx_small_light
    small_light_pattern_define thumb
    dw=600,dh=400,jpeghint=y;
    JPEGҎ֎ͷը૾Λॲཧ͢Δࡍ͸ແࢹ͞Ε·͢
    

    View Slide

34. libjpeg-turbo
    

    View Slide

35. libjpeg-turbo
    • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg
    • SIMD໋ྩ(MMX, SSE2)Λར༻
    • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ
    • e.g. LD_LIBRARY_PATH
    

    View Slide

36. libjpegͱlibjpeg-turbo
    

    View Slide

37. OpenMP
    

    View Slide

38. OpenMP
    • ImageMagick͸σϑΥϧτͰOpenMP͕༗ޮ
    • Ϛϧνϓϩηε؀ڥͩͱܶతʹεϩʔμ΢ϯ͢Δ
    • OpenMPΛແޮʹ͢Δํ๏
    • —disable-openmp෇͖Ͱ࠶ίϯύΠϧ
    • OMP_NUM_THREADS=1
    • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ
    

    View Slide

39. OMP_NUM_THREADS=1
    ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ
    

    View Slide

40. OMP_NUM_THREADS=1
    ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ
    

    View Slide

41. OMP_NUM_THREADS=1 with nginx
    env OMP_NUM_THREADS=1;
    nginx͸ىಈ࣌ʹTZΛআ͘͢΂ͯͷ؀ڥม਺Λ
    ࡟আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏
    

    View Slide

42. WebP
    

    View Slide

43. WebP
    • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ
    • ωοτϫʔΫଳҬ࡟ݮͱදࣔ଎౓ͷ୹ॖ͕໨త
    • JPEG΍PNG౳ͱൺ΂͔ͯͳΓαΠζ͕খ͍͞
    • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ୺຤͕গͳ͍ͷ͕೉఺
    

    View Slide

44. ngx_small_lightͷWebPαϙʔτ
    • ImageMagickɺGDͰར༻Մೳ
    • ImageMagickͳΒlibwebp
    • GDͳΒlibvpx͕ඞཁ
    • of=webpͬͯॻ͚ͩ͘
    

    View Slide

45. ngx_small_light & WebP࠾༻ࣄྫ
    ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢
    

    View Slide

46. Cache
    

    View Slide

47. ngx_small_lightͷϨεϙϯεΛ
    nginxͰΩϟογϡ͢Δ؆୯ͳྫ
    proxy_cache_path /tmp/cache
    levels=1:2 key_zone=images:5m inactive=1h max_size=10G;
    server {
    server_name origin_server;
    …
    location ~ / {
    proxy_cache images;
    proxy_cache_valid 200 1h;
    proxy_cache_key “$scheme://$host$request_uri$is_args$args”;
    proxy_pass http://small_light_server;
    }
    }
    server {
    server_name small_light_server;
    …
    small_light on;
    location ~ small_light[^/]*/(.+)$ {
    set $file $1;
    rewrite ^ /$file;
    }
    }
    

    View Slide

48. ·ͱΊ
    • ngx_small_light͸ಈతͳαϜωΠϧੜ੒͕Ͱ
    ͖Δػೳ๛෋ͳnginxϞδϡʔϧ
    • ࠷ۙ͸࠾༻ࣄྫΛฉ͘͜ͱ΋ͪΒ΄Β
    • ը૾ม׵͸ॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά
    ΍ΩϟογϡΛߦ͍·͠ΐ͏
    

    View Slide