Upgrade to Pro — share decks privately, control downloads, hide ads and more …

実践ngx_small_light入門

 実践ngx_small_light入門

at 画像変換Night

Tatsuhiko Kubo

February 18, 2015
Tweet

More Decks by Tatsuhiko Kubo

Other Decks in Technology

Transcript

  1. ࣮ફngx_small_lightೖ໳
    Tatsuhiko Kubo@cubicdaiya
    ը૾ม׵Night@2015/02/18

    View full-size slide

  2. ࣗݾ঺հ
    • ٱอୡ඙(Tatsuhiko Kubo)
    • bokko@cubicdaiya
    • Software Engineer in Infrastructure Engineering
    • Mercari, Inc. (ࡢ೥ͷࠓࠒ͸·ͩpixiv, Inc.Ͱͨ͠)
    • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…

    View full-size slide

  3. ngx_small_light

    View full-size slide

  4. ngx_small_light
    • nginxͰಈతͳαϜωΠϧੜ੒͕Ͱ͖Δ֦ுϞδϡʔϧ
    • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ
    • ImageMagick / Imlib2 / GD
    • JPEG / GIF / PNG / WebP
    • https://github.com/cubicdaiya/ngx_small_light

    View full-size slide

  5. ngx_small_lightͰͰ͖Δ͜ͱ
    • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹੒ / ճస
    • ϑΥʔϚοτม׵(e.g. PNG -> JPEG)
    • ϒϥʔ / (Ξϯ)γϟʔϓ౳ͷՃ޻ॲཧ
    • etc…

    View full-size slide

  6. Setup
    cd ngx_small_light
    # enable ImageMagick
    ./setup
    # enable ImageMagick and Imlib2
    ./setup —-with-imlib2
    # enable ImageMagick and GD
    ./setup —-with-gd
    # enable ImageMagick and Imlib2 and GD
    ./setup —-with-imlib2 —with—gd

    View full-size slide

  7. Install
    cd ${ngx_small_light_dir}
    ./setup
    cd ${nginx_dir}
    ./configure ¥
    —-add-moduole=${ngx_small_light_dir}
    make
    sudo make install

    View full-size slide

  8. Quick Start
    # server context in nginx.conf
    small_light on;
    location ~ small_light[^/]*/(.+)$ {
    set $file $1;
    rewrite ^ /$file;
    }
    ݩը૾ͷ63* JNHJNBHFKQH
    ม׵༻ͷ63* TNBMM@MJHIU EX EI
    JNHJNBHFKQH

    View full-size slide

  9. σΟϨΫςΟϒҰཡ
    σΟϨΫςΟϒ໊ ղઆ
    small_light small_lightͷon/off
    small_light_pattern_define αϜωΠϧੜ੒ύλʔϯΛఆٛ
    small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜ੒ύλʔϯΛఆٛ
    small_light_material_dir ߹੒ૉࡐͷσΟϨΫτϦΛࢦఆ
    small_light_imlib2_temp_dir imlib2͕ੜ੒͢ΔςϯϙϥϦϑΝΠϧͷύε
    small_light_buffer Content-Length͕ۭͷ৔߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ

    View full-size slide

  10. ngx_small_lightΛ༗ޮʹ͢Δ
    small_light on;
    serverίϯςΩετͰͷΈར༻ՄೳͰ͢

    View full-size slide

  11. αϜωΠϧੜ੒ύλʔϯΛఆٛ
    # in server context
    small_light_pattern_define medium dw=600,dh=400;
    small_light_pattern_define small dw=300,dh=200;
    /small_light(dw=300,dh=200)/img/image.jpg
    /small_light(p=small)/img/image.jpg

    ■ઃఆ
    ■URI

    View full-size slide

  12. جຊతͳύϥϝʔλ
    ύϥϝʔλ໊ ղઆ
    dw, dh ੜ੒͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200)
    of ੜ੒͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg / gif / png / webp)
    q ը࣭(e.g. 95)
    p small_light_pattern_defineͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛ୅ೖ
    e ը૾ม׵ϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)

    View full-size slide

  13. small_light_getparam_mode
    small_light_getparam_mode on;
    σϑΥϧτ͸offͰ͢
    /small_light(dw=300,dh=200)/img/image.jpg
    /img/image.jpg?dw=300&dh=200
    ■ઃఆ
    ■URI

    View full-size slide

  14. small_light_material_dir
    # not trailing slash
    small_light_material_dir /var/materials;
    # there is the icon.jpg in /var/materials.
    small_light_pattern_define embed
    dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15;
    /small_light(p=embed)/img/image.jpg
    + =
    ■URI
    ■ઃఆ

    View full-size slide

  15. small_light_imlib2_temp_dir
    # default setting
    small_light_imlib2_temp_dir /tmp 1 2 0;
    ■ઃఆ
    • Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε
    • Imlib2͸ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍
    • ઃఆͷॻࣜ͸proxy_temp_pathͱҰॹ

    View full-size slide

  16. small_light_buffer
    # default setting
    small_light_imlib2_temp_dir /tmp 1 2 0;
    ■ઃఆ
    • ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ
    • ௨ৗ͸ར༻͞Εͳ͍(Content-Length෼֬อ͢Δ)
    • ྫ͑͹ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ
    Content-Lengthؚ͕·Ε͍ͯͳ͍৔߹ʹར༻͢Δ
    # default setting
    small_light_buffer 1m;

    View full-size slide

  17. ֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ
    ngx_small_lightͰϦαΠζ
    # server context in nginx.conf
    small_light on;
    location ~ small_light[^/]*/(.+)$ {
    set $file $1;
    rewrite ^ /$file;
    }
    location ~ /img {
    proxy_pass http://external_host;
    }
    /small_light(dw=300,dh=200)/img/image.jpg
    ■ઃఆ
    ■URI

    View full-size slide

  18. υΩϡϝϯτ͸ͪ͜Β

    https://github.com/cubicdaiya/
    ngx_small_light/wiki/Configuration

    View full-size slide

  19. ը૾ϥΠϒϥϦͷൺֱ
    ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭
    ImageMagick ˕ ˚ ˕
    Imlib2 ˚ ˕ ʓ
    GD ʓ ʓ ʓ

    View full-size slide

  20. ͦͷଞͷը૾ม׵Ϟδϡʔϧ
    • mod_small_light
    • ngx_http_image_filter
    • go-thumber

    View full-size slide

  21. mod_small_light
    • ݩ૆SmalllightͷApacheϞδϡʔϧ
    • https://github.com/yamac/smalllight
    • ngx_small_light͸͜ͷϞδϡʔϧͷnginx޲͚
    ϙʔςΟϯάͱͯ͠ελʔτ
    • ͔ͭͯpixivͰ΋ຐվ଄ͯ͠ར༻͞Ε͍ͯ·ͨ͠

    View full-size slide

  22. Quick Start

    SetOutputFilter SMALL_LIGHT

    ݩը૾ͷ63* JNHJNBHFKQH
    ม׵༻ͷ63* TNBMM@MJHIU EX EI
    JNHJNBHFKQH

    View full-size slide

  23. ngx_small_lightͱͷҧ͍
    • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ
    • ngx_small_lightͰ͸of͕ͳ͍৔߹ɺ
    ೖྗ/ग़ྗϑΥʔϚοτ͸ಉ͡ʹͳΔ
    • ར༻Մೳͳػೳ΍ύϥϝʔλ͕एׯগͳΊ

    View full-size slide

  24. ngx_http_image_filter
    • ΋͏Ұͭͷnginxͷը૾ม׵Ϟδϡʔϧ
    • ͳΜͱඪ४ϞδϡʔϧͰ͢
    • nginx/src/http/modules/ngx_http_image_filter_module.c

    View full-size slide

  25. Quick Start
    location ~ \.(png|gif|jpe?g)$ {
    image_filter resize 150 150;
    }
    (mod|ngx)_small_lightͷΑ͏ͳ
    ม׵༻URIͷ֓೦͕ͳ͍

    View full-size slide

  26. ngx_small_lightͱͷҧ͍
    • ػೳগͳΊ
    • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only
    • image_filter_buffer͕ৗʹར༻͞ΕΔ
    • σϑΥϧτ஋͕1mͱখ͍͞ͷͰ஫ҙ
    • ࣮૷Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ

    View full-size slide

  27. go-thumber
    • pixivۘ੡ը૾ม׵ϓϩΩγαʔό
    • ੍໿ͱੑೳ
    • JPEG & Resize only
    • High Speed & High Quality
    • ಛఆͷ৚݅ԼͰઈେͳޮՌΛൃش͢Δ

    View full-size slide

  28. Image Convert
    Optimization

    View full-size slide

  29. worker_(processes|connections)

    View full-size slide

  30. worker_(proccesses|connections)
    • worker_processes͸ଟΊʹ
    • worker_connections͸গͳΊʹ
    • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ
    • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱ͸๨ΕΔ

    View full-size slide

  31. JPEG Hinting

    View full-size slide

  32. JPEG Hinting
    • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴ଎Խ͢Δ࢓૊Έ
    • ৄ͘͠͸ʮຊ౰͸଎͍ImageMagickʯͰάάΖ͏
    • ImageMagickͱImlib2Ͱར༻Մೳ
    • jpeghint=yΛࢦఆ͢Δ͚ͩ
    • ਺ഒʙ10ഒ଎͘ͳΔέʔε΋͋Δ

    View full-size slide

  33. JPEG Hinting
    with ngx_small_light
    small_light_pattern_define thumb
    dw=600,dh=400,jpeghint=y;
    JPEGҎ֎ͷը૾Λॲཧ͢Δࡍ͸ແࢹ͞Ε·͢

    View full-size slide

  34. libjpeg-turbo

    View full-size slide

  35. libjpeg-turbo
    • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg
    • SIMD໋ྩ(MMX, SSE2)Λར༻
    • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ
    • e.g. LD_LIBRARY_PATH

    View full-size slide

  36. libjpegͱlibjpeg-turbo

    View full-size slide

  37. OpenMP
    • ImageMagick͸σϑΥϧτͰOpenMP͕༗ޮ
    • Ϛϧνϓϩηε؀ڥͩͱܶతʹεϩʔμ΢ϯ͢Δ
    • OpenMPΛແޮʹ͢Δํ๏
    • —disable-openmp෇͖Ͱ࠶ίϯύΠϧ
    • OMP_NUM_THREADS=1
    • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ

    View full-size slide

  38. OMP_NUM_THREADS=1
    ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ

    View full-size slide

  39. OMP_NUM_THREADS=1
    ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ

    View full-size slide

  40. OMP_NUM_THREADS=1 with nginx
    env OMP_NUM_THREADS=1;
    nginx͸ىಈ࣌ʹTZΛআ͘͢΂ͯͷ؀ڥม਺Λ
    ࡟আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏

    View full-size slide

  41. WebP
    • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ
    • ωοτϫʔΫଳҬ࡟ݮͱදࣔ଎౓ͷ୹ॖ͕໨త
    • JPEG΍PNG౳ͱൺ΂͔ͯͳΓαΠζ͕খ͍͞
    • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ୺຤͕গͳ͍ͷ͕೉఺

    View full-size slide

  42. ngx_small_lightͷWebPαϙʔτ
    • ImageMagickɺGDͰར༻Մೳ
    • ImageMagickͳΒlibwebp
    • GDͳΒlibvpx͕ඞཁ
    • of=webpͬͯॻ͚ͩ͘

    View full-size slide

  43. ngx_small_light & WebP࠾༻ࣄྫ
    ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢

    View full-size slide

  44. ngx_small_lightͷϨεϙϯεΛ
    nginxͰΩϟογϡ͢Δ؆୯ͳྫ
    proxy_cache_path /tmp/cache
    levels=1:2 key_zone=images:5m inactive=1h max_size=10G;
    server {
    server_name origin_server;

    location ~ / {
    proxy_cache images;
    proxy_cache_valid 200 1h;
    proxy_cache_key “$scheme://$host$request_uri$is_args$args”;
    proxy_pass http://small_light_server;
    }
    }
    server {
    server_name small_light_server;

    small_light on;
    location ~ small_light[^/]*/(.+)$ {
    set $file $1;
    rewrite ^ /$file;
    }
    }

    View full-size slide

  45. ·ͱΊ
    • ngx_small_light͸ಈతͳαϜωΠϧੜ੒͕Ͱ
    ͖Δػೳ๛෋ͳnginxϞδϡʔϧ
    • ࠷ۙ͸࠾༻ࣄྫΛฉ͘͜ͱ΋ͪΒ΄Β
    • ը૾ม׵͸ॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά
    ΍ΩϟογϡΛߦ͍·͠ΐ͏

    View full-size slide