実践ngx_small_light入門

Slide 1

Slide 1 text

࣮ફngx_small_lightೖ໳ Tatsuhiko Kubo@cubicdaiya ը૾ม׵Night@2015/02/18

Slide 2

Slide 2 text

ࣗݾ঺հ • ٱอୡ඙(Tatsuhiko Kubo) • bokko@cubicdaiya • Software Engineer in Infrastructure Engineering • Mercari, Inc. (ࡢ೥ͷࠓࠒ͸·ͩpixiv, Inc.Ͱͨ͠) • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…

Slide 3

Slide 3 text

ngx_small_light

Slide 4

Slide 4 text

ngx_small_light • nginxͰಈతͳαϜωΠϧੜ੒͕Ͱ͖Δ֦ுϞδϡʔϧ • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ • ImageMagick / Imlib2 / GD • JPEG / GIF / PNG / WebP • https://github.com/cubicdaiya/ngx_small_light

Slide 5

Slide 5 text

ngx_small_lightͰͰ͖Δ͜ͱ • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹੒ / ճస • ϑΥʔϚοτม׵(e.g. PNG -> JPEG) • ϒϥʔ / (Ξϯ)γϟʔϓ౳ͷՃ޻ॲཧ • etc…

Slide 6

Slide 6 text

Setup cd ngx_small_light # enable ImageMagick ./setup # enable ImageMagick and Imlib2 ./setup —-with-imlib2 # enable ImageMagick and GD ./setup —-with-gd # enable ImageMagick and Imlib2 and GD ./setup —-with-imlib2 —with—gd

Slide 7

Slide 7 text

Install cd ${ngx_small_light_dir} ./setup cd ${nginx_dir} ./configure ¥ —-add-moduole=${ngx_small_light_dir} make sudo make install

Slide 8

Slide 8 text

Quick Start # server context in nginx.conf small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } ݩը૾ͷ63* JNHJNBHFKQH ม׵༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH

Slide 9

Slide 9 text

σΟϨΫςΟϒҰཡ σΟϨΫςΟϒ໊ ղઆ small_light small_lightͷon/off small_light_pattern_deﬁne αϜωΠϧੜ੒ύλʔϯΛఆٛ small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜ੒ύλʔϯΛఆٛ small_light_material_dir ߹੒ૉࡐͷσΟϨΫτϦΛࢦఆ small_light_imlib2_temp_dir imlib2͕ੜ੒͢ΔςϯϙϥϦϑΝΠϧͷύε small_light_buffer Content-Length͕ۭͷ৔߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ

Slide 10

Slide 10 text

ngx_small_lightΛ༗ޮʹ͢Δ small_light on; serverίϯςΩετͰͷΈར༻ՄೳͰ͢

Slide 11

Slide 11 text

αϜωΠϧੜ੒ύλʔϯΛఆٛ # in server context small_light_pattern_define medium dw=600,dh=400; small_light_pattern_define small dw=300,dh=200; /small_light(dw=300,dh=200)/img/image.jpg /small_light(p=small)/img/image.jpg → ■ઃఆ ■URI

Slide 12

Slide 12 text

جຊతͳύϥϝʔλ ύϥϝʔλ໊ ղઆ dw, dh ੜ੒͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200) of ੜ੒͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg / gif / png / webp) q ը࣭(e.g. 95) p small_light_pattern_deﬁneͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛ୅ೖ e ը૾ม׵ϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)

Slide 13

Slide 13 text

small_light_getparam_mode small_light_getparam_mode on; σϑΥϧτ͸offͰ͢ /small_light(dw=300,dh=200)/img/image.jpg /img/image.jpg?dw=300&dh=200 ■ઃఆ ■URI

Slide 14

Slide 14 text

small_light_material_dir # not trailing slash small_light_material_dir /var/materials; # there is the icon.jpg in /var/materials. small_light_pattern_define embed dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15; /small_light(p=embed)/img/image.jpg + = ■URI ■ઃఆ

Slide 15

Slide 15 text

small_light_imlib2_temp_dir # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ■ઃఆ • Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε • Imlib2͸ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍ • ઃఆͷॻࣜ͸proxy_temp_pathͱҰॹ

Slide 16

Slide 16 text

small_light_buffer # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ■ઃఆ • ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ • ௨ৗ͸ར༻͞Εͳ͍(Content-Length෼֬อ͢Δ) • ྫ͑͹ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ Content-Lengthؚ͕·Ε͍ͯͳ͍৔߹ʹར༻͢Δ # default setting small_light_buffer 1m;

Slide 17

Slide 17 text

֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ ngx_small_lightͰϦαΠζ # server context in nginx.conf small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location ~ /img { proxy_pass http://external_host; } /small_light(dw=300,dh=200)/img/image.jpg ■ઃఆ ■URI

Slide 18

Slide 18 text

υΩϡϝϯτ͸ͪ͜Β ↓ https://github.com/cubicdaiya/ ngx_small_light/wiki/Conﬁguration

Slide 19

Slide 19 text

ը૾ϥΠϒϥϦͷൺֱ ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭ ImageMagick ˕ ˚ ˕ Imlib2 ˚ ˕ ʓ GD ʓ ʓ ʓ

Slide 20

Slide 20 text

ͦͷଞͷը૾ม׵Ϟδϡʔϧ • mod_small_light • ngx_http_image_ﬁlter • go-thumber

Slide 21

Slide 21 text

mod_small_light • ݩ૆SmalllightͷApacheϞδϡʔϧ • https://github.com/yamac/smalllight • ngx_small_light͸͜ͷϞδϡʔϧͷnginx޲͚ ϙʔςΟϯάͱͯ͠ελʔτ • ͔ͭͯpixivͰ΋ຐվ଄ͯ͠ར༻͞Ε͍ͯ·ͨ͠

Slide 22

Slide 22 text

Quick Start SetOutputFilter SMALL_LIGHT ݩը૾ͷ63* JNHJNBHFKQH ม׵༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH

Slide 23

Slide 23 text

ngx_small_lightͱͷҧ͍ • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ • ngx_small_lightͰ͸of͕ͳ͍৔߹ɺ ೖྗ/ग़ྗϑΥʔϚοτ͸ಉ͡ʹͳΔ • ར༻Մೳͳػೳ΍ύϥϝʔλ͕एׯগͳΊ

Slide 24

Slide 24 text

ngx_http_image_ﬁlter • ΋͏Ұͭͷnginxͷը૾ม׵Ϟδϡʔϧ • ͳΜͱඪ४ϞδϡʔϧͰ͢ • nginx/src/http/modules/ngx_http_image_ﬁlter_module.c

Slide 25

Slide 25 text

Quick Start location ~ \.(png|gif|jpe?g)$ { image_filter resize 150 150; } (mod|ngx)_small_lightͷΑ͏ͳ ม׵༻URIͷ֓೦͕ͳ͍

Slide 26

Slide 26 text

ngx_small_lightͱͷҧ͍ • ػೳগͳΊ • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only • image_ﬁlter_buffer͕ৗʹར༻͞ΕΔ • σϑΥϧτ஋͕1mͱখ͍͞ͷͰ஫ҙ • ࣮૷Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ

Slide 27

Slide 27 text

go-thumber • pixivۘ੡ը૾ม׵ϓϩΩγαʔό • ੍໿ͱੑೳ • JPEG & Resize only • High Speed & High Quality • ಛఆͷ৚݅ԼͰઈେͳޮՌΛൃش͢Δ

Slide 28

Slide 28 text

Image Convert Optimization

Slide 29

Slide 29 text

worker_(processes|connections)

Slide 30

Slide 30 text

worker_(proccesses|connections) • worker_processes͸ଟΊʹ • worker_connections͸গͳΊʹ • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱ͸๨ΕΔ

Slide 31

Slide 31 text

JPEG Hinting

Slide 32

Slide 32 text

JPEG Hinting • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴ଎Խ͢Δ࢓૊Έ • ৄ͘͠͸ʮຊ౰͸଎͍ImageMagickʯͰάάΖ͏ • ImageMagickͱImlib2Ͱར༻Մೳ • jpeghint=yΛࢦఆ͢Δ͚ͩ • ਺ഒʙ10ഒ଎͘ͳΔέʔε΋͋Δ

Slide 33

Slide 33 text

JPEG Hinting with ngx_small_light small_light_pattern_define thumb dw=600,dh=400,jpeghint=y; JPEGҎ֎ͷը૾Λॲཧ͢Δࡍ͸ແࢹ͞Ε·͢

Slide 34

Slide 34 text

libjpeg-turbo

Slide 35

Slide 35 text

libjpeg-turbo • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg • SIMD໋ྩ(MMX, SSE2)Λར༻ • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ • e.g. LD_LIBRARY_PATH

Slide 36

Slide 36 text

libjpegͱlibjpeg-turbo

Slide 37

Slide 37 text

OpenMP

Slide 38

Slide 38 text

OpenMP • ImageMagick͸σϑΥϧτͰOpenMP͕༗ޮ • Ϛϧνϓϩηε؀ڥͩͱܶతʹεϩʔμ΢ϯ͢Δ • OpenMPΛແޮʹ͢Δํ๏ • —disable-openmp෇͖Ͱ࠶ίϯύΠϧ • OMP_NUM_THREADS=1 • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ

Slide 39

Slide 39 text

OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ

Slide 40

Slide 40 text

OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ

Slide 41

Slide 41 text

OMP_NUM_THREADS=1 with nginx env OMP_NUM_THREADS=1; nginx͸ىಈ࣌ʹTZΛআ͘͢΂ͯͷ؀ڥม਺Λ ࡟আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏

Slide 42

Slide 42 text

WebP

Slide 43

Slide 43 text

WebP • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ • ωοτϫʔΫଳҬ࡟ݮͱදࣔ଎౓ͷ୹ॖ͕໨త • JPEG΍PNG౳ͱൺ΂͔ͯͳΓαΠζ͕খ͍͞ • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ୺຤͕গͳ͍ͷ͕೉఺

Slide 44

Slide 44 text

ngx_small_lightͷWebPαϙʔτ • ImageMagickɺGDͰར༻Մೳ • ImageMagickͳΒlibwebp • GDͳΒlibvpx͕ඞཁ • of=webpͬͯॻ͚ͩ͘

Slide 45

Slide 45 text

ngx_small_light & WebP࠾༻ࣄྫ ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢

Slide 46

Slide 46 text

Cache

Slide 47

Slide 47 text

ngx_small_lightͷϨεϙϯεΛ nginxͰΩϟογϡ͢Δ؆୯ͳྫ proxy_cache_path /tmp/cache levels=1:2 key_zone=images:5m inactive=1h max_size=10G; server { server_name origin_server; … location ~ / { proxy_cache images; proxy_cache_valid 200 1h; proxy_cache_key “$scheme://$host$request_uri$is_args$args”; proxy_pass http://small_light_server; } } server { server_name small_light_server; … small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } }

Slide 48

Slide 48 text

·ͱΊ • ngx_small_light͸ಈతͳαϜωΠϧੜ੒͕Ͱ ͖Δػೳ๛෋ͳnginxϞδϡʔϧ • ࠷ۙ͸࠾༻ࣄྫΛฉ͘͜ͱ΋ͪΒ΄Β • ը૾ม׵͸ॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά ΍ΩϟογϡΛߦ͍·͠ΐ͏