Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
実践ngx_small_light入門
Search
Tatsuhiko Kubo
February 18, 2015
Technology
12
15k
実践ngx_small_light入門
at 画像変換Night
Tatsuhiko Kubo
February 18, 2015
Tweet
Share
More Decks by Tatsuhiko Kubo
See All by Tatsuhiko Kubo
Handling a tremendous amount of images with Fastly / Yamagoya Traverse 2020
cubicdaiya
2
1.4k
System Integration with Fastly
cubicdaiya
0
530
実例で学ぶ画像最適化集 with ImageFlux / ImageFlux meetup#2
cubicdaiya
4
19k
Software Engineer, Infrastructure
cubicdaiya
4
3.1k
High Performance Count Up!
cubicdaiya
0
280
ImageFluxを利用した画像配信の最適化 / ImageFlux meetup 201801
cubicdaiya
0
2.8k
Building high performance push notification server in Go
cubicdaiya
5
3.1k
メルカリのデータ分析基盤 / mercari data analysis infrastructure
cubicdaiya
11
11k
On-call Engineering
cubicdaiya
8
6.4k
Other Decks in Technology
See All in Technology
たくさん本を読んだけど 1年後には綺麗サッパリ!を乗り越えて 学習の鬼になるぞ👹
yum3
0
160
Datadog Cloud SIEMを使ってAWS環境の脅威を可視化した話/lifeistech-datadog-cloud-siem
gidajun
0
480
ゆめみのアクセシビリティの現在地と今後
ryokatsuse
3
290
AI研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
130
可視化プラットフォームGrafanaの基本と活用方法の全て
hamadakoji
0
230
成長期に歩みを止めないための創業期の開発文化形成
mayah
6
420
[I/O Extended Android 2024] What`s new in Android 2024
kyeongwan
0
220
開発生産性をむしろ向上させる セキュリティパートナーの作り方 / Dev Productivity Con 2024
flatt_security
0
360
初中級者用如何使用backlog -VALE TUDOEDITION-
in0u
0
140
[NIKKEI Tech Talk]Bias for Action!! 実践から学ぶための仕組とコミュニティ / Community for Practice and Learning
kanamasa
0
270
エンジニアの生存戦略 〜クラウド潮流の経験から紐解く技術トレンドのメカニズムと乗りこなし方〜
shimy
9
1.9k
「我々はどこに向かっているのか」を問い続けるための仕組みづくり / Establishing a System for Continuous Inquiry about where we are
daitasu
0
170
Featured
See All Featured
Building Effective Engineering Teams - LeadDev
addyosmani
47
2.2k
Pencils Down: Stop Designing & Start Developing
hursman
118
11k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
228
16k
Product Roadmaps are Hard
iamctodd
PRO
48
10k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
29
2.5k
Teambox: Starting and Learning
jrom
130
8.6k
jQuery: Nuts, Bolts and Bling
dougneiner
61
7.4k
Thoughts on Productivity
jonyablonski
64
4.1k
Bootstrapping a Software Product
garrettdimon
PRO
304
110k
The MySQL Ecosystem @ GitHub 2015
samlambert
248
12k
The Pragmatic Product Professional
lauravandoore
29
6.1k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Transcript
࣮ફngx_small_lightೖ Tatsuhiko Kubo@cubicdaiya ը૾มNight@2015/02/18
ࣗݾհ • ٱอୡ(Tatsuhiko Kubo) • bokko@cubicdaiya • Software Engineer in
Infrastructure Engineering • Mercari, Inc. (ࡢͷࠓࠒ·ͩpixiv, Inc.Ͱͨ͠) • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…
ngx_small_light
ngx_small_light • nginxͰಈతͳαϜωΠϧੜ͕Ͱ͖Δ֦ுϞδϡʔϧ • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ • ImageMagick / Imlib2 /
GD • JPEG / GIF / PNG / WebP • https://github.com/cubicdaiya/ngx_small_light
ngx_small_lightͰͰ͖Δ͜ͱ • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹ / ճస •
ϑΥʔϚοτม(e.g. PNG -> JPEG) • ϒϥʔ / (Ξϯ)γϟʔϓͷՃॲཧ • etc…
Setup cd ngx_small_light # enable ImageMagick ./setup # enable ImageMagick
and Imlib2 ./setup —-with-imlib2 # enable ImageMagick and GD ./setup —-with-gd # enable ImageMagick and Imlib2 and GD ./setup —-with-imlib2 —with—gd
Install cd ${ngx_small_light_dir} ./setup cd ${nginx_dir} ./configure ¥ —-add-moduole=${ngx_small_light_dir} make
sudo make install
Quick Start # server context in nginx.conf small_light on; location
~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } ݩը૾ͷ63* JNHJNBHFKQH ม༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH
σΟϨΫςΟϒҰཡ σΟϨΫςΟϒ໊ ղઆ small_light small_lightͷon/off small_light_pattern_define αϜωΠϧੜύλʔϯΛఆٛ small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜύλʔϯΛఆٛ small_light_material_dir
߹ૉࡐͷσΟϨΫτϦΛࢦఆ small_light_imlib2_temp_dir imlib2͕ੜ͢ΔςϯϙϥϦϑΝΠϧͷύε small_light_buffer Content-Length͕ۭͷ߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ
ngx_small_lightΛ༗ޮʹ͢Δ small_light on; serverίϯςΩετͰͷΈར༻ՄೳͰ͢
αϜωΠϧੜύλʔϯΛఆٛ # in server context small_light_pattern_define medium dw=600,dh=400; small_light_pattern_define small
dw=300,dh=200; /small_light(dw=300,dh=200)/img/image.jpg /small_light(p=small)/img/image.jpg → ▪ઃఆ ▪URI
جຊతͳύϥϝʔλ ύϥϝʔλ໊ ղઆ dw, dh ੜ͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200) of ੜ͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg
/ gif / png / webp) q ը࣭(e.g. 95) p small_light_pattern_defineͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛೖ e ը૾มϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)
small_light_getparam_mode small_light_getparam_mode on; σϑΥϧτoffͰ͢ /small_light(dw=300,dh=200)/img/image.jpg /img/image.jpg?dw=300&dh=200 ▪ઃఆ ▪URI
small_light_material_dir # not trailing slash small_light_material_dir /var/materials; # there is
the icon.jpg in /var/materials. small_light_pattern_define embed dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15; /small_light(p=embed)/img/image.jpg + = ▪URI ▪ઃఆ
small_light_imlib2_temp_dir # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ
• Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε • Imlib2ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍ • ઃఆͷॻࣜproxy_temp_pathͱҰॹ
small_light_buffer # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ
• ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ • ௨ৗར༻͞Εͳ͍(Content-Length֬อ͢Δ) • ྫ͑ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ Content-Lengthؚ͕·Ε͍ͯͳ͍߹ʹར༻͢Δ # default setting small_light_buffer 1m;
֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ ngx_small_lightͰϦαΠζ # server context in nginx.conf small_light on;
location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location ~ /img { proxy_pass http://external_host; } /small_light(dw=300,dh=200)/img/image.jpg ▪ઃఆ ▪URI
υΩϡϝϯτͪ͜Β ↓ https://github.com/cubicdaiya/ ngx_small_light/wiki/Configuration
ը૾ϥΠϒϥϦͷൺֱ ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭ ImageMagick ˕ ˚ ˕ Imlib2
˚ ˕ ʓ GD ʓ ʓ ʓ
ͦͷଞͷը૾มϞδϡʔϧ • mod_small_light • ngx_http_image_filter • go-thumber
mod_small_light • ݩSmalllightͷApacheϞδϡʔϧ • https://github.com/yamac/smalllight • ngx_small_light͜ͷϞδϡʔϧͷnginx͚ ϙʔςΟϯάͱͯ͠ελʔτ • ͔ͭͯpixivͰຐվͯ͠ར༻͞Ε͍ͯ·ͨ͠
Quick Start <LocationMatch ^/small_light[^/]*/> SetOutputFilter SMALL_LIGHT </LocationMatch> ݩը૾ͷ63* JNHJNBHFKQH ม༻ͷ63*
TNBMM@MJHIU EX EI JNHJNBHFKQH
ngx_small_lightͱͷҧ͍ • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ • ngx_small_lightͰof͕ͳ͍߹ɺ ೖྗ/ग़ྗϑΥʔϚοτಉ͡ʹͳΔ • ར༻Մೳͳػೳύϥϝʔλ͕एׯগͳΊ
ngx_http_image_filter • ͏Ұͭͷnginxͷը૾มϞδϡʔϧ • ͳΜͱඪ४ϞδϡʔϧͰ͢ • nginx/src/http/modules/ngx_http_image_filter_module.c
Quick Start location ~ \.(png|gif|jpe?g)$ { image_filter resize 150 150;
} (mod|ngx)_small_lightͷΑ͏ͳ ม༻URIͷ֓೦͕ͳ͍
ngx_small_lightͱͷҧ͍ • ػೳগͳΊ • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only • image_filter_buffer͕ৗʹར༻͞ΕΔ • σϑΥϧτ͕1mͱখ͍͞ͷͰҙ
• ࣮Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ
go-thumber • pixivۘը૾มϓϩΩγαʔό • ੍ͱੑೳ • JPEG & Resize only
• High Speed & High Quality • ಛఆͷ݅ԼͰઈେͳޮՌΛൃش͢Δ
Image Convert Optimization
worker_(processes|connections)
worker_(proccesses|connections) • worker_processesଟΊʹ • worker_connectionsগͳΊʹ • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱΕΔ
JPEG Hinting
JPEG Hinting • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴԽ͢ΔΈ • ৄ͘͠ʮຊ͍ImageMagickʯͰάάΖ͏ • ImageMagickͱImlib2Ͱར༻Մೳ • jpeghint=yΛࢦఆ͢Δ͚ͩ
• ഒʙ10ഒ͘ͳΔέʔε͋Δ
JPEG Hinting with ngx_small_light small_light_pattern_define thumb dw=600,dh=400,jpeghint=y; JPEGҎ֎ͷը૾Λॲཧ͢Δࡍແࢹ͞Ε·͢
libjpeg-turbo
libjpeg-turbo • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg • SIMD໋ྩ(MMX, SSE2)Λར༻ • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ • e.g.
LD_LIBRARY_PATH
libjpegͱlibjpeg-turbo
OpenMP
OpenMP • ImageMagickσϑΥϧτͰOpenMP͕༗ޮ • Ϛϧνϓϩηεڥͩͱܶతʹεϩʔμϯ͢Δ • OpenMPΛແޮʹ͢Δํ๏ • —disable-openmp͖Ͱ࠶ίϯύΠϧ •
OMP_NUM_THREADS=1 • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ
OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ
OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ
OMP_NUM_THREADS=1 with nginx env OMP_NUM_THREADS=1; nginxىಈ࣌ʹTZΛআͯ͘͢ͷڥมΛ আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏
WebP
WebP • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ • ωοτϫʔΫଳҬݮͱදࣔͷॖ͕త • JPEGPNGͱൺ͔ͯͳΓαΠζ͕খ͍͞ • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ͕গͳ͍ͷ͕
ngx_small_lightͷWebPαϙʔτ • ImageMagickɺGDͰར༻Մೳ • ImageMagickͳΒlibwebp • GDͳΒlibvpx͕ඞཁ • of=webpͬͯॻ͚ͩ͘
ngx_small_light & WebP࠾༻ࣄྫ ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢
Cache
ngx_small_lightͷϨεϙϯεΛ nginxͰΩϟογϡ͢Δ؆୯ͳྫ proxy_cache_path /tmp/cache levels=1:2 key_zone=images:5m inactive=1h max_size=10G; server {
server_name origin_server; … location ~ / { proxy_cache images; proxy_cache_valid 200 1h; proxy_cache_key “$scheme://$host$request_uri$is_args$args”; proxy_pass http://small_light_server; } } server { server_name small_light_server; … small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } }
·ͱΊ • ngx_small_lightಈతͳαϜωΠϧੜ͕Ͱ ͖Δػೳ๛ͳnginxϞδϡʔϧ • ࠷ۙ࠾༻ࣄྫΛฉ͘͜ͱͪ´ • ը૾มॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά ΩϟογϡΛߦ͍·͠ΐ͏