Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
実践ngx_small_light入門
Tatsuhiko Kubo
February 18, 2015
Technology
12
14k
実践ngx_small_light入門
at 画像変換Night
Tatsuhiko Kubo
February 18, 2015
Tweet
Share
More Decks by Tatsuhiko Kubo
See All by Tatsuhiko Kubo
Handling a tremendous amount of images with Fastly / Yamagoya Traverse 2020
cubicdaiya
2
970
System Integration with Fastly
cubicdaiya
0
400
実例で学ぶ画像最適化集 with ImageFlux / ImageFlux meetup#2
cubicdaiya
4
17k
Software Engineer, Infrastructure
cubicdaiya
4
2.7k
High Performance Count Up!
cubicdaiya
0
200
ImageFluxを利用した画像配信の最適化 / ImageFlux meetup 201801
cubicdaiya
0
2.2k
Building high performance push notification server in Go
cubicdaiya
5
2.7k
メルカリのデータ分析基盤 / mercari data analysis infrastructure
cubicdaiya
11
11k
On-call Engineering
cubicdaiya
8
5.7k
Other Decks in Technology
See All in Technology
XRとシビックテック
satoshirobatofujimoto
0
110
JAWS-UG re:Habilitaion 報告 / JAWS-UG OITA rehabilitation
hiranofumio
0
140
誰が正解を知っているのか / Who knows the right answer
takaking22
1
250
How to start with DDD when you have a Monolith
javujavichi
0
370
Design for Humans: How to make better modernization decisions
indualagarsamy
2
130
20220622_FinJAWS_あのときにAWSがあったらこうできた
taketakekaho
0
120
FoodTechにおける商流・金流・物流の進化/Evolution of Commercial, Financial, and Logistics in FoodTech
dskst
0
420
Building smarter apps with machine learning, from magic to reality
picardparis
4
3.2k
サーバレスECにおける Step Functions の使い方 〜ステートマシン全部見せます!〜
miu_crescent
0
200
UIKitのアップデート #WWDC22
akatsuki174
4
360
開発組織の生産性を可視化する State of DevOpsとFour Keysとは / deep dive into State of DevOps
yfcgpsebp
0
310
Introduction To Technical Writing
olawanle_joel
0
100
Featured
See All Featured
The Brand Is Dead. Long Live the Brand.
mthomps
46
2.7k
Design by the Numbers
sachag
271
17k
Clear Off the Table
cherdarchuk
79
280k
Why You Should Never Use an ORM
jnunemaker
PRO
47
7.6k
Embracing the Ebb and Flow
colly
73
3.4k
The World Runs on Bad Software
bkeepers
PRO
57
5.3k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
15
950
Fashionably flexible responsive web design (full day workshop)
malarkey
396
62k
Become a Pro
speakerdeck
PRO
3
850
Gamification - CAS2011
davidbonilla
75
3.9k
Imperfection Machines: The Place of Print at Facebook
scottboms
253
12k
GraphQLの誤解/rethinking-graphql
sonatard
28
6.6k
Transcript
࣮ફngx_small_lightೖ Tatsuhiko Kubo@cubicdaiya ը૾มNight@2015/02/18
ࣗݾհ • ٱอୡ(Tatsuhiko Kubo) • bokko@cubicdaiya • Software Engineer in
Infrastructure Engineering • Mercari, Inc. (ࡢͷࠓࠒ·ͩpixiv, Inc.Ͱͨ͠) • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…
ngx_small_light
ngx_small_light • nginxͰಈతͳαϜωΠϧੜ͕Ͱ͖Δ֦ுϞδϡʔϧ • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ • ImageMagick / Imlib2 /
GD • JPEG / GIF / PNG / WebP • https://github.com/cubicdaiya/ngx_small_light
ngx_small_lightͰͰ͖Δ͜ͱ • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹ / ճస •
ϑΥʔϚοτม(e.g. PNG -> JPEG) • ϒϥʔ / (Ξϯ)γϟʔϓͷՃॲཧ • etc…
Setup cd ngx_small_light # enable ImageMagick ./setup # enable ImageMagick
and Imlib2 ./setup —-with-imlib2 # enable ImageMagick and GD ./setup —-with-gd # enable ImageMagick and Imlib2 and GD ./setup —-with-imlib2 —with—gd
Install cd ${ngx_small_light_dir} ./setup cd ${nginx_dir} ./configure ¥ —-add-moduole=${ngx_small_light_dir} make
sudo make install
Quick Start # server context in nginx.conf small_light on; location
~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } ݩը૾ͷ63* JNHJNBHFKQH ม༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH
σΟϨΫςΟϒҰཡ σΟϨΫςΟϒ໊ ղઆ small_light small_lightͷon/off small_light_pattern_define αϜωΠϧੜύλʔϯΛఆٛ small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜύλʔϯΛఆٛ small_light_material_dir
߹ૉࡐͷσΟϨΫτϦΛࢦఆ small_light_imlib2_temp_dir imlib2͕ੜ͢ΔςϯϙϥϦϑΝΠϧͷύε small_light_buffer Content-Length͕ۭͷ߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ
ngx_small_lightΛ༗ޮʹ͢Δ small_light on; serverίϯςΩετͰͷΈར༻ՄೳͰ͢
αϜωΠϧੜύλʔϯΛఆٛ # in server context small_light_pattern_define medium dw=600,dh=400; small_light_pattern_define small
dw=300,dh=200; /small_light(dw=300,dh=200)/img/image.jpg /small_light(p=small)/img/image.jpg → ▪ઃఆ ▪URI
جຊతͳύϥϝʔλ ύϥϝʔλ໊ ղઆ dw, dh ੜ͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200) of ੜ͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg
/ gif / png / webp) q ը࣭(e.g. 95) p small_light_pattern_defineͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛೖ e ը૾มϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)
small_light_getparam_mode small_light_getparam_mode on; σϑΥϧτoffͰ͢ /small_light(dw=300,dh=200)/img/image.jpg /img/image.jpg?dw=300&dh=200 ▪ઃఆ ▪URI
small_light_material_dir # not trailing slash small_light_material_dir /var/materials; # there is
the icon.jpg in /var/materials. small_light_pattern_define embed dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15; /small_light(p=embed)/img/image.jpg + = ▪URI ▪ઃఆ
small_light_imlib2_temp_dir # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ
• Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε • Imlib2ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍ • ઃఆͷॻࣜproxy_temp_pathͱҰॹ
small_light_buffer # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ
• ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ • ௨ৗར༻͞Εͳ͍(Content-Length֬อ͢Δ) • ྫ͑ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ Content-Lengthؚ͕·Ε͍ͯͳ͍߹ʹར༻͢Δ # default setting small_light_buffer 1m;
֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ ngx_small_lightͰϦαΠζ # server context in nginx.conf small_light on;
location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location ~ /img { proxy_pass http://external_host; } /small_light(dw=300,dh=200)/img/image.jpg ▪ઃఆ ▪URI
υΩϡϝϯτͪ͜Β ↓ https://github.com/cubicdaiya/ ngx_small_light/wiki/Configuration
ը૾ϥΠϒϥϦͷൺֱ ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭ ImageMagick ˕ ˚ ˕ Imlib2
˚ ˕ ʓ GD ʓ ʓ ʓ
ͦͷଞͷը૾มϞδϡʔϧ • mod_small_light • ngx_http_image_filter • go-thumber
mod_small_light • ݩSmalllightͷApacheϞδϡʔϧ • https://github.com/yamac/smalllight • ngx_small_light͜ͷϞδϡʔϧͷnginx͚ ϙʔςΟϯάͱͯ͠ελʔτ • ͔ͭͯpixivͰຐվͯ͠ར༻͞Ε͍ͯ·ͨ͠
Quick Start <LocationMatch ^/small_light[^/]*/> SetOutputFilter SMALL_LIGHT </LocationMatch> ݩը૾ͷ63* JNHJNBHFKQH ม༻ͷ63*
TNBMM@MJHIU EX EI JNHJNBHFKQH
ngx_small_lightͱͷҧ͍ • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ • ngx_small_lightͰof͕ͳ͍߹ɺ ೖྗ/ग़ྗϑΥʔϚοτಉ͡ʹͳΔ • ར༻Մೳͳػೳύϥϝʔλ͕एׯগͳΊ
ngx_http_image_filter • ͏Ұͭͷnginxͷը૾มϞδϡʔϧ • ͳΜͱඪ४ϞδϡʔϧͰ͢ • nginx/src/http/modules/ngx_http_image_filter_module.c
Quick Start location ~ \.(png|gif|jpe?g)$ { image_filter resize 150 150;
} (mod|ngx)_small_lightͷΑ͏ͳ ม༻URIͷ֓೦͕ͳ͍
ngx_small_lightͱͷҧ͍ • ػೳগͳΊ • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only • image_filter_buffer͕ৗʹར༻͞ΕΔ • σϑΥϧτ͕1mͱখ͍͞ͷͰҙ
• ࣮Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ
go-thumber • pixivۘը૾มϓϩΩγαʔό • ੍ͱੑೳ • JPEG & Resize only
• High Speed & High Quality • ಛఆͷ݅ԼͰઈେͳޮՌΛൃش͢Δ
Image Convert Optimization
worker_(processes|connections)
worker_(proccesses|connections) • worker_processesଟΊʹ • worker_connectionsগͳΊʹ • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱΕΔ
JPEG Hinting
JPEG Hinting • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴԽ͢ΔΈ • ৄ͘͠ʮຊ͍ImageMagickʯͰάάΖ͏ • ImageMagickͱImlib2Ͱར༻Մೳ • jpeghint=yΛࢦఆ͢Δ͚ͩ
• ഒʙ10ഒ͘ͳΔέʔε͋Δ
JPEG Hinting with ngx_small_light small_light_pattern_define thumb dw=600,dh=400,jpeghint=y; JPEGҎ֎ͷը૾Λॲཧ͢Δࡍແࢹ͞Ε·͢
libjpeg-turbo
libjpeg-turbo • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg • SIMD໋ྩ(MMX, SSE2)Λར༻ • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ • e.g.
LD_LIBRARY_PATH
libjpegͱlibjpeg-turbo
OpenMP
OpenMP • ImageMagickσϑΥϧτͰOpenMP͕༗ޮ • Ϛϧνϓϩηεڥͩͱܶతʹεϩʔμϯ͢Δ • OpenMPΛແޮʹ͢Δํ๏ • —disable-openmp͖Ͱ࠶ίϯύΠϧ •
OMP_NUM_THREADS=1 • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ
OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ
OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ
OMP_NUM_THREADS=1 with nginx env OMP_NUM_THREADS=1; nginxىಈ࣌ʹTZΛআͯ͘͢ͷڥมΛ আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏
WebP
WebP • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ • ωοτϫʔΫଳҬݮͱදࣔͷॖ͕త • JPEGPNGͱൺ͔ͯͳΓαΠζ͕খ͍͞ • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ͕গͳ͍ͷ͕
ngx_small_lightͷWebPαϙʔτ • ImageMagickɺGDͰར༻Մೳ • ImageMagickͳΒlibwebp • GDͳΒlibvpx͕ඞཁ • of=webpͬͯॻ͚ͩ͘
ngx_small_light & WebP࠾༻ࣄྫ ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢
Cache
ngx_small_lightͷϨεϙϯεΛ nginxͰΩϟογϡ͢Δ؆୯ͳྫ proxy_cache_path /tmp/cache levels=1:2 key_zone=images:5m inactive=1h max_size=10G; server {
server_name origin_server; … location ~ / { proxy_cache images; proxy_cache_valid 200 1h; proxy_cache_key “$scheme://$host$request_uri$is_args$args”; proxy_pass http://small_light_server; } } server { server_name small_light_server; … small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } }
·ͱΊ • ngx_small_lightಈతͳαϜωΠϧੜ͕Ͱ ͖Δػೳ๛ͳnginxϞδϡʔϧ • ࠷ۙ࠾༻ࣄྫΛฉ͘͜ͱͪ´ • ը૾มॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά ΩϟογϡΛߦ͍·͠ΐ͏