Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
実践ngx_small_light入門
Search
Tatsuhiko Kubo
February 18, 2015
Technology
12
16k
実践ngx_small_light入門
at 画像変換Night
Tatsuhiko Kubo
February 18, 2015
Tweet
Share
More Decks by Tatsuhiko Kubo
See All by Tatsuhiko Kubo
Mackerel in さくらのクラウド
cubicdaiya
1
750
Handling a tremendous amount of images with Fastly / Yamagoya Traverse 2020
cubicdaiya
2
1.5k
System Integration with Fastly
cubicdaiya
0
640
実例で学ぶ画像最適化集 with ImageFlux / ImageFlux meetup#2
cubicdaiya
4
19k
Software Engineer, Infrastructure
cubicdaiya
4
3.2k
High Performance Count Up!
cubicdaiya
0
380
ImageFluxを利用した画像配信の最適化 / ImageFlux meetup 201801
cubicdaiya
0
3k
Building high performance push notification server in Go
cubicdaiya
5
3.3k
メルカリのデータ分析基盤 / mercari data analysis infrastructure
cubicdaiya
11
12k
Other Decks in Technology
See All in Technology
extension 現場で使えるXcodeショートカット一覧
ktombow
0
210
LLM時代にデータエンジニアの役割はどう変わるか?
ikkimiyazaki
1
560
生成AIを活用したZennの取り組み事例
ryosukeigarashi
0
210
フルカイテン株式会社 エンジニア向け採用資料
fullkaiten
0
9.1k
Exadata Database Service on Dedicated Infrastructure(ExaDB-D) UI スクリーン・キャプチャ集
oracle4engineer
PRO
2
5.4k
いまさら聞けない ABテスト入門
skmr2348
1
210
AWSにおけるTrend Vision Oneの効果について
shimak
0
130
Large Vision Language Modelを用いた 文書画像データ化作業自動化の検証、運用 / shibuya_AI
sansan_randd
0
110
組織観点からIAM Identity CenterとIAMの設計を考える
nrinetcom
PRO
1
180
綺麗なデータマートをつくろう_データ整備を前向きに考える会 / Let's create clean data mart
brainpadpr
2
190
ユニットテストに対する考え方の変遷 / Everyone should watch his live coding
mdstoy
0
130
10年の共創が示す、これからの開発者と企業の関係 ~ Crossroad
soracom
PRO
1
360
Featured
See All Featured
Six Lessons from altMBA
skipperchong
28
4k
Statistics for Hackers
jakevdp
799
220k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
9
960
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.1k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
61k
Facilitating Awesome Meetings
lara
56
6.6k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
The Straight Up "How To Draw Better" Workshop
denniskardys
237
140k
Intergalactic Javascript Robots from Outer Space
tanoku
273
27k
Mobile First: as difficult as doing things right
swwweet
224
10k
Gamification - CAS2011
davidbonilla
81
5.5k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
127
53k
Transcript
࣮ફngx_small_lightೖ Tatsuhiko Kubo@cubicdaiya ը૾มNight@2015/02/18
ࣗݾհ • ٱอୡ(Tatsuhiko Kubo) • bokko@cubicdaiya • Software Engineer in
Infrastructure Engineering • Mercari, Inc. (ࡢͷࠓࠒ·ͩpixiv, Inc.Ͱͨ͠) • Go / C / mruby / nginx / ngx_(lua|mruby) / etc…
ngx_small_light
ngx_small_light • nginxͰಈతͳαϜωΠϧੜ͕Ͱ͖Δ֦ுϞδϡʔϧ • ↓ͷը૾ϥΠϒϥϦɺϑΥʔϚοτ͕ར༻Մೳ • ImageMagick / Imlib2 /
GD • JPEG / GIF / PNG / WebP • https://github.com/cubicdaiya/ngx_small_light
ngx_small_lightͰͰ͖Δ͜ͱ • ը૾ͷϦαΠζ / Ϋϩοϓ / ߹ / ճస •
ϑΥʔϚοτม(e.g. PNG -> JPEG) • ϒϥʔ / (Ξϯ)γϟʔϓͷՃॲཧ • etc…
Setup cd ngx_small_light # enable ImageMagick ./setup # enable ImageMagick
and Imlib2 ./setup —-with-imlib2 # enable ImageMagick and GD ./setup —-with-gd # enable ImageMagick and Imlib2 and GD ./setup —-with-imlib2 —with—gd
Install cd ${ngx_small_light_dir} ./setup cd ${nginx_dir} ./configure ¥ —-add-moduole=${ngx_small_light_dir} make
sudo make install
Quick Start # server context in nginx.conf small_light on; location
~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } ݩը૾ͷ63* JNHJNBHFKQH ม༻ͷ63* TNBMM@MJHIU EX EI JNHJNBHFKQH
σΟϨΫςΟϒҰཡ σΟϨΫςΟϒ໊ ղઆ small_light small_lightͷon/off small_light_pattern_define αϜωΠϧੜύλʔϯΛఆٛ small_light_getparam_mode GETύϥϝʔλͰαϜωΠϧੜύλʔϯΛఆٛ small_light_material_dir
߹ૉࡐͷσΟϨΫτϦΛࢦఆ small_light_imlib2_temp_dir imlib2͕ੜ͢ΔςϯϙϥϦϑΝΠϧͷύε small_light_buffer Content-Length͕ۭͷ߹ʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ
ngx_small_lightΛ༗ޮʹ͢Δ small_light on; serverίϯςΩετͰͷΈར༻ՄೳͰ͢
αϜωΠϧੜύλʔϯΛఆٛ # in server context small_light_pattern_define medium dw=600,dh=400; small_light_pattern_define small
dw=300,dh=200; /small_light(dw=300,dh=200)/img/image.jpg /small_light(p=small)/img/image.jpg → ▪ઃఆ ▪URI
جຊతͳύϥϝʔλ ύϥϝʔλ໊ ղઆ dw, dh ੜ͢ΔαϜωΠϧͷ෯ͱߴ͞(e.g. dw=300, dh=200) of ੜ͢ΔαϜωΠϧͷϑΥʔϚοτ(jpg
/ gif / png / webp) q ը࣭(e.g. 95) p small_light_pattern_defineͰఆٛ͞ΕͨαϜωΠϧύλʔϯΛೖ e ը૾มϥΠϒϥϦΛࢦఆ(imagemagick / imlib2 / gd)
small_light_getparam_mode small_light_getparam_mode on; σϑΥϧτoffͰ͢ /small_light(dw=300,dh=200)/img/image.jpg /img/image.jpg?dw=300&dh=200 ▪ઃఆ ▪URI
small_light_material_dir # not trailing slash small_light_material_dir /var/materials; # there is
the icon.jpg in /var/materials. small_light_pattern_define embed dw=300,dh=200,embedicon=icon.jpg,ix=10,iy=15; /small_light(p=embed)/img/image.jpg + = ▪URI ▪ઃఆ
small_light_imlib2_temp_dir # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ
• Imlib2͕อଘ͢ΔςϯϙϥϦϑΝΠϧͷύε • Imlib2ϑΝΠϧ୯ҐͰ͔͠ը૾Λѻ͑ͳ͍ • ઃఆͷॻࣜproxy_temp_pathͱҰॹ
small_light_buffer # default setting small_light_imlib2_temp_dir /tmp 1 2 0; ▪ઃఆ
• ֤ϦΫΤετຖʹ֬อ͢ΔϝϞϦόοϑΝͷαΠζ • ௨ৗར༻͞Εͳ͍(Content-Length֬อ͢Δ) • ྫ͑ɺϓϩΩγઌͷαʔόͷϨεϙϯεʹ Content-Lengthؚ͕·Ε͍ͯͳ͍߹ʹར༻͢Δ # default setting small_light_buffer 1m;
֎෦αʔό(e.g. S3)ʹ͋Δը૾Λ ngx_small_lightͰϦαΠζ # server context in nginx.conf small_light on;
location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } location ~ /img { proxy_pass http://external_host; } /small_light(dw=300,dh=200)/img/image.jpg ▪ઃఆ ▪URI
υΩϡϝϯτͪ͜Β ↓ https://github.com/cubicdaiya/ ngx_small_light/wiki/Configuration
ը૾ϥΠϒϥϦͷൺֱ ϥΠϒϥϦ໊ ػೳ εϐʔυ ը࣭ ImageMagick ˕ ˚ ˕ Imlib2
˚ ˕ ʓ GD ʓ ʓ ʓ
ͦͷଞͷը૾มϞδϡʔϧ • mod_small_light • ngx_http_image_filter • go-thumber
mod_small_light • ݩSmalllightͷApacheϞδϡʔϧ • https://github.com/yamac/smalllight • ngx_small_light͜ͷϞδϡʔϧͷnginx͚ ϙʔςΟϯάͱͯ͠ελʔτ • ͔ͭͯpixivͰຐվͯ͠ར༻͞Ε͍ͯ·ͨ͠
Quick Start <LocationMatch ^/small_light[^/]*/> SetOutputFilter SMALL_LIGHT </LocationMatch> ݩը૾ͷ63* JNHJNBHFKQH ม༻ͷ63*
TNBMM@MJHIU EX EI JNHJNBHFKQH
ngx_small_lightͱͷҧ͍ • of͕ࢦఆ͞Εͯͳ͍ͱৗʹJPEGग़ྗ • ngx_small_lightͰof͕ͳ͍߹ɺ ೖྗ/ग़ྗϑΥʔϚοτಉ͡ʹͳΔ • ར༻Մೳͳػೳύϥϝʔλ͕एׯগͳΊ
ngx_http_image_filter • ͏Ұͭͷnginxͷը૾มϞδϡʔϧ • ͳΜͱඪ४ϞδϡʔϧͰ͢ • nginx/src/http/modules/ngx_http_image_filter_module.c
Quick Start location ~ \.(png|gif|jpe?g)$ { image_filter resize 150 150;
} (mod|ngx)_small_lightͷΑ͏ͳ ม༻URIͷ֓೦͕ͳ͍
ngx_small_lightͱͷҧ͍ • ػೳগͳΊ • ར༻Մೳͳը૾ϥΠϒϥϦ͕GD only • image_filter_buffer͕ৗʹར༻͞ΕΔ • σϑΥϧτ͕1mͱখ͍͞ͷͰҙ
• ࣮Λ؆୯ʹͯ͠nginx.confͰؤுΔΠϝʔδ
go-thumber • pixivۘը૾มϓϩΩγαʔό • ੍ͱੑೳ • JPEG & Resize only
• High Speed & High Quality • ಛఆͷ݅ԼͰઈେͳޮՌΛൃش͢Δ
Image Convert Optimization
worker_(processes|connections)
worker_(proccesses|connections) • worker_processesଟΊʹ • worker_connectionsগͳΊʹ • preforkͷApacheͩͱࢥͬͯઃఆ͢Δ • nginx͕ΠϕϯτۦಈαʔόͰ͋Δ͜ͱΕΔ
JPEG Hinting
JPEG Hinting • JPEGը૾ͷಡΈࠐΈॲཧΛେ෯ʹߴԽ͢ΔΈ • ৄ͘͠ʮຊ͍ImageMagickʯͰάάΖ͏ • ImageMagickͱImlib2Ͱར༻Մೳ • jpeghint=yΛࢦఆ͢Δ͚ͩ
• ഒʙ10ഒ͘ͳΔέʔε͋Δ
JPEG Hinting with ngx_small_light small_light_pattern_define thumb dw=600,dh=400,jpeghint=y; JPEGҎ֎ͷը૾Λॲཧ͢Δࡍແࢹ͞Ε·͢
libjpeg-turbo
libjpeg-turbo • x86ͱx86_64༻ʹ࠷దԽ͞Εͨlibjpeg • SIMD໋ྩ(MMX, SSE2)Λར༻ • libjpegͱ؆୯ʹࠩ͠ସ͑Δ͜ͱ͕Մೳ • e.g.
LD_LIBRARY_PATH
libjpegͱlibjpeg-turbo
OpenMP
OpenMP • ImageMagickσϑΥϧτͰOpenMP͕༗ޮ • Ϛϧνϓϩηεڥͩͱܶతʹεϩʔμϯ͢Δ • OpenMPΛແޮʹ͢Δํ๏ • —disable-openmp͖Ͱ࠶ίϯύΠϧ •
OMP_NUM_THREADS=1 • ࠶ίϯύΠϧ͕ෆཁͳޙऀ͕Φεεϝ
OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷCPUάϥϑ
OMP_NUM_THREADS=1 ੲ͍ͨձࣾͷappαʔόͷLAάϥϑ
OMP_NUM_THREADS=1 with nginx env OMP_NUM_THREADS=1; nginxىಈ࣌ʹTZΛআͯ͘͢ͷڥมΛ আ͢ΔͷͰenvσΟϨΫςΟϒΛར༻͠·͠ΐ͏
WebP
WebP • Google͕։ൃ͍ͯ͠Δը૾ϑΥʔϚοτ • ωοτϫʔΫଳҬݮͱදࣔͷॖ͕త • JPEGPNGͱൺ͔ͯͳΓαΠζ͕খ͍͞ • ϞόΠϧͩͱ·ͩαϙʔτ͍ͯ͠Δ͕গͳ͍ͷ͕
ngx_small_lightͷWebPαϙʔτ • ImageMagickɺGDͰར༻Մೳ • ImageMagickͳΒlibwebp • GDͳΒlibvpx͕ඞཁ • of=webpͬͯॻ͚ͩ͘
ngx_small_light & WebP࠾༻ࣄྫ ͋ͱͰWantedlyͷํ͕LTͯ͘͠Ε·͢
Cache
ngx_small_lightͷϨεϙϯεΛ nginxͰΩϟογϡ͢Δ؆୯ͳྫ proxy_cache_path /tmp/cache levels=1:2 key_zone=images:5m inactive=1h max_size=10G; server {
server_name origin_server; … location ~ / { proxy_cache images; proxy_cache_valid 200 1h; proxy_cache_key “$scheme://$host$request_uri$is_args$args”; proxy_pass http://small_light_server; } } server { server_name small_light_server; … small_light on; location ~ small_light[^/]*/(.+)$ { set $file $1; rewrite ^ /$file; } }
·ͱΊ • ngx_small_lightಈతͳαϜωΠϧੜ͕Ͱ ͖Δػೳ๛ͳnginxϞδϡʔϧ • ࠷ۙ࠾༻ࣄྫΛฉ͘͜ͱͪ´ • ը૾มॏ͍ͷͰͦΕ૬Ԡͷνϡʔχϯά ΩϟογϡΛߦ͍·͠ΐ͏