Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Docker Performance on Web Application
Search
Yuuki Tsubouchi (yuuk1)
January 17, 2015
Technology
7
130k
Docker Performance on Web Application
Docker Meetup Tokyo #4
Yuuki Tsubouchi (yuuk1)
January 17, 2015
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
とあるSREの博士「過程」 / A Certain SRE’s Ph.D. Journey
yuukit
9
4.1k
eBPFを用いたAIネットワーク監視システム論文の実装 / eBPF Japan Meetup #4
yuukit
3
1k
クラウドのテレメトリーシステム研究動向2025年
yuukit
3
1k
博士論文公聴会: Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining / PhD Defence
yuukit
1
230
博士学位論文予備審査 / Scaling Telemetry Workloads in Cloud Applications: Techniques for Instrumentation, Storage, and Mining
yuukit
1
2k
MetricSifter:クラウドアプリケーションにおける故障箇所特定の効率化のための多変量時系列データの特徴量削減 / FIT 2024
yuukit
2
290
工学としてのSRE再訪 / Revisiting SRE as Engineering
yuukit
19
14k
Cloudless Computingの論文紹介
yuukit
2
570
#SRE論文紹介 Detection is Better Than Cure: A Cloud Incidents Perspective V. Ganatra et. al., ESEC/FSE’23
yuukit
3
2.2k
Other Decks in Technology
See All in Technology
ソフトウェア エンジニアとしての 姿勢と心構え
recruitengineers
PRO
1
120
Yahoo!ニュースにおけるソフトウェア開発
lycorptech_jp
PRO
0
300
AIドリブンのソフトウェア開発 - うまいやり方とまずいやり方
okdt
PRO
9
550
サイボウズフロントエンドの横断活動から考える AI時代にできること
mugi_uno
4
1.4k
広島発!スタートアップ開発の裏側
tsankyo
0
220
メルカリIBIS:AIが拓く次世代インシデント対応
0gm
2
520
Yahoo!広告ビジネス基盤におけるバックエンド開発
lycorptech_jp
PRO
1
250
広島銀行におけるAWS活用の取り組みについて
masakimori
0
120
プロジェクトマネジメントは不確実性との対話だ
hisashiwatanabe
0
200
Understanding Go GC #coefl_go_jp
bengo4com
0
1.1k
ZOZOTOWNフロントエンドにおけるディレクトリの分割戦略
zozotech
PRO
15
5k
開発と脆弱性と脆弱性診断についての話
su3158
1
1.1k
Featured
See All Featured
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.6k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
32
1.4k
The World Runs on Bad Software
bkeepers
PRO
70
11k
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Code Reviewing Like a Champion
maltzj
525
40k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
The Pragmatic Product Professional
lauravandoore
36
6.8k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
It's Worth the Effort
3n
187
28k
Building Applications with DynamoDB
mza
96
6.6k
The Cult of Friendly URLs
andyhume
79
6.5k
A better future with KSS
kneath
239
17k
Transcript
Docker Performance on Web Application JEZ@VVLJ %PDLFS.FFUVQ5PLZP
id:y_uuki ӡ༻ @y_uuk1
Docker Engine্Ͱɹ ΞϓϦέʔγϣϯΛɹ ಈ͔ͯ͠ੑೳྼԽ ͠ͳ͍ͷʁ
None
1 DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱͳʹ͔
Linux Containers
/FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF
1*% /BNFTQBDF
/FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF
1*% /BNFTQBDF -JOVYΧʔωϧͷ໊લۭؒػೳͷ ू·Γ
Linux Containers Χʔωϧͷ֤ϦιʔεΛ໊લۭؒͰִ͠ ۭͨؒͰϓϩηεΛ࣮ߦ͢Δͱ͍͏Ϟσϧ ϑΝΠϧγεςϜɼϢʔβɼϓϩηεςʔ ϒϧɼωοτϫʔΫ… etc LXC Linux
Containers ͷϑϩϯτΤϯυ
Linux Containers Overhead ୯ମͷLinux ΧʔωϧͰ݁͢ΔͷͰ ΦϒδΣΫτͷڞ༗ޮ͕Α͍ Hypervisor ͷΑ͏ʹ֤ϦιʔεΛɹɹ ೋॏॲཧ͠ͳͯ͘Α͍
example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ 4JOHMF ,FSOFM
example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ /*$ ,FSOFM ,FSOFM
$PQZ $PQZ 4JOHMF ,FSOFM )ZQFSWJTPS 9FO ,7. 6TFSMBOE $PQZ ˞ 43*07ͳͲ)ZQFSWJTPSͰύέοτίϐʔΦʔόϔουΛগͳ͘ ͢Δ࣮͋Δ
Linux Containers ͩͱ Single Kernel Ͱ͍
Docker Filesystem -JOVY$POUBJOFST
UNION Filesystem -JOVY$POUBJOFST %PDLFS'JMFTZTUFN
IUUQTEPDTEPDLFSDPNUFSNTJNBHFTEPDLFSpMFTZTUFNTNVMUJMBZFSQOH UNION Filesystem Image
8SJUF*0 3FBE*0 ADBUFUDEFCJBO@WFSTJPOA
8SJUF*0 3FBE*0 8SJUF*0࠷্ॻ͖ࠐΈ 3FBE*0֘ϑΝΠϧΛ ֨ೲ͢ΔΛ୳ࡧ
UNION Filesystem • AUFS, Btrfs, OverlayFS … ͳͲΛ࣮ͬͯݱ • Copy
On Write: ݪຊΛෳͨ͠;ΓΛͯ͠ ͓͍ͯ৽نॻ͖ࠐΈ͕͋ͬͨΒෳΛ࡞Δ • LinuxΧʔωϧඪ४ͷ Device MapperΛͬ ࣮ͯݱͰ͖Δ
Storage drivers -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN
EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT
EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT 6OJPO'4ͷ࣮ΛબͰ͖Δ
Storage drivers devicemapper, aufs, btrfs, overrayfs (vfs) Χʔωϧඪ४ͳͨΊRHELܥͰͷɹɹ σϑΥϧτ devicemapper
Device Mapper ϒϩοΫσόΠεͷI/Oʹ༷ʑͳมΛՃ͑Δ͜ͱ͕ Ͱ͖Δ (҉߸ԽɺετϥΠϓɺϛϥʔ ͳͲ) ϒϩοΫσόΠευϥΠόϨϕϧͷ࣮ͳͷͰɹɹɹɹ ಛఆͷϑΝΠϧγεςϜʹґଘ͠ͳ͍ LVMͳͲͰΘΕ͍ͯΔ docker
commit ͕ΔͱsnapshotσόΠεΛ࡞ͯ͠Π ϝʔδͱ͢Δ
Volume -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST
Volume ίϯςφؒͰσΟϨΫτϦΛڞ༗͢ΔͨΊͷͷ ίϯςφ͝ͱͰͳ͘DockerάϩʔόϧͳྖҬʹ֨ ೲ͞ΕΔ ࢦఆͨ͠σΟϨΫτϦҎԼͷΠϝʔδΛόΠύε͠ ͯΞΫηε I/Oཁٻ͕Union FS෦Λ௨Βͳ͍ͷͰΦʔόϔου ͕গͳ͍
howto/Volume EPDLFSSVOWWBSMJCNZTRMNZTRM 70-6.&WBSMJCNZTRM EPDLFSSVONZTRM %PDLFSpMF 4IFMM 4IFMM PS
Docker Network -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF
Portmapper -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL
Portmapper ίϯςφؒ௨৴ίϯςφɾϗετؒ௨৴ϗετଆ ͷ iptables ͰNAT ! iptables͕ͳ͍ڥͩͱಠࣗͷϢʔβϥϯυͷϓϩη ε(docker-proxy)Ͱίϯςφؒ௨৴ "%0$,&3JEPDLFSQUDQNUDQEQPSUK%/"5 UPEFTUJOBUJPO
EPDLFSQSPYZQSPUPUDQIPTUJQIPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSU
Host Networking -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS
Host Networking ίϯςφ༻ͷNetwork NamespaceΛ࡞Β ͣʹϗετͷωοτϫʔΫΛͦͷ··͏ ίϯςφϗετଆͷportΛlisten͢Δ iptablesdocker-proxyΛܦ༝͠ͳͯ͘Α ͘ͳΓɺΦʔόϔου͕খ͘͞ͳΔ
howto/Host Networking ! FYFDESJWFSOBUJWF PS -9$Ҏ্ͰFYFDESJWFSMYD ͕ඞཁ EPDLFSSVOOFUIPTUNZTRM
-JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS )PTU/FUXPSLJOH DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱͳʹ͔
2 DockerԽͨ͠ ISUCONΞϓϦέʔγϣϯ ͷϕϯνϚʔΫ
ISUCON *JLBOKJOJ4QFFE6Q$0/UFTU
/HJOY CFODI NBSLFS .Z42- "QQ .FNDBDIFE ISUCON4 ༧બ
Machine & Software Spec instance type: m3.xlarge CPU: Xeon E5-2670
v2 @ 2.50GHz 4 vCPU Memory: 16GB RAM Storage: Magnetic volume OS: Ubuntu 14.04 LTS Kernel 3.18.0 Docker: 1.4.1 (latest) MySQL: 5.5.40, memcached: 1.4.14, Nginx:1.4.6
νϡʔχϯά༰ • είΞ 38446 ( 3000 req/s) • ࢼߦ͝ͱʹ +-1000
είΞఔͷޡࠩ • ༧બಥഁϨϕϧ • σʔλશ෦ϝϞϦʹΔ • ηογϣϯใͳͲ memcached • Nginx Ͱ੩తϑΝΠϧΛฦ͢ • ωοτϫʔΫελοΫɺNginx, MySQL ී௨ͷ νϡʔχϯά
Evaluation
• Nginx ͱ MySQL ΛͦΕͧΕDockerԽ • ҎԼͷ֤ํࣜΛnative(default)ͱൺֱ • Nginx͚ͩDockerԽ •
—net=host ͱ —net=bridge • MySQL͚ͩDockerԽ • storage-driver=devicemapper ͱ storage-driver=overlayfs • Volume ͷ ON/OFF
Result
EFGBVMU /HJOY OFUCSJEHF /HJOY OFUIPTU
/HJOYʹύέοτ͕ू͢Δ /"15͢ΔΦʔόϔου͕ߴ͍
EFGBVMU .Z42- EFWJDFNBQQFS WPMVNFP⒎ .Z42- EFWJDFNBQQFS WPMVNFPO .Z42- PWFSMBZGT WPMVNFP⒎
.Z42- PWFSMBZGT WPMVNFPO
7PMVNFͷ0/0''ͱ TUPSBHFESJWFSʹΑΔ ੑೳྼԽͳ͠
7PMVNF0''ͰมΘΒͳ͍ 3FBE*0ϝϞϦʹશ෦ͷΔ 8SJUF*0࠷্͚ͩ
NAPTͷߴԽ
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j
DOCKER
-A OUTPUT E -m addrtype --dst-type LOCAL -j DOCKER
127.0.0.1Ѽͩͱiptablesͷ DockerνΣʔϯʹೖΒͳ͍
docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA
docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA CPU 50%
benchmarker bench --host 172.31.8.235 --init ~/init.sh --workload 8
32172 37931
ͳͥ userland ͷ proxy ͕͍Δͷ͔
IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
Hairpin NAT
)PTU $POUBJOFS EPDLFS -*45&/ FUI FUI MP FUI FUI
ETU MP FUI FUI )PTU $POUBJOFS EPDLFS -*45&/ FUI
FUI
ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS
-*45&/ FUI FUI
ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS
-*45&/ -*45&/ Ծbridge(docker0)Λ Hairpin NAT modeʹ͠ͳ͍ͱ NAT͞Εͳ͍
FDIPTZTDMBTTOFUCSCSJGFUIIBJSQJO@NPEF
IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
None
3)&-BOE$FOU04 SFBEPOMZTZT
1. DockerͷύϑΥʔϚϯεʹ͍ͭͯ ॏཁͳࣄͳʹ͔ 2. DockerԽͨ͠ISUCONΞϓϦέʔ γϣϯͷϕϯνϚʔΫ
Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker
Network Portmapper Host Networking
Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker
Network Portmapper (Performance issue) Host Networking
None
IUUQIBUFOBDPSQKQSFDSVJUDBSFFSTBMFTFOHJOFFS ηʔϧεΤϯδχΞ৬
! 8FCΦϖϨʔγϣϯΤϯδχΞ Λืू͓ͯ͠Γ·͢