Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Docker Performance on Web Application
Yuuki Tsubouchi (yuuk1)
January 17, 2015
Technology
7
130k
Docker Performance on Web Application
Docker Meetup Tokyo #4
Yuuki Tsubouchi (yuuk1)
January 17, 2015
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
AIOps研究録―SREのための システム障害の自動原因診断 / SRE NEXT 2022
yuukit
7
5k
Interactive AIOps
yuukit
0
850
Meltria: マイクロサービスにおける 異常検知・原因分析のための データセットの動的生成システム / Meltria in IOTS2021
yuukit
1
720
AIOpsの研究動向と AIOps向けデータセットの動的生成の研究 / Introducing AIOps and A Dynamic Datasets Generating System
yuukit
1
280
分散アプリケーションの高信頼化のための 運用技術に関する研究 / A Study on Operation Technology for High Reliability of Distributed Applications
yuukit
1
390
ネットワークサービスの依存発見に向いた TCP/UDP通信の低負荷なトレース手法 / Low Overhead TCP-UDP Tracing in Kernel
yuukit
5
2.9k
クラウドのシステム運用技術に機械学習を応用する研究 / CLOUD AI
yuukit
4
2.4k
TSifter: マイクロサービスにおける性能異常の迅速な診断に向いた時系列データの次元削減手法 / TSifter in proceedings of IOTS2020
yuukit
1
1.5k
TSifter: マイクロサービスにおける性能異常の 迅速な診断に向いた時系列データの次元削減手法
yuukit
0
98
Other Decks in Technology
See All in Technology
Data in Google I/O - IO Extended GDG Seoul
kennethanceyer
0
160
OCI Service Mesh 概要 / OCI Service Mesh ovewview
oracle4engineer
PRO
0
110
雑な攻撃からELBを守る一工夫 +おまけ / Know-how to protect servers from miscellaneous attacks
hiroga
0
700
HoloLens2とMetaQuest2どちらも動くWebXRアプリをBabylon.jsで作る
iwaken71
0
210
Persistence in Serverless Applications - ServerlessDays NYC
marcduiker
0
270
Building smarter apps with machine learning, from magic to reality
picardparis
4
3.2k
アーキテクチャを明文化して開発に臨んだ話
akkie76
0
360
Swift Regex Builder
kumamotone
1
110
情報の世界 2022年度 第11回「都市のデータ」 #情報の世界 / Data of City 2022
yumulab
0
110
【SAP知らない人向け】SAP on AWS 個人学習メモ/sap-on-aws-study
emiki
3
2.4k
データ分析基盤のはじめかた
chanyou0311
0
130
Autonomous Database Cloud 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
10
18k
Featured
See All Featured
The World Runs on Bad Software
bkeepers
PRO
57
5.3k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
39
13k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
15
950
The Invisible Side of Design
smashingmag
290
48k
How New CSS Is Changing Everything About Graphic Design on the Web
jensimmons
213
11k
What’s in a name? Adding method to the madness
productmarketing
11
1.6k
The Power of CSS Pseudo Elements
geoffreycrofte
47
3.9k
The Art of Programming - Codeland 2020
erikaheidi
32
11k
Reflections from 52 weeks, 52 projects
jeffersonlam
337
17k
GraphQLの誤解/rethinking-graphql
sonatard
28
6.6k
Docker and Python
trallard
27
1.6k
Testing 201, or: Great Expectations
jmmastey
21
5.4k
Transcript
Docker Performance on Web Application JEZ@VVLJ %PDLFS.FFUVQ5PLZP
id:y_uuki ӡ༻ @y_uuk1
Docker Engine্Ͱɹ ΞϓϦέʔγϣϯΛɹ ಈ͔ͯ͠ੑೳྼԽ ͠ͳ͍ͷʁ
None
1 DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱͳʹ͔
Linux Containers
/FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF
1*% /BNFTQBDF
/FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF
1*% /BNFTQBDF -JOVYΧʔωϧͷ໊લۭؒػೳͷ ू·Γ
Linux Containers Χʔωϧͷ֤ϦιʔεΛ໊લۭؒͰִ͠ ۭͨؒͰϓϩηεΛ࣮ߦ͢Δͱ͍͏Ϟσϧ ϑΝΠϧγεςϜɼϢʔβɼϓϩηεςʔ ϒϧɼωοτϫʔΫ… etc LXC Linux
Containers ͷϑϩϯτΤϯυ
Linux Containers Overhead ୯ମͷLinux ΧʔωϧͰ݁͢ΔͷͰ ΦϒδΣΫτͷڞ༗ޮ͕Α͍ Hypervisor ͷΑ͏ʹ֤ϦιʔεΛɹɹ ೋॏॲཧ͠ͳͯ͘Α͍
example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ 4JOHMF ,FSOFM
example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ /*$ ,FSOFM ,FSOFM
$PQZ $PQZ 4JOHMF ,FSOFM )ZQFSWJTPS 9FO ,7. 6TFSMBOE $PQZ ˞ 43*07ͳͲ)ZQFSWJTPSͰύέοτίϐʔΦʔόϔουΛগͳ͘ ͢Δ࣮͋Δ
Linux Containers ͩͱ Single Kernel Ͱ͍
Docker Filesystem -JOVY$POUBJOFST
UNION Filesystem -JOVY$POUBJOFST %PDLFS'JMFTZTUFN
IUUQTEPDTEPDLFSDPNUFSNTJNBHFTEPDLFSpMFTZTUFNTNVMUJMBZFSQOH UNION Filesystem Image
8SJUF*0 3FBE*0 ADBUFUDEFCJBO@WFSTJPOA
8SJUF*0 3FBE*0 8SJUF*0࠷্ॻ͖ࠐΈ 3FBE*0֘ϑΝΠϧΛ ֨ೲ͢ΔΛ୳ࡧ
UNION Filesystem • AUFS, Btrfs, OverlayFS … ͳͲΛ࣮ͬͯݱ • Copy
On Write: ݪຊΛෳͨ͠;ΓΛͯ͠ ͓͍ͯ৽نॻ͖ࠐΈ͕͋ͬͨΒෳΛ࡞Δ • LinuxΧʔωϧඪ४ͷ Device MapperΛͬ ࣮ͯݱͰ͖Δ
Storage drivers -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN
EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT
EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT 6OJPO'4ͷ࣮ΛબͰ͖Δ
Storage drivers devicemapper, aufs, btrfs, overrayfs (vfs) Χʔωϧඪ४ͳͨΊRHELܥͰͷɹɹ σϑΥϧτ devicemapper
Device Mapper ϒϩοΫσόΠεͷI/Oʹ༷ʑͳมΛՃ͑Δ͜ͱ͕ Ͱ͖Δ (҉߸ԽɺετϥΠϓɺϛϥʔ ͳͲ) ϒϩοΫσόΠευϥΠόϨϕϧͷ࣮ͳͷͰɹɹɹɹ ಛఆͷϑΝΠϧγεςϜʹґଘ͠ͳ͍ LVMͳͲͰΘΕ͍ͯΔ docker
commit ͕ΔͱsnapshotσόΠεΛ࡞ͯ͠Π ϝʔδͱ͢Δ
Volume -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST
Volume ίϯςφؒͰσΟϨΫτϦΛڞ༗͢ΔͨΊͷͷ ίϯςφ͝ͱͰͳ͘DockerάϩʔόϧͳྖҬʹ֨ ೲ͞ΕΔ ࢦఆͨ͠σΟϨΫτϦҎԼͷΠϝʔδΛόΠύε͠ ͯΞΫηε I/Oཁٻ͕Union FS෦Λ௨Βͳ͍ͷͰΦʔόϔου ͕গͳ͍
howto/Volume EPDLFSSVOWWBSMJCNZTRMNZTRM 70-6.&WBSMJCNZTRM EPDLFSSVONZTRM %PDLFSpMF 4IFMM 4IFMM PS
Docker Network -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF
Portmapper -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL
Portmapper ίϯςφؒ௨৴ίϯςφɾϗετؒ௨৴ϗετଆ ͷ iptables ͰNAT ! iptables͕ͳ͍ڥͩͱಠࣗͷϢʔβϥϯυͷϓϩη ε(docker-proxy)Ͱίϯςφؒ௨৴ "%0$,&3JEPDLFSQUDQNUDQEQPSUK%/"5 UPEFTUJOBUJPO
EPDLFSQSPYZQSPUPUDQIPTUJQIPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSU
Host Networking -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS
Host Networking ίϯςφ༻ͷNetwork NamespaceΛ࡞Β ͣʹϗετͷωοτϫʔΫΛͦͷ··͏ ίϯςφϗετଆͷportΛlisten͢Δ iptablesdocker-proxyΛܦ༝͠ͳͯ͘Α ͘ͳΓɺΦʔόϔου͕খ͘͞ͳΔ
howto/Host Networking ! FYFDESJWFSOBUJWF PS -9$Ҏ্ͰFYFDESJWFSMYD ͕ඞཁ EPDLFSSVOOFUIPTUNZTRM
-JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS )PTU/FUXPSLJOH DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱͳʹ͔
2 DockerԽͨ͠ ISUCONΞϓϦέʔγϣϯ ͷϕϯνϚʔΫ
ISUCON *JLBOKJOJ4QFFE6Q$0/UFTU
/HJOY CFODI NBSLFS .Z42- "QQ .FNDBDIFE ISUCON4 ༧બ
Machine & Software Spec instance type: m3.xlarge CPU: Xeon E5-2670
v2 @ 2.50GHz 4 vCPU Memory: 16GB RAM Storage: Magnetic volume OS: Ubuntu 14.04 LTS Kernel 3.18.0 Docker: 1.4.1 (latest) MySQL: 5.5.40, memcached: 1.4.14, Nginx:1.4.6
νϡʔχϯά༰ • είΞ 38446 ( 3000 req/s) • ࢼߦ͝ͱʹ +-1000
είΞఔͷޡࠩ • ༧બಥഁϨϕϧ • σʔλશ෦ϝϞϦʹΔ • ηογϣϯใͳͲ memcached • Nginx Ͱ੩తϑΝΠϧΛฦ͢ • ωοτϫʔΫελοΫɺNginx, MySQL ී௨ͷ νϡʔχϯά
Evaluation
• Nginx ͱ MySQL ΛͦΕͧΕDockerԽ • ҎԼͷ֤ํࣜΛnative(default)ͱൺֱ • Nginx͚ͩDockerԽ •
—net=host ͱ —net=bridge • MySQL͚ͩDockerԽ • storage-driver=devicemapper ͱ storage-driver=overlayfs • Volume ͷ ON/OFF
Result
EFGBVMU /HJOY OFUCSJEHF /HJOY OFUIPTU
/HJOYʹύέοτ͕ू͢Δ /"15͢ΔΦʔόϔου͕ߴ͍
EFGBVMU .Z42- EFWJDFNBQQFS WPMVNFP⒎ .Z42- EFWJDFNBQQFS WPMVNFPO .Z42- PWFSMBZGT WPMVNFP⒎
.Z42- PWFSMBZGT WPMVNFPO
7PMVNFͷ0/0''ͱ TUPSBHFESJWFSʹΑΔ ੑೳྼԽͳ͠
7PMVNF0''ͰมΘΒͳ͍ 3FBE*0ϝϞϦʹશ෦ͷΔ 8SJUF*0࠷্͚ͩ
NAPTͷߴԽ
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j
DOCKER
-A OUTPUT E -m addrtype --dst-type LOCAL -j DOCKER
127.0.0.1Ѽͩͱiptablesͷ DockerνΣʔϯʹೖΒͳ͍
docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA
docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA CPU 50%
benchmarker bench --host 172.31.8.235 --init ~/init.sh --workload 8
32172 37931
ͳͥ userland ͷ proxy ͕͍Δͷ͔
IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
Hairpin NAT
)PTU $POUBJOFS EPDLFS -*45&/ FUI FUI MP FUI FUI
ETU MP FUI FUI )PTU $POUBJOFS EPDLFS -*45&/ FUI
FUI
ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS
-*45&/ FUI FUI
ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS
-*45&/ -*45&/ Ծbridge(docker0)Λ Hairpin NAT modeʹ͠ͳ͍ͱ NAT͞Εͳ͍
FDIPTZTDMBTTOFUCSCSJGFUIIBJSQJO@NPEF
IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
None
3)&-BOE$FOU04 SFBEPOMZTZT
1. DockerͷύϑΥʔϚϯεʹ͍ͭͯ ॏཁͳࣄͳʹ͔ 2. DockerԽͨ͠ISUCONΞϓϦέʔ γϣϯͷϕϯνϚʔΫ
Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker
Network Portmapper Host Networking
Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker
Network Portmapper (Performance issue) Host Networking
None
IUUQIBUFOBDPSQKQSFDSVJUDBSFFSTBMFTFOHJOFFS ηʔϧεΤϯδχΞ৬
! 8FCΦϖϨʔγϣϯΤϯδχΞ Λืू͓ͯ͠Γ·͢