Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Docker Performance on Web Application
Search
Yuuki Tsubouchi (yuuk1)
January 17, 2015
Technology
7
130k
Docker Performance on Web Application
Docker Meetup Tokyo #4
Yuuki Tsubouchi (yuuk1)
January 17, 2015
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
Cloudless Computingの論文紹介
yuukit
1
300
#SRE論文紹介 Detection is Better Than Cure: A Cloud Incidents Perspective V. Ganatra et. al., ESEC/FSE’23
yuukit
3
950
エンジニアのためのSRE論文への招待 / Introduction to SRE Papers for Engineers
yuukit
1
10k
博士課程での研究まとめ 2023年1月版 / Summary of my research in the PhD course
yuukit
1
200
AI時代に向けたクラウドにおける信頼性エンジニアリングの未来構想 / DICOMO2022 6A-1
yuukit
7
2.7k
AIOps研究録―SREのための システム障害の自動原因診断 / SRE NEXT 2022
yuukit
10
12k
Interactive AIOps
yuukit
0
1.9k
Meltria: マイクロサービスにおける 異常検知・原因分析のための データセットの動的生成システム / Meltria in IOTS2021
yuukit
1
1.5k
AIOpsの研究動向と AIOps向けデータセットの動的生成の研究 / Introducing AIOps and A Dynamic Datasets Generating System
yuukit
1
690
Other Decks in Technology
See All in Technology
Git 研修 Basic【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
310
ゆめみのアクセシビリティの現在地と今後
ryokatsuse
3
290
セキュリティ研修 Day1【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
160
AIアシスタントの活用で品質の向上と開発ワークフローのスピードアップ
nagix
1
200
開発生産性をむしろ向上させる セキュリティパートナーの作り方 / Dev Productivity Con 2024
flatt_security
0
370
コンテナ・K8s研修 - 前半 コンテナ基礎・ハンズオン【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
170
AWSで”最小権限の原則”を実現するための考え方 /20240722-ssmjp-aws-least-privilege
opelab
10
4.3k
Azure AI ことはじめ
tsubakimoto_s
0
130
エンジニア向け会社紹介資料
caddi_eng
14
220k
エンジニアリングマネージャーはどう学んでいくのか #devsumi / How Do Engineering Managers Continue to Learn and Grow?
expajp
4
1.3k
What if...? 처음부터 다시 LLM 어플리케이션을 개발한다면
huffon
0
1k
AI研修【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
130
Featured
See All Featured
Designing on Purpose - Digital PM Summit 2013
jponch
113
6.6k
How To Stay Up To Date on Web Technology
chriscoyier
784
250k
Building Adaptive Systems
keathley
34
2k
Producing Creativity
orderedlist
PRO
340
39k
Principles of Awesome APIs and How to Build Them.
keavy
124
16k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
248
20k
What's new in Ruby 2.0
geeforr
338
31k
Code Review Best Practice
trishagee
58
16k
Debugging Ruby Performance
tmm1
71
11k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
35
6.3k
Automating Front-end Workflow
addyosmani
1362
200k
Facilitating Awesome Meetings
lara
46
5.8k
Transcript
Docker Performance on Web Application JEZ@VVLJ %PDLFS.FFUVQ5PLZP
id:y_uuki ӡ༻ @y_uuk1
Docker Engine্Ͱɹ ΞϓϦέʔγϣϯΛɹ ಈ͔ͯ͠ੑೳྼԽ ͠ͳ͍ͷʁ
None
1 DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱͳʹ͔
Linux Containers
/FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF
1*% /BNFTQBDF
/FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF
1*% /BNFTQBDF -JOVYΧʔωϧͷ໊લۭؒػೳͷ ू·Γ
Linux Containers Χʔωϧͷ֤ϦιʔεΛ໊લۭؒͰִ͠ ۭͨؒͰϓϩηεΛ࣮ߦ͢Δͱ͍͏Ϟσϧ ϑΝΠϧγεςϜɼϢʔβɼϓϩηεςʔ ϒϧɼωοτϫʔΫ… etc LXC Linux
Containers ͷϑϩϯτΤϯυ
Linux Containers Overhead ୯ମͷLinux ΧʔωϧͰ݁͢ΔͷͰ ΦϒδΣΫτͷڞ༗ޮ͕Α͍ Hypervisor ͷΑ͏ʹ֤ϦιʔεΛɹɹ ೋॏॲཧ͠ͳͯ͘Α͍
example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ 4JOHMF ,FSOFM
example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ /*$ ,FSOFM ,FSOFM
$PQZ $PQZ 4JOHMF ,FSOFM )ZQFSWJTPS 9FO ,7. 6TFSMBOE $PQZ ˞ 43*07ͳͲ)ZQFSWJTPSͰύέοτίϐʔΦʔόϔουΛগͳ͘ ͢Δ࣮͋Δ
Linux Containers ͩͱ Single Kernel Ͱ͍
Docker Filesystem -JOVY$POUBJOFST
UNION Filesystem -JOVY$POUBJOFST %PDLFS'JMFTZTUFN
IUUQTEPDTEPDLFSDPNUFSNTJNBHFTEPDLFSpMFTZTUFNTNVMUJMBZFSQOH UNION Filesystem Image
8SJUF*0 3FBE*0 ADBUFUDEFCJBO@WFSTJPOA
8SJUF*0 3FBE*0 8SJUF*0࠷্ॻ͖ࠐΈ 3FBE*0֘ϑΝΠϧΛ ֨ೲ͢ΔΛ୳ࡧ
UNION Filesystem • AUFS, Btrfs, OverlayFS … ͳͲΛ࣮ͬͯݱ • Copy
On Write: ݪຊΛෳͨ͠;ΓΛͯ͠ ͓͍ͯ৽نॻ͖ࠐΈ͕͋ͬͨΒෳΛ࡞Δ • LinuxΧʔωϧඪ४ͷ Device MapperΛͬ ࣮ͯݱͰ͖Δ
Storage drivers -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN
EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT
EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT 6OJPO'4ͷ࣮ΛબͰ͖Δ
Storage drivers devicemapper, aufs, btrfs, overrayfs (vfs) Χʔωϧඪ४ͳͨΊRHELܥͰͷɹɹ σϑΥϧτ devicemapper
Device Mapper ϒϩοΫσόΠεͷI/Oʹ༷ʑͳมΛՃ͑Δ͜ͱ͕ Ͱ͖Δ (҉߸ԽɺετϥΠϓɺϛϥʔ ͳͲ) ϒϩοΫσόΠευϥΠόϨϕϧͷ࣮ͳͷͰɹɹɹɹ ಛఆͷϑΝΠϧγεςϜʹґଘ͠ͳ͍ LVMͳͲͰΘΕ͍ͯΔ docker
commit ͕ΔͱsnapshotσόΠεΛ࡞ͯ͠Π ϝʔδͱ͢Δ
Volume -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST
Volume ίϯςφؒͰσΟϨΫτϦΛڞ༗͢ΔͨΊͷͷ ίϯςφ͝ͱͰͳ͘DockerάϩʔόϧͳྖҬʹ֨ ೲ͞ΕΔ ࢦఆͨ͠σΟϨΫτϦҎԼͷΠϝʔδΛόΠύε͠ ͯΞΫηε I/Oཁٻ͕Union FS෦Λ௨Βͳ͍ͷͰΦʔόϔου ͕গͳ͍
howto/Volume EPDLFSSVOWWBSMJCNZTRMNZTRM 70-6.&WBSMJCNZTRM EPDLFSSVONZTRM %PDLFSpMF 4IFMM 4IFMM PS
Docker Network -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF
Portmapper -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL
Portmapper ίϯςφؒ௨৴ίϯςφɾϗετؒ௨৴ϗετଆ ͷ iptables ͰNAT ! iptables͕ͳ͍ڥͩͱಠࣗͷϢʔβϥϯυͷϓϩη ε(docker-proxy)Ͱίϯςφؒ௨৴ "%0$,&3JEPDLFSQUDQNUDQEQPSUK%/"5 UPEFTUJOBUJPO
EPDLFSQSPYZQSPUPUDQIPTUJQIPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSU
Host Networking -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS
Host Networking ίϯςφ༻ͷNetwork NamespaceΛ࡞Β ͣʹϗετͷωοτϫʔΫΛͦͷ··͏ ίϯςφϗετଆͷportΛlisten͢Δ iptablesdocker-proxyΛܦ༝͠ͳͯ͘Α ͘ͳΓɺΦʔόϔου͕খ͘͞ͳΔ
howto/Host Networking ! FYFDESJWFSOBUJWF PS -9$Ҏ্ͰFYFDESJWFSMYD ͕ඞཁ EPDLFSSVOOFUIPTUNZTRM
-JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS )PTU/FUXPSLJOH DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱͳʹ͔
2 DockerԽͨ͠ ISUCONΞϓϦέʔγϣϯ ͷϕϯνϚʔΫ
ISUCON *JLBOKJOJ4QFFE6Q$0/UFTU
/HJOY CFODI NBSLFS .Z42- "QQ .FNDBDIFE ISUCON4 ༧બ
Machine & Software Spec instance type: m3.xlarge CPU: Xeon E5-2670
v2 @ 2.50GHz 4 vCPU Memory: 16GB RAM Storage: Magnetic volume OS: Ubuntu 14.04 LTS Kernel 3.18.0 Docker: 1.4.1 (latest) MySQL: 5.5.40, memcached: 1.4.14, Nginx:1.4.6
νϡʔχϯά༰ • είΞ 38446 ( 3000 req/s) • ࢼߦ͝ͱʹ +-1000
είΞఔͷޡࠩ • ༧બಥഁϨϕϧ • σʔλશ෦ϝϞϦʹΔ • ηογϣϯใͳͲ memcached • Nginx Ͱ੩తϑΝΠϧΛฦ͢ • ωοτϫʔΫελοΫɺNginx, MySQL ී௨ͷ νϡʔχϯά
Evaluation
• Nginx ͱ MySQL ΛͦΕͧΕDockerԽ • ҎԼͷ֤ํࣜΛnative(default)ͱൺֱ • Nginx͚ͩDockerԽ •
—net=host ͱ —net=bridge • MySQL͚ͩDockerԽ • storage-driver=devicemapper ͱ storage-driver=overlayfs • Volume ͷ ON/OFF
Result
EFGBVMU /HJOY OFUCSJEHF /HJOY OFUIPTU
/HJOYʹύέοτ͕ू͢Δ /"15͢ΔΦʔόϔου͕ߴ͍
EFGBVMU .Z42- EFWJDFNBQQFS WPMVNFP⒎ .Z42- EFWJDFNBQQFS WPMVNFPO .Z42- PWFSMBZGT WPMVNFP⒎
.Z42- PWFSMBZGT WPMVNFPO
7PMVNFͷ0/0''ͱ TUPSBHFESJWFSʹΑΔ ੑೳྼԽͳ͠
7PMVNF0''ͰมΘΒͳ͍ 3FBE*0ϝϞϦʹશ෦ͷΔ 8SJUF*0࠷্͚ͩ
NAPTͷߴԽ
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j
DOCKER
-A OUTPUT E -m addrtype --dst-type LOCAL -j DOCKER
127.0.0.1Ѽͩͱiptablesͷ DockerνΣʔϯʹೖΒͳ͍
docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA
docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA CPU 50%
benchmarker bench --host 172.31.8.235 --init ~/init.sh --workload 8
32172 37931
ͳͥ userland ͷ proxy ͕͍Δͷ͔
IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
Hairpin NAT
)PTU $POUBJOFS EPDLFS -*45&/ FUI FUI MP FUI FUI
ETU MP FUI FUI )PTU $POUBJOFS EPDLFS -*45&/ FUI
FUI
ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS
-*45&/ FUI FUI
ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS
-*45&/ -*45&/ Ծbridge(docker0)Λ Hairpin NAT modeʹ͠ͳ͍ͱ NAT͞Εͳ͍
FDIPTZTDMBTTOFUCSCSJGFUIIBJSQJO@NPEF
IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
None
3)&-BOE$FOU04 SFBEPOMZTZT
1. DockerͷύϑΥʔϚϯεʹ͍ͭͯ ॏཁͳࣄͳʹ͔ 2. DockerԽͨ͠ISUCONΞϓϦέʔ γϣϯͷϕϯνϚʔΫ
Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker
Network Portmapper Host Networking
Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker
Network Portmapper (Performance issue) Host Networking
None
IUUQIBUFOBDPSQKQSFDSVJUDBSFFSTBMFTFOHJOFFS ηʔϧεΤϯδχΞ৬
! 8FCΦϖϨʔγϣϯΤϯδχΞ Λืू͓ͯ͠Γ·͢