Docker Meetup Tokyo #4
Docker Performanceon Web ApplicationJEZ@VVLJ%PDLFS.FFUVQ5PLZP
View Slide
id:y_uukiӡ༻@y_uuk1
Docker Engine্ͰɹΞϓϦέʔγϣϯΛɹಈ͔ͯ͠ੑೳྼԽ͠ͳ͍ͷʁ
1DockerͷύϑΥʔϚϯεʹ͓͍ͯॏཁͳ͜ͱͳʹ͔
Linux Containers
/FUXPSL/BNFTQBDF*1$/BNFTQBDF654/BNFTQBDF.PVOU/BNFTQBDF6TFS/BNFTQBDF1*%/BNFTQBDF
/FUXPSL/BNFTQBDF*1$/BNFTQBDF654/BNFTQBDF.PVOU/BNFTQBDF6TFS/BNFTQBDF1*%/BNFTQBDF-JOVYΧʔωϧͷ໊લۭؒػೳͷू·Γ
Linux ContainersΧʔωϧͷ֤ϦιʔεΛ໊લۭؒͰִۭͨؒ͠ͰϓϩηεΛ࣮ߦ͢Δͱ͍͏ϞσϧϑΝΠϧγεςϜɼϢʔβɼϓϩηεςʔϒϧɼωοτϫʔΫ… etcLXC Linux Containers ͷϑϩϯτΤϯυ
Linux Containers Overhead୯ମͷLinux ΧʔωϧͰ݁͢ΔͷͰΦϒδΣΫτͷڞ༗ޮ͕Α͍Hypervisor ͷΑ͏ʹ֤ϦιʔεΛɹɹೋॏॲཧ͠ͳͯ͘Α͍
example/packet receive/*$ ,FSOFM 6TFSMBOE$PQZ $PQZ4JOHMF,FSOFM
example/packet receive/*$ ,FSOFM 6TFSMBOE$PQZ $PQZ/*$ ,FSOFM ,FSOFM$PQZ $PQZ4JOHMF,FSOFM)ZQFSWJTPS9FO ,7. 6TFSMBOE$PQZ˞ 43*07ͳͲ)ZQFSWJTPSͰύέοτίϐʔΦʔόϔουΛগͳ͘͢Δ࣮͋Δ
Linux Containers ͩͱSingle Kernel Ͱ͍
Docker Filesystem-JOVY$POUBJOFST
UNION Filesystem-JOVY$POUBJOFST%PDLFS'JMFTZTUFN
IUUQTEPDTEPDLFSDPNUFSNTJNBHFTEPDLFSpMFTZTUFNTNVMUJMBZFSQOHUNION Filesystem Image
8SJUF*0 3FBE*0 ADBUFUDEFCJBO@WFSTJPOA
8SJUF*0 3FBE*08SJUF*0࠷্ॻ͖ࠐΈ3FBE*0֘ϑΝΠϧΛ֨ೲ͢ΔΛ୳ࡧ
UNION Filesystem• AUFS, Btrfs, OverlayFS … ͳͲΛ࣮ͬͯݱ• Copy On Write: ݪຊΛෳͨ͠;ΓΛ͓͍ͯͯ͠৽نॻ͖ࠐΈ͕͋ͬͨΒෳΛ࡞Δ• LinuxΧʔωϧඪ४ͷ Device MapperΛ࣮ͬͯݱͰ͖Δ
Storage drivers-JOVY$POUBJOFST%PDLFS'JMFTZTUFN6/*0/'JMFTZTUFN
EFWJDFNBQQFSBVGTCUSGTPWFSMBZGT
EFWJDFNBQQFSBVGTCUSGTPWFSMBZGT6OJPO'4ͷ࣮ΛબͰ͖Δ
Storage driversdevicemapper, aufs, btrfs, overrayfs(vfs)Χʔωϧඪ४ͳͨΊRHELܥͰͷɹɹσϑΥϧτ devicemapper
Device MapperϒϩοΫσόΠεͷI/Oʹ༷ʑͳมΛՃ͑Δ͜ͱ͕Ͱ͖Δ (҉߸ԽɺετϥΠϓɺϛϥʔ ͳͲ)ϒϩοΫσόΠευϥΠόϨϕϧͷ࣮ͳͷͰɹɹɹɹಛఆͷϑΝΠϧγεςϜʹґଘ͠ͳ͍LVMͳͲͰΘΕ͍ͯΔdocker commit ͕ΔͱsnapshotσόΠεΛ࡞ͯ͠Πϝʔδͱ͢Δ
Volume-JOVY$POUBJOFST%PDLFS'JMFTZTUFN6/*0/'JMFTZTUFN4UPSBHF%SJWFST
VolumeίϯςφؒͰσΟϨΫτϦΛڞ༗͢ΔͨΊͷͷίϯςφ͝ͱͰͳ͘DockerάϩʔόϧͳྖҬʹ֨ೲ͞ΕΔࢦఆͨ͠σΟϨΫτϦҎԼͷΠϝʔδΛόΠύεͯ͠ΞΫηεI/Oཁٻ͕Union FS෦Λ௨Βͳ͍ͷͰΦʔόϔου͕গͳ͍
howto/VolumeEPDLFSSVOWWBSMJCNZTRMNZTRM70-6.&WBSMJCNZTRMEPDLFSSVONZTRM%PDLFSpMF4IFMM4IFMMPS
Docker Network-JOVY$POUBJOFST%PDLFS'JMFTZTUFN6/*0/'JMFTZTUFN4UPSBHF%SJWFST7PMVNF
Portmapper-JOVY$POUBJOFST%PDLFS'JMFTZTUFN6/*0/'JMFTZTUFN4UPSBHF%SJWFST7PMVNF%PDLFS/FUXPSL
Portmapperίϯςφؒ௨৴ίϯςφɾϗετؒ௨৴ϗετଆͷ iptables ͰNAT!iptables͕ͳ͍ڥͩͱಠࣗͷϢʔβϥϯυͷϓϩηε(docker-proxy)Ͱίϯςφؒ௨৴"%0$,&3JEPDLFSQUDQNUDQEQPSUK%/"5UPEFTUJOBUJPOEPDLFSQSPYZQSPUPUDQIPTUJQIPTUQPSUDPOUBJOFSJQDPOUBJOFSQPSU
Host Networking-JOVY$POUBJOFST%PDLFS'JMFTZTUFN6/*0/'JMFTZTUFN4UPSBHF%SJWFST7PMVNF%PDLFS/FUXPSL1PSUNBQQFS
Host Networkingίϯςφ༻ͷNetwork NamespaceΛ࡞ΒͣʹϗετͷωοτϫʔΫΛͦͷ··͏ίϯςφϗετଆͷportΛlisten͢Δiptablesdocker-proxyΛܦ༝͠ͳͯ͘Α͘ͳΓɺΦʔόϔου͕খ͘͞ͳΔ
howto/Host Networking!FYFDESJWFSOBUJWFPS-9$Ҏ্ͰFYFDESJWFSMYD͕ඞཁEPDLFSSVOOFUIPTUNZTRM
-JOVY$POUBJOFST%PDLFS'JMFTZTUFN6/*0/'JMFTZTUFN4UPSBHF%SJWFST7PMVNF%PDLFS/FUXPSL1PSUNBQQFS)PTU/FUXPSLJOHDockerͷύϑΥʔϚϯεʹ͓͍ͯॏཁͳ͜ͱͳʹ͔
2DockerԽͨ͠ISUCONΞϓϦέʔγϣϯͷϕϯνϚʔΫ
ISUCON*JLBOKJOJ4QFFE6Q$0/UFTU
/HJOYCFODINBSLFS.Z42-"QQ.FNDBDIFEISUCON4 ༧બ
Machine & Software Specinstance type: m3.xlargeCPU: Xeon E5-2670 v2 @ 2.50GHz 4 vCPUMemory: 16GB RAMStorage: Magnetic volumeOS: Ubuntu 14.04 LTS Kernel 3.18.0Docker: 1.4.1 (latest)MySQL: 5.5.40, memcached: 1.4.14, Nginx:1.4.6
νϡʔχϯά༰• είΞ 38446 ( 3000 req/s)• ࢼߦ͝ͱʹ +-1000 είΞఔͷޡࠩ• ༧બಥഁϨϕϧ• σʔλશ෦ϝϞϦʹΔ• ηογϣϯใͳͲ memcached• Nginx Ͱ੩తϑΝΠϧΛฦ͢• ωοτϫʔΫελοΫɺNginx, MySQL ී௨ͷνϡʔχϯά
Evaluation
• Nginx ͱ MySQL ΛͦΕͧΕDockerԽ• ҎԼͷ֤ํࣜΛnative(default)ͱൺֱ• Nginx͚ͩDockerԽ• —net=host ͱ —net=bridge• MySQL͚ͩDockerԽ• storage-driver=devicemapper ͱstorage-driver=overlayfs• Volume ͷ ON/OFF
Result
EFGBVMU/HJOYOFUCSJEHF/HJOYOFUIPTU
/HJOYʹύέοτ͕ू͢Δ/"15͢ΔΦʔόϔου͕ߴ͍
EFGBVMU.Z42-EFWJDFNBQQFSWPMVNFP⒎.Z42-EFWJDFNBQQFSWPMVNFPO.Z42-PWFSMBZGTWPMVNFP⒎.Z42-PWFSMBZGTWPMVNFPO
7PMVNFͷ0/0''ͱTUPSBHFESJWFSʹΑΔੑೳྼԽͳ͠
7PMVNF0''ͰมΘΒͳ͍3FBE*0ϝϞϦʹશ෦ͷΔ8SJUF*0࠷্͚ͩ
NAPTͷߴԽ
-A OUTPUT ! -d 127.0.0.0/8-m addrtype --dst-type LOCAL-j DOCKER
-A OUTPUT E-m addrtype --dst-type LOCAL-j DOCKER
127.0.0.1ѼͩͱiptablesͷDockerνΣʔϯʹೖΒͳ͍
docker-proxyEPDLFSQSPYZQSPUPUDQIPTUJQIPTUQPSUDPOUBJOFSJQDPOUBJOFSQPSUA
docker-proxyEPDLFSQSPYZQSPUPUDQIPTUJQIPTUQPSUDPOUBJOFSJQDPOUBJOFSQPSUACPU 50%
benchmarker bench--host 172.31.8.235--init ~/init.sh--workload 8
32172 37931
ͳͥ userland ͷproxy ͕͍Δͷ͔
IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
Hairpin NAT
)PTU $POUBJOFSEPDLFS-*45&/FUIFUIMPFUIFUI
ETUMPFUIFUI)PTU $POUBJOFSEPDLFS-*45&/FUIFUI
ETU"%0$,&3JEPDLFSQUDQNUDQEQPSUK%/"5UPEFTUJOBUJPO)PTU $POUBJOFSMPFUIFUIEPDLFS-*45&/FUIFUI
ETU"%0$,&3JEPDLFSQUDQNUDQEQPSUK%/"5UPEFTUJOBUJPO)PTU $POUBJOFSMPFUIFUIEPDLFS-*45&/-*45&/Ծbridge(docker0)ΛHairpin NAT modeʹ͠ͳ͍ͱNAT͞Εͳ͍
FDIPTZTDMBTTOFUCSCSJGFUIIBJSQJO@NPEF
3)&-BOE$FOU04SFBEPOMZTZT
1. DockerͷύϑΥʔϚϯεʹ͍ͭͯॏཁͳࣄͳʹ͔2. DockerԽͨ͠ISUCONΞϓϦέʔγϣϯͷϕϯνϚʔΫ
Linux ContainersDocker FilesystemUNION FilesystemStorage driversVolumeDocker NetworkPortmapperHost Networking
Linux ContainersDocker FilesystemUNION FilesystemStorage driversVolumeDocker NetworkPortmapper (Performance issue)Host Networking
IUUQIBUFOBDPSQKQSFDSVJUDBSFFSTBMFTFOHJOFFSηʔϧεΤϯδχΞ৬
!8FCΦϖϨʔγϣϯΤϯδχΞΛืू͓ͯ͠Γ·͢
yuukit
miu_crescent
daikimori
whywaita
k_koheyi
ryomaru0825
gl_tsukasa
coe
mashiike
kohbis
andpad
kobayang
yamahiro
sferik
chriscoyier
phodgson
stephaniewalter
smashingmag
samlambert
hatefulcrawdad
orderedlist
mongodb
denniskardys
kastner
productmarketing