Docker Performance on Web Application

Docker Meetup Tokyo #4

Transcript

1. Docker Performance on Web Application JEZ@VVLJ %PDLFS
   .FFUVQ
   5PLZP
   
2. id:y_uuki ӡ༻ @y_uuk1
3. Docker Engine্Ͱɹ ΞϓϦέʔγϣϯΛɹ ಈ͔ͯ͠ੑೳྼԽ ͠ͳ͍ͷʁ
5. 1 DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱ͸ͳʹ͔
6. Linux Containers
7. /FUXPSL
   /BNFTQBDF *1$
   /BNFTQBDF 654
   /BNFTQBDF .PVOU
   /BNFTQBDF 6TFS
   /BNFTQBDF 1*%
   /BNFTQBDF
8. /FUXPSL
   /BNFTQBDF *1$
   /BNFTQBDF 654
   /BNFTQBDF .PVOU
   /BNFTQBDF 6TFS
   /BNFTQBDF 1*%
   /BNFTQBDF -JOVYΧʔωϧͷ໊લۭؒػೳͷ
   ू·Γ
9. Linux Containers Χʔωϧͷ֤ϦιʔεΛ໊લۭؒͰִ཭͠ ۭͨؒͰϓϩηεΛ࣮ߦ͢Δͱ͍͏Ϟσϧ ϑΝΠϧγεςϜɼϢʔβɼϓϩηεςʔ ϒϧɼωοτϫʔΫ… etc LXC ͸ Linux Containers ͷϑϩϯτΤϯυ
10. Linux Containers Overhead ୯ମͷLinux ΧʔωϧͰ׬݁͢ΔͷͰ ΦϒδΣΫτͷڞ༗ޮ཰͕Α͍ Hypervisor ͷΑ͏ʹ֤ϦιʔεΛɹɹ ೋॏॲཧ͠ͳͯ͘Α͍
11. example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ 4JOHMF
    
    ,FSOFM
12. example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ /*$ ,FSOFM ,FSOFM $PQZ $PQZ 4JOHMF
    
    ,FSOFM )ZQFSWJTPS
    9FO ,7. 6TFSMBOE $PQZ ˞ 43*07ͳͲ)ZQFSWJTPSͰ΋ύέοτίϐʔΦʔόϔουΛগͳ͘ ͢Δ࣮૷͸͋Δ
13. Linux Containers ͩͱ Single Kernel Ͱ଎͍
14. Docker Filesystem -JOVY
    $POUBJOFST
15. UNION Filesystem -JOVY
    $POUBJOFST %PDLFS
    'JMFTZTUFN
16. IUUQTEPDTEPDLFSDPNUFSNTJNBHFTEPDLFSpMFTZTUFNTNVMUJMBZFSQOH UNION Filesystem Image
17. 8SJUF
    *0 3FBE
    *0
    ADBU
    FUDEFCJBO@WFSTJPOA
18. 8SJUF
    *0 3FBE
    *0
    8SJUF
    *0
    ͸࠷্૚΁ॻ͖ࠐΈ 3FBE
    *0
    ͸֘౰ϑΝΠϧΛ
    ֨ೲ͢Δ૚Λ୳ࡧ
19. UNION Filesystem • AUFS, Btrfs, OverlayFS … ͳͲΛ࢖࣮ͬͯݱ • Copy On Write: ݪຊΛෳ੡ͨ͠;ΓΛͯ͠ ͓͍ͯ৽نॻ͖ࠐΈ͕͋ͬͨΒෳ੡Λ࡞Δ • LinuxΧʔωϧඪ४ͷ Device MapperΛ࢖ͬ ͯ΋࣮ݱͰ͖Δ
20. Storage drivers -JOVY
    $POUBJOFST %PDLFS
    'JMFTZTUFN 6/*0/
    'JMFTZTUFN
21. EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT
22. EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT 6OJPO
    '4ͷ࣮૷Λબ୒Ͱ͖Δ
23. Storage drivers devicemapper, aufs, btrfs, overrayfs (vfs) Χʔωϧඪ४ͳͨΊRHELܥͰͷɹɹ σϑΥϧτ͸ devicemapper
24. Device Mapper ϒϩοΫσόΠε΁ͷI/Oʹ༷ʑͳม׵ΛՃ͑Δ͜ͱ͕ Ͱ͖Δ (҉߸ԽɺετϥΠϓɺϛϥʔ ͳͲ) ϒϩοΫσόΠευϥΠόϨϕϧͷ࣮૷ͳͷͰɹɹɹɹ ಛఆͷϑΝΠϧγεςϜʹґଘ͠ͳ͍ LVMͳͲͰ࢖ΘΕ͍ͯΔ docker commit ͕૸ΔͱsnapshotσόΠεΛ࡞੒ͯ͠Π ϝʔδ૚ͱ͢Δ
25. Volume -JOVY
    $POUBJOFST %PDLFS
    'JMFTZTUFN 6/*0/
    'JMFTZTUFN 4UPSBHF
    %SJWFST
26. Volume ίϯςφؒͰσΟϨΫτϦΛڞ༗͢ΔͨΊͷ΋ͷ ίϯςφ͝ͱͰ͸ͳ͘DockerάϩʔόϧͳྖҬʹ֨ ೲ͞ΕΔ ࢦఆͨ͠σΟϨΫτϦҎԼͷΠϝʔδ૚ΛόΠύε͠ ͯΞΫηε I/Oཁٻ͕Union FS෦෼Λ௨Βͳ͍ͷͰΦʔόϔου ͕গͳ͍
27. howto/Volume EPDLFS
    SVO
    W
    WBSMJCNZTRM
    NZTRM 70-6.&
    WBSMJCNZTRM EPDLFS
    SVO
    NZTRM %PDLFSpMF 4IFMM 4IFMM PS
28. Docker Network -JOVY
    $POUBJOFST %PDLFS
    'JMFTZTUFN 6/*0/
    'JMFTZTUFN 4UPSBHF
    %SJWFST 7PMVNF
29. Portmapper -JOVY
    $POUBJOFST %PDLFS
    'JMFTZTUFN 6/*0/
    'JMFTZTUFN 4UPSBHF
    %SJWFST 7PMVNF %PDLFS
    /FUXPSL
30. Portmapper ίϯςφؒ௨৴΍ίϯςφɾϗετؒ௨৴͸ϗετଆ ͷ iptables ͰNAT ! iptables͕ͳ͍؀ڥͩͱಠࣗͷϢʔβϥϯυͷϓϩη ε(docker-proxy)Ͱίϯςφؒ௨৴ "
    %0$,&3
    
    J
    EPDLFS
    Q
    UDQ
    N
    UDQ
    ŠEQPSU
    
    K
    %/"5
    
    ŠUPEFTUJOBUJPO
     EPDLFSQSPYZ
    QSPUP
    UDQ
    IPTUJQ
    
    IPTUQPSU
    
     DPOUBJOFSJQ
    
    DPOUBJOFSQPSU
    
31. Host Networking -JOVY
    $POUBJOFST %PDLFS
    'JMFTZTUFN 6/*0/
    'JMFTZTUFN 4UPSBHF
    %SJWFST 7PMVNF %PDLFS
    /FUXPSL 1PSUNBQQFS
32. Host Networking ίϯςφ༻ͷNetwork NamespaceΛ࡞Β ͣʹϗετͷωοτϫʔΫΛͦͷ··࢖͏ ίϯςφ͸ϗετଆͷportΛlisten͢Δ iptables΍docker-proxyΛܦ༝͠ͳͯ͘Α ͘ͳΓɺΦʔόϔου͕খ͘͞ͳΔ
33. howto/Host Networking ! ŠFYFDESJWFSOBUJWF
    
    PS
    
    -9$
    
    Ҏ্Ͱ
    ŠFYFDESJWFSMYD
    ͕ඞཁ EPDLFS
    SVO
    ŠOFUIPTU
    NZTRM
34. -JOVY
    $POUBJOFST %PDLFS
    'JMFTZTUFN 6/*0/
    'JMFTZTUFN 4UPSBHF
    %SJWFST 7PMVNF %PDLFS
    /FUXPSL 1PSUNBQQFS )PTU
    /FUXPSLJOH DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱ͸ͳʹ͔
35. 2 DockerԽͨ͠ ISUCONΞϓϦέʔγϣϯ ͷϕϯνϚʔΫ
36. ISUCON *JLBOKJOJ
    4QFFE
    6Q
    $0/UFTU
37. /HJOY CFODI
    NBSLFS .Z42- "QQ .FNDBDIFE ISUCON4 ༧બ
38. Machine & Software Spec instance type: m3.xlarge CPU: Xeon E5-2670 v2 @ 2.50GHz 4 vCPU Memory: 16GB RAM Storage: Magnetic volume OS: Ubuntu 14.04 LTS Kernel 3.18.0 Docker: 1.4.1 (latest) MySQL: 5.5.40, memcached: 1.4.14, Nginx:1.4.6
39. νϡʔχϯά಺༰ • είΞ 38446 (໿ 3000 req/s) • ࢼߦ͝ͱʹ +-1000 είΞఔ౓ͷޡࠩ • ༧બಥഁϨϕϧ • σʔλ͸શ෦ϝϞϦʹ৐Δ • ηογϣϯ৘ใͳͲ͸ memcached • Nginx Ͱ੩తϑΝΠϧΛฦ͢ • ωοτϫʔΫελοΫɺNginx, MySQL ͸ී௨ͷ νϡʔχϯά
40. Evaluation
41. • Nginx ͱ MySQL ΛͦΕͧΕDockerԽ • ҎԼͷ֤ํࣜΛnative(default)ͱൺֱ • Nginx͚ͩDockerԽ • —net=host ͱ —net=bridge • MySQL͚ͩDockerԽ • storage-driver=devicemapper ͱ storage-driver=overlayfs • Volume ͷ ON/OFF
42. Result
43. EFGBVMU /HJOY
    OFUCSJEHF /HJOY
    OFUIPTU   
44. /HJOY
    ʹύέοτ͕ू໿͢Δ
    /"15
    ͢ΔΦʔόϔου͕ߴ͍
45. EFGBVMU .Z42-
    EFWJDFNBQQFS
    WPMVNF
    P⒎ .Z42-
    EFWJDFNBQQFS
    WPMVNF
    PO .Z42-
    PWFSMBZGT
    WPMVNF
    P⒎ .Z42-
    PWFSMBZGT
    WPMVNF
    PO     
46. 7PMVNF
    ͷ
    0/0''ͱ
    TUPSBHF
    ESJWFSʹΑΔ
    ੑೳྼԽͳ͠
47. 7PMVNF
    
    0''
    ͰมΘΒͳ͍ 3FBE
    *0͸ϝϞϦʹશ෦ͷΔ
    8SJUF
    *0
    ͸࠷্૚͚ͩ
48. NAPTͷߴ଎Խ
49. -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
50. -A OUTPUT 
    E
     -m addrtype --dst-type LOCAL -j DOCKER
51. 127.0.0.1Ѽͩͱiptablesͷ DockerνΣʔϯʹೖΒͳ͍
52. docker-proxy EPDLFSQSPYZ
    QSPUP
    UDQ
    IPTUJQ
    
    
    IPTUQPSU
    
    
    DPOUBJOFSJQ
    
    DPOUBJOFSQPSU
    A
53. docker-proxy EPDLFSQSPYZ
    QSPUP
    UDQ
    IPTUJQ
    
    IPTUQPSU
    
    DPOUBJOFSJQ
    
    DPOUBJOFSQPSU
    A CPU 50%
54. benchmarker bench --host 172.31.8.235 --init ~/init.sh --workload 8
55. 32172 37931
56. ͳͥ userland ͷ proxy ͕͍Δͷ͔
57. IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
58. Hairpin NAT
59. )PTU $POUBJOFS EPDLFS  -*45&/
     FUI FUI MP
    
    
    
    
     FUI
     FUI
    
60. ETU
     MP
    
    
    
    
     FUI
     FUI
     )PTU $POUBJOFS EPDLFS  -*45&/
     FUI FUI
61. ETU
     "
    %0$,&3
    
    J
    EPDLFS
    Q
    UDQ
    N
    UDQ
    EQPSU
    
    
    K
    %/"5
    UPEFTUJOBUJPO
     )PTU $POUBJOFS MP
    
    
    
    
     FUI
     FUI
     EPDLFS  -*45&/
     FUI FUI
62. ETU
     "
    %0$,&3
    
    J
    EPDLFS
    Q
    UDQ
    N
    UDQ
    EQPSU
    
    
    K
    %/"5
    UPEFTUJOBUJPO
     )PTU $POUBJOFS MP
    
    
    
    
     FUI
    FUI
    EPDLFS  -*45&/
     -*45&/
     Ծ૝bridge(docker0)Λ Hairpin NAT modeʹ͠ͳ͍ͱ NAT͞Εͳ͍
63. FDIP
    
    
    TZTDMBTTOFUCSCSJGFUIIBJSQJO@NPEF
64. IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM
66. 3)&-
    
    BOE
    $FOU04
     SFBEPOMZ
    TZT
67. 1. DockerͷύϑΥʔϚϯεʹ͍ͭͯ ॏཁͳࣄ͸ͳʹ͔ 2. DockerԽͨ͠ISUCONΞϓϦέʔ γϣϯͷϕϯνϚʔΫ
68. Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker Network Portmapper Host Networking
69. Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker Network Portmapper (Performance issue) Host Networking
71. IUUQIBUFOBDPSQKQSFDSVJUDBSFFSTBMFTFOHJOFFS ηʔϧεΤϯδχΞ৬
72. ! 8FCΦϖϨʔγϣϯΤϯδχΞ
    Λืू͓ͯ͠Γ·͢