Docker Performance on Web Application

A658ec7f1badf73819dfa501165016c1?s=47 Yuuki Tsubouchi (yuuk1)
January 17, 2015
Technology
7
130k

Docker Performance on Web Application

Docker Meetup Tokyo #4

A658ec7f1badf73819dfa501165016c1?s=128

Yuuki Tsubouchi (yuuk1)

January 17, 2015
Tweet

Want more?

A658ec7f1badf73819dfa501165016c1?s=48 yuukit
4
670
A658ec7f1badf73819dfa501165016c1?s=48 yuukit
0
37
A658ec7f1badf73819dfa501165016c1?s=48 yuukit
0
25
3ab1249be442027903e1180025340b3f?s=48 reverentgeek
2
140
A9a491b0fcbe0fbce3d64063a37add99?s=48 rmw
1
71
766b9746b59e6f09e280cd33cf4ed419?s=48 skipperchong
0
68
61857dafbd287b3027c4dcea9008ad3c?s=48 carmenhchung
5
140
06f8b41980eb4c577fa40c41d5030c19?s=48 keathley
4
150
C0390e8b11079f8761c0cd3274ed61cb?s=48 productmarketing
2
220
C35e01544a0dd94a2cf1619ee8a42ebb?s=48 clairelew
2
420
6bb82b13cda86d3b821fa44100335ddf?s=48 thoeni
1
110
B3ace07412a5178ea778e7eec161fc0a?s=48 jcasabona
4
200

Transcript

  1. 1.

    Docker Performance on Web Application JEZ@VVLJ %PDLFS.FFUVQ5PLZP

  2. 2.

    id:y_uuki ӡ༻ @y_uuk1

  3. 3.

    Docker Engine্Ͱɹ ΞϓϦέʔγϣϯΛɹ ಈ͔ͯ͠ੑೳྼԽ ͠ͳ͍ͷʁ

  4. 4.
    None
  5. 5.

    1 DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱ͸ͳʹ͔

  6. 6.

    Linux Containers

  7. 7.

    /FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF

    1*% /BNFTQBDF
  8. 8.

    /FUXPSL /BNFTQBDF *1$ /BNFTQBDF 654 /BNFTQBDF .PVOU /BNFTQBDF 6TFS /BNFTQBDF

    1*% /BNFTQBDF -JOVYΧʔωϧͷ໊લۭؒػೳͷ ू·Γ
  9. 9.

    Linux Containers Χʔωϧͷ֤ϦιʔεΛ໊લۭؒͰִ཭͠ ۭͨؒͰϓϩηεΛ࣮ߦ͢Δͱ͍͏Ϟσϧ ϑΝΠϧγεςϜɼϢʔβɼϓϩηεςʔ ϒϧɼωοτϫʔΫ… etc LXC ͸ Linux

    Containers ͷϑϩϯτΤϯυ
  10. 10.

    Linux Containers Overhead ୯ମͷLinux ΧʔωϧͰ׬݁͢ΔͷͰ ΦϒδΣΫτͷڞ༗ޮ཰͕Α͍ Hypervisor ͷΑ͏ʹ֤ϦιʔεΛɹɹ ೋॏॲཧ͠ͳͯ͘Α͍

  11. 11.

    example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ 4JOHMF ,FSOFM

  12. 12.

    example/packet receive /*$ ,FSOFM 6TFSMBOE $PQZ $PQZ /*$ ,FSOFM ,FSOFM

    $PQZ $PQZ 4JOHMF ,FSOFM )ZQFSWJTPS 9FO ,7. 6TFSMBOE $PQZ ˞ 43*07ͳͲ)ZQFSWJTPSͰ΋ύέοτίϐʔΦʔόϔουΛগͳ͘ ͢Δ࣮૷͸͋Δ
  13. 13.

    Linux Containers ͩͱ Single Kernel Ͱ଎͍

  14. 14.

    Docker Filesystem -JOVY$POUBJOFST

  15. 15.

    UNION Filesystem -JOVY$POUBJOFST %PDLFS'JMFTZTUFN

  16. 16.

    IUUQTEPDTEPDLFSDPNUFSNTJNBHFTEPDLFSpMFTZTUFNTNVMUJMBZFSQOH UNION Filesystem Image

  17. 17.

    8SJUF*0 3FBE*0 ADBUFUDEFCJBO@WFSTJPOA

  18. 18.

    8SJUF*0 3FBE*0 8SJUF*0͸࠷্૚΁ॻ͖ࠐΈ 3FBE*0͸֘౰ϑΝΠϧΛ ֨ೲ͢Δ૚Λ୳ࡧ

  19. 19.

    UNION Filesystem • AUFS, Btrfs, OverlayFS … ͳͲΛ࢖࣮ͬͯݱ • Copy

    On Write: ݪຊΛෳ੡ͨ͠;ΓΛͯ͠ ͓͍ͯ৽نॻ͖ࠐΈ͕͋ͬͨΒෳ੡Λ࡞Δ • LinuxΧʔωϧඪ४ͷ Device MapperΛ࢖ͬ ͯ΋࣮ݱͰ͖Δ
  20. 20.

    Storage drivers -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN

  21. 21.

    EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT

  22. 22.

    EFWJDFNBQQFS BVGT CUSGT PWFSMBZGT 6OJPO'4ͷ࣮૷Λબ୒Ͱ͖Δ

  23. 23.

    Storage drivers devicemapper, aufs, btrfs, overrayfs (vfs) Χʔωϧඪ४ͳͨΊRHELܥͰͷɹɹ σϑΥϧτ͸ devicemapper

  24. 24.

    Device Mapper ϒϩοΫσόΠε΁ͷI/Oʹ༷ʑͳม׵ΛՃ͑Δ͜ͱ͕ Ͱ͖Δ (҉߸ԽɺετϥΠϓɺϛϥʔ ͳͲ) ϒϩοΫσόΠευϥΠόϨϕϧͷ࣮૷ͳͷͰɹɹɹɹ ಛఆͷϑΝΠϧγεςϜʹґଘ͠ͳ͍ LVMͳͲͰ࢖ΘΕ͍ͯΔ docker

    commit ͕૸ΔͱsnapshotσόΠεΛ࡞੒ͯ͠Π ϝʔδ૚ͱ͢Δ
  25. 25.

    Volume -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST

  26. 26.

    Volume ίϯςφؒͰσΟϨΫτϦΛڞ༗͢ΔͨΊͷ΋ͷ ίϯςφ͝ͱͰ͸ͳ͘DockerάϩʔόϧͳྖҬʹ֨ ೲ͞ΕΔ ࢦఆͨ͠σΟϨΫτϦҎԼͷΠϝʔδ૚ΛόΠύε͠ ͯΞΫηε I/Oཁٻ͕Union FS෦෼Λ௨Βͳ͍ͷͰΦʔόϔου ͕গͳ͍

  27. 27.

    howto/Volume EPDLFSSVOWWBSMJCNZTRMNZTRM 70-6.&WBSMJCNZTRM EPDLFSSVONZTRM %PDLFSpMF 4IFMM 4IFMM PS

  28. 28.

    Docker Network -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF

  29. 29.

    Portmapper -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL

  30. 30.

    Portmapper ίϯςφؒ௨৴΍ίϯςφɾϗετؒ௨৴͸ϗετଆ ͷ iptables ͰNAT ! iptables͕ͳ͍؀ڥͩͱಠࣗͷϢʔβϥϯυͷϓϩη ε(docker-proxy)Ͱίϯςφؒ௨৴ "%0$,&3JEPDLFSQUDQNUDQŠEQPSUK%/"5 ŠUPEFTUJOBUJPO

    EPDLFSQSPYZQSPUPUDQIPTUJQIPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSU
  31. 31.

    Host Networking -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS

  32. 32.

    Host Networking ίϯςφ༻ͷNetwork NamespaceΛ࡞Β ͣʹϗετͷωοτϫʔΫΛͦͷ··࢖͏ ίϯςφ͸ϗετଆͷportΛlisten͢Δ iptables΍docker-proxyΛܦ༝͠ͳͯ͘Α ͘ͳΓɺΦʔόϔου͕খ͘͞ͳΔ

  33. 33.

    howto/Host Networking ! ŠFYFDESJWFSOBUJWF PS -9$Ҏ্ͰŠFYFDESJWFSMYD ͕ඞཁ EPDLFSSVOŠOFUIPTUNZTRM

  34. 34.

    -JOVY$POUBJOFST %PDLFS'JMFTZTUFN 6/*0/'JMFTZTUFN 4UPSBHF%SJWFST 7PMVNF %PDLFS/FUXPSL 1PSUNBQQFS )PTU/FUXPSLJOH DockerͷύϑΥʔϚϯεʹ ͓͍ͯॏཁͳ͜ͱ͸ͳʹ͔

  35. 35.

    2 DockerԽͨ͠ ISUCONΞϓϦέʔγϣϯ ͷϕϯνϚʔΫ

  36. 36.

    ISUCON *JLBOKJOJ4QFFE6Q$0/UFTU

  37. 37.

    /HJOY CFODI NBSLFS .Z42- "QQ .FNDBDIFE ISUCON4 ༧બ

  38. 38.

    Machine & Software Spec instance type: m3.xlarge CPU: Xeon E5-2670

    v2 @ 2.50GHz 4 vCPU Memory: 16GB RAM Storage: Magnetic volume OS: Ubuntu 14.04 LTS Kernel 3.18.0 Docker: 1.4.1 (latest) MySQL: 5.5.40, memcached: 1.4.14, Nginx:1.4.6
  39. 39.

    νϡʔχϯά಺༰ • είΞ 38446 (໿ 3000 req/s) • ࢼߦ͝ͱʹ +-1000

    είΞఔ౓ͷޡࠩ • ༧બಥഁϨϕϧ • σʔλ͸શ෦ϝϞϦʹ৐Δ • ηογϣϯ৘ใͳͲ͸ memcached • Nginx Ͱ੩తϑΝΠϧΛฦ͢ • ωοτϫʔΫελοΫɺNginx, MySQL ͸ී௨ͷ νϡʔχϯά
  40. 40.

    Evaluation

  41. 41.

    • Nginx ͱ MySQL ΛͦΕͧΕDockerԽ • ҎԼͷ֤ํࣜΛnative(default)ͱൺֱ • Nginx͚ͩDockerԽ •

    —net=host ͱ —net=bridge • MySQL͚ͩDockerԽ • storage-driver=devicemapper ͱ storage-driver=overlayfs • Volume ͷ ON/OFF
  42. 42.

    Result

  43. 43.

    EFGBVMU /HJOY OFUCSJEHF /HJOY OFUIPTU   

  44. 44.

    /HJOYʹύέοτ͕ू໿͢Δ /"15͢ΔΦʔόϔου͕ߴ͍

  45. 45.

    EFGBVMU .Z42- EFWJDFNBQQFS WPMVNFP⒎ .Z42- EFWJDFNBQQFS WPMVNFPO .Z42- PWFSMBZGT WPMVNFP⒎

    .Z42- PWFSMBZGT WPMVNFPO     
  46. 46.

    7PMVNFͷ0/0''ͱ TUPSBHFESJWFSʹΑΔ ੑೳྼԽͳ͠

  47. 47.

    7PMVNF0''ͰมΘΒͳ͍ 3FBE*0͸ϝϞϦʹશ෦ͷΔ 8SJUF*0͸࠷্૚͚ͩ

  48. 48.

    NAPTͷߴ଎Խ

  49. 49.

    -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j

    DOCKER
  50. 50.

    -A OUTPUT E -m addrtype --dst-type LOCAL -j DOCKER

  51. 51.

    127.0.0.1Ѽͩͱiptablesͷ DockerνΣʔϯʹೖΒͳ͍

  52. 52.

    docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA

  53. 53.

    docker-proxy EPDLFSQSPYZQSPUPUDQIPTUJQ IPTUQPSU DPOUBJOFSJQDPOUBJOFSQPSUA CPU 50%

  54. 54.

    benchmarker bench --host 172.31.8.235 --init ~/init.sh --workload 8

  55. 55.

    32172 37931

  56. 56.

    ͳͥ userland ͷ proxy ͕͍Δͷ͔

  57. 57.

    IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM

  58. 58.

    Hairpin NAT

  59. 59.

    )PTU $POUBJOFS EPDLFS  -*45&/ FUI FUI MP FUI FUI

  60. 60.

    ETU MP FUI FUI )PTU $POUBJOFS EPDLFS  -*45&/ FUI

    FUI
  61. 61.

    ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS 

    -*45&/ FUI FUI
  62. 62.

    ETU "%0$,&3JEPDLFSQUDQNUDQEQPSU K%/"5UPEFTUJOBUJPO )PTU $POUBJOFS MP FUI FUI EPDLFS 

    -*45&/ -*45&/ Ծ૝bridge(docker0)Λ Hairpin NAT modeʹ͠ͳ͍ͱ NAT͞Εͳ͍
  63. 63.

    FDIPTZTDMBTTOFUCSCSJGFUIIBJSQJO@NPEF

  64. 64.

    IUUQTHJUIVCDPNEPDLFSEPDLFSQVMM

  65. 65.
    None
  66. 66.

    3)&-BOE$FOU04 SFBEPOMZTZT

  67. 67.

    1. DockerͷύϑΥʔϚϯεʹ͍ͭͯ ॏཁͳࣄ͸ͳʹ͔ 2. DockerԽͨ͠ISUCONΞϓϦέʔ γϣϯͷϕϯνϚʔΫ

  68. 68.

    Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker

    Network Portmapper Host Networking
  69. 69.

    Linux Containers Docker Filesystem UNION Filesystem Storage drivers Volume Docker

    Network Portmapper (Performance issue) Host Networking
  70. 70.
    None
  71. 71.

    IUUQIBUFOBDPSQKQSFDSVJUDBSFFSTBMFTFOHJOFFS ηʔϧεΤϯδχΞ৬

  72. 72.

    ! 8FCΦϖϨʔγϣϯΤϯδχΞ Λืू͓ͯ͠Γ·͢