Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cookpad's deployment and auto scaling

Cookpad's deployment and auto scaling

クックパッドのデプロイとオートスケール
2014.03.15 JAWS DAYS 2014 Immutable Infrastructure track
http://jawsdays2014.jaws-ug.jp/speaker/i_naruta/

Issei Naruta

March 15, 2014
Tweet

More Decks by Issei Naruta

Other Decks in Technology

Transcript

  1. ΫοΫύουͷ
    σϓϩΠͱ
    Φʔτεέʔϧ
    ΫοΫύουגࣜձࣾ੒ాҰੜ
    +"84%":4

    View full-size slide

  2. KBXTEBZTJOGSB

    View full-size slide

  3. ੒ాҰੜ
    ͳΔͨ ͍͍ͬͤ
    !NJSBLVJ
    ΫοΫύουגࣜձࣾ
    ΠϯϑϥετϥΫνϟʔ෦௕

    View full-size slide

  4. ࠓ೔ͷ࿩
    ΫοΫύουʹ͍ͭͯ
    ϓϩϏδϣχϯάʢαʔϏεσΟεΧόϦʣ
    ΞϓϦέʔγϣϯͷσϓϩΠ
    Φʔτεέʔϧ
    !

    View full-size slide

  5. ΫοΫύουʹ͍ͭͯ

    View full-size slide

  6. 'VMM"84
    2011೥3݄ ౦ژϦʔδϣϯΦʔϓϯ
    2011೥8݄ DC͔ΒAWS΁Ҡߦ

    View full-size slide

  7. .6#NP
    .14VTFST
    .DPPLJOHSFDJQFT
    ˞೥݄ݱࡏ
    !"
    #
    #

    View full-size slide

  8. &$JOTUBODFT
    &-#T
    4#VDLFUT
    3PVUF3FDPSET

    View full-size slide

  9. 3FETIJGUGPS#*
    UBCFNJSVDPN
    4/4GPS.PCJMF1VTI/PUJpDBUJPO

    View full-size slide

  10. 0QT
    %FWFMPQFST
    %FQMPZTEBZ

    #

    View full-size slide

  11. ΞϓϦέʔγϣϯίʔυͷσϓϩΠ

    View full-size slide

  12. $
    QVMMSFRVFTU
    ()&
    #
    EFWFMPQFS

    View full-size slide

  13. $
    QVMMSFRVFTU
    NFOUJPO
    ()&
    #
    EFWFMPQFS
    #
    SFWJFXFS

    View full-size slide

  14. $
    -(5.
    QVMMSFRVFTU
    NFOUJPO
    ()&
    #
    EFWFMPQFS
    #
    SFWJFXFS

    View full-size slide

  15. $
    -(5.
    QVMMSFRVFTU
    NFOUJPO
    ()&
    #
    EFWFMPQFS
    #
    SFWJFXFS
    NFSHF

    View full-size slide

  16. $
    -(5.
    QVMMSFRVFTU
    NFOUJPO
    ()&
    #
    EFWFMPQFS
    #
    SFWJFXFS
    $*
    +FOLJOT
    NFSHF

    View full-size slide

  17. $
    TVDDFTT
    -(5.
    QVMMSFRVFTU
    NFOUJPO
    ()&
    #
    EFWFMPQFS
    #
    SFWJFXFS
    $*
    +FOLJOT
    NFSHF

    View full-size slide

  18. $
    TVDDFTT
    -(5.
    QVMMSFRVFTU
    NFOUJPO
    EFQMPZ
    &$
    ()&
    #
    EFWFMPQFS
    #
    SFWJFXFS
    $*
    +FOLJOT
    NFSHF

    View full-size slide

  19. σϓϩΠϧʔϧʢҰ෦ʣ
    • CI Λύεͨ͠ϦϏδϣϯͷΈσϓϩΠͯ͠Α͍
    • σϓϩΠ͸ίʔυΛ push ͨ͠։ൃऀࣗ਎͕ߦ͏
    • Ӧۀ࣌ؒ಺ͷΈσϓϩΠՄೳ
    • σϓϩΠޙ͸։ൃऀ͕ಈ࡞֬ೝ͠ɺ

    ෆ۩߹Λݟ͚ͭͨΒ͙͢ʹϩʔϧόοΫ͢Δ

    View full-size slide

  20. σϓϩΠޙʹΤϥʔ΍ϨεϙϯελΠϜྼԽ͕ແ͍͜ͱΛ֬ೝ
    ϦΞϧλΠϜϞχλϦϯάπʔϧlJNPOz

    View full-size slide

  21. ΫοΫύουͷ
    αʔόϓϩϏδϣχϯά

    View full-size slide

  22. αʔϏεσΟεΧόϦ໰୊
    αʔό͕ಈతʹ૿͑ͨΓݮͬͨΓ͢Δঢ়گͰ͸

    ੩తͳ”αʔό؅ཧ୆ா”͸ӡ༻Ͱ͖ͳ͍
    !
    αʔό͕௥Ճ͞ΕͨΒݕ஌͢Δ࢓૊Έ͕ඞཁ
    →Serf, Zookeeper ౳
    %

    View full-size slide

  23. &$λάʹΑΔσΟεΧόϦ
    Name λά
    Role λά
    Status λά
    &

    View full-size slide

  24. &$ͷλάػೳʹ͍ͭͯ
    key: value ͷϖΞ

    View full-size slide

  25. ΫοΫύουͰͷ&$λά
    ͦͷαʔόͷଐੑ΍ঢ়ଶΛهड़
    !
    Πϯελϯε͸ࣗ෼ࣗ਎΍ɺଞΠϯελϯεͷλάΛݟͯ
    ࣗ཯ڠௐతʹಈ͘

    View full-size slide

  26. /BNFλά
    ͦͷΠϯελϯεͷϗετ໊
    DNSαʔό͕ࣗಈݕ஌͠ɺAϨίʔυΛ௥Ճ
    !
    ྫ:
    Name: app-001
    &

    View full-size slide

  27. ' /BNFEC
    ' /BNFEC


    View full-size slide

  28. ' /BNFEC
    ' /BNFEC


    1PXFS%/4

    View full-size slide

  29. ' /BNFEC
    ' /BNFEC


    1PXFS%/4

    (

    View full-size slide

  30. ' /BNFEC
    ' /BNFEC


    1PXFS%/4

    (
    EC"
    EC"
    %/4αʔό͕ఆظతʹશ/BNFλάΛ૸ࠪ͠ɺ
    %/4ϨίʔυΛߋ৽

    View full-size slide

  31. 3PMFλά
    Πϯελϯεͷ໾ׂΛهड़
    !
    ྫ:
    Role: app
    &

    View full-size slide

  32. QVQQFU
    NBTUFS )

    View full-size slide

  33. ' 3PMFBQQ
    QVQQFU
    NBTUFS )

    View full-size slide

  34. ' 3PMFBQQ
    BQQMZlBQQz
    QVQQFU
    NBTUFS )

    View full-size slide

  35. ' 3PMFBQQ
    ' 3PMFEC
    BQQMZlBQQz
    QVQQFU
    NBTUFS )

    View full-size slide

  36. ' 3PMFBQQ
    ' 3PMFEC
    BQQMZlBQQz
    QVQQFU
    NBTUFS
    BQQMZlECz
    )

    View full-size slide

  37. ' 3PMFBQQ
    ' 3PMFEC
    BQQMZlBQQz
    QVQQFU
    NBTUFS
    BQQMZlECz
    QVQQFUNBTUFS͸ద༻ઌͷ3PMFλάΛݟͯ

    ద༻͢ΔΫϥεΛࣗಈͰܾఆ
    )

    View full-size slide

  38. 4UBUVTλά
    ͦͷΠϯελϯε͕όϥϯαʔʹ

    ௥Ճ͍͍ͯ͠΋ͷ͔Ͳ͏͔Λఆٛ
    !
    ྫ:
    Status: working
    &

    View full-size slide

  39. 3PMFBQQ
    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ

    View full-size slide

  40. 3PMFBQQ
    ' 4UBUVTXPSLJOH
    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ

    View full-size slide

  41. 3PMFBQQ
    ' 4UBUVTXPSLJOH

    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ

    View full-size slide

  42. 3PMFBQQ
    ' 4UBUVTXPSLJOH

    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ

    View full-size slide

  43. 3PMFBQQ
    ' 4UBUVTXPSLJOH

    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ
    ' 4UBUVTXPSLJOH

    View full-size slide

  44. 3PMFBQQ
    ' 4UBUVTXPSLJOH

    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ
    ' 4UBUVTXPSLJOH

    View full-size slide

  45. 3PMFBQQ
    ' 4UBUVTXPSLJOH

    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ
    ' 4UBUVTXPSLJOH

    View full-size slide

  46. 3PMFBQQ
    ' 4UBUVTXPSLJOH

    *
    IBQSPYZDGH
    BQQIBQSPYZ
    3PMFBQQ
    ' 4UBUVTXPSLJOH
    XPSLJOHঢ়ଶͷΠϯελϯεΛࣗಈͰαʔϏεΠϯ

    View full-size slide

  47. '
    ͳͥλάͰશ෦΍Δͷ͔
    Zookeeper, Serf ౳ͷಛผͳϛυϧ΢ΣΞΛඞཁͱͤͣʹ

    αʔϏεσΟεΧόϦ͕Մೳ
    !
    ඇৗʹγϯϓϧ͔ͭϓϩάϥϚϒϧ

    View full-size slide

  48. λάͷϋϚϦͲ͜Ζ
    DescribeTags ͷ API Limit
    1 Ϧιʔεʹλά͸10ݸ·Ͱ
    %
    -JNJU
    DBDIF
    (

    View full-size slide

  49. πʔϧ
    λάϑϨϯυϦʔͳ؅ཧπʔϧ͕ඞཁ
    ElasticFox-ec2tag

    View full-size slide

  50. Φʔτεέʔϧ

    View full-size slide

  51. τϥϑΟοΫ
    &$Πϯελϯε਺
    ࣌ؒ
    τϥϑΟοΫͷมԽʹԠͯ͡Πϯελϯε਺Λ૿ݮ

    View full-size slide

  52. ͳͥΦʔτεέʔϧͳͷ͔
    '

    View full-size slide

  53. *OTUBODFIPVS
    EC2ͷ࠷খ՝ۚ୯Ґ
    ىಈதͷΠϯελϯε਺ x ىಈ࣌ؒʢ1࣌ؒ୯Ґʣ
    ΦϯσϚϯυɺ౦ژϦʔδϣϯ೥݄೔ݱࡏIUUQBXTBNB[PODPNKQFDQSJDJOH

    View full-size slide

  54. &$Πϯελϯε਺
    I
    Πϯελϯε࣌
    ( JOTUBODFIPVS
    Φʔτεέʔϧ
    ͳ͠
    Φʔτεέʔϧ
    ͋Γ

    View full-size slide

  55. "NB[PO"VUP4DBMJOH

    View full-size slide

  56. $MPVE8BUDI
    "VUP
    4DBMJOH
    "VUP4DBMJOH(SPVQ
    ".*

    View full-size slide

  57. ͨͱ͑͹
    "NB[PO"VUP4DBMJOH
    Λ΍ΊΔ

    View full-size slide

  58. ͳͥ"NB[PO"VUP4DBMJOHͰ͸ͳ͍ͷ͔
    CloudWatch 1෼ͷน
    Graceful Terminate ໰୊
    ELB Ͱ͸ͳ͘ haproxy Λ࢖͍͍ͨ
    σϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͏ͨΊ

    View full-size slide

  59. $MPVE8BUDIͷੑೳ΁ͷෆຬ
    ෼ͷนɹϝτϦΫε͕औಘͰ͖Δͷ͸࠷଎Ͱ΋1෼ޙ
    +

    View full-size slide

  60. (SBDFGVM5FSNJOBUF໰୊
    desired_capacity ͕ݮΔͱ͖ɺ

    Πϯελϯε͸໰౴ແ༻ͰTerminate ͞Εͯ͠·͏
    ҆શͳαʔϏεΞ΢τΛߦ͏ʹ͸޻෉͕ඞཁ

    View full-size slide

  61. *OUFSOBM&-#Ͱ͸ͳ͘IBQSPYZΛ࢖͍͍ͨ
    ELB ͷ IP ΞυϨεͷಈతͳมԽʹ

    ϛυϧ΢ΣΞ͕ରԠ͖͠Εͳ͍
    !
    ϒϥοΫϘοΫε౓͕ߴ͘ɺ໰୊͕೺Ѳͮ͠Β͍
    ˞&-#ͳ͠Ͱ΋"NB[PO"VUP4DBMJOH͸࢖͑·͢

    View full-size slide

  62. "NB[PO"VUP4DBMJOH
    WT
    EFQMPZTEBZ


    View full-size slide

  63. W W W
    W
    W

    W W W
    #

    View full-size slide

  64. W W W W W W
    W
    W

    W W W
    #

    View full-size slide

  65. W W W W W W
    W W W
    ݹ͍όʔδϣϯͷΠϯελϯε͕࡞ΒΕͯ͠·͏

    View full-size slide

  66. "NB[PO"VUP4DBMJOHͰഉଞ੍ޚ
    σϓϩΠத͸ AutoScaling ΛࢭΊɺ
    AutoScaling த͸σϓϩΠΛېࢭ͍ͨ͠
    !
    AutoScaling ͕ΠϯελϯεΛ૿΍͢ͱ͖ɾݮΒ͢ͱ͖ʹ
    ॲཧΛׂΓ͜·ͤΔඞཁ͕͋Δ
    →ࠔ೉

    View full-size slide

  67. CloudWatch 1෼ͷน
    Graceful Terminate ໰୊
    ELB Ͱ͸ͳ͘ haproxy Λ࢖͍͍ͨ
    σϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͍͍ͨ
    ࣗ෼Ͱશ෦ॻ͍ͯ΋͋·Γखؒ͸มΘΒͳͦ͏
    "NB[PO"VUP4DBMJOHͰղܾෆՄೳͰ͸ͳ͍͚Ͳɺ
    ଟ͘ͷίʔυΛॻ͖ɺΧελϚΠζ͢Δඞཁ͕͋Δ
    +
    +

    View full-size slide

  68. $PPLQBE"VUP4DBMJOHͷ࣮૷
    ,

    View full-size slide

  69. JNPO "4
    #
    W
    W
    EFWFMPQFS
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  70. JNPO "4
    #

    W
    W
    EFWFMPQFS
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  71. W
    W
    JNPO "4
    #

    W
    W
    EFWFMPQFS
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  72. W
    W
    JNPO "4
    #
    W
    W
    EFWFMPQFS
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  73. W
    W
    JNPO "4
    #
    W
    W
    EFWFMPQFS
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  74. W
    W
    JNPO "4
    #
    W
    W
    EFWFMPQFS
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  75. W
    W
    JNPO "4
    #
    W
    W
    EFWFMPQFS
    W
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  76. W
    W
    JNPO "4
    #
    W
    W
    EFWFMPQFS
    W
    W
    W
    W
    W
    W
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  77. W
    W
    JNPO "4
    #
    W
    W
    EFWFMPQFS
    W
    W
    W
    W
    W
    W
    SQT
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH
    '4UBUVTXPSLJOH

    View full-size slide

  78. Φʔτεέʔϧͷ͋Δ฻Β͠

    View full-size slide

  79. ͱ͋ΔΠϯελϯεͷ
    ୆਺

    Φʔτεέʔϧ

    View full-size slide

  80. ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫத
    σϓϩΠத

    View full-size slide

  81. (
    (
    ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫத
    σϓϩΠத

    View full-size slide

  82. Φʔτεέʔϧத
    σϓϩΠத

    View full-size slide

  83. - ೥Ҏ্ӡ༻Ͱ͖ͯΔ

    View full-size slide

  84. ೥ͷӡ༻࣮੷Λ௨ͯ͠
    ۤ࿑ͨ͠఺ɺ޻෉ɺؾ͍ͮͨ͜ͱ

    View full-size slide

  85. ϩοΫ࣌ؒɾճ਺Λ࠷খʹ͢Δ
    σϓϩΠΛϩοΫʹՁ஋Λಧ͚ΔνϟϯεΛݮΒ͢
    .

    View full-size slide

  86. ୆਺
    ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ

    View full-size slide

  87. ୆਺
    ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ

    αʔϏε"ϩοΫճ
    αʔϏε#ϩοΫճ

    View full-size slide

  88. ୆਺
    Ұ౓ʹ૿ݮ͢Δ୆਺Λௐ੔͢Δ͜ͱͰ
    ۀ຿࣌ؒதͷσϓϩΠϩοΫճ਺Λ࠷খݶʹ
    ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ

    αʔϏε"ϩοΫճ
    αʔϏε#ϩοΫճ

    View full-size slide

  89. ୆਺
    Ұ౓ʹ૿ݮ͢Δ୆਺Λௐ੔͢Δ͜ͱͰ
    ۀ຿࣌ؒதͷσϓϩΠϩοΫճ਺Λ࠷খݶʹ
    ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ

    αʔϏε"ϩοΫճ
    αʔϏε#ϩοΫճ
    ۀ຿࣌ؒ֎ͷͨΊɺ
    ݮΒ͢ͱ͖͸খࠁΈʹ

    View full-size slide

  90. ςετΛॻ͘
    Φʔτεέʔϧ͸αʔόΛམͱ͢ϓϩάϥϜ
    /

    View full-size slide

  91. ఆظతʹઃఆΛݟ௚͢
    ์͓͍ͬͯͯ΋εέʔϧ͢ΔΑ͏ʹݟ͑Δ͕…
    !
    ΞϓϦαʔόͷ୆਺૿ˠόοΫΤϯυ΁ͷίωΫγϣϯ૿
    ΞϓϦέʔγϣϯͷύϑΥʔϚϯε΋ৗʹมԽ
    0

    View full-size slide

  92. TUBUJDOPEFTEZOBNJDOPEFTߏ੒
    ৗʹཱͪଓ͚ͯΔΠϯελϯε͕͍ͨ΄͏͕
    ͳΜ͔ͩΜͩͰศར
    TUBUJD
    EZOBNJD

    View full-size slide

  93. TUBUJDOPEFT EZOBNJDOPEFT
    Φʔτεέʔϧର৅֎
    ୆਺͸ݻఆ
    TIVUEPXO͞Εͳ͍
    Φʔτεέʔϧର৅
    ୆਺͸Մม
    CBMBODFS

    View full-size slide

  94. αʔό໊Ͳ͏͠Α͏໰୊
    dynamic nodes Ͱ͋Ζ͏ͱɺ

    ݁ہ hostname ͸͋ͬͨ΄͏͕؅ཧ͠΍͍͢
    !
    10.0.123.45 ͩͬͨΒ app-10-0-123-45 ͱ͍͏

    Name λά͕ͭ͘Α͏ʹ͍ͯ͠Δ
    &

    View full-size slide

  95. Φʔτεέʔϧͷ෭࢈෺
    ΦʔτεέʔϧΛ૊Ήͱɺ Immutable ͳ

    Golden Image ͕࡞ΕΔ
    !
    ͍ͭͰ΋ҰൃͰ࠷৽ͷαʔό্ཱ͕ͪ͛ΒΕΔ͜ͱ͕

    อূ͞ΕΔ +

    View full-size slide

  96. EFQMPZTEBZWT
    "VUP4DBMJOH
    10 deploys/day ͳ؀ڥͰ΋Φʔτεέʔϧ͸Մೳ
    ίετͱ։ൃ଎౓ͷτϨʔυΦϑΛߟྀͨ͠νϡʔχϯά

    View full-size slide

  97. "VUP4DBMJOHBT
    *NNVUBCMF*OGSBTUSVDUVSF
    Immutable Infrastructure ͸ະདྷͷ࿩͡Όͳ͘ɺ

    ΫοΫύουͰ͸ΦʔτεέʔϧʹΑͬͯ͢Ͱʹ
    ݱ࣮ʹͳͬͯΔ

    View full-size slide

  98. DSFEJUT
    http://aws.amazon.com/jp/architecture/icons/
    http://www.entypo.com/
    http://fortawesome.github.io/Font-Awesome/
    http://www.flickr.com/photos/sora_h/8916379094/

    View full-size slide