Pro Yearly is on sale from $80 to $50! »

Cookpad's deployment and auto scaling

Cookpad's deployment and auto scaling

クックパッドのデプロイとオートスケール
2014.03.15 JAWS DAYS 2014 Immutable Infrastructure track
http://jawsdays2014.jaws-ug.jp/speaker/i_naruta/

6f8e101c016fa7ebbe58e693dbd86b7d?s=128

Issei Naruta

March 15, 2014
Tweet

Transcript

  1. ΫοΫύουͷ σϓϩΠͱ Φʔτεέʔϧ ΫοΫύουגࣜձࣾ੒ాҰੜ +"84%":4

  2. KBXTEBZTJOGSB

  3. ੒ాҰੜ ͳΔͨ ͍͍ͬͤ !NJSBLVJ ΫοΫύουגࣜձࣾ ΠϯϑϥετϥΫνϟʔ෦௕

  4. ࠓ೔ͷ࿩ ΫοΫύουʹ͍ͭͯ ϓϩϏδϣχϯάʢαʔϏεσΟεΧόϦʣ ΞϓϦέʔγϣϯͷσϓϩΠ Φʔτεέʔϧ !

  5. ΫοΫύουʹ͍ͭͯ

  6. 'VMM"84 2011೥3݄ ౦ژϦʔδϣϯΦʔϓϯ 2011೥8݄ DC͔ΒAWS΁Ҡߦ ✈

  7. .6#NP .14VTFST .DPPLJOHSFDJQFT ˞೥݄ݱࡏ !" # #

  8. &$JOTUBODFT &-#T 4#VDLFUT 3PVUF3FDPSET

  9. 3FETIJGUGPS#* UBCFNJSVDPN 4/4GPS.PCJMF1VTI/PUJpDBUJPO

  10. 0QT %FWFMPQFST %FQMPZTEBZ ⚒ #

  11. ΞϓϦέʔγϣϯίʔυͷσϓϩΠ

  12. $ QVMMSFRVFTU ()& # EFWFMPQFS

  13. $ QVMMSFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS

  14. $ -(5. QVMMSFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS

  15. $ -(5. QVMMSFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS NFSHF

  16. $ -(5. QVMMSFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS $*

    +FOLJOT NFSHF
  17. $ TVDDFTT -(5. QVMMSFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS

    $* +FOLJOT NFSHF
  18. $ TVDDFTT -(5. QVMMSFRVFTU NFOUJPO EFQMPZ &$ ()& # EFWFMPQFS

    # SFWJFXFS $* +FOLJOT NFSHF
  19. σϓϩΠϧʔϧʢҰ෦ʣ • CI Λύεͨ͠ϦϏδϣϯͷΈσϓϩΠͯ͠Α͍ • σϓϩΠ͸ίʔυΛ push ͨ͠։ൃऀࣗ਎͕ߦ͏ • Ӧۀ࣌ؒ಺ͷΈσϓϩΠՄೳ

    • σϓϩΠޙ͸։ൃऀ͕ಈ࡞֬ೝ͠ɺ
 ෆ۩߹Λݟ͚ͭͨΒ͙͢ʹϩʔϧόοΫ͢Δ
  20. σϓϩΠޙʹΤϥʔ΍ϨεϙϯελΠϜྼԽ͕ແ͍͜ͱΛ֬ೝ ϦΞϧλΠϜϞχλϦϯάπʔϧlJNPOz

  21. ΫοΫύουͷ αʔόϓϩϏδϣχϯά

  22. αʔϏεσΟεΧόϦ໰୊ αʔό͕ಈతʹ૿͑ͨΓݮͬͨΓ͢Δঢ়گͰ͸
 ੩తͳ”αʔό؅ཧ୆ா”͸ӡ༻Ͱ͖ͳ͍ ! αʔό͕௥Ճ͞ΕͨΒݕ஌͢Δ࢓૊Έ͕ඞཁ →Serf, Zookeeper ౳ %

  23. &$λάʹΑΔσΟεΧόϦ Name λά Role λά Status λά &

  24. &$ͷλάػೳʹ͍ͭͯ key: value ͷϖΞ

  25. ΫοΫύουͰͷ&$λά ͦͷαʔόͷଐੑ΍ঢ়ଶΛهड़ ! Πϯελϯε͸ࣗ෼ࣗ਎΍ɺଞΠϯελϯεͷλάΛݟͯ ࣗ཯ڠௐతʹಈ͘

  26. /BNFλά ͦͷΠϯελϯεͷϗετ໊ DNSαʔό͕ࣗಈݕ஌͠ɺAϨίʔυΛ௥Ճ ! ྫ: Name: app-001 &

  27.  

  28. ' /BNFEC  

  29. ' /BNFEC ' /BNFEC  

  30. ' /BNFEC ' /BNFEC   1PXFS%/4

  31. ' /BNFEC ' /BNFEC   1PXFS%/4 ⟲ (

  32. ' /BNFEC ' /BNFEC   1PXFS%/4 ⟲ ( EC"

    EC" %/4αʔό͕ఆظతʹશ/BNFλάΛ૸ࠪ͠ɺ %/4ϨίʔυΛߋ৽
  33. 3PMFλά Πϯελϯεͷ໾ׂΛهड़ ! ྫ: Role: app &

  34. QVQQFU NBTUFS )

  35. ' 3PMFBQQ QVQQFU NBTUFS )

  36. ' 3PMFBQQ BQQMZlBQQz QVQQFU NBTUFS )

  37. ' 3PMFBQQ ' 3PMFEC BQQMZlBQQz QVQQFU NBTUFS )

  38. ' 3PMFBQQ ' 3PMFEC BQQMZlBQQz QVQQFU NBTUFS BQQMZlECz )

  39. ' 3PMFBQQ ' 3PMFEC BQQMZlBQQz QVQQFU NBTUFS BQQMZlECz QVQQFUNBTUFS͸ద༻ઌͷ3PMFλάΛݟͯ
 ద༻͢ΔΫϥεΛࣗಈͰܾఆ

    )
  40. 4UBUVTλά ͦͷΠϯελϯε͕όϥϯαʔʹ
 ௥Ճ͍͍ͯ͠΋ͷ͔Ͳ͏͔Λఆٛ ! ྫ: Status: working &

  41. 3PMFBQQ * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ

  42. 3PMFBQQ ' 4UBUVTXPSLJOH * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ

  43. 3PMFBQQ ' 4UBUVTXPSLJOH ⟲ * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ

  44. 3PMFBQQ ' 4UBUVTXPSLJOH ⟲ * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ

  45. 3PMFBQQ ' 4UBUVTXPSLJOH ⟲ * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ ' 4UBUVTXPSLJOH

  46. 3PMFBQQ ' 4UBUVTXPSLJOH ⟲ * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ ' 4UBUVTXPSLJOH

  47. 3PMFBQQ ' 4UBUVTXPSLJOH ⟲ * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ ' 4UBUVTXPSLJOH

  48. 3PMFBQQ ' 4UBUVTXPSLJOH ⟲ * IBQSPYZDGH BQQIBQSPYZ 3PMFBQQ ' 4UBUVTXPSLJOH

    XPSLJOHঢ়ଶͷΠϯελϯεΛࣗಈͰαʔϏεΠϯ
  49. ' ͳͥλάͰશ෦΍Δͷ͔ Zookeeper, Serf ౳ͷಛผͳϛυϧ΢ΣΞΛඞཁͱͤͣʹ
 αʔϏεσΟεΧόϦ͕Մೳ ! ඇৗʹγϯϓϧ͔ͭϓϩάϥϚϒϧ

  50. λάͷϋϚϦͲ͜Ζ DescribeTags ͷ API Limit 1 Ϧιʔεʹλά͸10ݸ·Ͱ % -JNJU DBDIF

    (
  51. πʔϧ λάϑϨϯυϦʔͳ؅ཧπʔϧ͕ඞཁ ElasticFox-ec2tag

  52. Φʔτεέʔϧ

  53. τϥϑΟοΫ &$Πϯελϯε਺ ࣌ؒ τϥϑΟοΫͷมԽʹԠͯ͡Πϯελϯε਺Λ૿ݮ

  54. ͳͥΦʔτεέʔϧͳͷ͔ '

  55. *OTUBODFIPVS EC2ͷ࠷খ՝ۚ୯Ґ ىಈதͷΠϯελϯε਺ x ىಈ࣌ؒʢ1࣌ؒ୯Ґʣ ΦϯσϚϯυɺ౦ژϦʔδϣϯ೥݄೔ݱࡏIUUQBXTBNB[PODPNKQFDQSJDJOH

  56. &$Πϯελϯε਺ I        

     Πϯελϯε࣌  ( JOTUBODFIPVS Φʔτεέʔϧ ͳ͠ Φʔτεέʔϧ ͋Γ
  57. "NB[PO"VUP4DBMJOH

  58. $MPVE8BUDI "VUP 4DBMJOH "VUP4DBMJOH(SPVQ ".*

  59. ͨͱ͑͹ "NB[PO"VUP4DBMJOH Λ΍ΊΔ

  60. ͳͥ"NB[PO"VUP4DBMJOHͰ͸ͳ͍ͷ͔ CloudWatch 1෼ͷน Graceful Terminate ໰୊ ELB Ͱ͸ͳ͘ haproxy Λ࢖͍͍ͨ

    σϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͏ͨΊ
  61. $MPVE8BUDIͷੑೳ΁ͷෆຬ ෼ͷนɹϝτϦΫε͕औಘͰ͖Δͷ͸࠷଎Ͱ΋1෼ޙ +

  62. (SBDFGVM5FSNJOBUF໰୊ desired_capacity ͕ݮΔͱ͖ɺ
 Πϯελϯε͸໰౴ແ༻ͰTerminate ͞Εͯ͠·͏ ҆શͳαʔϏεΞ΢τΛߦ͏ʹ͸޻෉͕ඞཁ

  63. *OUFSOBM&-#Ͱ͸ͳ͘IBQSPYZΛ࢖͍͍ͨ ELB ͷ IP ΞυϨεͷಈతͳมԽʹ
 ϛυϧ΢ΣΞ͕ରԠ͖͠Εͳ͍ ! ϒϥοΫϘοΫε౓͕ߴ͘ɺ໰୊͕೺Ѳͮ͠Β͍ ˞&-#ͳ͠Ͱ΋"NB[PO"VUP4DBMJOH͸࢖͑·͢

  64. "NB[PO"VUP4DBMJOH WT EFQMPZTEBZ

  65. W W W W #

  66. W W W W W #

  67. W W W W W #

  68. W W W W W W W W #

  69. W W W W W W W W W W

    W #
  70. W W W W W W W W W ݹ͍όʔδϣϯͷΠϯελϯε͕࡞ΒΕͯ͠·͏

  71. "NB[PO"VUP4DBMJOHͰഉଞ੍ޚ σϓϩΠத͸ AutoScaling ΛࢭΊɺ AutoScaling த͸σϓϩΠΛېࢭ͍ͨ͠ ! AutoScaling ͕ΠϯελϯεΛ૿΍͢ͱ͖ɾݮΒ͢ͱ͖ʹ ॲཧΛׂΓ͜·ͤΔඞཁ͕͋Δ

    →ࠔ೉
  72. CloudWatch 1෼ͷน Graceful Terminate ໰୊ ELB Ͱ͸ͳ͘ haproxy Λ࢖͍͍ͨ σϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͍͍ͨ

    ࣗ෼Ͱશ෦ॻ͍ͯ΋͋·Γखؒ͸มΘΒͳͦ͏ "NB[PO"VUP4DBMJOHͰղܾෆՄೳͰ͸ͳ͍͚Ͳɺ ଟ͘ͷίʔυΛॻ͖ɺΧελϚΠζ͢Δඞཁ͕͋Δ + +
  73. $PPLQBE"VUP4DBMJOHͷ࣮૷ ,

  74. JNPO "4 # W W EFWFMPQFS W SQT '4UBUVTXPSLJOH '4UBUVTXPSLJOH

  75. JNPO "4 # W W EFWFMPQFS W SQT '4UBUVTXPSLJOH '4UBUVTXPSLJOH

  76. W W JNPO "4 # W W EFWFMPQFS W SQT

    '4UBUVTXPSLJOH '4UBUVTXPSLJOH
  77. W W JNPO "4 # W W EFWFMPQFS W SQT

    '4UBUVTXPSLJOH '4UBUVTXPSLJOH
  78. W W JNPO "4 # W W EFWFMPQFS W SQT

    '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH
  79. W W JNPO "4 # W W EFWFMPQFS W SQT

    '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH
  80. W W JNPO "4 # W W EFWFMPQFS W W

    SQT '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH
  81. W W JNPO "4 # W W EFWFMPQFS W W

    W W W W W SQT '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH
  82. W W JNPO "4 # W W EFWFMPQFS W W

    W W W W SQT '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH '4UBUVTXPSLJOH
  83. Φʔτεέʔϧͷ͋Δ฻Β͠ ✈

  84. ͱ͋ΔΠϯελϯεͷ ୆਺ ೔ Φʔτεέʔϧ

  85. ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫத σϓϩΠத

  86. ( ( ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫத σϓϩΠத

  87. Φʔτεέʔϧத σϓϩΠத

  88. None
  89. None
  90. None
  91. - ೥Ҏ্ӡ༻Ͱ͖ͯΔ 

  92. ೥ͷӡ༻࣮੷Λ௨ͯ͠ ۤ࿑ͨ͠఺ɺ޻෉ɺؾ͍ͮͨ͜ͱ

  93. ϩοΫ࣌ؒɾճ਺Λ࠷খʹ͢Δ σϓϩΠΛϩοΫʹՁ஋Λಧ͚ΔνϟϯεΛݮΒ͢ .

  94. ୆਺

  95. ୆਺ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ

  96. ୆਺ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ αʔϏε"ϩοΫճ αʔϏε#ϩοΫճ

  97. ୆਺ Ұ౓ʹ૿ݮ͢Δ୆਺Λௐ੔͢Δ͜ͱͰ ۀ຿࣌ؒதͷσϓϩΠϩοΫճ਺Λ࠷খݶʹ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ αʔϏε"ϩοΫճ αʔϏε#ϩοΫճ

  98. ୆਺ Ұ౓ʹ૿ݮ͢Δ୆਺Λௐ੔͢Δ͜ͱͰ ۀ຿࣌ؒதͷσϓϩΠϩοΫճ਺Λ࠷খݶʹ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ αʔϏε"ϩοΫճ αʔϏε#ϩοΫճ ۀ຿࣌ؒ֎ͷͨΊɺ ݮΒ͢ͱ͖͸খࠁΈʹ

  99. ςετΛॻ͘ Φʔτεέʔϧ͸αʔόΛམͱ͢ϓϩάϥϜ /

  100. ఆظతʹઃఆΛݟ௚͢ ์͓͍ͬͯͯ΋εέʔϧ͢ΔΑ͏ʹݟ͑Δ͕… ! ΞϓϦαʔόͷ୆਺૿ˠόοΫΤϯυ΁ͷίωΫγϣϯ૿ ΞϓϦέʔγϣϯͷύϑΥʔϚϯε΋ৗʹมԽ 0

  101. TUBUJDOPEFT EZOBNJDOPEFTߏ੒ ৗʹཱͪଓ͚ͯΔΠϯελϯε͕͍ͨ΄͏͕ ͳΜ͔ͩΜͩͰศར TUBUJD EZOBNJD

  102. None
  103. TUBUJDOPEFT EZOBNJDOPEFT Φʔτεέʔϧର৅֎ ୆਺͸ݻఆ TIVUEPXO͞Εͳ͍ Φʔτεέʔϧର৅ ୆਺͸Մม CBMBODFS

  104. αʔό໊Ͳ͏͠Α͏໰୊ dynamic nodes Ͱ͋Ζ͏ͱɺ
 ݁ہ hostname ͸͋ͬͨ΄͏͕؅ཧ͠΍͍͢ ! 10.0.123.45 ͩͬͨΒ

    app-10-0-123-45 ͱ͍͏
 Name λά͕ͭ͘Α͏ʹ͍ͯ͠Δ &
  105. Φʔτεέʔϧͷ෭࢈෺ ΦʔτεέʔϧΛ૊Ήͱɺ Immutable ͳ
 Golden Image ͕࡞ΕΔ ! ͍ͭͰ΋ҰൃͰ࠷৽ͷαʔό্ཱ͕ͪ͛ΒΕΔ͜ͱ͕
 อূ͞ΕΔ

    +
  106. ·ͱΊ

  107. EFQMPZTEBZWT "VUP4DBMJOH 10 deploys/day ͳ؀ڥͰ΋Φʔτεέʔϧ͸Մೳ ίετͱ։ൃ଎౓ͷτϨʔυΦϑΛߟྀͨ͠νϡʔχϯά

  108. "VUP4DBMJOHBT *NNVUBCMF*OGSBTUSVDUVSF Immutable Infrastructure ͸ະདྷͷ࿩͡Όͳ͘ɺ
 ΫοΫύουͰ͸ΦʔτεέʔϧʹΑͬͯ͢Ͱʹ ݱ࣮ʹͳͬͯΔ

  109. 2" '

  110. DSFEJUT http://aws.amazon.com/jp/architecture/icons/ http://www.entypo.com/ http://fortawesome.github.io/Font-Awesome/ http://www.flickr.com/photos/sora_h/8916379094/