Cookpad's deployment and auto scaling

クックパッドのデプロイとオートスケール
2014.03.15 JAWS DAYS 2014 Immutable Infrastructure track
http://jawsdays2014.jaws-ug.jp/speaker/i_naruta/

Transcript

1. ΫοΫύουͷ
   σϓϩΠͱ
   Φʔτεέʔϧ ΫοΫύουגࣜձࣾ
   ੒ాҰੜ
   
   +"84
   %":4
   
2. KBXTEBZT
   JOGSB
3. ੒ా
   Ұੜ ͳΔͨ ͍͍ͬͤ !NJSBLVJ ΫοΫύουגࣜձࣾ
   ΠϯϑϥετϥΫνϟʔ෦௕
4. ࠓ೔ͷ࿩ ΫοΫύουʹ͍ͭͯ ϓϩϏδϣχϯάʢαʔϏεσΟεΧόϦʣ ΞϓϦέʔγϣϯͷσϓϩΠ Φʔτεέʔϧ !
5. ΫοΫύουʹ͍ͭͯ
6. 'VMM
   "84 2011೥3݄ ౦ژϦʔδϣϯΦʔϓϯ 2011೥8݄ DC͔ΒAWS΁Ҡߦ ✈
7. .
   6#NP .
   14
   VTFST .
   DPPLJOH
   SFDJQFT ˞೥݄ݱࡏ !" # #
8. 
   &$
   JOTUBODFT 
   &-#T 
   4
   #VDLFUT 
   3PVUF
   3FDPSET
9. 3FETIJGU
   GPS
   #* UBCFNJSVDPN 4/4
   GPS
   .PCJMF
   1VTI
   /PUJpDBUJPO
10. 
    0QT 
    %FWFMPQFST 
    %FQMPZTEBZ ⚒ #
11. ΞϓϦέʔγϣϯίʔυͷσϓϩΠ
12. $ QVMM
    SFRVFTU ()& # EFWFMPQFS
13. $ QVMM
    SFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS
14. $ -(5. QVMM
    SFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS
15. $ -(5. QVMM
    SFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS NFSHF
16. $ -(5. QVMM
    SFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS $* +FOLJOT NFSHF
17. $ TVDDFTT -(5. QVMM
    SFRVFTU NFOUJPO ()& # EFWFMPQFS # SFWJFXFS $* +FOLJOT NFSHF
18. $ TVDDFTT -(5. QVMM
    SFRVFTU NFOUJPO EFQMPZ &$ ()& # EFWFMPQFS # SFWJFXFS $* +FOLJOT NFSHF
19. σϓϩΠϧʔϧʢҰ෦ʣ • CI Λύεͨ͠ϦϏδϣϯͷΈσϓϩΠͯ͠Α͍ • σϓϩΠ͸ίʔυΛ push ͨ͠։ൃऀࣗ਎͕ߦ͏ • Ӧۀ࣌ؒ಺ͷΈσϓϩΠՄೳ • σϓϩΠޙ͸։ൃऀ͕ಈ࡞֬ೝ͠ɺ
 ෆ۩߹Λݟ͚ͭͨΒ͙͢ʹϩʔϧόοΫ͢Δ
20. σϓϩΠޙʹΤϥʔ΍ϨεϙϯελΠϜྼԽ͕ແ͍͜ͱΛ֬ೝ ϦΞϧλΠϜϞχλϦϯάπʔϧ
    lJNPOz
21. ΫοΫύουͷ
    αʔόϓϩϏδϣχϯά
22. αʔϏεσΟεΧόϦ໰୊ αʔό͕ಈతʹ૿͑ͨΓݮͬͨΓ͢Δঢ়گͰ͸
 ੩తͳ”αʔό؅ཧ୆ா”͸ӡ༻Ͱ͖ͳ͍ ! αʔό͕௥Ճ͞ΕͨΒݕ஌͢Δ࢓૊Έ͕ඞཁ →Serf, Zookeeper ౳ %
23. &$λάʹΑΔσΟεΧόϦ Name λά Role λά Status λά &
24. &$ͷλάػೳʹ͍ͭͯ key: value ͷϖΞ
25. ΫοΫύουͰͷ&$λά ͦͷαʔόͷଐੑ΍ঢ়ଶΛهड़ ! Πϯελϯε͸ࣗ෼ࣗ਎΍ɺଞΠϯελϯεͷλάΛݟͯ ࣗ཯ڠௐతʹಈ͘
26. /BNF
    λά ͦͷΠϯελϯεͷϗετ໊ DNSαʔό͕ࣗಈݕ஌͠ɺAϨίʔυΛ௥Ճ ! ྫ: Name: app-001 &
27.  
28. ' /BNF
    EC  
29. ' /BNF
    EC ' /BNF
    EC  
30. ' /BNF
    EC ' /BNF
    EC   1PXFS%/4
31. ' /BNF
    EC ' /BNF
    EC   1PXFS%/4 ⟲ (
32. ' /BNF
    EC ' /BNF
    EC   1PXFS%/4 ⟲ ( EC
    "
    
    EC
    "
     %/4
    αʔό͕ఆظతʹશ
    /BNF
    λάΛ૸ࠪ͠ɺ
    %/4ϨίʔυΛߋ৽
33. 3PMF
    λά Πϯελϯεͷ໾ׂΛهड़ ! ྫ: Role: app &
34. QVQQFU
    NBTUFS )
35. ' 3PMF
    BQQ QVQQFU
    NBTUFS )
36. ' 3PMF
    BQQ BQQMZ
    lBQQz QVQQFU
    NBTUFS )
37. ' 3PMF
    BQQ ' 3PMF
    EC BQQMZ
    lBQQz QVQQFU
    NBTUFS )
38. ' 3PMF
    BQQ ' 3PMF
    EC BQQMZ
    lBQQz QVQQFU
    NBTUFS BQQMZ
    lECz )
39. ' 3PMF
    BQQ ' 3PMF
    EC BQQMZ
    lBQQz QVQQFU
    NBTUFS BQQMZ
    lECz QVQQFU
    NBTUFS
    ͸ద༻ઌͷ
    3PMF
    λάΛݟͯ
 ద༻͢ΔΫϥεΛࣗಈͰܾఆ )
40. 4UBUVT
    λά ͦͷΠϯελϯε͕όϥϯαʔʹ
 ௥Ճ͍͍ͯ͠΋ͷ͔Ͳ͏͔Λఆٛ ! ྫ: Status: working &
41. 3PMF
    BQQ * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ
42. 3PMF
    BQQ ' 4UBUVT
    XPSLJOH * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ
43. 3PMF
    BQQ ' 4UBUVT
    XPSLJOH ⟲ * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ
44. 3PMF
    BQQ ' 4UBUVT
    XPSLJOH ⟲ * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ
45. 3PMF
    BQQ ' 4UBUVT
    XPSLJOH ⟲ * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ ' 4UBUVT
    XPSLJOH
46. 3PMF
    BQQ ' 4UBUVT
    XPSLJOH ⟲ * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ ' 4UBUVT
    XPSLJOH
47. 3PMF
    BQQ ' 4UBUVT
    XPSLJOH ⟲ * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ ' 4UBUVT
    XPSLJOH
48. 3PMF
    BQQ ' 4UBUVT
    XPSLJOH ⟲ * IBQSPYZDGH BQQ
    IBQSPYZ 3PMF
    BQQ ' 4UBUVT
    XPSLJOH XPSLJOH
    ঢ়ଶͷΠϯελϯεΛࣗಈͰαʔϏεΠϯ
49. ' ͳͥλάͰશ෦΍Δͷ͔ Zookeeper, Serf ౳ͷಛผͳϛυϧ΢ΣΞΛඞཁͱͤͣʹ
 αʔϏεσΟεΧόϦ͕Մೳ ! ඇৗʹγϯϓϧ͔ͭϓϩάϥϚϒϧ
50. λάͷϋϚϦͲ͜Ζ DescribeTags ͷ API Limit 1 Ϧιʔεʹλά͸10ݸ·Ͱ % -JNJU DBDIF (
51. πʔϧ λάϑϨϯυϦʔͳ؅ཧπʔϧ͕ඞཁ ElasticFox-ec2tag
52. Φʔτεέʔϧ
53. τϥϑΟοΫ &$Πϯελϯε਺ ࣌ؒ τϥϑΟοΫͷมԽʹԠͯ͡Πϯελϯε਺Λ૿ݮ
54. ͳͥΦʔτεέʔϧͳͷ͔ '
55. *OTUBODFIPVS EC2ͷ࠷খ՝ۚ୯Ґ ىಈதͷΠϯελϯε਺ x ىಈ࣌ؒʢ1࣌ؒ୯Ґʣ ΦϯσϚϯυɺ౦ژϦʔδϣϯ
    ೥݄೔ݱࡏ
    IUUQBXTBNB[PODPNKQFDQSJDJOH
56. &$Πϯελϯε਺ I          Πϯελϯε࣌  ( 
    JOTUBODFIPVS Φʔτεέʔϧ
    ͳ͠ Φʔτεέʔϧ
    ͋Γ
57. "NB[PO
    "VUP
    4DBMJOH
58. $MPVE8BUDI "VUP
    4DBMJOH "VUP
    4DBMJOH
    (SPVQ ".*
59. ͨͱ͑͹
    "NB[PO
    "VUP4DBMJOH
    Λ΍ΊΔ
60. ͳͥ
    "NB[PO
    "VUP4DBMJOH
    Ͱ͸ͳ͍ͷ͔ CloudWatch 1෼ͷน Graceful Terminate ໰୊ ELB Ͱ͸ͳ͘ haproxy Λ࢖͍͍ͨ σϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͏ͨΊ
61. $MPVE8BUDI
    ͷੑೳ΁ͷෆຬ ෼ͷนɹϝτϦΫε͕औಘͰ͖Δͷ͸࠷଎Ͱ΋1෼ޙ +
62. (SBDFGVM
    5FSNJOBUF
    ໰୊ desired_capacity ͕ݮΔͱ͖ɺ
 Πϯελϯε͸໰౴ແ༻ͰTerminate ͞Εͯ͠·͏ ҆શͳαʔϏεΞ΢τΛߦ͏ʹ͸޻෉͕ඞཁ
63. *OUFSOBM
    &-#
    Ͱ͸ͳ͘
    IBQSPYZ
    Λ࢖͍͍ͨ ELB ͷ IP ΞυϨεͷಈతͳมԽʹ
 ϛυϧ΢ΣΞ͕ରԠ͖͠Εͳ͍ ! ϒϥοΫϘοΫε౓͕ߴ͘ɺ໰୊͕೺Ѳͮ͠Β͍ ˞&-#ͳ͠Ͱ΋
    "NB[PO
    "VUP4DBMJOH
    ͸࢖͑·͢
64. "NB[PO
    "VUP4DBMJOH
    WT
    
    EFQMPZTEBZ
65. W W W W #
66. W W W W W #
67. W W W W W #
68. W W W W W W W W #
69. W W W W W W W W W W W #
70. W W W W W W W W W ݹ͍όʔδϣϯͷΠϯελϯε͕࡞ΒΕͯ͠·͏
71. "NB[PO
    "VUP4DBMJOH
    Ͱഉଞ੍ޚ σϓϩΠத͸ AutoScaling ΛࢭΊɺ AutoScaling த͸σϓϩΠΛېࢭ͍ͨ͠ ! AutoScaling ͕ΠϯελϯεΛ૿΍͢ͱ͖ɾݮΒ͢ͱ͖ʹ ॲཧΛׂΓ͜·ͤΔඞཁ͕͋Δ →ࠔ೉
72. CloudWatch 1෼ͷน Graceful Terminate ໰୊ ELB Ͱ͸ͳ͘ haproxy Λ࢖͍͍ͨ σϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͍͍ͨ ࣗ෼Ͱશ෦ॻ͍ͯ΋͋·Γखؒ͸มΘΒͳͦ͏ "NB[PO
    "VUP4DBMJOH
    ͰղܾෆՄೳͰ͸ͳ͍͚Ͳɺ
    ଟ͘ͷίʔυΛॻ͖ɺΧελϚΠζ͢Δඞཁ͕͋Δ + +
73. $PPLQBE
    "VUP4DBMJOH
    ͷ࣮૷ ,
74. JNPO "4 # W W EFWFMPQFS W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
75. JNPO "4 # W W EFWFMPQFS W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
76. W W JNPO "4 # W W EFWFMPQFS W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
77. W W JNPO "4 # W W EFWFMPQFS W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
78. W W JNPO "4 # W W EFWFMPQFS W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
79. W W JNPO "4 # W W EFWFMPQFS W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
80. W W JNPO "4 # W W EFWFMPQFS W W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
81. W W JNPO "4 # W W EFWFMPQFS W W W W W W W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
82. W W JNPO "4 # W W EFWFMPQFS W W W W W W SQT '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH '4UBUVT
    XPSLJOH
83. Φʔτεέʔϧͷ͋Δ฻Β͠ ✈
84. ͱ͋ΔΠϯελϯεͷ
    ୆਺ ೔ Φʔτεέʔϧ
85. ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫத σϓϩΠத
86. ( ( ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫத σϓϩΠத
87. Φʔτεέʔϧத σϓϩΠத
91. - ೥Ҏ্ӡ༻Ͱ͖ͯΔ 
92. ೥ͷӡ༻࣮੷Λ௨ͯ͠ ۤ࿑ͨ͠఺ɺ޻෉ɺؾ͍ͮͨ͜ͱ
93. ϩοΫ࣌ؒɾճ਺Λ࠷খʹ͢Δ σϓϩΠΛϩοΫʹՁ஋Λಧ͚ΔνϟϯεΛݮΒ͢ .
94. ୆਺
95. ୆਺ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ
96. ୆਺ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ αʔϏε"
    ϩοΫճ αʔϏε#
    ϩοΫճ
97. ୆਺ Ұ౓ʹ૿ݮ͢Δ୆਺Λௐ੔͢Δ͜ͱͰ
    ۀ຿࣌ؒதͷσϓϩΠϩοΫճ਺Λ࠷খݶʹ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ αʔϏε"
    ϩοΫճ αʔϏε#
    ϩοΫճ
98. ୆਺ Ұ౓ʹ૿ݮ͢Δ୆਺Λௐ੔͢Δ͜ͱͰ
    ۀ຿࣌ؒதͷσϓϩΠϩοΫճ਺Λ࠷খݶʹ ۀ຿࣌ؒ σϓϩΠՄೳ࣌ؒଳ αʔϏε"
    ϩοΫճ αʔϏε#
    ϩοΫճ ۀ຿࣌ؒ֎ͷͨΊɺ
    ݮΒ͢ͱ͖͸খࠁΈʹ
99. ςετΛॻ͘ Φʔτεέʔϧ͸αʔόΛམͱ͢ϓϩάϥϜ /
100. ఆظతʹઃఆΛݟ௚͢ ์͓͍ͬͯͯ΋εέʔϧ͢ΔΑ͏ʹݟ͑Δ͕… ! ΞϓϦαʔόͷ୆਺૿ˠόοΫΤϯυ΁ͷίωΫγϣϯ૿ ΞϓϦέʔγϣϯͷύϑΥʔϚϯε΋ৗʹมԽ 0
101. TUBUJD
     OPEFT
     
     EZOBNJD
     OPEFT
     ߏ੒ ৗʹཱͪଓ͚ͯΔΠϯελϯε͕͍ͨ΄͏͕ ͳΜ͔ͩΜͩͰศར TUBUJD EZOBNJD
103. TUBUJD
     OPEFT EZOBNJD
     OPEFT Φʔτεέʔϧର৅֎
     ୆਺͸ݻఆ
     TIVUEPXO
     ͞Εͳ͍ Φʔτεέʔϧର৅
     ୆਺͸Մม CBMBODFS
104. αʔό໊Ͳ͏͠Α͏໰୊ dynamic nodes Ͱ͋Ζ͏ͱɺ
 ݁ہ hostname ͸͋ͬͨ΄͏͕؅ཧ͠΍͍͢ ! 10.0.123.45 ͩͬͨΒ app-10-0-123-45 ͱ͍͏
 Name λά͕ͭ͘Α͏ʹ͍ͯ͠Δ &
105. Φʔτεέʔϧͷ෭࢈෺ ΦʔτεέʔϧΛ૊Ήͱɺ Immutable ͳ
 Golden Image ͕࡞ΕΔ ! ͍ͭͰ΋ҰൃͰ࠷৽ͷαʔό্ཱ͕ͪ͛ΒΕΔ͜ͱ͕
 อূ͞ΕΔ +
106. ·ͱΊ
107. 
     EFQMPZTEBZ
     WT
     "VUP
     4DBMJOH 10 deploys/day ͳ؀ڥͰ΋Φʔτεέʔϧ͸Մೳ ίετͱ։ൃ଎౓ͷτϨʔυΦϑΛߟྀͨ͠νϡʔχϯά
108. "VUP
     4DBMJOH
     BT
     *NNVUBCMF
     *OGSBTUSVDUVSF Immutable Infrastructure ͸ະདྷͷ࿩͡Όͳ͘ɺ
 ΫοΫύουͰ͸ΦʔτεέʔϧʹΑͬͯ͢Ͱʹ ݱ࣮ʹͳͬͯΔ
109. 2" '
110. DSFEJUT http://aws.amazon.com/jp/architecture/icons/ http://www.entypo.com/ http://fortawesome.github.io/Font-Awesome/ http://www.flickr.com/photos/sora_h/8916379094/