クックパッドのデプロイとオートスケール 2014.03.15 JAWS DAYS 2014 Immutable Infrastructure track http://jawsdays2014.jaws-ug.jp/speaker/i_naruta/
ΫοΫύουͷσϓϩΠͱΦʔτεέʔϧΫοΫύουגࣜձࣾాҰੜ+"84%":4
View Slide
KBXTEBZTJOGSB
ాҰੜͳΔͨ ͍͍ͬͤ!NJSBLVJΫοΫύουגࣜձࣾΠϯϑϥετϥΫνϟʔ෦
ࠓͷΫοΫύουʹ͍ͭͯϓϩϏδϣχϯάʢαʔϏεσΟεΧόϦʣΞϓϦέʔγϣϯͷσϓϩΠΦʔτεέʔϧ!
ΫοΫύουʹ͍ͭͯ
'VMM"8420113݄ ౦ژϦʔδϣϯΦʔϓϯ20118݄ DC͔ΒAWSҠߦ✈
.6#NP.14VTFST.DPPLJOHSFDJQFT˞݄ݱࡏ!"##
&$JOTUBODFT&-#T4#VDLFUT3PVUF3FDPSET
3FETIJGUGPS#*UBCFNJSVDPN4/4GPS.PCJMF1VTI/PUJpDBUJPO
0QT%FWFMPQFST%FQMPZTEBZ⚒#
ΞϓϦέʔγϣϯίʔυͷσϓϩΠ
$QVMMSFRVFTU()&#EFWFMPQFS
$QVMMSFRVFTUNFOUJPO()&#EFWFMPQFS#SFWJFXFS
$-(5.QVMMSFRVFTUNFOUJPO()&#EFWFMPQFS#SFWJFXFS
$-(5.QVMMSFRVFTUNFOUJPO()&#EFWFMPQFS#SFWJFXFSNFSHF
$-(5.QVMMSFRVFTUNFOUJPO()&#EFWFMPQFS#SFWJFXFS$*+FOLJOTNFSHF
$TVDDFTT-(5.QVMMSFRVFTUNFOUJPO()&#EFWFMPQFS#SFWJFXFS$*+FOLJOTNFSHF
$TVDDFTT-(5.QVMMSFRVFTUNFOUJPOEFQMPZ&$()&#EFWFMPQFS#SFWJFXFS$*+FOLJOTNFSHF
σϓϩΠϧʔϧʢҰ෦ʣ• CI Λύεͨ͠ϦϏδϣϯͷΈσϓϩΠͯ͠Α͍• σϓϩΠίʔυΛ push ͨ͠։ൃऀ͕ࣗߦ͏• Ӧۀ࣌ؒͷΈσϓϩΠՄೳ• σϓϩΠޙ։ൃऀ͕ಈ࡞֬ೝ͠ɺ ෆ۩߹Λݟ͚ͭͨΒ͙͢ʹϩʔϧόοΫ͢Δ
σϓϩΠޙʹΤϥʔϨεϙϯελΠϜྼԽ͕ແ͍͜ͱΛ֬ೝϦΞϧλΠϜϞχλϦϯάπʔϧlJNPOz
ΫοΫύουͷαʔόϓϩϏδϣχϯά
αʔϏεσΟεΧόϦαʔό͕ಈతʹ૿͑ͨΓݮͬͨΓ͢Δঢ়گͰ ੩తͳ”αʔόཧா”ӡ༻Ͱ͖ͳ͍!αʔό͕Ճ͞ΕͨΒݕ͢ΔΈ͕ඞཁ→Serf, Zookeeper %
&$λάʹΑΔσΟεΧόϦName λάRole λάStatus λά&
&$ͷλάػೳʹ͍ͭͯkey: value ͷϖΞ
ΫοΫύουͰͷ&$λάͦͷαʔόͷଐੑঢ়ଶΛهड़!ΠϯελϯεࣗࣗɺଞΠϯελϯεͷλάΛݟͯࣗڠௐతʹಈ͘
/BNFλάͦͷΠϯελϯεͷϗετ໊DNSαʔό͕ࣗಈݕ͠ɺAϨίʔυΛՃ!ྫ:Name: app-001&
' /BNFEC
' /BNFEC' /BNFEC
' /BNFEC' /BNFEC1PXFS%/4
' /BNFEC' /BNFEC1PXFS%/4⟲(
' /BNFEC' /BNFEC1PXFS%/4⟲(EC"EC"%/4αʔό͕ఆظతʹશ/BNFλάΛࠪ͠ɺ%/4ϨίʔυΛߋ৽
3PMFλάΠϯελϯεͷׂΛهड़!ྫ:Role: app&
QVQQFUNBTUFS )
' 3PMFBQQQVQQFUNBTUFS )
' 3PMFBQQBQQMZlBQQzQVQQFUNBTUFS )
' 3PMFBQQ' 3PMFECBQQMZlBQQzQVQQFUNBTUFS )
' 3PMFBQQ' 3PMFECBQQMZlBQQzQVQQFUNBTUFSBQQMZlECz)
' 3PMFBQQ' 3PMFECBQQMZlBQQzQVQQFUNBTUFSBQQMZlECzQVQQFUNBTUFSద༻ઌͷ3PMFλάΛݟͯ ద༻͢ΔΫϥεΛࣗಈͰܾఆ)
4UBUVTλάͦͷΠϯελϯε͕όϥϯαʔʹ Ճ͍͍ͯ͠ͷ͔Ͳ͏͔Λఆٛ!ྫ:Status: working&
3PMFBQQ*IBQSPYZDGHBQQIBQSPYZ3PMFBQQ
3PMFBQQ' 4UBUVTXPSLJOH*IBQSPYZDGHBQQIBQSPYZ3PMFBQQ
3PMFBQQ' 4UBUVTXPSLJOH⟲*IBQSPYZDGHBQQIBQSPYZ3PMFBQQ
3PMFBQQ' 4UBUVTXPSLJOH⟲*IBQSPYZDGHBQQIBQSPYZ3PMFBQQ' 4UBUVTXPSLJOH
3PMFBQQ' 4UBUVTXPSLJOH⟲*IBQSPYZDGHBQQIBQSPYZ3PMFBQQ' 4UBUVTXPSLJOHXPSLJOHঢ়ଶͷΠϯελϯεΛࣗಈͰαʔϏεΠϯ
'ͳͥλάͰશ෦Δͷ͔Zookeeper, Serf ͷಛผͳϛυϧΣΞΛඞཁͱͤͣʹ αʔϏεσΟεΧόϦ͕Մೳ!ඇৗʹγϯϓϧ͔ͭϓϩάϥϚϒϧ
λάͷϋϚϦͲ͜ΖDescribeTags ͷ API Limit1 Ϧιʔεʹλά10ݸ·Ͱ%-JNJUDBDIF(
πʔϧλάϑϨϯυϦʔͳཧπʔϧ͕ඞཁElasticFox-ec2tag
Φʔτεέʔϧ
τϥϑΟοΫ&$Πϯελϯε࣌ؒτϥϑΟοΫͷมԽʹԠͯ͡ΠϯελϯεΛ૿ݮ
ͳͥΦʔτεέʔϧͳͷ͔'
*OTUBODFIPVSEC2ͷ࠷খ՝ۚ୯ҐىಈதͷΠϯελϯε x ىಈ࣌ؒʢ1࣌ؒ୯ҐʣΦϯσϚϯυɺ౦ژϦʔδϣϯ݄ݱࡏIUUQBXTBNB[PODPNKQFDQSJDJOH
&$ΠϯελϯεI Πϯελϯε࣌ ( JOTUBODFIPVSΦʔτεέʔϧͳ͠Φʔτεέʔϧ͋Γ
"NB[PO"VUP4DBMJOH
$MPVE8BUDI"VUP4DBMJOH"VUP4DBMJOH(SPVQ".*
ͨͱ͑"NB[PO"VUP4DBMJOHΛΊΔ
ͳͥ"NB[PO"VUP4DBMJOHͰͳ͍ͷ͔CloudWatch 1ͷนGraceful Terminate ELB Ͱͳ͘ haproxy Λ͍͍ͨσϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͏ͨΊ
$MPVE8BUDIͷੑೳͷෆຬͷนɹϝτϦΫε͕औಘͰ͖Δͷ࠷Ͱ1ޙ+
(SBDFGVM5FSNJOBUFdesired_capacity ͕ݮΔͱ͖ɺ Πϯελϯεແ༻ͰTerminate ͞Εͯ͠·͏҆શͳαʔϏεΞτΛߦ͏ʹ͕ඞཁ
*OUFSOBM&-#Ͱͳ͘IBQSPYZΛ͍͍ͨELB ͷ IP ΞυϨεͷಈతͳมԽʹ ϛυϧΣΞ͕ରԠ͖͠Εͳ͍!ϒϥοΫϘοΫε͕ߴ͘ɺ͕Ѳͮ͠Β͍˞&-#ͳ͠Ͱ"NB[PO"VUP4DBMJOH͑·͢
"NB[PO"VUP4DBMJOHWTEFQMPZTEBZ
W W WW#
W W WWW#
W W WWWW W W#
W W W W W WWWW W W#
W W W W W WW W Wݹ͍όʔδϣϯͷΠϯελϯε͕࡞ΒΕͯ͠·͏
"NB[PO"VUP4DBMJOHͰഉଞ੍ޚσϓϩΠத AutoScaling ΛࢭΊɺAutoScaling தσϓϩΠΛېࢭ͍ͨ͠!AutoScaling ͕ΠϯελϯεΛ૿͢ͱ͖ɾݮΒ͢ͱ͖ʹॲཧΛׂΓ͜·ͤΔඞཁ͕͋Δ→ࠔ
CloudWatch 1ͷนGraceful Terminate ELB Ͱͳ͘ haproxy Λ͍͍ͨσϓϩΠͱͷഉଞ੍ޚΛεϜʔζʹߦ͍͍ͨࣗͰશ෦ॻ͍ͯ͋·ΓखؒมΘΒͳͦ͏"NB[PO"VUP4DBMJOHͰղܾෆՄೳͰͳ͍͚Ͳɺଟ͘ͷίʔυΛॻ͖ɺΧελϚΠζ͢Δඞཁ͕͋Δ++
$PPLQBE"VUP4DBMJOHͷ࣮,
JNPO "4# WWEFWFMPQFSWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH
JNPO "4#WWEFWFMPQFSWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH
WWJNPO "4#WWEFWFMPQFSWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH
WWJNPO "4# WWEFWFMPQFSWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH
WWJNPO "4# WWEFWFMPQFSWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH
WWJNPO "4# WWEFWFMPQFS WSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH
WWJNPO "4# WWEFWFMPQFS WWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH
WWJNPO "4# WWEFWFMPQFS WWWWWWWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH
WWJNPO "4# WWEFWFMPQFSWWWWWWSQT'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH'4UBUVTXPSLJOH
Φʔτεέʔϧͷ͋ΔΒ͠✈
ͱ͋ΔΠϯελϯεͷΦʔτεέʔϧ
ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫதσϓϩΠத
((ΦʔτεέʔϧʹΑΔσϓϩΠϩοΫதσϓϩΠத
ΦʔτεέʔϧதσϓϩΠத
- Ҏ্ӡ༻Ͱ͖ͯΔ
ͷӡ༻࣮Λ௨ͯۤ͠࿑ͨ͠ɺɺؾ͍ͮͨ͜ͱ
ϩοΫ࣌ؒɾճΛ࠷খʹ͢ΔσϓϩΠΛϩοΫʹՁΛಧ͚ΔνϟϯεΛݮΒ͢.
ۀ࣌ؒ σϓϩΠՄೳ࣌ؒଳ
ۀ࣌ؒ σϓϩΠՄೳ࣌ؒଳαʔϏε"ϩοΫճαʔϏε#ϩοΫճ
Ұʹ૿ݮ͢ΔΛௐ͢Δ͜ͱͰۀ࣌ؒதͷσϓϩΠϩοΫճΛ࠷খݶʹۀ࣌ؒ σϓϩΠՄೳ࣌ؒଳαʔϏε"ϩοΫճαʔϏε#ϩοΫճ
Ұʹ૿ݮ͢ΔΛௐ͢Δ͜ͱͰۀ࣌ؒதͷσϓϩΠϩοΫճΛ࠷খݶʹۀ࣌ؒ σϓϩΠՄೳ࣌ؒଳαʔϏε"ϩοΫճαʔϏε#ϩοΫճۀ࣌ؒ֎ͷͨΊɺݮΒ͢ͱ͖খࠁΈʹ
ςετΛॻ͘ΦʔτεέʔϧαʔόΛམͱ͢ϓϩάϥϜ/
ఆظతʹઃఆΛݟ͢์͓͍ͬͯͯεέʔϧ͢ΔΑ͏ʹݟ͑Δ͕…!ΞϓϦαʔόͷ૿ˠόοΫΤϯυͷίωΫγϣϯ૿ΞϓϦέʔγϣϯͷύϑΥʔϚϯεৗʹมԽ0
TUBUJDOPEFTEZOBNJDOPEFTߏৗʹཱͪଓ͚ͯΔΠϯελϯε͕͍ͨ΄͏͕ͳΜ͔ͩΜͩͰศརTUBUJDEZOBNJD
TUBUJDOPEFT EZOBNJDOPEFTΦʔτεέʔϧର֎ݻఆTIVUEPXO͞Εͳ͍ΦʔτεέʔϧରՄมCBMBODFS
αʔό໊Ͳ͏͠Α͏dynamic nodes Ͱ͋Ζ͏ͱɺ ݁ہ hostname ͋ͬͨ΄͏͕ཧ͍͢͠!10.0.123.45 ͩͬͨΒ app-10-0-123-45 ͱ͍͏ Name λά͕ͭ͘Α͏ʹ͍ͯ͠Δ&
Φʔτεέʔϧͷ෭࢈ΦʔτεέʔϧΛΉͱɺ Immutable ͳ Golden Image ͕࡞ΕΔ!͍ͭͰҰൃͰ࠷৽ͷαʔό্ཱ͕ͪ͛ΒΕΔ͜ͱ͕ อূ͞ΕΔ +
·ͱΊ
EFQMPZTEBZWT"VUP4DBMJOH10 deploys/day ͳڥͰΦʔτεέʔϧՄೳίετͱ։ൃͷτϨʔυΦϑΛߟྀͨ͠νϡʔχϯά
"VUP4DBMJOHBT*NNVUBCMF*OGSBTUSVDUVSFImmutable Infrastructure ະདྷͷ͡Όͳ͘ɺ ΫοΫύουͰΦʔτεέʔϧʹΑͬͯ͢Ͱʹݱ࣮ʹͳͬͯΔ
2"'
DSFEJUThttp://aws.amazon.com/jp/architecture/icons/http://www.entypo.com/http://fortawesome.github.io/Font-Awesome/http://www.flickr.com/photos/sora_h/8916379094/