Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Multicloud deploy with Spinnaker / builderscon tokyo 2018
Daiki Matsui
September 08, 2018
Technology
2
3k
Multicloud deploy with Spinnaker / builderscon tokyo 2018
builderscon tokyo 2018の発表スライドです。
Daiki Matsui
September 08, 2018
Tweet
Share
More Decks by Daiki Matsui
See All by Daiki Matsui
ikemonn
0
510
ikemonn
2
12k
ikemonn
0
84
Other Decks in Technology
See All in Technology
satoryu
0
2k
clustervr
0
190
fujiihda
8
640
shotakashihara
1
1.3k
vkbaba
0
110
smzksts
0
200
soachr
1
150
ymas0315
0
110
pinboro
1
1.4k
toshinoritakai
1
190
yasuakiomokawa
0
290
hecateball
1
12k
Featured
See All Featured
morganepeng
92
13k
bryan
30
3.3k
searls
204
35k
rmw
11
740
mza
80
4.1k
davidbonilla
69
3.5k
lara
16
2.6k
62gerente
587
200k
phodgson
87
3.9k
matthewcrist
73
7.5k
ammeep
656
54k
sugarenia
233
830k
Transcript
Multi-cloud deploy with Spinnaker @ikemonn
@ikemonn SRE at
Spinnakerͷ͜ͱΛ ͬͯΔਓ✋
SpinnakerΛࢼͯ͠ΈΑ͏ͱ ࢥͬͯΒ͏ ࠓͷΰʔϧ
CXϓϥοτϑΥʔϜ KARTE
4IPFT" 4IPFT# d d 4)011*/($"35 ๚தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ
ίϛϡχέʔγϣϯΛఏڙ
ϢʔβΛϦΞϧλΠϜͰ֬ೝͰ͖Δ
ඵؒ࠷େΠϕϯτ events / sec 22,000 1ͷ߹ܭΠϕϯτ events / day 1billion
ղੳ࣌ؒ sec 0.x
ϚϧνΫϥυ(AWS/GCP)
ϦΞϧλΠϜੑ͕ٻΊΒΕΔɺ ߴෛՙͳେنࢄγεςϜ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ • ୭ͰσϓϩΠͰ͖Δ
• ͷಛఆͱRollback͘͢͢͠ΔͨΊ • ࠷11ճσϓϩΠ • ଟ͍࣌10ճҎ্ খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ https://medium.com/@Zaiku/continuous-delivery-in-a-nutshell-29f4213dabda
• վળαΠΫϧͷߴԽ & SRE:3ਓ͚ͩ • σϓϩΠͷࣗಈԽ • σϓϩΠ࣌ʹ͕͋ͬͨ࣌ࣗಈrollback ୭ͰσϓϩΠͰ͖Δ http://susunshun.hatenablog.com/entry/2015/10/30/011554
https://www.autorabit.com/salesforce-deployment-automation/ σϓϩΠπʔϧʹٻΊΔ͜ͱ
େنΠϯϑϥʹසൟʹ҆ఆͨ͠ ࣗಈσϓϩΠ͕ߦ͑Δ͜ͱ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυԽ
• CodeDeployΛແཧΓ͍ͬͯͨ • ϚϧνΫϥυωΠςΟϒͷπʔϧͰͳ͍ͨ Ίɺ͕ى͖ҠߦΛܾఆ ϚϧνΫϥυॳظ https://docs.aws.amazon.com/ja_jp/codedeploy/latest/userguide/instances-on-premises.html
ϚϧνΫϥυରԠͷ σϓϩΠπʔϧΛ୳͢
https://cloudplatform.googleblog.com/2017/06/spinnaker-10-continuous-delivery.html
Spinnaker is an open source, multi-cloud continuous delivery platform for
releasing software changes with high velocity and confidence. https://netflix.github.io/
https://www.spinnaker.io/
• ଞͷ࡞ۀ͠ͳ͕ΒɺSRE2ਓͰ3ϲ݄ • σϓϩΠϑϩʔͷઃܭ • όάͷճආํ๏ͷࡧɺPRͷϚʔδͪ Ҡߦίετ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• ✅ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ✅ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ओཁΫϥυϓϩόΠμk8sͰར༻Մೳ ϚϧνΫϥυରԠ Azure VM Openstack Amazon EC2 Amazon ECS
• ֤ΫϥυͷࠩΛҙࣝͤͣʹσϓϩΠͰ͖Δ ϚϧνΫϥυରԠ
• ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• σϓϩΠͷਐḿΛཧը໘Ͱ֬ೝͰ͖Δ ҆ఆͨ͠σϓϩΠ
• stage, pipelineͰॊೈʹσϓϩΠϑϩʔΛ ΊΔ ҆ఆͨ͠σϓϩΠ
• stageͷbuild-inػೳ͕ॆ࣮͓ͯ͠ΓɺΓͨ ͍͜ͱେͰ͖Δ(20+) ҆ఆͨ͠σϓϩΠ
• Blue-GreenσϓϩΠΧφϦΞσϓϩΠʹ ରԠ(spinnaker/kayenta) ҆ఆͨ͠σϓϩΠ https://cloudplatform.googleblog.com/2018/04/introducing-Kayenta-an-open-automated-canary-analysis-tool-from-Google-and-Netflix.html
• Immutable Infraͷڧ੍ • ιʔείʔυͰͳ͘imageΛΓସ͑Δࢥ • ιʔείʔυઃఆ͕ҟͳΔαʔό͕ଘࡏ͠ͳ͍ • ෭࣍తͳޮՌͱͯ͠Spin-upͷߴԽ •
CodeDeployʹൺͯ5ഒ ҆ఆͨ͠σϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• roer/dcd-specͰpipelineͷઃఆΛཧ • ։ൃऴ͍ྃͯ͠ΔͷͰɺ͜Ε͔Βspin σϓϩΠϑϩʔͷίʔυԽ https://blog.spinnaker.io/spin-and-roer-managed-pipeline-templates-4fde2951c648
https://docs.google.com/document/d/17Ifm562aypjLSHnNJ02COFv_VMv3spicH-7zMYd-oXE/edit
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Gate͕REST APIΛఏڙ͍ͯ͠Δ • Pipelineͷ࣮ߦ݁Ռͷऔಘ͕Մೳ ࣗಈσϓϩΠ https://cloudacademy.com/blog/netflix-spinnaker/
• Pipelineͷىಈ/ޭ/ࣦഊΛ௨Մೳ • Slack, Mail, HipChat, SMS… ࣗಈσϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Netflix, Google͕ੵۃతʹ։ൃ ։ൃ͕׆ൃ https://cloud.withgoogle.com/next18/sf/sessions/session/155951
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
None
SpinnakerͰ Firewall, LB, ASGΛ࡞Մೳ
৽͍͠αʔόάϧʔϓΛ࡞͠ ͍ͯ͘ࢥ
'JSFXBMM -# "VUP4DBMJOH (SPVQ Ҡߦલ Ҡߦޙ
• ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• σϑΥϧτͷઃఆͩͱrate exceededͰσϓϩ Π͕ࣦഊ͕ͪ͠ AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ clouddriver.yml
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
https://cloud.google.com/marketplace/ ϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ϚωʔδυɾαʔϏε͕ແ͍ͷͰࣗͰӡ༻ • GCSʹconfigใΛอଘ • diskͷϚϯτ֤छઃఆεΫϦϓτԽ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• UI্ͷόάɺvupʹࣦഊ͢ΔɺCPU༻͕ಥ વ্͕Δɺϓϩηε͕ಥવࢮ͵ όά͕ଟ͍
GCEʹσϓϩΠޙΠϯελϯεͷ ͕Ұؾʹmin·ͰݮΔ RPSͰεέʔϧ͍ͤͯ͞Δ߹ɺLB,Φʔτεέʔ ϥʔͱSpinnakerͷ૬ੑ͕ѱ͍ͷͰ10ͭ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• υΩϡϝϯτ͕ॆ࣮ͯ͠ͳ͍ • ϒϩάͷใগͳ͍orݹ͘ͳ͍ͬͯΔ • GithubͷissueΈͨΓɺίʔυಡΜͩΓɺSlack Ͱ࣭ͨ͠Γ͢Δͷ͕ૣ͍ υΩϡϝϯτ͕গͳ͍
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυͷIaaSͷσϓϩΠ ʹSpinnaker͕Φεεϝ
ϚϧνΫϥυରԠ
σϓϩΠؔ࿈ͷػೳ͕ॆ࣮ https://sweetcode.io/13115-2/
Immutable Infraͷڧ੍ https://www.merriam-webster.com/assets/mw/word-of-the-day/social/7bcf417841f7ce37cd7e7e1427c0504f.jpg
ΛΓӽ͑Δඞཁ͋Γ όάͷଟ͞ υΩϡϝϯτͷগͳ͞
σϓϩΠπʔϧͰΜͰΔͳΒ ੋඇݕ౼͍ͯͩ͘͠͞ʂ
͓खݩͷQRίʔυ͔Β ͥͻϑΟʔυόοΫΛ͓ئ͍͠·͢
͓·͚
શମͷpipelineߏ
deploy-all deploy-aws deploy-gcp deploy-role1 deploy-role2 … Pipeline (શମ) deploy-role1 deploy-role2
…
Pipeline (֤Ϋϥυ) αʔόͷroleຖʹσϓϩΠ & rollbackՄೳ
Pipeline (֤Ϋϥυ) Pipeline parameter & conditional on ExpressionͰpipelineΛ੍ޚ
֤छϚΠΫϩαʔϏεͷઃఆ • ~/.hal/$DEPLOYMENT/profiles/ ҎԼʹϑ ΝΠϧΛ࡞ & hal deploy apply ɾ/opt/spinnaker/configʹө͞ΕΔ(v1.9)