Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Multicloud deploy with Spinnaker / builderscon ...
Search
Daiki Matsui
September 08, 2018
Technology
2
4.3k
Multicloud deploy with Spinnaker / builderscon tokyo 2018
builderscon tokyo 2018の発表スライドです。
Daiki Matsui
September 08, 2018
Tweet
Share
More Decks by Daiki Matsui
See All by Daiki Matsui
The Fourth Age of SRE
ikemonn
0
200
マルチクラウドな大規模サービスKARTEを支えるDatadog/multi-cloud monitoring with Datadog
ikemonn
0
2.2k
KARTE を支えるマルチプラットフォームインフラ監視 /karte-multi-platform-monitoring
ikemonn
2
15k
Node Apps Performance Tuning
ikemonn
0
130
Other Decks in Technology
See All in Technology
2024年のナビゲーション・フォーカス対応:Composeでキーボード・ナビゲーションをサポートしよう
tahia910
0
100
Estrategias de escalabilidade para projetos web
jessilyneh
2
230
Tricentisにおけるテスト自動化へのAI活用ご紹介/20240910Shunsuke Katakura
shift_evolve
0
180
Jetpack Compose Modifier 徹底解説 / Jetpack Compose Modifier
wiroha
0
140
スタッフエンジニアの道: The Staff Engineer’s Path
snoozer05
PRO
41
13k
EitherT_with_Future
aoiroaoino
1
1.1k
Oracle Base Database Service:サービス概要のご紹介
oracle4engineer
PRO
0
13k
やってやろうじゃないかメカアジャイル! / Let's do it, mechanical agile!
psj59129
1
180
contenteditableと向き合う
kikuchikakeru
2
290
サーバー管理しないサーバーサービスManaged DevOps Pool
kkamegawa
0
110
OR学会2024秋_短期収益と将来のオフ方策評価性能を考慮したクーポン割当方策混合比の決定
recruitengineers
PRO
4
430
サプライチェーン攻撃に備える
ryunen344
0
160
Featured
See All Featured
What the flash - Photography Introduction
edds
67
11k
Fontdeck: Realign not Redesign
paulrobertlloyd
80
5.1k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
663
120k
How GitHub (no longer) Works
holman
310
140k
We Have a Design System, Now What?
morganepeng
48
7.1k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
123
18k
Debugging Ruby Performance
tmm1
72
12k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
26
1.3k
A better future with KSS
kneath
235
17k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
32k
Being A Developer After 40
akosma
84
590k
Transcript
Multi-cloud deploy with Spinnaker @ikemonn
@ikemonn SRE at
Spinnakerͷ͜ͱΛ ͬͯΔਓ✋
SpinnakerΛࢼͯ͠ΈΑ͏ͱ ࢥͬͯΒ͏ ࠓͷΰʔϧ
CXϓϥοτϑΥʔϜ KARTE
4IPFT" 4IPFT# d d 4)011*/($"35 ๚தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ
ίϛϡχέʔγϣϯΛఏڙ
ϢʔβΛϦΞϧλΠϜͰ֬ೝͰ͖Δ
ඵؒ࠷େΠϕϯτ events / sec 22,000 1ͷ߹ܭΠϕϯτ events / day 1billion
ղੳ࣌ؒ sec 0.x
ϚϧνΫϥυ(AWS/GCP)
ϦΞϧλΠϜੑ͕ٻΊΒΕΔɺ ߴෛՙͳେنࢄγεςϜ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ • ୭ͰσϓϩΠͰ͖Δ
• ͷಛఆͱRollback͘͢͢͠ΔͨΊ • ࠷11ճσϓϩΠ • ଟ͍࣌10ճҎ্ খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ https://medium.com/@Zaiku/continuous-delivery-in-a-nutshell-29f4213dabda
• վળαΠΫϧͷߴԽ & SRE:3ਓ͚ͩ • σϓϩΠͷࣗಈԽ • σϓϩΠ࣌ʹ͕͋ͬͨ࣌ࣗಈrollback ୭ͰσϓϩΠͰ͖Δ http://susunshun.hatenablog.com/entry/2015/10/30/011554
https://www.autorabit.com/salesforce-deployment-automation/ σϓϩΠπʔϧʹٻΊΔ͜ͱ
େنΠϯϑϥʹසൟʹ҆ఆͨ͠ ࣗಈσϓϩΠ͕ߦ͑Δ͜ͱ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυԽ
• CodeDeployΛແཧΓ͍ͬͯͨ • ϚϧνΫϥυωΠςΟϒͷπʔϧͰͳ͍ͨ Ίɺ͕ى͖ҠߦΛܾఆ ϚϧνΫϥυॳظ https://docs.aws.amazon.com/ja_jp/codedeploy/latest/userguide/instances-on-premises.html
ϚϧνΫϥυରԠͷ σϓϩΠπʔϧΛ୳͢
https://cloudplatform.googleblog.com/2017/06/spinnaker-10-continuous-delivery.html
Spinnaker is an open source, multi-cloud continuous delivery platform for
releasing software changes with high velocity and confidence. https://netflix.github.io/
https://www.spinnaker.io/
• ଞͷ࡞ۀ͠ͳ͕ΒɺSRE2ਓͰ3ϲ݄ • σϓϩΠϑϩʔͷઃܭ • όάͷճආํ๏ͷࡧɺPRͷϚʔδͪ Ҡߦίετ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• ✅ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ✅ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ओཁΫϥυϓϩόΠμk8sͰར༻Մೳ ϚϧνΫϥυରԠ Azure VM Openstack Amazon EC2 Amazon ECS
• ֤ΫϥυͷࠩΛҙࣝͤͣʹσϓϩΠͰ͖Δ ϚϧνΫϥυରԠ
• ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• σϓϩΠͷਐḿΛཧը໘Ͱ֬ೝͰ͖Δ ҆ఆͨ͠σϓϩΠ
• stage, pipelineͰॊೈʹσϓϩΠϑϩʔΛ ΊΔ ҆ఆͨ͠σϓϩΠ
• stageͷbuild-inػೳ͕ॆ࣮͓ͯ͠ΓɺΓͨ ͍͜ͱେͰ͖Δ(20+) ҆ఆͨ͠σϓϩΠ
• Blue-GreenσϓϩΠΧφϦΞσϓϩΠʹ ରԠ(spinnaker/kayenta) ҆ఆͨ͠σϓϩΠ https://cloudplatform.googleblog.com/2018/04/introducing-Kayenta-an-open-automated-canary-analysis-tool-from-Google-and-Netflix.html
• Immutable Infraͷڧ੍ • ιʔείʔυͰͳ͘imageΛΓସ͑Δࢥ • ιʔείʔυઃఆ͕ҟͳΔαʔό͕ଘࡏ͠ͳ͍ • ෭࣍తͳޮՌͱͯ͠Spin-upͷߴԽ •
CodeDeployʹൺͯ5ഒ ҆ఆͨ͠σϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• roer/dcd-specͰpipelineͷઃఆΛཧ • ։ൃऴ͍ྃͯ͠ΔͷͰɺ͜Ε͔Βspin σϓϩΠϑϩʔͷίʔυԽ https://blog.spinnaker.io/spin-and-roer-managed-pipeline-templates-4fde2951c648
https://docs.google.com/document/d/17Ifm562aypjLSHnNJ02COFv_VMv3spicH-7zMYd-oXE/edit
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Gate͕REST APIΛఏڙ͍ͯ͠Δ • Pipelineͷ࣮ߦ݁Ռͷऔಘ͕Մೳ ࣗಈσϓϩΠ https://cloudacademy.com/blog/netflix-spinnaker/
• Pipelineͷىಈ/ޭ/ࣦഊΛ௨Մೳ • Slack, Mail, HipChat, SMS… ࣗಈσϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Netflix, Google͕ੵۃతʹ։ൃ ։ൃ͕׆ൃ https://cloud.withgoogle.com/next18/sf/sessions/session/155951
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
None
SpinnakerͰ Firewall, LB, ASGΛ࡞Մೳ
৽͍͠αʔόάϧʔϓΛ࡞͠ ͍ͯ͘ࢥ
'JSFXBMM -# "VUP4DBMJOH (SPVQ Ҡߦલ Ҡߦޙ
• ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• σϑΥϧτͷઃఆͩͱrate exceededͰσϓϩ Π͕ࣦഊ͕ͪ͠ AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ clouddriver.yml
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
https://cloud.google.com/marketplace/ ϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ϚωʔδυɾαʔϏε͕ແ͍ͷͰࣗͰӡ༻ • GCSʹconfigใΛอଘ • diskͷϚϯτ֤छઃఆεΫϦϓτԽ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• UI্ͷόάɺvupʹࣦഊ͢ΔɺCPU༻͕ಥ વ্͕Δɺϓϩηε͕ಥવࢮ͵ όά͕ଟ͍
GCEʹσϓϩΠޙΠϯελϯεͷ ͕Ұؾʹmin·ͰݮΔ RPSͰεέʔϧ͍ͤͯ͞Δ߹ɺLB,Φʔτεέʔ ϥʔͱSpinnakerͷ૬ੑ͕ѱ͍ͷͰ10ͭ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• υΩϡϝϯτ͕ॆ࣮ͯ͠ͳ͍ • ϒϩάͷใগͳ͍orݹ͘ͳ͍ͬͯΔ • GithubͷissueΈͨΓɺίʔυಡΜͩΓɺSlack Ͱ࣭ͨ͠Γ͢Δͷ͕ૣ͍ υΩϡϝϯτ͕গͳ͍
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυͷIaaSͷσϓϩΠ ʹSpinnaker͕Φεεϝ
ϚϧνΫϥυରԠ
σϓϩΠؔ࿈ͷػೳ͕ॆ࣮ https://sweetcode.io/13115-2/
Immutable Infraͷڧ੍ https://www.merriam-webster.com/assets/mw/word-of-the-day/social/7bcf417841f7ce37cd7e7e1427c0504f.jpg
ΛΓӽ͑Δඞཁ͋Γ όάͷଟ͞ υΩϡϝϯτͷগͳ͞
σϓϩΠπʔϧͰΜͰΔͳΒ ੋඇݕ౼͍ͯͩ͘͠͞ʂ
͓खݩͷQRίʔυ͔Β ͥͻϑΟʔυόοΫΛ͓ئ͍͠·͢
͓·͚
શମͷpipelineߏ
deploy-all deploy-aws deploy-gcp deploy-role1 deploy-role2 … Pipeline (શମ) deploy-role1 deploy-role2
…
Pipeline (֤Ϋϥυ) αʔόͷroleຖʹσϓϩΠ & rollbackՄೳ
Pipeline (֤Ϋϥυ) Pipeline parameter & conditional on ExpressionͰpipelineΛ੍ޚ
֤छϚΠΫϩαʔϏεͷઃఆ • ~/.hal/$DEPLOYMENT/profiles/ ҎԼʹϑ ΝΠϧΛ࡞ & hal deploy apply ɾ/opt/spinnaker/configʹө͞ΕΔ(v1.9)