Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Multicloud deploy with Spinnaker / builderscon ...
Search
Daiki Matsui
September 08, 2018
Technology
2
4.6k
Multicloud deploy with Spinnaker / builderscon tokyo 2018
builderscon tokyo 2018の発表スライドです。
Daiki Matsui
September 08, 2018
Tweet
Share
More Decks by Daiki Matsui
See All by Daiki Matsui
The Fourth Age of SRE
ikemonn
0
530
マルチクラウドな大規模サービスKARTEを支えるDatadog/multi-cloud monitoring with Datadog
ikemonn
0
2.5k
KARTE を支えるマルチプラットフォームインフラ監視 /karte-multi-platform-monitoring
ikemonn
2
16k
Node Apps Performance Tuning
ikemonn
0
140
Other Decks in Technology
See All in Technology
アクセスピークを制するオートスケール再設計: 障害を乗り越えKEDAで実現したリソース管理の最適化
myamashii
1
130
OpenTelemetryセマンティック規約の恩恵とMackerel APMにおける活用例 / SRE NEXT 2025
mackerelio
2
770
VS CodeとGitHub Copilotで爆速開発!アップデートの波に乗るおさらい会 / Rapid Development with VS Code and GitHub Copilot: Catch the Latest Wave
yamachu
2
190
Lufthansa ®️ USA Contact Numbers: Complete 2025 Support Guide
lufthanahelpsupport
0
220
How Do I Contact HP Printer Support? [Full 2025 Guide for U.S. Businesses]
harrry1211
0
130
VGGT: Visual Geometry Grounded Transformer
peisuke
0
160
Claude Code に プロジェクト管理やらせたみた
unson
6
4.6k
[SRE NEXT] ARR150億円_エンジニア140名_27チーム_17プロダクトから始めるSLO.pdf
satos
2
620
United Airlines Customer Service– Call 1-833-341-3142 Now!
airhelp
0
170
【LT会登壇資料】TROCCO新コネクタ「スマレジ」を活用した直営店データの分析
kazari0425
1
110
american airlines®️ USA Contact Numbers: Complete 2025 Support Guide
supportflight
1
110
SEQUENCE object comparison - db tech showcase 2025 LT2
nori_shinoda
0
170
Featured
See All Featured
Building Applications with DynamoDB
mza
95
6.5k
Music & Morning Musume
bryan
46
6.6k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
RailsConf 2023
tenderlove
30
1.1k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Automating Front-end Workflow
addyosmani
1370
200k
A designer walks into a library…
pauljervisheath
207
24k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Typedesign – Prime Four
hannesfritz
42
2.7k
Embracing the Ebb and Flow
colly
86
4.7k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
138
34k
Optimizing for Happiness
mojombo
379
70k
Transcript
Multi-cloud deploy with Spinnaker @ikemonn
@ikemonn SRE at
Spinnakerͷ͜ͱΛ ͬͯΔਓ✋
SpinnakerΛࢼͯ͠ΈΑ͏ͱ ࢥͬͯΒ͏ ࠓͷΰʔϧ
CXϓϥοτϑΥʔϜ KARTE
4IPFT" 4IPFT# d d 4)011*/($"35 ๚தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ
ίϛϡχέʔγϣϯΛఏڙ
ϢʔβΛϦΞϧλΠϜͰ֬ೝͰ͖Δ
ඵؒ࠷େΠϕϯτ events / sec 22,000 1ͷ߹ܭΠϕϯτ events / day 1billion
ղੳ࣌ؒ sec 0.x
ϚϧνΫϥυ(AWS/GCP)
ϦΞϧλΠϜੑ͕ٻΊΒΕΔɺ ߴෛՙͳେنࢄγεςϜ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ • ୭ͰσϓϩΠͰ͖Δ
• ͷಛఆͱRollback͘͢͢͠ΔͨΊ • ࠷11ճσϓϩΠ • ଟ͍࣌10ճҎ্ খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ https://medium.com/@Zaiku/continuous-delivery-in-a-nutshell-29f4213dabda
• վળαΠΫϧͷߴԽ & SRE:3ਓ͚ͩ • σϓϩΠͷࣗಈԽ • σϓϩΠ࣌ʹ͕͋ͬͨ࣌ࣗಈrollback ୭ͰσϓϩΠͰ͖Δ http://susunshun.hatenablog.com/entry/2015/10/30/011554
https://www.autorabit.com/salesforce-deployment-automation/ σϓϩΠπʔϧʹٻΊΔ͜ͱ
େنΠϯϑϥʹසൟʹ҆ఆͨ͠ ࣗಈσϓϩΠ͕ߦ͑Δ͜ͱ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυԽ
• CodeDeployΛແཧΓ͍ͬͯͨ • ϚϧνΫϥυωΠςΟϒͷπʔϧͰͳ͍ͨ Ίɺ͕ى͖ҠߦΛܾఆ ϚϧνΫϥυॳظ https://docs.aws.amazon.com/ja_jp/codedeploy/latest/userguide/instances-on-premises.html
ϚϧνΫϥυରԠͷ σϓϩΠπʔϧΛ୳͢
https://cloudplatform.googleblog.com/2017/06/spinnaker-10-continuous-delivery.html
Spinnaker is an open source, multi-cloud continuous delivery platform for
releasing software changes with high velocity and confidence. https://netflix.github.io/
https://www.spinnaker.io/
• ଞͷ࡞ۀ͠ͳ͕ΒɺSRE2ਓͰ3ϲ݄ • σϓϩΠϑϩʔͷઃܭ • όάͷճආํ๏ͷࡧɺPRͷϚʔδͪ Ҡߦίετ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• ✅ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ✅ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ओཁΫϥυϓϩόΠμk8sͰར༻Մೳ ϚϧνΫϥυରԠ Azure VM Openstack Amazon EC2 Amazon ECS
• ֤ΫϥυͷࠩΛҙࣝͤͣʹσϓϩΠͰ͖Δ ϚϧνΫϥυରԠ
• ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• σϓϩΠͷਐḿΛཧը໘Ͱ֬ೝͰ͖Δ ҆ఆͨ͠σϓϩΠ
• stage, pipelineͰॊೈʹσϓϩΠϑϩʔΛ ΊΔ ҆ఆͨ͠σϓϩΠ
• stageͷbuild-inػೳ͕ॆ࣮͓ͯ͠ΓɺΓͨ ͍͜ͱେͰ͖Δ(20+) ҆ఆͨ͠σϓϩΠ
• Blue-GreenσϓϩΠΧφϦΞσϓϩΠʹ ରԠ(spinnaker/kayenta) ҆ఆͨ͠σϓϩΠ https://cloudplatform.googleblog.com/2018/04/introducing-Kayenta-an-open-automated-canary-analysis-tool-from-Google-and-Netflix.html
• Immutable Infraͷڧ੍ • ιʔείʔυͰͳ͘imageΛΓସ͑Δࢥ • ιʔείʔυઃఆ͕ҟͳΔαʔό͕ଘࡏ͠ͳ͍ • ෭࣍తͳޮՌͱͯ͠Spin-upͷߴԽ •
CodeDeployʹൺͯ5ഒ ҆ఆͨ͠σϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• roer/dcd-specͰpipelineͷઃఆΛཧ • ։ൃऴ͍ྃͯ͠ΔͷͰɺ͜Ε͔Βspin σϓϩΠϑϩʔͷίʔυԽ https://blog.spinnaker.io/spin-and-roer-managed-pipeline-templates-4fde2951c648
https://docs.google.com/document/d/17Ifm562aypjLSHnNJ02COFv_VMv3spicH-7zMYd-oXE/edit
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Gate͕REST APIΛఏڙ͍ͯ͠Δ • Pipelineͷ࣮ߦ݁Ռͷऔಘ͕Մೳ ࣗಈσϓϩΠ https://cloudacademy.com/blog/netflix-spinnaker/
• Pipelineͷىಈ/ޭ/ࣦഊΛ௨Մೳ • Slack, Mail, HipChat, SMS… ࣗಈσϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Netflix, Google͕ੵۃతʹ։ൃ ։ൃ͕׆ൃ https://cloud.withgoogle.com/next18/sf/sessions/session/155951
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
None
SpinnakerͰ Firewall, LB, ASGΛ࡞Մೳ
৽͍͠αʔόάϧʔϓΛ࡞͠ ͍ͯ͘ࢥ
'JSFXBMM -# "VUP4DBMJOH (SPVQ Ҡߦલ Ҡߦޙ
• ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• σϑΥϧτͷઃఆͩͱrate exceededͰσϓϩ Π͕ࣦഊ͕ͪ͠ AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ clouddriver.yml
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
https://cloud.google.com/marketplace/ ϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ϚωʔδυɾαʔϏε͕ແ͍ͷͰࣗͰӡ༻ • GCSʹconfigใΛอଘ • diskͷϚϯτ֤छઃఆεΫϦϓτԽ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• UI্ͷόάɺvupʹࣦഊ͢ΔɺCPU༻͕ಥ વ্͕Δɺϓϩηε͕ಥવࢮ͵ όά͕ଟ͍
GCEʹσϓϩΠޙΠϯελϯεͷ ͕Ұؾʹmin·ͰݮΔ RPSͰεέʔϧ͍ͤͯ͞Δ߹ɺLB,Φʔτεέʔ ϥʔͱSpinnakerͷ૬ੑ͕ѱ͍ͷͰ10ͭ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• υΩϡϝϯτ͕ॆ࣮ͯ͠ͳ͍ • ϒϩάͷใগͳ͍orݹ͘ͳ͍ͬͯΔ • GithubͷissueΈͨΓɺίʔυಡΜͩΓɺSlack Ͱ࣭ͨ͠Γ͢Δͷ͕ૣ͍ υΩϡϝϯτ͕গͳ͍
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυͷIaaSͷσϓϩΠ ʹSpinnaker͕Φεεϝ
ϚϧνΫϥυରԠ
σϓϩΠؔ࿈ͷػೳ͕ॆ࣮ https://sweetcode.io/13115-2/
Immutable Infraͷڧ੍ https://www.merriam-webster.com/assets/mw/word-of-the-day/social/7bcf417841f7ce37cd7e7e1427c0504f.jpg
ΛΓӽ͑Δඞཁ͋Γ όάͷଟ͞ υΩϡϝϯτͷগͳ͞
σϓϩΠπʔϧͰΜͰΔͳΒ ੋඇݕ౼͍ͯͩ͘͠͞ʂ
͓खݩͷQRίʔυ͔Β ͥͻϑΟʔυόοΫΛ͓ئ͍͠·͢
͓·͚
શମͷpipelineߏ
deploy-all deploy-aws deploy-gcp deploy-role1 deploy-role2 … Pipeline (શମ) deploy-role1 deploy-role2
…
Pipeline (֤Ϋϥυ) αʔόͷroleຖʹσϓϩΠ & rollbackՄೳ
Pipeline (֤Ϋϥυ) Pipeline parameter & conditional on ExpressionͰpipelineΛ੍ޚ
֤छϚΠΫϩαʔϏεͷઃఆ • ~/.hal/$DEPLOYMENT/profiles/ ҎԼʹϑ ΝΠϧΛ࡞ & hal deploy apply ɾ/opt/spinnaker/configʹө͞ΕΔ(v1.9)