Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Multicloud deploy with Spinnaker / builderscon ...
Search
Daiki Matsui
September 08, 2018
Technology
2
4.3k
Multicloud deploy with Spinnaker / builderscon tokyo 2018
builderscon tokyo 2018の発表スライドです。
Daiki Matsui
September 08, 2018
Tweet
Share
More Decks by Daiki Matsui
See All by Daiki Matsui
The Fourth Age of SRE
ikemonn
0
200
マルチクラウドな大規模サービスKARTEを支えるDatadog/multi-cloud monitoring with Datadog
ikemonn
0
2.2k
KARTE を支えるマルチプラットフォームインフラ監視 /karte-multi-platform-monitoring
ikemonn
2
15k
Node Apps Performance Tuning
ikemonn
0
130
Other Decks in Technology
See All in Technology
グイグイ系QAマネージャーの仕事
sadonosake
0
460
Passkey Autofill に賭けるマネーフォワード ID - Money Forward Tech Day 2024
nov
1
570
株式会社EventHub・エンジニア採用資料
eventhub
0
3k
サンプルサンプル株式会社 会社説明資料
takakuni
0
270
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
5
46k
AI前提のサービス運用ってなんだろう?
ryuichi1208
5
1.2k
係り受け解析を用いた法律文書中の略称規定の解析についての報告
puripuri2100
0
290
モノづくり品質珍道中
kzkhrk
1
110
持続可能なソフトウェア開発を支える『GitHub CI/CD実践ガイド』
tmknom
10
1.8k
Wasmコンテナを動かしてみた
stanaka26
0
100
Singularity/Spack どこでもだれでもHPC
kmorimotoptj
0
100
JEP 480: Structured Concurrency
aya_ebata
0
140
Featured
See All Featured
Done Done
chrislema
180
16k
Learning to Love Humans: Emotional Interface Design
aarron
271
40k
Happy Clients
brianwarren
96
6.6k
Making Projects Easy
brettharned
114
5.8k
Fashionably flexible responsive web design (full day workshop)
malarkey
401
65k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
230
17k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
26
4k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
24
610
Design by the Numbers
sachag
277
19k
Rebuilding a faster, lazier Slack
samanthasiow
78
8.6k
Why Our Code Smells
bkeepers
PRO
334
56k
4 Signs Your Business is Dying
shpigford
179
21k
Transcript
Multi-cloud deploy with Spinnaker @ikemonn
@ikemonn SRE at
Spinnakerͷ͜ͱΛ ͬͯΔਓ✋
SpinnakerΛࢼͯ͠ΈΑ͏ͱ ࢥͬͯΒ͏ ࠓͷΰʔϧ
CXϓϥοτϑΥʔϜ KARTE
4IPFT" 4IPFT# d d 4)011*/($"35 ๚தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ
ίϛϡχέʔγϣϯΛఏڙ
ϢʔβΛϦΞϧλΠϜͰ֬ೝͰ͖Δ
ඵؒ࠷େΠϕϯτ events / sec 22,000 1ͷ߹ܭΠϕϯτ events / day 1billion
ղੳ࣌ؒ sec 0.x
ϚϧνΫϥυ(AWS/GCP)
ϦΞϧλΠϜੑ͕ٻΊΒΕΔɺ ߴෛՙͳେنࢄγεςϜ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ • ୭ͰσϓϩΠͰ͖Δ
• ͷಛఆͱRollback͘͢͢͠ΔͨΊ • ࠷11ճσϓϩΠ • ଟ͍࣌10ճҎ্ খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ https://medium.com/@Zaiku/continuous-delivery-in-a-nutshell-29f4213dabda
• վળαΠΫϧͷߴԽ & SRE:3ਓ͚ͩ • σϓϩΠͷࣗಈԽ • σϓϩΠ࣌ʹ͕͋ͬͨ࣌ࣗಈrollback ୭ͰσϓϩΠͰ͖Δ http://susunshun.hatenablog.com/entry/2015/10/30/011554
https://www.autorabit.com/salesforce-deployment-automation/ σϓϩΠπʔϧʹٻΊΔ͜ͱ
େنΠϯϑϥʹසൟʹ҆ఆͨ͠ ࣗಈσϓϩΠ͕ߦ͑Δ͜ͱ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυԽ
• CodeDeployΛແཧΓ͍ͬͯͨ • ϚϧνΫϥυωΠςΟϒͷπʔϧͰͳ͍ͨ Ίɺ͕ى͖ҠߦΛܾఆ ϚϧνΫϥυॳظ https://docs.aws.amazon.com/ja_jp/codedeploy/latest/userguide/instances-on-premises.html
ϚϧνΫϥυରԠͷ σϓϩΠπʔϧΛ୳͢
https://cloudplatform.googleblog.com/2017/06/spinnaker-10-continuous-delivery.html
Spinnaker is an open source, multi-cloud continuous delivery platform for
releasing software changes with high velocity and confidence. https://netflix.github.io/
https://www.spinnaker.io/
• ଞͷ࡞ۀ͠ͳ͕ΒɺSRE2ਓͰ3ϲ݄ • σϓϩΠϑϩʔͷઃܭ • όάͷճආํ๏ͷࡧɺPRͷϚʔδͪ Ҡߦίετ
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
• ✅ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ✅ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• ओཁΫϥυϓϩόΠμk8sͰར༻Մೳ ϚϧνΫϥυରԠ Azure VM Openstack Amazon EC2 Amazon ECS
• ֤ΫϥυͷࠩΛҙࣝͤͣʹσϓϩΠͰ͖Δ ϚϧνΫϥυରԠ
• ϚϧνΫϥυରԠ • ✅҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• σϓϩΠͷਐḿΛཧը໘Ͱ֬ೝͰ͖Δ ҆ఆͨ͠σϓϩΠ
• stage, pipelineͰॊೈʹσϓϩΠϑϩʔΛ ΊΔ ҆ఆͨ͠σϓϩΠ
• stageͷbuild-inػೳ͕ॆ࣮͓ͯ͠ΓɺΓͨ ͍͜ͱେͰ͖Δ(20+) ҆ఆͨ͠σϓϩΠ
• Blue-GreenσϓϩΠΧφϦΞσϓϩΠʹ ରԠ(spinnaker/kayenta) ҆ఆͨ͠σϓϩΠ https://cloudplatform.googleblog.com/2018/04/introducing-Kayenta-an-open-automated-canary-analysis-tool-from-Google-and-Netflix.html
• Immutable Infraͷڧ੍ • ιʔείʔυͰͳ͘imageΛΓସ͑Δࢥ • ιʔείʔυઃఆ͕ҟͳΔαʔό͕ଘࡏ͠ͳ͍ • ෭࣍తͳޮՌͱͯ͠Spin-upͷߴԽ •
CodeDeployʹൺͯ5ഒ ҆ఆͨ͠σϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• roer/dcd-specͰpipelineͷઃఆΛཧ • ։ൃऴ͍ྃͯ͠ΔͷͰɺ͜Ε͔Βspin σϓϩΠϑϩʔͷίʔυԽ https://blog.spinnaker.io/spin-and-roer-managed-pipeline-templates-4fde2951c648
https://docs.google.com/document/d/17Ifm562aypjLSHnNJ02COFv_VMv3spicH-7zMYd-oXE/edit
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Gate͕REST APIΛఏڙ͍ͯ͠Δ • Pipelineͷ࣮ߦ݁Ռͷऔಘ͕Մೳ ࣗಈσϓϩΠ https://cloudacademy.com/blog/netflix-spinnaker/
• Pipelineͷىಈ/ޭ/ࣦഊΛ௨Մೳ • Slack, Mail, HipChat, SMS… ࣗಈσϓϩΠ
• ϚϧνΫϥυରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ
ظ௨Γͩͬͨ͜ͱ
• Netflix, Google͕ੵۃతʹ։ൃ ։ൃ͕׆ൃ https://cloud.withgoogle.com/next18/sf/sessions/session/155951
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• ❌ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
None
SpinnakerͰ Firewall, LB, ASGΛ࡞Մೳ
৽͍͠αʔόάϧʔϓΛ࡞͠ ͍ͯ͘ࢥ
'JSFXBMM -# "VUP4DBMJOH (SPVQ Ҡߦલ Ҡߦޙ
• ଞπʔϧͱͷׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• σϑΥϧτͷઃఆͩͱrate exceededͰσϓϩ Π͕ࣦഊ͕ͪ͠ AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ clouddriver.yml
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
https://cloud.google.com/marketplace/ ϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ϚωʔδυɾαʔϏε͕ແ͍ͷͰࣗͰӡ༻ • GCSʹconfigใΛอଘ • diskͷϚϯτ֤छઃఆεΫϦϓτԽ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• UI্ͷόάɺvupʹࣦഊ͢ΔɺCPU༻͕ಥ વ্͕Δɺϓϩηε͕ಥવࢮ͵ όά͕ଟ͍
GCEʹσϓϩΠޙΠϯελϯεͷ ͕Ұؾʹmin·ͰݮΔ RPSͰεέʔϧ͍ͤͯ͞Δ߹ɺLB,Φʔτεέʔ ϥʔͱSpinnakerͷ૬ੑ͕ѱ͍ͷͰ10ͭ
• ଞπʔϧͱͷׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍
ظͱҧ͍ͬͯͨ͜ͱ
• υΩϡϝϯτ͕ॆ࣮ͯ͠ͳ͍ • ϒϩάͷใগͳ͍orݹ͘ͳ͍ͬͯΔ • GithubͷissueΈͨΓɺίʔυಡΜͩΓɺSlack Ͱ࣭ͨ͠Γ͢Δͷ͕ૣ͍ υΩϡϝϯτ͕গͳ͍
Agenda • σϓϩΠͷࢥ • SpinnakerͷҠߦ • Spinnakerӡ༻Ͱಘͨݟ • ·ͱΊ
ϚϧνΫϥυͷIaaSͷσϓϩΠ ʹSpinnaker͕Φεεϝ
ϚϧνΫϥυରԠ
σϓϩΠؔ࿈ͷػೳ͕ॆ࣮ https://sweetcode.io/13115-2/
Immutable Infraͷڧ੍ https://www.merriam-webster.com/assets/mw/word-of-the-day/social/7bcf417841f7ce37cd7e7e1427c0504f.jpg
ΛΓӽ͑Δඞཁ͋Γ όάͷଟ͞ υΩϡϝϯτͷগͳ͞
σϓϩΠπʔϧͰΜͰΔͳΒ ੋඇݕ౼͍ͯͩ͘͠͞ʂ
͓खݩͷQRίʔυ͔Β ͥͻϑΟʔυόοΫΛ͓ئ͍͠·͢
͓·͚
શମͷpipelineߏ
deploy-all deploy-aws deploy-gcp deploy-role1 deploy-role2 … Pipeline (શମ) deploy-role1 deploy-role2
…
Pipeline (֤Ϋϥυ) αʔόͷroleຖʹσϓϩΠ & rollbackՄೳ
Pipeline (֤Ϋϥυ) Pipeline parameter & conditional on ExpressionͰpipelineΛ੍ޚ
֤छϚΠΫϩαʔϏεͷઃఆ • ~/.hal/$DEPLOYMENT/profiles/ ҎԼʹϑ ΝΠϧΛ࡞ & hal deploy apply ɾ/opt/spinnaker/configʹө͞ΕΔ(v1.9)