Multicloud deploy with Spinnaker / builderscon tokyo 2018

builderscon tokyo 2018の発表スライドです。

Transcript

1. Multi-cloud deploy with Spinnaker @ikemonn
2. @ikemonn SRE at
3. Spinnakerͷ͜ͱΛ ஌ͬͯΔਓ✋
4. SpinnakerΛࢼͯ͠ΈΑ͏ͱ ࢥͬͯ΋Β͏ ࠓ೔ͷΰʔϧ
5. CXϓϥοτϑΥʔϜ KARTE
6. 4IPFT" 4IPFT# d  d  4)011*/(
   $"35 ๚໰தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ ίϛϡχέʔγϣϯΛఏڙ
7. ϢʔβΛϦΞϧλΠϜͰ֬ೝͰ͖Δ
8. ඵؒ࠷େΠϕϯτ਺ events / sec 22,000 1೔ͷ߹ܭΠϕϯτ਺ events / day 1billion ղੳ࣌ؒ sec 0.x
9. ϚϧνΫϥ΢υ(AWS/GCP)
10. ϦΞϧλΠϜੑ͕ٻΊΒΕΔɺ ߴෛՙͳେن໛෼ࢄγεςϜ
11. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ
12. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ
13. • খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ • ୭Ͱ΋σϓϩΠͰ͖Δ
14. • ໰୊ͷಛఆͱRollback͠΍͘͢͢ΔͨΊ • ࠷௿1೔1ճ͸σϓϩΠ • ଟ͍࣌͸10ճҎ্ খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ https://medium.com/@Zaiku/continuous-delivery-in-a-nutshell-29f4213dabda
15. • վળαΠΫϧͷߴ଎Խ & SRE:3ਓ͚ͩ • σϓϩΠͷࣗಈԽ • σϓϩΠ࣌ʹ໰୊͕͋ͬͨ࣌͸ࣗಈrollback ୭Ͱ΋σϓϩΠͰ͖Δ http://susunshun.hatenablog.com/entry/2015/10/30/011554
16. https://www.autorabit.com/salesforce-deployment-automation/ σϓϩΠπʔϧʹٻΊΔ͜ͱ
17. େن໛Πϯϑϥʹසൟʹ҆ఆͨ͠ ࣗಈσϓϩΠ͕ߦ͑Δ͜ͱ
18. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ
19. ϚϧνΫϥ΢υԽ
20. • CodeDeployΛແཧ΍Γ࢖͍ͬͯͨ • ϚϧνΫϥ΢υωΠςΟϒͷπʔϧͰ͸ͳ͍ͨ Ίɺ໰୊͕ى͖ҠߦΛܾఆ ϚϧνΫϥ΢υॳظ https://docs.aws.amazon.com/ja_jp/codedeploy/latest/userguide/instances-on-premises.html
21. ϚϧνΫϥ΢υରԠͷ σϓϩΠπʔϧΛ୳͢
22. https://cloudplatform.googleblog.com/2017/06/spinnaker-10-continuous-delivery.html
23. Spinnaker is an open source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence. https://netflix.github.io/
24. https://www.spinnaker.io/
25. • ଞͷ࡞ۀ΋͠ͳ͕ΒɺSRE2ਓͰ3ϲ݄ • σϓϩΠϑϩʔͷઃܭ • όάͷճආํ๏ͷ໛ࡧɺPRͷϚʔδ଴ͪ Ҡߦίετ
26. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ
27. • ✅ϚϧνΫϥ΢υରԠ • ✅҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ ظ଴௨Γͩͬͨ͜ͱ
28. • ✅ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ ظ଴௨Γͩͬͨ͜ͱ
29. • ओཁΫϥ΢υϓϩόΠμ΍k8sͰ΋ར༻Մೳ ϚϧνΫϥ΢υରԠ Azure VM Openstack Amazon EC2 Amazon ECS
30. • ֤Ϋϥ΢υͷࠩ෼ΛҙࣝͤͣʹσϓϩΠͰ͖Δ ϚϧνΫϥ΢υରԠ
31. • ϚϧνΫϥ΢υରԠ • ✅҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ ظ଴௨Γͩͬͨ͜ͱ
32. • σϓϩΠͷਐḿΛ؅ཧը໘Ͱ֬ೝͰ͖Δ ҆ఆͨ͠σϓϩΠ
33. • stage, pipelineͰॊೈʹσϓϩΠϑϩʔΛ૊ ΊΔ ҆ఆͨ͠σϓϩΠ
34. • stageͷbuild-inػೳ͕ॆ࣮͓ͯ͠Γɺ΍Γͨ ͍͜ͱ͸େ఍Ͱ͖Δ(20+) ҆ఆͨ͠σϓϩΠ
35. • Blue-GreenσϓϩΠ΍ΧφϦΞσϓϩΠʹ΋ ରԠ(spinnaker/kayenta) ҆ఆͨ͠σϓϩΠ https://cloudplatform.googleblog.com/2018/04/introducing-Kayenta-an-open-automated-canary-analysis-tool-from-Google-and-Netflix.html
36. • Immutable Infraͷڧ੍ • ιʔείʔυͰ͸ͳ͘imageΛ੾Γସ͑Δࢥ૝ • ιʔείʔυ΍ઃఆ͕ҟͳΔαʔό͕ଘࡏ͠ͳ͍ • ෭࣍తͳޮՌͱͯ͠Spin-upͷߴ଎Խ • CodeDeployʹൺ΂ͯ5ഒ ҆ఆͨ͠σϓϩΠ
37. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ ظ଴௨Γͩͬͨ͜ͱ
38. • roer/dcd-specͰpipelineͷઃఆΛ؅ཧ • ։ൃऴ͍ྃͯ͠ΔͷͰɺ͜Ε͔Β͸spin σϓϩΠϑϩʔͷίʔυԽ https://blog.spinnaker.io/spin-and-roer-managed-pipeline-templates-4fde2951c648
39. https://docs.google.com/document/d/17Ifm562aypjLSHnNJ02COFv_VMv3spicH-7zMYd-oXE/edit
40. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ։ൃ͕׆ൃ ظ଴௨Γͩͬͨ͜ͱ
41. • Gate͕REST APIΛఏڙ͍ͯ͠Δ • Pipelineͷ࣮ߦ΍݁Ռͷऔಘ౳͕Մೳ ࣗಈσϓϩΠ https://cloudacademy.com/blog/netflix-spinnaker/
42. • Pipelineͷىಈ/੒ޭ/ࣦഊ౳Λ௨஌Մೳ • Slack, Mail, HipChat, SMS… ࣗಈσϓϩΠ
43. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ ظ଴௨Γͩͬͨ͜ͱ
44. • Netflix, Google͕ੵۃతʹ։ൃ ։ൃ͕׆ൃ https://cloud.withgoogle.com/next18/sf/sessions/session/155951
45. • ❌ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍ ظ଴ͱҧ͍ͬͯͨ͜ͱ
46. • ❌ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍ ظ଴ͱҧ͍ͬͯͨ͜ͱ
48. SpinnakerͰ΋ Firewall, LB, ASGΛ࡞੒Մೳ
49. ৽͍͠αʔόάϧʔϓΛ࡞੒͠ ͍ͯ͘ࢥ૝
50. 'JSFXBMM -# "VUP4DBMJOH
    (SPVQ Ҡߦલ Ҡߦޙ
51. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍ ظ଴ͱҧ͍ͬͯͨ͜ͱ
52. • σϑΥϧτͷઃఆͩͱrate exceededͰσϓϩ Π͕ࣦഊ͕ͪ͠ AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ clouddriver.yml
53. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍ ظ଴ͱҧ͍ͬͯͨ͜ͱ
54. https://cloud.google.com/marketplace/ ϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ϚωʔδυɾαʔϏε͕ແ͍ͷͰࣗ෼Ͱӡ༻ • GCSʹconfig৘ใΛอଘ • diskͷϚ΢ϯτ΍֤छઃఆ͸εΫϦϓτԽ
55. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍ ظ଴ͱҧ͍ͬͯͨ͜ͱ
56. • UI্ͷόάɺvupʹࣦഊ͢ΔɺCPU࢖༻཰͕ಥ વ্͕Δɺϓϩηε͕ಥવࢮ͵౳ όά͕ଟ͍
57. GCEʹσϓϩΠޙΠϯελϯεͷ ୆਺͕Ұؾʹmin·ͰݮΔ RPSͰεέʔϧ͍ͤͯ͞Δ৔߹ɺLB,Φʔτεέʔ ϥʔͱSpinnakerͷ૬ੑ͕ѱ͍ͷͰ10෼଴ͭ
58. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍ ظ଴ͱҧ͍ͬͯͨ͜ͱ
59. • υΩϡϝϯτ͕ॆ࣮ͯ͠ͳ͍ • ϒϩά౳ͷ৘ใ΋গͳ͍orݹ͘ͳ͍ͬͯΔ • GithubͷissueΈͨΓɺίʔυಡΜͩΓɺSlack Ͱ࣭໰ͨ͠Γ͢Δͷ͕ૣ͍ υΩϡϝϯτ͕গͳ͍
60. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ
61. ϚϧνΫϥ΢υͷIaaS΁ͷσϓϩΠ ʹ͸Spinnaker͕Φεεϝ
62. ϚϧνΫϥ΢υରԠ
63. σϓϩΠؔ࿈ͷػೳ͕ॆ࣮ https://sweetcode.io/13115-2/
64. Immutable Infraͷڧ੍ https://www.merriam-webster.com/assets/mw/word-of-the-day/social/7bcf417841f7ce37cd7e7e1427c0504f.jpg
65. Λ৐Γӽ͑Δඞཁ͋Γ όάͷଟ͞ υΩϡϝϯτͷগͳ͞
66. σϓϩΠπʔϧͰ೰ΜͰΔͳΒ ੋඇݕ౼͍ͯͩ͘͠͞ʂ
67. ͓खݩͷQRίʔυ͔Β ͥͻϑΟʔυόοΫΛ͓ئ͍͠·͢
68. ͓·͚
69. શମͷpipelineߏ੒
70. deploy-all deploy-aws deploy-gcp deploy-role1 deploy-role2 … Pipeline (શମ) deploy-role1 deploy-role2 …
71. Pipeline (֤Ϋϥ΢υ) αʔόͷroleຖʹσϓϩΠ & rollbackՄೳ
72. Pipeline (֤Ϋϥ΢υ) Pipeline parameter & conditional on ExpressionͰpipelineΛ੍ޚ
73. ֤छϚΠΫϩαʔϏεͷઃఆ • ~/.hal/$DEPLOYMENT/profiles/ ҎԼʹϑ ΝΠϧΛ࡞੒ & hal deploy apply ɾ/opt/spinnaker/configʹ൓ө͞ΕΔ(v1.9)