Multicloud deploy with Spinnaker / builderscon tokyo 2018

6de30acfd736745120a3d6457dc6d45c?s=47 Daiki Matsui
September 08, 2018

Multicloud deploy with Spinnaker / builderscon tokyo 2018

builderscon tokyo 2018の発表スライドです。

6de30acfd736745120a3d6457dc6d45c?s=128

Daiki Matsui

September 08, 2018
Tweet

Transcript

  1. Multi-cloud deploy with Spinnaker @ikemonn

  2. @ikemonn SRE at

  3. Spinnakerͷ͜ͱΛ ஌ͬͯΔਓ✋

  4. SpinnakerΛࢼͯ͠ΈΑ͏ͱ ࢥͬͯ΋Β͏ ࠓ೔ͷΰʔϧ

  5. CXϓϥοτϑΥʔϜ KARTE

  6. 4IPFT" 4IPFT# d  d  4)011*/($"35 ๚໰தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ

    ίϛϡχέʔγϣϯΛఏڙ
  7. ϢʔβΛϦΞϧλΠϜͰ֬ೝͰ͖Δ

  8. ඵؒ࠷େΠϕϯτ਺ events / sec 22,000 1೔ͷ߹ܭΠϕϯτ਺ events / day 1billion

    ղੳ࣌ؒ sec 0.x
  9. ϚϧνΫϥ΢υ(AWS/GCP)

  10. ϦΞϧλΠϜੑ͕ٻΊΒΕΔɺ ߴෛՙͳେن໛෼ࢄγεςϜ

  11. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

  12. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

  13. • খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ • ୭Ͱ΋σϓϩΠͰ͖Δ

  14. • ໰୊ͷಛఆͱRollback͠΍͘͢͢ΔͨΊ • ࠷௿1೔1ճ͸σϓϩΠ • ଟ͍࣌͸10ճҎ্ খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ https://medium.com/@Zaiku/continuous-delivery-in-a-nutshell-29f4213dabda

  15. • վળαΠΫϧͷߴ଎Խ & SRE:3ਓ͚ͩ • σϓϩΠͷࣗಈԽ • σϓϩΠ࣌ʹ໰୊͕͋ͬͨ࣌͸ࣗಈrollback ୭Ͱ΋σϓϩΠͰ͖Δ http://susunshun.hatenablog.com/entry/2015/10/30/011554

  16. https://www.autorabit.com/salesforce-deployment-automation/ σϓϩΠπʔϧʹٻΊΔ͜ͱ

  17. େن໛Πϯϑϥʹසൟʹ҆ఆͨ͠ ࣗಈσϓϩΠ͕ߦ͑Δ͜ͱ

  18. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

  19. ϚϧνΫϥ΢υԽ

  20. • CodeDeployΛແཧ΍Γ࢖͍ͬͯͨ • ϚϧνΫϥ΢υωΠςΟϒͷπʔϧͰ͸ͳ͍ͨ Ίɺ໰୊͕ى͖ҠߦΛܾఆ ϚϧνΫϥ΢υॳظ https://docs.aws.amazon.com/ja_jp/codedeploy/latest/userguide/instances-on-premises.html

  21. ϚϧνΫϥ΢υରԠͷ σϓϩΠπʔϧΛ୳͢

  22. https://cloudplatform.googleblog.com/2017/06/spinnaker-10-continuous-delivery.html

  23. Spinnaker is an open source, multi-cloud continuous delivery platform for

    releasing software changes with high velocity and confidence. https://netflix.github.io/
  24. https://www.spinnaker.io/

  25. • ଞͷ࡞ۀ΋͠ͳ͕ΒɺSRE2ਓͰ3ϲ݄ • σϓϩΠϑϩʔͷઃܭ • όάͷճආํ๏ͷ໛ࡧɺPRͷϚʔδ଴ͪ Ҡߦίετ

  26. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

  27. • ✅ϚϧνΫϥ΢υରԠ • ✅҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ
  28. • ✅ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ
  29. • ओཁΫϥ΢υϓϩόΠμ΍k8sͰ΋ར༻Մೳ ϚϧνΫϥ΢υରԠ Azure VM Openstack Amazon EC2 Amazon ECS

  30. • ֤Ϋϥ΢υͷࠩ෼ΛҙࣝͤͣʹσϓϩΠͰ͖Δ ϚϧνΫϥ΢υରԠ

  31. • ϚϧνΫϥ΢υରԠ • ✅҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ
  32. • σϓϩΠͷਐḿΛ؅ཧը໘Ͱ֬ೝͰ͖Δ ҆ఆͨ͠σϓϩΠ

  33. • stage, pipelineͰॊೈʹσϓϩΠϑϩʔΛ૊ ΊΔ ҆ఆͨ͠σϓϩΠ

  34. • stageͷbuild-inػೳ͕ॆ࣮͓ͯ͠Γɺ΍Γͨ ͍͜ͱ͸େ఍Ͱ͖Δ(20+) ҆ఆͨ͠σϓϩΠ

  35. • Blue-GreenσϓϩΠ΍ΧφϦΞσϓϩΠʹ΋ ରԠ(spinnaker/kayenta) ҆ఆͨ͠σϓϩΠ https://cloudplatform.googleblog.com/2018/04/introducing-Kayenta-an-open-automated-canary-analysis-tool-from-Google-and-Netflix.html

  36. • Immutable Infraͷڧ੍ • ιʔείʔυͰ͸ͳ͘imageΛ੾Γସ͑Δࢥ૝ • ιʔείʔυ΍ઃఆ͕ҟͳΔαʔό͕ଘࡏ͠ͳ͍ • ෭࣍తͳޮՌͱͯ͠Spin-upͷߴ଎Խ •

    CodeDeployʹൺ΂ͯ5ഒ ҆ఆͨ͠σϓϩΠ
  37. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ
  38. • roer/dcd-specͰpipelineͷઃఆΛ؅ཧ • ։ൃऴ͍ྃͯ͠ΔͷͰɺ͜Ε͔Β͸spin σϓϩΠϑϩʔͷίʔυԽ https://blog.spinnaker.io/spin-and-roer-managed-pipeline-templates-4fde2951c648

  39. https://docs.google.com/document/d/17Ifm562aypjLSHnNJ02COFv_VMv3spicH-7zMYd-oXE/edit

  40. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ
  41. • Gate͕REST APIΛఏڙ͍ͯ͠Δ • Pipelineͷ࣮ߦ΍݁Ռͷऔಘ౳͕Մೳ ࣗಈσϓϩΠ https://cloudacademy.com/blog/netflix-spinnaker/

  42. • Pipelineͷىಈ/੒ޭ/ࣦഊ౳Λ௨஌Մೳ • Slack, Mail, HipChat, SMS… ࣗಈσϓϩΠ

  43. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ
  44. • Netflix, Google͕ੵۃతʹ։ൃ ։ൃ͕׆ൃ https://cloud.withgoogle.com/next18/sf/sessions/session/155951

  45. • ❌ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ
  46. • ❌ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ
  47. None
  48. SpinnakerͰ΋ Firewall, LB, ASGΛ࡞੒Մೳ

  49. ৽͍͠αʔόάϧʔϓΛ࡞੒͠ ͍ͯ͘ࢥ૝

  50. 'JSFXBMM -# "VUP4DBMJOH (SPVQ Ҡߦલ Ҡߦޙ

  51. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ
  52. • σϑΥϧτͷઃఆͩͱrate exceededͰσϓϩ Π͕ࣦഊ͕ͪ͠ AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ clouddriver.yml

  53. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ
  54. https://cloud.google.com/marketplace/ ϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ϚωʔδυɾαʔϏε͕ແ͍ͷͰࣗ෼Ͱӡ༻ • GCSʹconfig৘ใΛอଘ • diskͷϚ΢ϯτ΍֤छઃఆ͸εΫϦϓτԽ

  55. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ
  56. • UI্ͷόάɺvupʹࣦഊ͢ΔɺCPU࢖༻཰͕ಥ વ্͕Δɺϓϩηε͕ಥવࢮ͵౳ όά͕ଟ͍

  57. GCEʹσϓϩΠޙΠϯελϯεͷ ୆਺͕Ұؾʹmin·ͰݮΔ RPSͰεέʔϧ͍ͤͯ͞Δ৔߹ɺLB,Φʔτεέʔ ϥʔͱSpinnakerͷ૬ੑ͕ѱ͍ͷͰ10෼଴ͭ

  58. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ
  59. • υΩϡϝϯτ͕ॆ࣮ͯ͠ͳ͍ • ϒϩά౳ͷ৘ใ΋গͳ͍orݹ͘ͳ͍ͬͯΔ • GithubͷissueΈͨΓɺίʔυಡΜͩΓɺSlack Ͱ࣭໰ͨ͠Γ͢Δͷ͕ૣ͍ υΩϡϝϯτ͕গͳ͍

  60. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

  61. ϚϧνΫϥ΢υͷIaaS΁ͷσϓϩΠ ʹ͸Spinnaker͕Φεεϝ

  62. ϚϧνΫϥ΢υରԠ

  63. σϓϩΠؔ࿈ͷػೳ͕ॆ࣮ https://sweetcode.io/13115-2/

  64. Immutable Infraͷڧ੍ https://www.merriam-webster.com/assets/mw/word-of-the-day/social/7bcf417841f7ce37cd7e7e1427c0504f.jpg

  65. Λ৐Γӽ͑Δඞཁ͋Γ όάͷଟ͞ υΩϡϝϯτͷগͳ͞

  66. σϓϩΠπʔϧͰ೰ΜͰΔͳΒ ੋඇݕ౼͍ͯͩ͘͠͞ʂ

  67. ͓खݩͷQRίʔυ͔Β ͥͻϑΟʔυόοΫΛ͓ئ͍͠·͢

  68. ͓·͚

  69. શମͷpipelineߏ੒

  70. deploy-all deploy-aws deploy-gcp deploy-role1 deploy-role2 … Pipeline (શମ) deploy-role1 deploy-role2

  71. Pipeline (֤Ϋϥ΢υ) αʔόͷroleຖʹσϓϩΠ & rollbackՄೳ

  72. Pipeline (֤Ϋϥ΢υ) Pipeline parameter & conditional on ExpressionͰpipelineΛ੍ޚ

  73. ֤छϚΠΫϩαʔϏεͷઃఆ • ~/.hal/$DEPLOYMENT/profiles/ ҎԼʹϑ ΝΠϧΛ࡞੒ & hal deploy apply ɾ/opt/spinnaker/configʹ൓ө͞ΕΔ(v1.9)