Multicloud deploy with Spinnaker / builderscon tokyo 2018

Transcript

1. Multi-cloud deploy with Spinnaker @ikemonn

2. @ikemonn SRE at

3. Spinnakerͷ͜ͱΛ ஌ͬͯΔਓ✋

4. SpinnakerΛࢼͯ͠ΈΑ͏ͱ ࢥͬͯ΋Β͏ ࠓ೔ͷΰʔϧ

5. CXϓϥοτϑΥʔϜ KARTE

6. 4IPFT" 4IPFT# d  d  4)011*/(
   $"35 ๚໰தͷϢʔβʔΛ ϦΞϧλΠϜղੳ ݸʑͷϢʔβʔʹ߹Θͤͨ

   ίϛϡχέʔγϣϯΛఏڙ

7. ϢʔβΛϦΞϧλΠϜͰ֬ೝͰ͖Δ

8. ඵؒ࠷େΠϕϯτ਺ events / sec 22,000 1೔ͷ߹ܭΠϕϯτ਺ events / day 1billion

   ղੳ࣌ؒ sec 0.x

9. ϚϧνΫϥ΢υ(AWS/GCP)

10. ϦΞϧλΠϜੑ͕ٻΊΒΕΔɺ ߴෛՙͳେن໛෼ࢄγεςϜ

11. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

12. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

13. • খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ • ୭Ͱ΋σϓϩΠͰ͖Δ

14. • ໰୊ͷಛఆͱRollback͠΍͘͢͢ΔͨΊ • ࠷௿1೔1ճ͸σϓϩΠ • ଟ͍࣌͸10ճҎ্ খ͞ͳ୯ҐͰසൟʹσϓϩΠ͢Δ https://medium.com/@Zaiku/continuous-delivery-in-a-nutshell-29f4213dabda

15. • վળαΠΫϧͷߴ଎Խ & SRE:3ਓ͚ͩ • σϓϩΠͷࣗಈԽ • σϓϩΠ࣌ʹ໰୊͕͋ͬͨ࣌͸ࣗಈrollback ୭Ͱ΋σϓϩΠͰ͖Δ http://susunshun.hatenablog.com/entry/2015/10/30/011554

16. https://www.autorabit.com/salesforce-deployment-automation/ σϓϩΠπʔϧʹٻΊΔ͜ͱ

17. େن໛Πϯϑϥʹසൟʹ҆ఆͨ͠ ࣗಈσϓϩΠ͕ߦ͑Δ͜ͱ

18. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

19. ϚϧνΫϥ΢υԽ

20. • CodeDeployΛແཧ΍Γ࢖͍ͬͯͨ • ϚϧνΫϥ΢υωΠςΟϒͷπʔϧͰ͸ͳ͍ͨ Ίɺ໰୊͕ى͖ҠߦΛܾఆ ϚϧνΫϥ΢υॳظ https://docs.aws.amazon.com/ja_jp/codedeploy/latest/userguide/instances-on-premises.html

21. ϚϧνΫϥ΢υରԠͷ σϓϩΠπʔϧΛ୳͢

22. https://cloudplatform.googleblog.com/2017/06/spinnaker-10-continuous-delivery.html

23. Spinnaker is an open source, multi-cloud continuous delivery platform for

    releasing software changes with high velocity and confidence. https://netflix.github.io/

24. https://www.spinnaker.io/

25. • ଞͷ࡞ۀ΋͠ͳ͕ΒɺSRE2ਓͰ3ϲ݄ • σϓϩΠϑϩʔͷઃܭ • όάͷճආํ๏ͷ໛ࡧɺPRͷϚʔδ଴ͪ Ҡߦίετ

26. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

27. • ✅ϚϧνΫϥ΢υରԠ • ✅҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ

28. • ✅ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ

29. • ओཁΫϥ΢υϓϩόΠμ΍k8sͰ΋ར༻Մೳ ϚϧνΫϥ΢υରԠ Azure VM Openstack Amazon EC2 Amazon ECS

30. • ֤Ϋϥ΢υͷࠩ෼ΛҙࣝͤͣʹσϓϩΠͰ͖Δ ϚϧνΫϥ΢υରԠ

31. • ϚϧνΫϥ΢υରԠ • ✅҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ

32. • σϓϩΠͷਐḿΛ؅ཧը໘Ͱ֬ೝͰ͖Δ ҆ఆͨ͠σϓϩΠ

33. • stage, pipelineͰॊೈʹσϓϩΠϑϩʔΛ૊ ΊΔ ҆ఆͨ͠σϓϩΠ

34. • stageͷbuild-inػೳ͕ॆ࣮͓ͯ͠Γɺ΍Γͨ ͍͜ͱ͸େ఍Ͱ͖Δ(20+) ҆ఆͨ͠σϓϩΠ

35. • Blue-GreenσϓϩΠ΍ΧφϦΞσϓϩΠʹ΋ ରԠ(spinnaker/kayenta) ҆ఆͨ͠σϓϩΠ https://cloudplatform.googleblog.com/2018/04/introducing-Kayenta-an-open-automated-canary-analysis-tool-from-Google-and-Netflix.html

36. • Immutable Infraͷڧ੍ • ιʔείʔυͰ͸ͳ͘imageΛ੾Γସ͑Δࢥ૝ • ιʔείʔυ΍ઃఆ͕ҟͳΔαʔό͕ଘࡏ͠ͳ͍ • ෭࣍తͳޮՌͱͯ͠Spin-upͷߴ଎Խ •

    CodeDeployʹൺ΂ͯ5ഒ ҆ఆͨ͠σϓϩΠ

37. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • ✅σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ

38. • roer/dcd-specͰpipelineͷઃఆΛ؅ཧ • ։ൃऴ͍ྃͯ͠ΔͷͰɺ͜Ε͔Β͸spin σϓϩΠϑϩʔͷίʔυԽ https://blog.spinnaker.io/spin-and-roer-managed-pipeline-templates-4fde2951c648

39. https://docs.google.com/document/d/17Ifm562aypjLSHnNJ02COFv_VMv3spicH-7zMYd-oXE/edit

40. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ✅ࣗಈσϓϩΠ • ։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ

41. • Gate͕REST APIΛఏڙ͍ͯ͠Δ • Pipelineͷ࣮ߦ΍݁Ռͷऔಘ౳͕Մೳ ࣗಈσϓϩΠ https://cloudacademy.com/blog/netflix-spinnaker/

42. • Pipelineͷىಈ/੒ޭ/ࣦഊ౳Λ௨஌Մೳ • Slack, Mail, HipChat, SMS… ࣗಈσϓϩΠ

43. • ϚϧνΫϥ΢υରԠ • ҆ఆͨ͠σϓϩΠ • σϓϩΠϑϩʔͷίʔυԽ • ࣗಈσϓϩΠ • ✅։ൃ͕׆ൃ

    ظ଴௨Γͩͬͨ͜ͱ

44. • Netflix, Google͕ੵۃతʹ։ൃ ։ൃ͕׆ൃ https://cloud.withgoogle.com/next18/sf/sessions/session/155951

45. • ❌ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ

46. • ❌ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ
47. None

48. SpinnakerͰ΋ Firewall, LB, ASGΛ࡞੒Մೳ

49. ৽͍͠αʔόάϧʔϓΛ࡞੒͠ ͍ͯ͘ࢥ૝

50. 'JSFXBMM -# "VUP4DBMJOH
    (SPVQ Ҡߦલ Ҡߦޙ

51. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • ❌AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ

52. • σϑΥϧτͷઃఆͩͱrate exceededͰσϓϩ Π͕ࣦഊ͕ͪ͠ AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ clouddriver.yml

53. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ❌ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ

54. https://cloud.google.com/marketplace/ ϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ϚωʔδυɾαʔϏε͕ແ͍ͷͰࣗ෼Ͱӡ༻ • GCSʹconfig৘ใΛอଘ • diskͷϚ΢ϯτ΍֤छઃఆ͸εΫϦϓτԽ

55. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • ❌όά͕ଟ͍ • υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ

56. • UI্ͷόάɺvupʹࣦഊ͢ΔɺCPU࢖༻཰͕ಥ વ্͕Δɺϓϩηε͕ಥવࢮ͵౳ όά͕ଟ͍

57. GCEʹσϓϩΠޙΠϯελϯεͷ ୆਺͕Ұؾʹmin·ͰݮΔ RPSͰεέʔϧ͍ͤͯ͞Δ৔߹ɺLB,Φʔτεέʔ ϥʔͱSpinnakerͷ૬ੑ͕ѱ͍ͷͰ10෼଴ͭ

58. • ଞπʔϧͱͷ໾ׂ͕ඃͬͨ • AWSͰσϓϩΠ͕ࣦഊ͕ͪ͠ • ηϧϑϗεςΟϯά͠ͳ͍ͱ͍͚ͳ͍ • όά͕ଟ͍ • ❌υΩϡϝϯτ͕গͳ͍

    ظ଴ͱҧ͍ͬͯͨ͜ͱ

59. • υΩϡϝϯτ͕ॆ࣮ͯ͠ͳ͍ • ϒϩά౳ͷ৘ใ΋গͳ͍orݹ͘ͳ͍ͬͯΔ • GithubͷissueΈͨΓɺίʔυಡΜͩΓɺSlack Ͱ࣭໰ͨ͠Γ͢Δͷ͕ૣ͍ υΩϡϝϯτ͕গͳ͍

60. Agenda • σϓϩΠͷࢥ૝ • Spinnaker΁ͷҠߦ • Spinnakerӡ༻Ͱಘͨ஌ݟ • ·ͱΊ

61. ϚϧνΫϥ΢υͷIaaS΁ͷσϓϩΠ ʹ͸Spinnaker͕Φεεϝ

62. ϚϧνΫϥ΢υରԠ

63. σϓϩΠؔ࿈ͷػೳ͕ॆ࣮ https://sweetcode.io/13115-2/

64. Immutable Infraͷڧ੍ https://www.merriam-webster.com/assets/mw/word-of-the-day/social/7bcf417841f7ce37cd7e7e1427c0504f.jpg

65. Λ৐Γӽ͑Δඞཁ͋Γ όάͷଟ͞ υΩϡϝϯτͷগͳ͞

66. σϓϩΠπʔϧͰ೰ΜͰΔͳΒ ੋඇݕ౼͍ͯͩ͘͠͞ʂ

67. ͓खݩͷQRίʔυ͔Β ͥͻϑΟʔυόοΫΛ͓ئ͍͠·͢

68. ͓·͚

69. શମͷpipelineߏ੒

70. deploy-all deploy-aws deploy-gcp deploy-role1 deploy-role2 … Pipeline (શମ) deploy-role1 deploy-role2

    …

71. Pipeline (֤Ϋϥ΢υ) αʔόͷroleຖʹσϓϩΠ & rollbackՄೳ

72. Pipeline (֤Ϋϥ΢υ) Pipeline parameter & conditional on ExpressionͰpipelineΛ੍ޚ

73. ֤छϚΠΫϩαʔϏεͷઃఆ • ~/.hal/$DEPLOYMENT/profiles/ ҎԼʹϑ ΝΠϧΛ࡞੒ & hal deploy apply ɾ/opt/spinnaker/configʹ൓ө͞ΕΔ(v1.9)