Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Fourth Age of SRE

The Fourth Age of SRE

Cloud Operator Days Tokyo 2021の登壇資料です

6de30acfd736745120a3d6457dc6d45c?s=128

Daiki Matsui

July 15, 2021
Tweet

Transcript

  1. 1 The Fourth Age of SRE No SREΛ໨ࢦͯ͠ ɹɹʛɹɹ© PLAID

    Inc. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ גࣜձࣾϓϨΠυ দҪେٍ
  2. 2 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ࣗݾ঺հ দҪ େٍ גࣜձࣾϓϨΠυ

    Lead SRE SREจԽΛͻΖΊΔ 2015/11ೖࣾ GCP/AWS twitter : @ikemonn 31 α΢φ͕޷͖
  3. 3 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. KARTEʹ͍ͭͯ $9 ސ٬ମݧ ϓϥοτϑΥʔϜ

  4. 4 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. KARTEʹ͍ͭͯ ֬ೝ͢Δ جຊαʔϏε΍͝ར༻ํ๏Λ͝঺հ͠·͢ɻ ͓ಘͳಛయ΍໾ཱͭ৘ใ͕ຬࡌͰ͢ɻ

    ॳΊͯͷํ΁ ְתּׅ然钠ׅ׷ 6*رؠ؎ش٦׌ֽוזַזַ♳麦׃זְծ➙ ״׶إٔؗ،حف׃׋ְהְֲ倯䗳铣דׅկ UIرؠ؎ش٦ָ濼׏גֶֻץֹ 7אךرؠ؎ٕٝ٦ٕ DESIGN RULES » CHECK ׆׏ה⢪ִ׷،؎ذي׌ֽ׾䲧ִת׃׋կ ֿך堣⠓׾ֶ鋅鷕׃זֻկ 窫㼎ծ 妜׃ְ ౙͷओ໾ɺ Ξ΢λʔ COLLECTION OUTER 嗚稊勴⟝׾㼰׃㢌刿ׅ׷׌ֽדծ֮ז׋ך椚 䟝ך暟⟝ח⳿⠓ִ׷〳腉䚍ָ넝ֻז׶תׅկ 勴⟝׾㢌ִג嗚稊׃ג׫גֻ׌ְׁկ ׀䋞劄ך暟⟝כ 鋅אַ׶תׇ׿ד׃׋ַ ꟗׄ׷ ҰਓҰਓʹ߹Θͤͨ ސ٬ମݧΛఏڙ WebαΠτͷ๚໰ऀͷߦಈΛ
 ސ٬͝ͱʹϦΞϧλΠϜʹղੳ $9 ސ٬ମݧ ϓϥοτϑΥʔϜ
  5. 5 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. Stats of KARTE Real-time,

    Large-Scale, Global to real-time actions 0.x sec stored 8+ PB events / sec 105,000 inserted / month 1 PB analyzed / month 180+ PB
  6. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ຊ೔ͷςʔϚ 6 ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͯ͠ ϓϨΠυ͕ߦ͍ͬͯΔ௅ઓ💪

    • The Fourth Age of SREͱ͸ • No SREΛ໨ࢦͨ͠ܦҢ • No SREΛ࣮ݱ͢ΔϙΠϯτ • ·ͱΊ ࿩͢͜ͱ
  7. 7 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of

    SREͱ͸
  8. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ The Fourth Age of SREͱ͸ 8 ɹɹʛɹɹ© PLAID

    Inc. 97 Things Every SRE Should Know( O’Reilly) The Third Age of SRE 1⃣ The First Age SRE͸Googleಠࣗͷ΋ͷͩͬͨ 2⃣ The Second Age SREຊ͕ൃച͞Εɺόζϫʔυʹͳͬͨ 3⃣ The Third Age ઐ೚ͷSRE͕࠷దԽ͞ΕɺSRE͚ͩͰͳ͘ ͢΂ͯͷΤϯδχΞ͕SREͷ໾ׂΛ୲͏
  9. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ The Fourth Age of SREͱ͸ 9 ɹɹʛɹɹ© PLAID

    Inc. 97 Things Every SRE Should Know( O’Reilly) The Third Age of SRE The First Age SRE͸Googleಠࣗͷ΋ͷͩͬͨ The Second Age SREຊ͕ൃച͞Εɺόζϫʔυʹͳͬͨ 3⃣ The Third Age ઐ೚ͷSRE͕ݮΓɺ͢΂ͯͷΤϯδχΞ͕ ࢓ࣄͷҰ෦ͱͯ͠SREͷ໾ׂΛ୲͏ ϓϨΠυ͸ࠓίί
  10. 10 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of

    SREͱ͸ SREͷจԽΛ ΤϯδχΞʹ޿Ί͍ͯΔஈ֊
  11. 11 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of

    SREͱ͸ ࠷ऴతʹ͸ɺ΋͏ҰาਐΜͩ The Fourth Age of SRE ΛΊ͍ͯ͟͠Δ🎯
  12. 12 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of

    SREͱ͸ The Fourth Age of SRE = No SRE
  13. 13 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of

    SREͱ͸ SREͷจԽ͕͖ࠜͮɺ SREͳ͠Ͱ΋ྑ͍΋ͷ͕ͭ͘ΕΔ࣌୅
  14. 14 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ

  15. 15 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ αʔϏεͱ૊৫ͷ੒௕Λ SRE͕Χόʔ͖͠Εͳ͘ͳΓͦ͏ͩͬͨ

  16. 16 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • 2018೥͔Β։ൃऀ͕ٸ૿ • SRE͸2,3໊

    • ࿈೔ͷো֐ରԠʹΑΔർ ฐ • ։ൃͱӡ༻͕෼཭ͨ͜͠ ͱʹΑΔαΠϩԽ No SREΛ໨ࢦͨ͠ܦҢ ΤϯδχΞ਺ͷਪҠ
  17. 17 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ؤுͬͯSREΛ΋ͬͱ࠾༻͢Δʁ

  18. 18 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ SREͷ࠾༻͸೉͍͠ Ծʹ࠾༻Ͱ͖ͯ΋ɺ͍͔ͭ͸ಉ͡ঢ়گ͕ى͜Δ

  19. 19 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ࣾ಺ͷΤϯδχΞʹ SREͷจԽΛ޿ΊΕ͹Α͍ͷͰ͸ʁ

  20. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 20 ɹɹʛɹɹ© PLAID Inc. ϓϨΠυͷ SREʹٻΊΒΕΔ͜ͱ

    • ϓϩμΫτͷج൫ઃܭ/ߏங • ϓϩμΫτͷ҆ఆӡ༻
  21. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 21 ɹɹʛɹɹ© PLAID Inc. ϓϩμΫτͷ ج൫ઃܭ/ߏங

    • Ϋϥ΢υΠϯϑϥͷઃܭ/ߏங • Microservicesͷج൫੔උ • etc …
  22. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 22 ɹɹʛɹɹ© PLAID Inc. ϓϩμΫτͷ ҆ఆӡ༻

    • ΞϓϦέʔγϣϯͷ҆ఆੑ޲্ • ϞχλϦϯά؀ڥͷ੔උ • ো֐ରԠ • ΦϖϨʔγϣϯͷࣗಈԽ • etc …
  23. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 23 ɹɹʛɹɹ© PLAID Inc. ͜ͷ2ͭΛ No

    SREͰ࣮ݱͰ͖ΔΑ͏ʹ • ϓϩμΫτͷج൫ઃܭ/ߏங • ϓϩμΫτͷ҆ఆӡ༻
  24. 24 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team 2019: ϓϩμΫτͷج൫ઃܭ/ߏங໘Λղܾ 2020: ϓϩμΫτͷ҆ఆӡ༻໘Λվળ 2021
  25. 25 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
  26. 26 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • VM based MultiCloud

    • ϚωδϝϯταʔϏεΛੵ ۃར༻ • ηϧϑϗετͷDB΋ӡ༻ • Πϯϑϥ͸terraformԽ • ։ൃऀ͕ٸ૿ • ϞϊϦγοΫͳΞʔΩςΫ νϟ͕ݪҼͰ։ൃ͕εέʔ ϧ͠ͳ͍ ը૾͕ೖΓ·͢ 2018 No SREΛ໨ࢦͨ͠ܦҢ
  27. 27 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
  28. 28 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • GKEΛ׆༻ͨ͠ϚΠΫϩ αʔϏε •

    ֤νʔϜ͕1ػೳΛ୲͏ • νʔϜ͝ͱʹαʔό΍ DBΛߏஙɾӡ༻Ͱ͖Δ ٕज़ج൫͸੔ͬͨ 2019 No SREΛ໨ࢦͨ͠ܦҢ
  29. 29 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
  30. 30 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ MicroservicesԽ͕ͨ͠ɺ SREͷෛ୲͸ݮΒͳ͍

  31. 31 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ٕज़ &

    ΞʔΩςΫνϟ໘͚ͩͰ͸ͳ͘ɺ ϚΠϯυηοτ໘΋ඞཁ
  32. 32 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ops teamͱ͍͏

    ։ൃऀͰߏ੒͞ΕͨνʔϜΛͭ͘Γ SREͷจԽͷ఻ಓࢣʹͳͬͯ΋Β͏🧙
  33. 33 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ops teamͱ͸ʁ

    • ໨త: SREͷจԽΛࣾ಺ʹ޿Ίͯ΋Β͏ • ϝϯόʔ: KARTEͷશମ૾Λ೺Ѳ͍ͯ͠Δ/ͯ͠΄͠ ͍ or Πϯϑϥͷ஌ݟ͕͋ΔΤϯδχΞ10໊΄Ͳ • ΍Δ͜ͱ: ো֐ͷҰ࣍ରԠΛߦ͍ɺSREͷ஌ݟΛಘΔ
  34. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 34 ɹɹʛɹɹ© PLAID Inc. ops teamͷӡ༻Ͱ

    ஫ҙͨ͜͠ͱ • ΦϯϘʔσΟϯά • ৼΓฦΓ • εϞʔϧελʔτ
  35. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 35 ɹɹʛɹɹ© PLAID Inc. εϞʔϧελʔτ •

    ཧ૝͸֤νʔϜͷো֐ରԠΛ೚ͤΔ • ؀ڥͷ੔උʹ޻਺͕͔͔ΔͷͰແཧ • ؂ࢹମ੍͕͔ͬ͠Γ੔͍ͬͯΔαʔ Ϗεͷো֐ରԠ͔Β࢝ΊΔ • ௚ۙ3ϲ݄Ͱසൃ͓ͯ͠ΓɺରԠํ ๏͕͋Δఔ౓ܾ·͍ͬͯΔ͕ɺਓؒ ͷ൑அ͕ඞཁͳো֐Λ5ݸ΄Ͳબ୒
  36. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 36 ɹɹʛɹɹ© PLAID Inc. ΦϯϘʔσΟϯά ‣

    Πϯϑϥߏ੒ ‣ ো֐ͷӨڹൣғ ‣ Dashboardͷݟํ ‣ ໰୊ͷ੾Γ෼͚ํ ‣ ରԠͷ࢓ํ • ର৅ো֐͝ͱʹɺԼهΛυΩϡϝϯ τʹ·ͱΊΔ • ࣮ࡍʹো֐࣌ͷάϥϑΛݟͳ͕Βର Ԡํ๏Λઆ໌ • ͠͹Β͘͸ো֐ൃੜ࣌ʹSREͷ୭͔ ͕ݟकΓͭͭରԠ
  37. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 37 ɹɹʛɹɹ© PLAID Inc. ৼΓฦΓ •

    ຖिɺͲΜͳো֐͕ଟ͔ͬͨͷ͔ɺ ୭͕ରԠͯ͘͠Ε͔ͨΛ֬ೝ • ॏେͳΠϯγσϯτʹؔͯ͠͸ɺ Github౳ʹৼΓฦΓ಺༰Λ·ͱΊΔ
  38. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 38 ɹɹʛɹɹ© PLAID Inc. ӡ༻ͯ͠Ͳ͏͔ͩͬͨʁ •

    SREνʔϜ͸ղࢄ🎊 • ϓϩμΫτͷߏங/ӡ༻͸֤νʔϜʹ͓ ೚ͤ • SLO/SLI/ΤϥʔόδΣοτ౳΋ঃʑʹ ઃఆ͞Ε͖ͯͨ • ো֐ରԠ͢ΔਓʹภΓ͕ͰΔ 😄 ྑ͔ͬͨ 😥 վળ͢΂͖
  39. 39 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ࣮ݱ͢ΔϙΠϯτ

  40. 40 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ࣮ݱ͢ΔϙΠϯτ • ٕज़

    & ΞʔΩςΫνϟ • ϚΠϯυηοτ ͷ͕྆࣠ඞཁ
  41. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ࣮ݱ͢ΔϙΠϯτ 41 ɹɹʛɹɹ© PLAID Inc. ٕज़ &

    ΞʔΩςΫνϟ • MicroservicesԽ • ػೳ͝ͱʹνʔϜΛ෼͚Δ͜ͱͰɺ੹೚ൣғ ͕໌֬ʹͳͬͨ • νʔϜ͝ͱʹGKEͷઃఆ΍ඞཁͳϦιʔεͷ ߏங/ӡ༻ΛͰ͖ΔΑ͏ʹͨ͠ • ϚωδϝϯταʔϏεͷਐԽ&ੵۃར༻ • Πϯϑϥͷந৅Խ͕ਐΈɺѻ͍΍͘͢ͳͬͨ • جຊతʹϚωδϝϯταʔϏεΛ࢖͏ • terraformͰ؆୯ʹߏஙͰ͖ΔΑ͏ʹ͢Δ
  42. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ࣮ݱ͢ΔϙΠϯτ 42 ɹɹʛɹɹ© PLAID Inc. ϚΠϯυηοτ •

    ٕज़͚ͩͰ͸ແཧ • ֤νʔϜ͕ΠϯϑϥΛͦΕͧΕͰӡ༻Ͱ͖Δ ج൫͕੔͍ͬͯΔ͚ͩͰ͸ɺࣗ෼ͨͪͰ໘౗ ΛΈΔํ޲ʹ͸͢͢·ͳ͍ • ো֐ରԠΛ௨ֶͯ͠ͼΛਂΊΔ • ؂ࢹମ੍͕੔ͬͨ؀ڥͰɺো֐ରԠΛ࣮ફ͠ ͯ΋Β͏͜ͱͰϕετϓϥΫςΟεֶ͕΂Δ • ࣗ෼ͰखΛಈ͔͢ͷͰɺࣗνʔϜͰ͸Ͳ͏׆ ͔ͤΔ͔Λߟ͑Δ͖͔͚ͬʹͳΔ • SREͷจԽΛ࣮ફ͠ɺ఻͍͑ͯ͘ਓ͕ඞཁ
  43. 43 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ

  44. 44 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ SREͷจԽΛ ΤϯδχΞʹ޿Ί͍ͯΔஈ֊

  45. 45 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ SREͷจԽ͕͖ࠜͮɺ SREͳ͠Ͱ΋ྑ͍΋ͷ͕ͭ͘ΕΔঢ়ଶ Λ໨ࢦ͍ͯ͠Δ🎯

  46. 46 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ • ٕज़ &

    ΞʔΩςΫνϟ • ϚΠϯυηοτ ͷ͕྆࣠ඞཁ
  47. None