Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Fourth Age of SRE

The Fourth Age of SRE

Cloud Operator Days Tokyo 2021の登壇資料です

Daiki Matsui

July 15, 2021
Tweet

More Decks by Daiki Matsui

Other Decks in Programming

Transcript

  1. 1 The Fourth Age of SRE No SREΛ໨ࢦͯ͠ ɹɹʛɹɹ© PLAID

    Inc. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ גࣜձࣾϓϨΠυ দҪେٍ
  2. 2 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ࣗݾ঺հ দҪ େٍ גࣜձࣾϓϨΠυ

    Lead SRE SREจԽΛͻΖΊΔ 2015/11ೖࣾ GCP/AWS twitter : @ikemonn 31 α΢φ͕޷͖
  3. 4 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. KARTEʹ͍ͭͯ ֬ೝ͢Δ جຊαʔϏε΍͝ར༻ํ๏Λ͝঺հ͠·͢ɻ ͓ಘͳಛయ΍໾ཱͭ৘ใ͕ຬࡌͰ͢ɻ

    ॳΊͯͷํ΁ ְתּׅ然钠ׅ׷ 6*رؠ؎ش٦׌ֽוזַזַ♳麦׃זְծ➙ ״׶إٔؗ،حف׃׋ְהְֲ倯䗳铣דׅկ UIرؠ؎ش٦ָ濼׏גֶֻץֹ 7אךرؠ؎ٕٝ٦ٕ DESIGN RULES » CHECK ׆׏ה⢪ִ׷،؎ذي׌ֽ׾䲧ִת׃׋կ ֿך堣⠓׾ֶ鋅鷕׃זֻկ 窫㼎ծ 妜׃ְ ౙͷओ໾ɺ Ξ΢λʔ COLLECTION OUTER 嗚稊勴⟝׾㼰׃㢌刿ׅ׷׌ֽדծ֮ז׋ך椚 䟝ך暟⟝ח⳿⠓ִ׷〳腉䚍ָ넝ֻז׶תׅկ 勴⟝׾㢌ִג嗚稊׃ג׫גֻ׌ְׁկ ׀䋞劄ך暟⟝כ 鋅אַ׶תׇ׿ד׃׋ַ ꟗׄ׷ ҰਓҰਓʹ߹Θͤͨ ސ٬ମݧΛఏڙ WebαΠτͷ๚໰ऀͷߦಈΛ
 ސ٬͝ͱʹϦΞϧλΠϜʹղੳ $9 ސ٬ମݧ ϓϥοτϑΥʔϜ
  4. 5 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. Stats of KARTE Real-time,

    Large-Scale, Global to real-time actions 0.x sec stored 8+ PB events / sec 105,000 inserted / month 1 PB analyzed / month 180+ PB
  5. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ຊ೔ͷςʔϚ 6 ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͯ͠ ϓϨΠυ͕ߦ͍ͬͯΔ௅ઓ💪

    • The Fourth Age of SREͱ͸ • No SREΛ໨ࢦͨ͠ܦҢ • No SREΛ࣮ݱ͢ΔϙΠϯτ • ·ͱΊ ࿩͢͜ͱ
  6. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ The Fourth Age of SREͱ͸ 8 ɹɹʛɹɹ© PLAID

    Inc. 97 Things Every SRE Should Know( O’Reilly) The Third Age of SRE 1⃣ The First Age SRE͸Googleಠࣗͷ΋ͷͩͬͨ 2⃣ The Second Age SREຊ͕ൃച͞Εɺόζϫʔυʹͳͬͨ 3⃣ The Third Age ઐ೚ͷSRE͕࠷దԽ͞ΕɺSRE͚ͩͰͳ͘ ͢΂ͯͷΤϯδχΞ͕SREͷ໾ׂΛ୲͏
  7. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ The Fourth Age of SREͱ͸ 9 ɹɹʛɹɹ© PLAID

    Inc. 97 Things Every SRE Should Know( O’Reilly) The Third Age of SRE The First Age SRE͸Googleಠࣗͷ΋ͷͩͬͨ The Second Age SREຊ͕ൃച͞Εɺόζϫʔυʹͳͬͨ 3⃣ The Third Age ઐ೚ͷSRE͕ݮΓɺ͢΂ͯͷΤϯδχΞ͕ ࢓ࣄͷҰ෦ͱͯ͠SREͷ໾ׂΛ୲͏ ϓϨΠυ͸ࠓίί
  8. 11 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of

    SREͱ͸ ࠷ऴతʹ͸ɺ΋͏ҰาਐΜͩ The Fourth Age of SRE ΛΊ͍ͯ͟͠Δ🎯
  9. 13 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of

    SREͱ͸ SREͷจԽ͕͖ࠜͮɺ SREͳ͠Ͱ΋ྑ͍΋ͷ͕ͭ͘ΕΔ࣌୅
  10. 16 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • 2018೥͔Β։ൃऀ͕ٸ૿ • SRE͸2,3໊

    • ࿈೔ͷো֐ରԠʹΑΔർ ฐ • ։ൃͱӡ༻͕෼཭ͨ͜͠ ͱʹΑΔαΠϩԽ No SREΛ໨ࢦͨ͠ܦҢ ΤϯδχΞ਺ͷਪҠ
  11. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 21 ɹɹʛɹɹ© PLAID Inc. ϓϩμΫτͷ ج൫ઃܭ/ߏங

    • Ϋϥ΢υΠϯϑϥͷઃܭ/ߏங • Microservicesͷج൫੔උ • etc …
  12. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 22 ɹɹʛɹɹ© PLAID Inc. ϓϩμΫτͷ ҆ఆӡ༻

    • ΞϓϦέʔγϣϯͷ҆ఆੑ޲্ • ϞχλϦϯά؀ڥͷ੔උ • ো֐ରԠ • ΦϖϨʔγϣϯͷࣗಈԽ • etc …
  13. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 23 ɹɹʛɹɹ© PLAID Inc. ͜ͷ2ͭΛ No

    SREͰ࣮ݱͰ͖ΔΑ͏ʹ • ϓϩμΫτͷج൫ઃܭ/ߏங • ϓϩμΫτͷ҆ఆӡ༻
  14. 24 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team 2019: ϓϩμΫτͷج൫ઃܭ/ߏங໘Λղܾ 2020: ϓϩμΫτͷ҆ఆӡ༻໘Λվળ 2021
  15. 25 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
  16. 26 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • VM based MultiCloud

    • ϚωδϝϯταʔϏεΛੵ ۃར༻ • ηϧϑϗετͷDB΋ӡ༻ • Πϯϑϥ͸terraformԽ • ։ൃऀ͕ٸ૿ • ϞϊϦγοΫͳΞʔΩςΫ νϟ͕ݪҼͰ։ൃ͕εέʔ ϧ͠ͳ͍ ը૾͕ೖΓ·͢ 2018 No SREΛ໨ࢦͨ͠ܦҢ
  17. 27 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
  18. 28 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • GKEΛ׆༻ͨ͠ϚΠΫϩ αʔϏε •

    ֤νʔϜ͕1ػೳΛ୲͏ • νʔϜ͝ͱʹαʔό΍ DBΛߏஙɾӡ༻Ͱ͖Δ ٕज़ج൫͸੔ͬͨ 2019 No SREΛ໨ࢦͨ͠ܦҢ
  19. 29 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020

    2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
  20. 32 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ops teamͱ͍͏

    ։ൃऀͰߏ੒͞ΕͨνʔϜΛͭ͘Γ SREͷจԽͷ఻ಓࢣʹͳͬͯ΋Β͏🧙
  21. 33 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ops teamͱ͸ʁ

    • ໨త: SREͷจԽΛࣾ಺ʹ޿Ίͯ΋Β͏ • ϝϯόʔ: KARTEͷશମ૾Λ೺Ѳ͍ͯ͠Δ/ͯ͠΄͠ ͍ or Πϯϑϥͷ஌ݟ͕͋ΔΤϯδχΞ10໊΄Ͳ • ΍Δ͜ͱ: ো֐ͷҰ࣍ରԠΛߦ͍ɺSREͷ஌ݟΛಘΔ
  22. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 34 ɹɹʛɹɹ© PLAID Inc. ops teamͷӡ༻Ͱ

    ஫ҙͨ͜͠ͱ • ΦϯϘʔσΟϯά • ৼΓฦΓ • εϞʔϧελʔτ
  23. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 35 ɹɹʛɹɹ© PLAID Inc. εϞʔϧελʔτ •

    ཧ૝͸֤νʔϜͷো֐ରԠΛ೚ͤΔ • ؀ڥͷ੔උʹ޻਺͕͔͔ΔͷͰແཧ • ؂ࢹମ੍͕͔ͬ͠Γ੔͍ͬͯΔαʔ Ϗεͷো֐ରԠ͔Β࢝ΊΔ • ௚ۙ3ϲ݄Ͱසൃ͓ͯ͠ΓɺରԠํ ๏͕͋Δఔ౓ܾ·͍ͬͯΔ͕ɺਓؒ ͷ൑அ͕ඞཁͳো֐Λ5ݸ΄Ͳબ୒
  24. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 36 ɹɹʛɹɹ© PLAID Inc. ΦϯϘʔσΟϯά ‣

    Πϯϑϥߏ੒ ‣ ো֐ͷӨڹൣғ ‣ Dashboardͷݟํ ‣ ໰୊ͷ੾Γ෼͚ํ ‣ ରԠͷ࢓ํ • ର৅ো֐͝ͱʹɺԼهΛυΩϡϝϯ τʹ·ͱΊΔ • ࣮ࡍʹো֐࣌ͷάϥϑΛݟͳ͕Βର Ԡํ๏Λઆ໌ • ͠͹Β͘͸ো֐ൃੜ࣌ʹSREͷ୭͔ ͕ݟकΓͭͭରԠ
  25. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 37 ɹɹʛɹɹ© PLAID Inc. ৼΓฦΓ •

    ຖिɺͲΜͳো֐͕ଟ͔ͬͨͷ͔ɺ ୭͕ରԠͯ͘͠Ε͔ͨΛ֬ೝ • ॏେͳΠϯγσϯτʹؔͯ͠͸ɺ Github౳ʹৼΓฦΓ಺༰Λ·ͱΊΔ
  26. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 38 ɹɹʛɹɹ© PLAID Inc. ӡ༻ͯ͠Ͳ͏͔ͩͬͨʁ •

    SREνʔϜ͸ղࢄ🎊 • ϓϩμΫτͷߏங/ӡ༻͸֤νʔϜʹ͓ ೚ͤ • SLO/SLI/ΤϥʔόδΣοτ౳΋ঃʑʹ ઃఆ͞Ε͖ͯͨ • ো֐ରԠ͢ΔਓʹภΓ͕ͰΔ 😄 ྑ͔ͬͨ 😥 վળ͢΂͖
  27. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ࣮ݱ͢ΔϙΠϯτ 41 ɹɹʛɹɹ© PLAID Inc. ٕज़ &

    ΞʔΩςΫνϟ • MicroservicesԽ • ػೳ͝ͱʹνʔϜΛ෼͚Δ͜ͱͰɺ੹೚ൣғ ͕໌֬ʹͳͬͨ • νʔϜ͝ͱʹGKEͷઃఆ΍ඞཁͳϦιʔεͷ ߏங/ӡ༻ΛͰ͖ΔΑ͏ʹͨ͠ • ϚωδϝϯταʔϏεͷਐԽ&ੵۃར༻ • Πϯϑϥͷந৅Խ͕ਐΈɺѻ͍΍͘͢ͳͬͨ • جຊతʹϚωδϝϯταʔϏεΛ࢖͏ • terraformͰ؆୯ʹߏஙͰ͖ΔΑ͏ʹ͢Δ
  28. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ࣮ݱ͢ΔϙΠϯτ 42 ɹɹʛɹɹ© PLAID Inc. ϚΠϯυηοτ •

    ٕज़͚ͩͰ͸ແཧ • ֤νʔϜ͕ΠϯϑϥΛͦΕͧΕͰӡ༻Ͱ͖Δ ج൫͕੔͍ͬͯΔ͚ͩͰ͸ɺࣗ෼ͨͪͰ໘౗ ΛΈΔํ޲ʹ͸͢͢·ͳ͍ • ো֐ରԠΛ௨ֶͯ͠ͼΛਂΊΔ • ؂ࢹମ੍͕੔ͬͨ؀ڥͰɺো֐ରԠΛ࣮ફ͠ ͯ΋Β͏͜ͱͰϕετϓϥΫςΟεֶ͕΂Δ • ࣗ෼ͰखΛಈ͔͢ͷͰɺࣗνʔϜͰ͸Ͳ͏׆ ͔ͤΔ͔Λߟ͑Δ͖͔͚ͬʹͳΔ • SREͷจԽΛ࣮ફ͠ɺ఻͍͑ͯ͘ਓ͕ඞཁ