Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SRE Practices in Organizations

SRE Practices in Organizations

Infra Study 2nd #7「SREと組織」の登壇資料です。
https://forkwell.connpass.com/event/228038/

Narimichi Takamura

November 16, 2021
Tweet

More Decks by Narimichi Takamura

Other Decks in Technology

Transcript

  1. Motivation • SRE ͷ৴৚͸ཧղͨ͠ && SRE ͷϓϥΫςΟε΋ཧղͨ͠ • ҰํͰɺͲͷΑ͏ʹͯࣗࣾ͠ʹ SRE

    Λಋೖ͢Ε͹͍͍ͷ͔ϐϯͱ͜ͳ͍ • IT ٕज़Ҏ֎ʹ΋ཁૉ͕ඞཁͦ͏͕ͩɺ۩ମతʹͲͷΑ͏ͳ΋ͷ͕͋Δͷ͔ • SRE ʹؔ͢Δଞࣾࣄྫ͸ࢀߟʹͳΔ • ҰํͰɺࣗࣾ΁ద༻͢ΔͨΊʹɺͲͷΑ͏ͳ؍఺Ͱݕ౼͢Ε͹Α͍ͷ͔ → ιϑτεΩϧ ͱ SRE ૊৫ͷઃܭϙΠϯτ ʹ͍ͭͯ࿩͠·͢ → ࠓճͷൃද͕ ࣗࣾͷ SRE ૊৫ͷ্ཱͪ͛ɾशख़ ͷҰॿʹͳΕ͹޾͍Ͱ͢
  2. Table of Contents • Why is Organization Important in SRE?

    • Soft Skills required to implement SRE • SRE Organization Design
  3. Business metrics Include Engineering metrics 1 1 Mohit Suley and

    Kurt Andersen, Understanding Business Metrics Can Make You a Better SRE, 2019, SREcon
  4. “A good SRE has an ability to critically examine a

    system and use that to guide them when asking questions of the system.” — Jamie Wilkinson, SRE at Google
  5. Top 5 Soft Skills in SRE3 1. Problem Solving 2.

    Teamwork 3. Composure underpressure 4. Written communication 5. Verbal communicaiton 3 Catchpoint, 2018 SRE report
  6. Case Soft Skill Postmortem Blameless, Critical Thinking... SLI/SLO Organizational Behavior...

    Building consensus with managers Facilitation, Negotiation...
  7. جૅཧ࿦ͷ۩ମྫ • ݸਓ • ex. εϖϯαʔʮණࢁϞσϧʯɺϘϠςΟζʮίϯϐςϯγʔ֓೦ ਤʯɺόϯσϡʔϥʮࣗݾޮྗײͷߏ੒ཁૉʯ • ूஂ •

    ex. ϨϰΟϯʮ૊৫มֵϓϩηεʯɺλοΫϚϯϞσϧ • Ϧʔμʔγοϓ࿦ • ex. ΧϦεϚϦʔμʔγοϓɺαʔόϯτϦʔμʔγοϓ...
  8. OB ͷ࣮ફྫ: SLI/SLO ͷஈ֊తͳಋೖ • ૊৫ͷಛੑΛ೺Ѳ ্ͨ͠Ͱɺղౚˠมֵˠ࠶ౚ݁ͷεςοϓ Λ ܦͭͭಋೖ͢Δ •

    Dev ͷߦಈݪཧΛཧղ ্ͨ͠ͰɺSLI/SLO ͷಋೖোนΛԼ͛Δ • SLI/SLO ΛτϦΨʔʹΞΫγϣϯͰ͖ΔΑ͏ʹɺߦಈม༰Λଅ ͢ࢪࡦ ʹऔΓ૊Ή
  9. SLI/SLO ಋೖ: ϑΣʔζ1 ·ͣ͸ SRE ͕ओମͱͳͬͯ૊৫ʹ SLI/SLO Λಋೖ͠ɺՁ஋ݕূΛߦ͏͜ͱΛ໨ࢦ ͢ɻ ӡ༻͸શମΛר͖ࠐΈͭͭɺSRE

    ͕ίϯτϩʔϧͰ͖ΔൣғͰ͸͡ΊΔͱΑ͍ɻ 1. SLI/SLO ͕ఆٛ͞Ε͍ͯΔ 2. SLI/SLO ʹؔ͢ΔϫʔΫϑϩʔ͕ఆٛ͞Ε͍ͯΔ 3. αʔϏενʔϜΛר͖ࠐΈͭͭɺSRE ͕ओମͱͳͬͯ SLO ͷӡ༻͕ߦΘΕ͍ͯΔ • SLO ͷ஋ΛτϦΨʔʹΞϥʔτ௨஌Λ͢Δ • ৼΓฦΓձΛߦ͏
  10. SLI/SLO ಋೖ: ϑΣʔζ2 SRE ͷ௚઀తͳࢧԉͳ͠Ͱ SLI/SLO ͕ӡ༻͞ΕΔମ੍Λ໨ࢦ͢ɻ ϑΣʔζ1ͰɺSLI/SLO ʹର͢ΔՁ஋͕ೝΊΒΕ͔ͯΒ͜ͷϑΣʔζʹҠߦ͢Δɻ ר͖ࠐΉਓ਺΍ϩʔϧ͕૿͍͑ͯΔ఺͕ϑΣʔζ1ͱ͸ҟͳΔɻ

    ΑΓଟ͘ͷਓ͕ސ٬ࢹ఺Λ࣋ͬͯ SLI/SLO Λӡ༻͢Δঢ়ଶΛ໨ࢦ͢ɻ 1. PdM ΍ࣄۀ੹೚ऀͳͲͱͱ΋ʹɺࣄۀࢹ఺Λ౿·͑ͯ SLI/SLO ΛఆΊΔ͜ͱ ͕Ͱ͖Δ 2. αʔϏενʔϜ͕ओମͱͳͬͯ SLO ͷӡ༻͕ߦΘΕ͍ͯΔ 3. Embedded SRE ͱͯ͠αʔϏενʔϜΛϑΥϩʔ͢Δମ੍͕͋Δ
  11. it's difficult to find someone who's lucky enough to only

    have useful, effective meetings. This is equally true for SRE. — Chapter 31 - Communication and Collaboration in SRE
  12. ୅දతͳ 2 ͭͷϩʔϧ6 6 New Relic, SRE-iously: Defining the Principles,

    Habits, and Practices of Site Reliability Engineering , 2018
  13. Responsibilities • ۀ຿ͷ෼୲΍੹೚ͷॴࡏΛ໌֬ʹ͢Δ • RACIϚτϦΫε͸ҎԼͷ4ͭͷཁૉΛ໌֬ʹࣔ͢ࡍʹ༗ޮ • RʢResponsibleʣ: ࣮ߦ੹೚ऀ • AʢAccountableʣ:

    આ໌੹೚ऀ • CʢConsultedʣ: ૬ஊઌ • IʢInformedʣ: ใࠂઌ • Google ͷهࣄͰ΋ RACI ༻ޠ͕ར༻͞Ε͍ͯΔ7 7 Alex Bramley, Are we there yet? Thoughts on assessing an SRE team’s maturity, 2021
  14. Mindset • ૊৫ͷ৴པੑʹ͸ 5 ͭͷجຊతஈ֊͕͋Γɺ͋Δ࣌఺ͷ૊৫ͷϚΠϯυηοτΛද͢9 • Absent: ૊৫ʹͱͬͯ৴པੑ͸ޙճ͠ʹͳ͍ͬͯΔঢ়ଶ • Reactive:

    ௚ۙͰੜͨ͡৴པੑͷ໰୊ͷϑΥϩʔ͕ߦΘΕΔ͕ɺγεςϜ΁ͷ௕ظతͳ౤ࢿ͸͠ͳ͍ • Proactive: ఆظతͳ૊৫ϓϩηεΛ௨ͯ͡જࡏతͳ৴པੑϦεΫ͕ಛఆ͞Εରॲ͞ΕΔ • Strategic: ΞʔΩςΫνϟɺϓϩμΫτɺϓϩηεΛମܥతʹมߋ͢Δ͜ͱͰϦεΫͷΫϥεΛ؅ཧ ͢Δ • Visionary: ৴པੑͷ࠷ߴҐʹ౸ୡ͓ͯ͠Γɺ৴པੑ΁ͷ෯޿͍औΓ૊ΈΛϕετϓϥΫςΟε͓Αͼ ܦݧʹج͍ͮͯࣾ಺֎ͰਪਐͰ͖Δ 9 What’s your org’s reliability mindset? Insights from Google SREs
  15. Mindset ͷཁ఺ • ඞͣ͠΋ Strategic ϑΣʔζ΍ Visionary ϑΣʔζʹ͍Δඞཁ͸ͳ͍ • ෳ਺ͷϑΣʔζʹ·͕ͨΔଐੑΛ͍࣋ͬͯΔ͜ͱ΋Ұൠత

    • େ෦෼͸डಈత͕ͩҰ෦͸ੵۃతଐੑΛ࣋ͭύλʔϯ΋͋Δ • ϚΠϯυηοτ͸૊৫ͷঢ়ଶʹ߹ΘͤͯมԽ͍ͤͯ͘͞ඞཁ͕͋Δ • e.g. डಈతˠੵۃతˠઓུత • ࡞ۀΛந৅Խ͠ɺٕೳΛ఻ঝ͠ɺߟ͑Λ໌จԽ͠ͳ͕ΒϑΣʔζΛ্͛ ͍ͯ͘
  16. Why is Organization Important in SRE? • ৴པੑ͸Ϗδωεʹ͓͍ͯॏཁͳࢦඪͰ͋ΓɺاۀશମʹӨڹ͕͋Δ ͨΊ •

    ৴པੑ͸ސ٬ʹڧ͘ඥ෇͍͓ͯΓɺSRE νʔϜ୯ମͰ؅ཧ͢Δͷ͸ࠔ೉ • SRE ͷ࣮ફ͸ɺଟ͘ͷίϥϘϨʔγϣϯΛ௨ͯ͡૊৫తʹऔΓ૊Ήඞཁ͕ ͋Δ ͨΊ • Ұ؏ͨ͠৴৚ʹج͍ͮͨϓϥΫςΟεͷ࣮ફʹ͸ɺจԽͷৢ੒ͱՁ஋؍ͷ ڞ༗͕ඞཁෆՄ Ͱ͋ΔͨΊ • ݸਓͰ͸ͳ͘ɺ૊৫తʹऔΓ૊Ήඞཁ͕͋Δ
  17. Soft Skills required to implement SRE • SRE ʹ͸ϋʔυεΩϧ͚ͩͰͳ͘ιϑτεΩϧ΋ॏཁ •

    ૊৫ʹ SRE Λಋೖ͢Δ্ͰॏཁͳιϑτεΩϧͷྫΛ঺հ • SLI/SLO ΍ϙετϞʔςϜͳͲͷϓϥΫςΟεͷ࣮ફʹ໾ཱ ͭεΩϧͱͯ͠ɺOrganizational Behavior ͱ Facilitation Λઆ ໌
  18. SRE Organization Design • ࣗࣾʹͱͬͯద੾ͳ SRE ૊৫Λͭ͘Δࡍʹॏཁͳ3ͭͷϙΠϯτΛ঺հ • গͣͭ͠ਐΊΔͨΊʹ͸֤ϙΠϯτΛஈ֊తʹҠߦ͍ͯ͘͠ͱΑ͍ •

    Roles: ·ͣ͸ Pure SRE ͔Β͸͡Ίͯɺঃʑʹ Embedded SRE Λݕ౼͢Δ • Responsibilities: ·ͣ͸ SRE ͕ R Λ୲͍ͳ͕Βɺগͣͭ͠ݖݶҠৡΛਐ Ίͯ A ΍ C ʹҠߦ͢Δ • Mindset: ·ͣ͸ Absent Λղফ͠ɺม༰Ͱ͖Δ෦෼Λݟ͚ͭͯ Reactive ΍ Proactive ʹ͍ͯ͘͠