SRE Practices in Organizations

Infra Study 2nd #7「SREと組織」の登壇資料です。
https://forkwell.connpass.com/event/228038/

Transcript

1. View Slide

2. View Slide

3. about:me
   

   View Slide

4. View Slide

5. View Slide

6. View Slide

7. View Slide

8. Motivation
   • SRE ͷ৴৚͸ཧղͨ͠ && SRE ͷϓϥΫςΟε΋ཧղͨ͠
   • ҰํͰɺͲͷΑ͏ʹͯࣗࣾ͠ʹ SRE Λಋೖ͢Ε͹͍͍ͷ͔ϐϯͱ͜ͳ͍
   • IT ٕज़Ҏ֎ʹ΋ཁૉ͕ඞཁͦ͏͕ͩɺ۩ମతʹͲͷΑ͏ͳ΋ͷ͕͋Δͷ͔
   • SRE ʹؔ͢Δଞࣾࣄྫ͸ࢀߟʹͳΔ
   • ҰํͰɺࣗࣾ΁ద༻͢ΔͨΊʹɺͲͷΑ͏ͳ؍఺Ͱݕ౼͢Ε͹Α͍ͷ͔
   → ιϑτεΩϧ ͱ SRE ૊৫ͷઃܭϙΠϯτ ʹ͍ͭͯ࿩͠·͢
   → ࠓճͷൃද͕ ࣗࣾͷ SRE ૊৫ͷ্ཱͪ͛ɾशख़ ͷҰॿʹͳΕ͹޾͍Ͱ͢
   

   View Slide

9. Table of Contents
   • Why is Organization Important in SRE?
   • Soft Skills required to implement SRE
   • SRE Organization Design
   

   View Slide

10. Why is Organization Important in
    SRE?
    

    View Slide

11. Business metrics Include Engineering metrics 1
    1 Mohit Suley and Kurt Andersen, Understanding Business Metrics Can Make You a Better SRE, 2019, SREcon
    

    View Slide

12. SRE collaborate a lot!!
    

    View Slide

13. Culture beats strategy
    every time
    — Chapter 31 - Communication and
    Collaboration in SRE
    

    View Slide

14. Soft Skills required to
    implement SRE
    

    View Slide

15. Why are soft skills so important?
    

    View Slide

16. Chapter 31 -
    Communication and
    Collaboration in SRE
    

    View Slide

17. “A good SRE has an ability to
    critically examine a system and use
    that to guide them when asking
    questions of the system.”
    — Jamie Wilkinson, SRE at Google
    

    View Slide

18. Top 5 Soft Skills in SRE3
    1. Problem Solving
    2. Teamwork
    3. Composure underpressure
    4. Written communication
    5. Verbal communicaiton
    3 Catchpoint, 2018 SRE report
    

    View Slide

19. SRE ʹٻΊΒΕΔιϑτεΩϧ
    • ໰୊ΛޮՌతʹղܾ͢ΔͨΊʹ͸ɺଞऀͱ͏·͘ڠྗ͢Δೳྗ
    ͕ඞཁͰ͋Δ
    • ͢΂ͯͷ౴͑Λ஌͍ͬͯΔ͜ͱΛظ଴͞Ε͍ͯΔͷͰ͸ͳ͘ɺ
    νʔϜ΍૊৫ͷதͰ୭ʹॿ͚ΛٻΊΕ͹Α͍ͷ͔ɺͲͷΑ͏ʹ
    ίϛϡχέʔγϣϯΛͱΕ͹Α͍ͷ͔Λ஌͍ͬͯΔඞཁ͕͋Δ
    

    View Slide

20. Soft Skills Example in Implement SRE
    

    View Slide

21. Case Soft Skill
    Postmortem Blameless, Critical Thinking...
    SLI/SLO Organizational Behavior...
    Building consensus with
    managers
    Facilitation, Negotiation...
    

    View Slide

22. Organizational Behavior
    • ਓΛಈ͔ͨ͢ΊͷΞϓϩʔν͸2ͭʹ෼ྨ͞ΕΔ
    • HRM: ࢓૊ΈʹΑΔΞϓϩʔν
    • OB: ରਓతͳΞϓϩʔν
    • SLI/SLO ΍ϙετϞʔςϜͳͲɺଞνʔϜΛר͖ࠐΉΑ͏ͳ
    γʔϯͰ OB ͸໾ʹཱͭ
    

    View Slide

23. View Slide

24. ϕʔεͱͳΔߦಈݪཧ
    

    View Slide

25. ॏཁͱͳΔ3ͭͷجૅ஌ࣝΧςΰϦ
    

    View Slide

26. جૅཧ࿦ͷ۩ମྫ
    • ݸਓ
    • ex. εϖϯαʔʮණࢁϞσϧʯɺϘϠςΟζʮίϯϐςϯγʔ֓೦
    ਤʯɺόϯσϡʔϥʮࣗݾޮྗײͷߏ੒ཁૉʯ
    • ूஂ
    • ex. ϨϰΟϯʮ૊৫มֵϓϩηεʯɺλοΫϚϯϞσϧ
    • Ϧʔμʔγοϓ࿦
    • ex. ΧϦεϚϦʔμʔγοϓɺαʔόϯτϦʔμʔγοϓ...
    

    View Slide

27. όϯσϡʔϥʮࣗݾޮྗײͷߏ੒ཁૉʯ5
    5 GLOBIS ஌ݟ࿥࣮࿥ʂ, MBAᶈ ࣗ෼มֵ͸ɺߦ͖ͭ໭Γͭɺগͣͭ͠ʲ࠷ऴճʳ, 2015
    

    View Slide

28. ϨϰΟϯʮ૊৫มֵϓϩηεʯ
    

    View Slide

29. OB ͷ࣮ફྫ: SLI/SLO ͷஈ֊తͳಋೖ
    • ૊৫ͷಛੑΛ೺Ѳ ্ͨ͠Ͱɺղౚˠมֵˠ࠶ౚ݁ͷεςοϓ Λ
    ܦͭͭಋೖ͢Δ
    • Dev ͷߦಈݪཧΛཧղ ্ͨ͠ͰɺSLI/SLO ͷಋೖোนΛԼ͛Δ
    • SLI/SLO ΛτϦΨʔʹΞΫγϣϯͰ͖ΔΑ͏ʹɺߦಈม༰Λଅ
    ͢ࢪࡦ ʹऔΓ૊Ή
    

    View Slide

30. SLI/SLO ಋೖͷϑΣʔζ෼͚ͷྫ
    

    View Slide

31. SLI/SLO ಋೖ: ϑΣʔζ1
    ·ͣ͸ SRE ͕ओମͱͳͬͯ૊৫ʹ SLI/SLO Λಋೖ͠ɺՁ஋ݕূΛߦ͏͜ͱΛ໨ࢦ
    ͢ɻ
    ӡ༻͸શମΛר͖ࠐΈͭͭɺSRE ͕ίϯτϩʔϧͰ͖ΔൣғͰ͸͡ΊΔͱΑ͍ɻ
    1. SLI/SLO ͕ఆٛ͞Ε͍ͯΔ
    2. SLI/SLO ʹؔ͢ΔϫʔΫϑϩʔ͕ఆٛ͞Ε͍ͯΔ
    3. αʔϏενʔϜΛר͖ࠐΈͭͭɺSRE ͕ओମͱͳͬͯ SLO ͷӡ༻͕ߦΘΕ͍ͯΔ
    • SLO ͷ஋ΛτϦΨʔʹΞϥʔτ௨஌Λ͢Δ
    • ৼΓฦΓձΛߦ͏
    

    View Slide

32. SLI/SLO ಋೖ: ϑΣʔζ2
    SRE ͷ௚઀తͳࢧԉͳ͠Ͱ SLI/SLO ͕ӡ༻͞ΕΔମ੍Λ໨ࢦ͢ɻ
    ϑΣʔζ1ͰɺSLI/SLO ʹର͢ΔՁ஋͕ೝΊΒΕ͔ͯΒ͜ͷϑΣʔζʹҠߦ͢Δɻ
    ר͖ࠐΉਓ਺΍ϩʔϧ͕૿͍͑ͯΔ఺͕ϑΣʔζ1ͱ͸ҟͳΔɻ
    ΑΓଟ͘ͷਓ͕ސ٬ࢹ఺Λ࣋ͬͯ SLI/SLO Λӡ༻͢Δঢ়ଶΛ໨ࢦ͢ɻ
    1. PdM ΍ࣄۀ੹೚ऀͳͲͱͱ΋ʹɺࣄۀࢹ఺Λ౿·͑ͯ SLI/SLO ΛఆΊΔ͜ͱ
    ͕Ͱ͖Δ
    2. αʔϏενʔϜ͕ओମͱͳͬͯ SLO ͷӡ༻͕ߦΘΕ͍ͯΔ
    3. Embedded SRE ͱͯ͠αʔϏενʔϜΛϑΥϩʔ͢Δମ੍͕͋Δ
    

    View Slide

33. Facilitation
    • ೲಘײͷ͋Δ݁࿦ʹ౸ୡ͢Δ ͨΊͷεΩϧ
    • ޮՌతͳ ձٞͷ४උͱਐߦ Λߦ͏ͨΊʹඞཁͳձٞϚωδϝϯ
    τͷఆੴ
    

    View Slide

34. it's difficult to find
    someone who's lucky
    enough to only have
    useful, effective
    meetings. This is
    equally true for SRE.
    — Chapter 31 - Communication and
    Collaboration in SRE
    

    View Slide

35. View Slide

36. View Slide

37. View Slide

38. SRE Organization Design
    

    View Slide

39. ཧ૝ͱݱ࣮ͷΪϟοϓʹର͢Δղ૾౓Λ্͛Δ
    1. SRE ͸धཁʹରͯ͠Ϧιʔεෆ଍ʹؕΔ͜ͱ͕ଟ͍
    2. εέʔϧ͢Δߏ੒ΛऔΔඞཁ͕͋Δ
    3. εέʔϥϏϦςΟΛอͱ͏ͱ͢Δͱ༷ʑͳϓϥΫςΟε͕ඞཁʹͳΔ
    4. ࣮ࡍʹ͸Ϧιʔε͕গͳ͍ͷͰɺগͣͭ͠ਐΊΔඞཁ͕͋Δʢཁ͸ό
    ϥϯεʣɺͰࢥߟΛࢭΊͳ͍
    5. → SRE ૊৫Λߏங͢Δ্Ͱɺ޻෉Ͱ͖ΔϙΠϯτ͸Ͳ͜ʹ͋Δ͔Λ
    ཧղ͢Δ
    

    View Slide

40. SRE ૊৫Λߏங͢Δࡍʹॏཁͳ3ͭͷϙΠϯτ
    • Roles
    • Responsibilities
    • Mindset
    

    View Slide

41. ୅දతͳ 2 ͭͷϩʔϧ6
    6 New Relic, SRE-iously: Defining the Principles, Habits, and Practices of Site Reliability Engineering
    , 2018
    

    View Slide

42. Responsibilities
    • ۀ຿ͷ෼୲΍੹೚ͷॴࡏΛ໌֬ʹ͢Δ
    • RACIϚτϦΫε͸ҎԼͷ4ͭͷཁૉΛ໌֬ʹࣔ͢ࡍʹ༗ޮ
    • RʢResponsibleʣ: ࣮ߦ੹೚ऀ
    • AʢAccountableʣ: આ໌੹೚ऀ
    • CʢConsultedʣ: ૬ஊઌ
    • IʢInformedʣ: ใࠂઌ
    • Google ͷهࣄͰ΋ RACI ༻ޠ͕ར༻͞Ε͍ͯΔ7
    7 Alex Bramley, Are we there yet? Thoughts on assessing an SRE team’s maturity, 2021
    

    View Slide

43. RACI Matrix example8
    8 Devops Raci Matrix Ppt Powerpoint Presentation File Format
    

    View Slide

44. Mindset
    • ૊৫ͷ৴པੑʹ͸ 5 ͭͷجຊతஈ֊͕͋Γɺ͋Δ࣌఺ͷ૊৫ͷϚΠϯυηοτΛද͢9
    • Absent: ૊৫ʹͱͬͯ৴པੑ͸ޙճ͠ʹͳ͍ͬͯΔঢ়ଶ
    • Reactive: ௚ۙͰੜͨ͡৴པੑͷ໰୊ͷϑΥϩʔ͕ߦΘΕΔ͕ɺγεςϜ΁ͷ௕ظతͳ౤ࢿ͸͠ͳ͍
    • Proactive: ఆظతͳ૊৫ϓϩηεΛ௨ͯ͡જࡏతͳ৴པੑϦεΫ͕ಛఆ͞Εରॲ͞ΕΔ
    • Strategic: ΞʔΩςΫνϟɺϓϩμΫτɺϓϩηεΛମܥతʹมߋ͢Δ͜ͱͰϦεΫͷΫϥεΛ؅ཧ
    ͢Δ
    • Visionary: ৴པੑͷ࠷ߴҐʹ౸ୡ͓ͯ͠Γɺ৴པੑ΁ͷ෯޿͍औΓ૊ΈΛϕετϓϥΫςΟε͓Αͼ
    ܦݧʹج͍ͮͯࣾ಺֎ͰਪਐͰ͖Δ
    9 What’s your org’s reliability mindset? Insights from Google SREs
    

    View Slide

45. Mindset ͷཁ఺
    • ඞͣ͠΋ Strategic ϑΣʔζ΍ Visionary ϑΣʔζʹ͍Δඞཁ͸ͳ͍
    • ෳ਺ͷϑΣʔζʹ·͕ͨΔଐੑΛ͍࣋ͬͯΔ͜ͱ΋Ұൠత
    • େ෦෼͸डಈత͕ͩҰ෦͸ੵۃతଐੑΛ࣋ͭύλʔϯ΋͋Δ
    • ϚΠϯυηοτ͸૊৫ͷঢ়ଶʹ߹ΘͤͯมԽ͍ͤͯ͘͞ඞཁ͕͋Δ
    • e.g. डಈతˠੵۃతˠઓུత
    • ࡞ۀΛந৅Խ͠ɺٕೳΛ఻ঝ͠ɺߟ͑Λ໌จԽ͠ͳ͕ΒϑΣʔζΛ্͛
    ͍ͯ͘
    

    View Slide

46. Lessons Learned
    

    View Slide

47. Why is Organization Important in SRE?
    • ৴པੑ͸Ϗδωεʹ͓͍ͯॏཁͳࢦඪͰ͋ΓɺاۀશମʹӨڹ͕͋Δ ͨΊ
    • ৴པੑ͸ސ٬ʹڧ͘ඥ෇͍͓ͯΓɺSRE νʔϜ୯ମͰ؅ཧ͢Δͷ͸ࠔ೉
    • SRE ͷ࣮ફ͸ɺଟ͘ͷίϥϘϨʔγϣϯΛ௨ͯ͡૊৫తʹऔΓ૊Ήඞཁ͕
    ͋Δ ͨΊ
    • Ұ؏ͨ͠৴৚ʹج͍ͮͨϓϥΫςΟεͷ࣮ફʹ͸ɺจԽͷৢ੒ͱՁ஋؍ͷ
    ڞ༗͕ඞཁෆՄ Ͱ͋ΔͨΊ
    • ݸਓͰ͸ͳ͘ɺ૊৫తʹऔΓ૊Ήඞཁ͕͋Δ
    

    View Slide

48. Soft Skills required to implement SRE
    • SRE ʹ͸ϋʔυεΩϧ͚ͩͰͳ͘ιϑτεΩϧ΋ॏཁ
    • ૊৫ʹ SRE Λಋೖ͢Δ্ͰॏཁͳιϑτεΩϧͷྫΛ঺հ
    • SLI/SLO ΍ϙετϞʔςϜͳͲͷϓϥΫςΟεͷ࣮ફʹ໾ཱ
    ͭεΩϧͱͯ͠ɺOrganizational Behavior ͱ Facilitation Λઆ
    ໌
    

    View Slide

49. SRE Organization Design
    • ࣗࣾʹͱͬͯద੾ͳ SRE ૊৫Λͭ͘Δࡍʹॏཁͳ3ͭͷϙΠϯτΛ঺հ
    • গͣͭ͠ਐΊΔͨΊʹ͸֤ϙΠϯτΛஈ֊తʹҠߦ͍ͯ͘͠ͱΑ͍
    • Roles: ·ͣ͸ Pure SRE ͔Β͸͡Ίͯɺঃʑʹ Embedded SRE Λݕ౼͢Δ
    • Responsibilities: ·ͣ͸ SRE ͕ R Λ୲͍ͳ͕Βɺগͣͭ͠ݖݶҠৡΛਐ
    Ίͯ A ΍ C ʹҠߦ͢Δ
    • Mindset: ·ͣ͸ Absent Λղফ͠ɺม༰Ͱ͖Δ෦෼Λݟ͚ͭͯ Reactive
    ΍ Proactive ʹ͍ͯ͘͠
    

    View Slide

50. We are Hiring!
    topotal.com/careers/software_engineer_sre
    

    View Slide