Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reading "The Site Reliability Workbook"

Takeshi Kondo
November 28, 2018

Reading "The Site Reliability Workbook"

Takeshi Kondo

November 28, 2018
Tweet

More Decks by Takeshi Kondo

Other Decks in Technology

Transcript

  1. 3FBEJOH
    l5IF4JUF3FMJBCJMJUZ
    8PSLCPPLz
    43&-PVOHF
    5BLFTIJ,POEP!2VJQQFS

    View Slide

  2. %JEZPVSFBEl43&CPPLz ✋

    View Slide

  3. %JEZPVSFBEl5IF4JUF3FMJBCJMJUZ8PSLCPPLz ✋

    View Slide

  4. %JEZPVSFBEl5IF4JUF3FMJBCJMJUZ8PSLCPPLz ✋
    5PEBZ`T
    5PQJD

    View Slide

  5. 8IZ
    w ͸͡Ίͯͷ43&
    w ਺ϲ݄ಇ͍ͯΈͯɺ43&ͱͯ͠ͷ࣮ྗෆ଍Λ࣮ײ
    w ઌਓͷ஌ܙΛआΓ͍ͨ
    w 43&࣮ફͷίπ΍צॴΛ஌Γ͍ͨ

    View Slide

  6. $POUFOUT
    ͸͡Ίʹ
    2VJQQFS2VJQQFS43&ʹ͍ͭͯ
    5IF4JUF3FMJBCJMJUZ8PSLCPPLʹ͍ͭͯ
    ͓ΘΓʹ
    ͓·͚ɿӳޠٕज़ॻͲ͏ಡΉʁ

    View Slide

  7. (PBM
    w l5IF4JUF3FMJBCJMJUZ8PSLCPPLzʹڵຯ͕͋Δͻͱ͕
    ͬ͘͟Γ಺༰Λ஌ΓɺಡΜͰΈΑ͏ͱࢥ͏͜ͱ
    w ӳޠ͕ۤखͳͻͱ͕ӳޠٕज़ॻΛಡΜͰΈΑ͏ͱࢥ͏
    ͜ͱ

    View Slide

  8. $POUFOUT
    ͸͡Ίʹ
    2VJQQFS2VJQQFS43&ʹ͍ͭͯ
    5IF4JUF3FMJBCJMJUZ8PSLCPPLʹ͍ͭͯ
    ͓ΘΓʹ
    ͓·͚ɿӳޠٕज़ॻͲ͏ಡΉʁ

    View Slide

  9. "CPVU2VJQQFS
    IUUQTHJUIVCDPNRVJQQFSIBOECPPLCMPCNBTUFSDPNQBOZQSPpMFKBNE

    View Slide

  10. "CPVU2VJQQFS
    IUUQTHJUIVCDPNRVJQQFSIBOECPPLCMPCNBTUFSDPNQBOZQSPpMFKBNE

    View Slide

  11. 2VJQQFS43&5FBN
    w .FNCFS
    w .JTTJPO
    w 5PNBLF4UVEZ4BQVSJ2VJQQFSTUBCMFBOESFMJBCMF
    w 3FTQPOTJCJMJUZ
    w 'JYFTTFSWFSJOTUBCJMJUZGBJMVSF
    w 3FTQPOTFUP!TSFJOTMBDLHJUIVC
    w 1SPBDUJWFBQQSPBDIGPSTUBCJMJUZXJUINPOJUPSJOHTFSWJDF
    -POEPO $50&OHJOFFSJOH.BOBHFS

    5PLZP
    .BOJMB

    View Slide

  12. 2VJQQFS43&5FBN
    w .FNCFS
    w .JTTJPO
    w 5PNBLF4UVEZ4BQVSJ2VJQQFSTUBCMFBOESFMJBCMF
    w 3FTQPOTJCJMJUZ
    w 'JYFTTFSWFSJOTUBCJMJUZGBJMVSF
    w 3FTQPOTFUP!TSFJOTMBDLHJUIVC
    w 1SPBDUJWFBQQSPBDIGPSTUBCJMJUZ
    -POEPO
    $50&OHJOFFSJOH.BOBHFS


    5PLZP
    .BOJMB


    View Slide

  13. 3FDFOUMZ

    View Slide

  14. $POUFOUT
    ͸͡Ίʹ
    2VJQQFS2VJQQFS43&ʹ͍ͭͯ
    5IF4JUF3FMJBCJMJUZ8PSLCPPLʹ͍ͭͯ
    ͓ΘΓʹ
    ͓·͚ɿӳޠٕज़ॻͲ͏ಡΉʁ

    View Slide

  15. 1PTJUJPOPGUIJTCPPL
    The Site Reliability Workbook is the hands-on
    companion to the bestselling Site Reliability
    Engineering book and uses concrete examples to
    show how to put SRE principles and practices to work.
    This book contains practical examples from
    Google’s experiences and case studies from Google’s
    Cloud Platform customers.
    IUUQTMBOEJOHHPPHMFDPNTSFCPPLT

    View Slide

  16. (PBMPGUIJTCPPL
    The purpose of this second SRE book is (a) to add more
    implementation detail to the principles outlined
    in the first volume, and (b) to dispel the idea that SRE
    is implementable only at “Google scale” or in “Google
    culture.”
    5IF4JUF3FMJBCJMJUZ8PSLCPPL1SFGBDF

    View Slide

  17. 1BSU'PVOEBUJPOT
    w 4-0&OHJOFFSJOH$BTF4UVEJFT
    w .POJUPSJOH
    w "MFSUJOHPO4-0T
    w &MJNJOBUJOH5PJM
    w 4JNQMJDJUZ

    View Slide

  18. 1BSU'PVOEBUJPOT
    w 4-0&OHJOFFSJOH$BTF4UVEJFT
    w .POJUPSJOH
    w "MFSUJOHPO4-0T
    w &MJNJOBUJOH5PJM
    w 4JNQMJDJUZ
    43&ͷίΞ֓೦

    View Slide

  19. 1BSU1SBDUJDFT
    w 0O$BMM
    w *ODJEFOU3FTQPOTF
    w 1PTUNPSUFN$VMUVSF-FBSOJOHGSPN'BJMVSF
    w .BOBHJOH-PBE
    w *OUSPEVDJOH/PO"CTUSBDU-BSHF4ZTUFN%FTJHO
    w %BUB1SPDFTTJOH1JQFMJOFT
    w $POpHVSBUJPO%FTJHOBOE#FTU1SBDUJDFT
    w $POpHVSBUJPO4QFDJpDT
    w $BOBSZJOH3FMFBTFT

    View Slide

  20. 1BSU1SBDUJDFT
    w 0O$BMM
    w *ODJEFOU3FTQPOTF
    w 1PTUNPSUFN$VMUVSF-FBSOJOHGSPN'BJMVSF
    w .BOBHJOH-PBE
    w *OUSPEVDJOH/PO"CTUSBDU-BSHF4ZTUFN%FTJHO
    w %BUB1SPDFTTJOH1JQFMJOFT
    w $POpHVSBUJPO%FTJHOBOE#FTU1SBDUJDFT
    w $POpHVSBUJPO4QFDJpDT
    w $BOBSZJOH3FMFBTFT
    ૊৫αʔϏε͕֦େ͢Δʹैͬ
    ͯߟ͑Δඞཁ͕͋ΔτϐοΫ

    View Slide

  21. 1BSU1SPDFTTFT
    w *EFOUJGZJOHBOE3FDPWFSJOHGSPN0WFSMPBE
    w 43&&OHBHFNFOU.PEFM
    w 43&3FBDIJOH#FZPOE:PVS8BMMT
    w 43&5FBN-JGFDZDMFT
    w 0SHBOJ[BUJPOBM$IBOHF.BOBHFNFOUJO43&
    43&νʔϜͱͯ͜͠Ε·Ͱͷ಺༰
    ΛͲ͏࣮ߦ͍͔ͯ͘͠

    View Slide

  22. 1BSU1SBDUJDFT
    w 0O$BMM
    w *ODJEFOU3FTQPOTF
    w 1PTUNPSUFN$VMUVSF-FBSOJOHGSPN'BJMVSF
    w .BOBHJOH-PBE
    w *OUSPEVDJOH/PO"CTUSBDU-BSHF4ZTUFN%FTJHO
    w %BUB1SPDFTTJOH1JQFMJOFT
    w $POpHVSBUJPO%FTJHOBOE#FTU1SBDUJDFT
    w $POpHVSBUJPO4QFDJpDT
    w $BOBSZJOH3FMFBTFT
    ʢݸਓతʹʣ໘ന͔ͬͨ
    UPQJDΛ঺հ

    View Slide

  23. /PO"CTUSBDU
    -BSHF4ZTUFN%FTJHO

    View Slide

  24. 8IBUJTl/PO"CTUSBDU-BSHF4ZTUFN%FTJHOz
    w Մ༻ੑͷߴ͍෼ࢄγεςϜΛઃܭ͢Δख๏
    w େن໛γεςϜΛઃܭ͢Δ43&ʹෆՄܽͳεΩϧ
    w ҎԼͷཁૉΛ૊Έ߹Θ͍ͤͯΔ
    w ΩϟύγςΟϓϥϯχϯά
    w ίϯϙʔωϯτ෼཭
    w γεςϜͷ҆શͳॖখ
    w ϗϫΠτϘʔυΛ࢖ͬͨ൓෮తͳΞϓϩʔν
    w ԾઆͱධՁΛ܁Γฦ͢

    View Slide

  25. 5XP1IBTFBOE'PVS2VFTUJPOT
    w #BTJDEFTJHOQIBTF
    w *TJUQPTTJCMF ෺ཧతʹे෼Մೳ͔ʁ
    w $BOXFEPCFUUFS ΑΓΑ͘Ͱ͖Δ͔ʁ
    w 4DBMFVQQIBTF
    w *TJUGFFTJCMF ίετ)8੍໿ԽͰ࣮ݱՄೳ͔ʁ
    w *TJUSFTJMJFOU EPXOͯ͠΋଱͑ΒΕΔ͔
    w $BOXFEPCFUUFS ΑΓΑ͘Ͱ͖Δ͔ʁ

    View Slide

  26. (PPHMF"E8BSETFYBNQMF
    w (PPHMF8FC4FBSDIʹςΩετ޿ࠂΛࡌͤΔγεςϜ
    w ͲͷݕࡧΫΤϦͰදࣔ͞Ε͔ͨͱΫϦοΫ཰Λ஌ΒͤΔ
    w ݕࡧγεςϜͱ޿ࠂγεςϜʹϩάͱͯ͠࢒͢
    w ཁ݅
    w ຖඵճͷ޿ࠂΫϦοΫ
    w ඵ͋ͨΓ݅ͷݕࡧΫΤϦΛ૝ఆ
    w 4-0
    w PGEBTICPBSERVFSJFTDPNQMFUFJOTFDPOE
    w PGUIFUJNF UIF$53EBUBEJTQMBZFEJTMFTTUIBO
    NJOVUFTPME

    View Slide

  27. (PPHMF"E8BSETFYBNQMF
    w ୆ͷϚγϯͰߟ͑Δ
    w ౰વͳ͕Β༷ʑͳ41P'͕͋Γɺ࣮ݱ͸೉͍͠
    w ͕ͩ͜ͷεςοϓ͕ඇৗʹॏཁ
    w ෳ਺ͷϚγϯͰ෼ࢄγεςϜΛߟ͑Δ
    w .BQ3FEVDF
    w -PH+PJOFS
    w 4IBSEFE-PH+PJOFS
    w .VMUJ%BUBDFOUFS

    View Slide

  28. ·ͱΊ
    w ͳͥ/PO"CTUSBDU͔ʁ
    w ͢΂ͯͷγεςϜ͸࣮ࡍͷωοτϫʔΫ΍ίϯϐϡʔλʔϦ
    ιʔεΛ࢖ͬͯಈ͍͍ͯΔ
    w ࠷ॳʹ୆ͷϚγϯͰߟ͑ɺ۩ମతͳϦιʔεΛߟ͍͑ͯ͘
    ͜ͱ͕ॏཁ

    View Slide

  29. $POpHVSBUJPO%FTJHO
    BOE#FTU1SBDUJDFT

    View Slide

  30. $POpHVSBUJPO%FTJHO
    w γεςϜͷҰཁૉͰ͋Γɺ43&ͷλεΫͰ΋͋Δ
    $POpHVSBUJPOΛ҆શ͔ͭ࣋ଓՄೳઃܭ͢Δํ๏Λ঺հ
    ͢Δ
    w γεςϜΛ࡞Δɺ͋Δ͍͸औΓѻ͏ࡍʹ஫ҙ͢΂͖͜ͱ

    View Slide

  31. 4FQBSBUJOH1IJMPTPQIZBOE.FDIBOJDT
    w 1IJMPTPQIZ
    w ઃఆͦͷ΋ͷ͚ͩʹ஫໨
    w ਖ਼͘͠ந৅Խ͢Δ
    w ଟ༷ͳϢʔεέʔεΛαϙʔτ͢Δ
    w .FDIBOJDT
    w ݴޠઃܭɺσϓϩΠઓུɺଞͱͷ࿈ܞ
    w Ͳ͏ઃఆ͢Δ͔ɺͲ͏ద༻͢Δ͔

    View Slide

  32. $POpHVSBUJPO1IJMPTPQIZ
    w ཧ૝ͷ$POpHVSBUJPO͸Ұ੾$POpHVSBUJPO͞Εͳ͍͜ͱ
    w $POpHVSBUJPO͸Ϣʔβʹ࣭໰Λ͢Δ
    w Ϣʔβͷ໨ඪʹ͍࣭ۙ໰Λ͢Δ
    w গͳ͍ઃఆ߲໨ͰϢʔβͷ໨ඪΛ׎͑Δ
    w Ͱ͖Δ͚ͩSFRVJSFΛ΁Β͠ɺPQUJPOBMΛ૿΍͢
    w ͦͷͨΊʹ͸҆શͳEFGBVMU͕ඞཁ
    w ͍͍ͨͯ͸TUBUJD͕ͩɺEZOBNJDʹܾΊΔ͜ͱ΋͋Δ

    View Slide

  33. $POpHVSBUJPO.FDIBOJDT
    w Ϣʔβ͕Ͳ͏΍ͬͯઃఆΛมߋ͢Δ͔
    w $POpHVSBUJPOͱ%BUB͸෼཭ͨ͠΄͏͕ྑ͍
    w %BUB͸෼ੳʹ࢖͑Δ
    w πʔϧ͸ॏཁɺҎԼͷཁૉ͕͋Δͱ͍͍
    w 4FNBOUJDWBMJEBUJPO
    w จ๏͕͍͋ͬͯͯ΋ແҙຯͳઃఆ͕͋Δ͔΋͠Εͳ͍
    w $POpHVSBUJPOTZOUBY
    w TZOUBYIJHIMJHIU
    w MJOUFS
    w BVUPNBUJDTZOUBYGPSNBUUFS

    View Slide

  34. 4BGF$POpHVSBUJPO$IBOHF"QQMJDBUJPO
    w ঃʑʹσϓϩΠͰ͖ΔΑ͏ʹ͠ɺBMMPSOPUIJOHͳมߋ
    Λආ͚Δ
    w ͷఀࢭΛආ͚ΔͨΊ
    w JF,VCFSOFUFST͕QPEΛSPMMJOHVQEBUF
    w ةݥͰ͋Δ͜ͱ͕൑໌ͨ͠ΒมߋΛϩʔϧόοΫ͢Δ
    w Πϯγσϯτͷܧଓ࣌ؒΛ୹ॖ͢ΔͨΊ
    w มߋʹΑͬͯΦϖϨʔλ੍ޚ͕ࣦΘΕͨ৔߹ɺࣗಈϩʔ
    ϧόοΫ͢Δʢ͋Δ͍͸࠷௿Ͱ΋ɺਐߦΛఀࢭ͢Δʣ

    View Slide

  35. ·ͱΊ
    w ֦େɾ࣋ଓՄೳͳγεςϜʹ͢ΔͨΊʹɺ1IJMPTPQIZ
    ͱ.FDBOJDTΛ෼཭ͯ͠ߟ͑Δ
    w 1IJMPTPQIZ
    w Ϣʔβͷೝ஌ෛՙΛ͋͛ͳ͍
    w ઃఆ߲໨ΛͰ͖Δ͚ͩݮΒ͢
    w .FDIBOJDT
    w Ϣʔβ͕ઃఆมߋલʹςετ͕Ͱ͖ΔΑ͏ʹ͢Δ
    w ঃʑʹద༻͠ɺةݥͳͱ͖͸ϩʔϧόοΫ͢Δ

    View Slide

  36. 2VJQQFSͰͷߏ੒؅ཧ
    w ࢖͍ͬͯΔϛυϧ΢ΣΞɾπʔϧ
    w Πϯϑϥ͸΄΅͕ίʔυԽ
    w 3FWFSTF1SPYZʹ/HJOY
    w ΞϓϦέʔγϣϯج൫͸,VCFSOFUFT
    w ,VCFBXT $MPVE'PSNBUJPO

    w ,VCFSOFUFTNBOJGFTU
    w σϓϩΠ͸$JSDMF$*ͰࣗಈԽ
    w ҆શͳߏ੒มߋͷͨΊʹ
    w εςʔδϯάʹઌʹ͋ͯΔ
    w ·͔ͣͬͨΒ͙͢ʹϩʔϧόοΫͰ͖Δ

    View Slide

  37. 4JUF3FMJBCJMJUZ8PSLCPPL$PODMVTJPO
    w 43&*OTFSU0UIFS%JTDJQMJOF
    w 43&ݪଇΛଞͷ෼໺ʹద༻Ͱ͖Δͷ͔Ͳ͏͔Λ໛ࡧ͍ͯ͠Δ
    w ຊॻࣥච࣌఺Ͱ͸%FW4FD0QTͱ͍͏ݴ༿͕ొ৔͍ͯ͠Δ
    w ঺հͨ͠ݪଇͱ࣮ફ͸ଞ෼໺Ͱ΋Ձ஋͕͋Δͱ৴͍ͯ͡Δ
    w 43&#FMPOHTUP"--PG6T
    w ࠷ॳͷຊͰ͸43&Λઆ໌͢Δ͜ͱ͕ಈػͩͬͨ
    w 43&͸ݱࡏHPPHMFҎ֎Ͱ΋޿͕͍ͬͯΔίϛϡχςΟ
    w 43&DPO NFFUVQT ຊͳͲޓ͍ʹֶͿػձ͕ͨ͘͞Μ͋Δ
    w ͜ͷຊ͕͞Βʹձ࿩Λ޿͛ͯ͘ΕΔ͜ͱΛئ͍ͬͯΔ

    View Slide

  38. ͓ΘΓʹ
    w ຊॻ͸ඞͣ͋ͳͨͷڵຯ͕͋ΔτϐοΫ͕͋Δ͸ͣͳͷ
    ͰɺಡΜͰΈΑ͏ͱࢥ͏͖͔͚ͬʹͳΕ͹޾͍Ͱ͢
    w Ҿ͖ଓ͖֤ষ͝ͱʹϒϩάʹϝϞ͍ͯ͜͠͏ͱࢥ͍·͢
    w ॻ੶঺հ͸CMPHͷ΄͏͕޲͍͍ͯΔʜ
    w ߟ͑ɾݪଇΛֶͿ͜ͱͱɺ࣮ࡍʹखΛಈ͔͢͜ͱ྆ํ
    େࣄ
    w ཁ݅Λ۩ମԽ͢Δ͜ͱ͸ܦݧʹཪଧͪ͞ΕͨεΩϧ
    w Ҿ͖ଓ͖ຏ͍͍͖ͯ·͠ΐ͏

    View Slide

  39. 5PEBZ`TDPOUFOUT
    ͸͡Ίʹ
    2VJQQFS2VJQQFS43&ʹ͍ͭͯ
    5IF4JUF3FMJBCJMJUZ8PSLCPPLʹ͍ͭͯ
    ͓ΘΓʹ
    ͓·͚ɿӳޠٕज़ॻͲ͏ಡΉʁ

    View Slide

  40. ༨ஊɿຊΛಡΉͱ͸ʁ
    w ʮಡΜͩʯͱʮಡΜͰͳ͍ʯΛ͸ͬ
    ͖Γ۠ผͰ͖ͳ͍
    w ڭཆ͋Δਓ͕ؒ஌Ζ͏ͱͭͱΊΔ
    ΂͖͸ɺ͞·͟·ͳॻ෺ͷ͍͋ͩ
    ͷʮ࿈བྷʯ΍ʮ઀ଓʯͰ͋ͬͯɺ
    ݸผͷॻ෺Ͱ͸ͳ͍
    w ڭཆͱ͍͏؍఺Ͱ͸ʮશମͷݟ੖
    Β͠ʯͱ͍͏ߟ͕͑ॏཁ

    View Slide

  41. ٕज़΋ಉ͡Ͱ͸ʁ
    w ݸผͷٕज़ɺද໘తͳڍಈ͚ͩΛݟͯ͸͍͚ͳ͍
    w ٕज़ͷཱͪҐஔΛ௫Ή͜ͱ͕େࣄ
    w ͳͥͦͷٕज़͕ݱΕͨͷ͔
    w Կͷ՝୊Λղܾ͢Δͷ͔
    w Ͳͷٕज़ͱͲ͏͍͏ؔ܎ੑʹ͋Δͷ͔

    View Slide

  42. ٕज़ॻ΋ಉ͡ʁ
    w ʮ͸͡ΊʹʯΛಡΜͰຊͷ໨తͱཱͪҐஔΛ஌Δ
    w શ໨࣍ΛோΊͯɺ಺༰Λਪଌ͢Δ
    w શষͷ͸͡Ίͱ݁࿦ΛݟͯɺಡΈ͍ͨষΛۛຯ
    w ಡΈ͍ͨষΛಡΉ

    View Slide

  43. ೔ຊޠͰ΋ӳޠͰ΋ಉ͡
    w ೔ຊޠͩͱzಡΊΔzͷͰ௨ಡͰ͖Δ͕ɻɻɻ
    w ೔ຊޠͷٕज़ॻͰ΋ҎԼͷಡΈํ͸Ͱ͖Δ
    w ʮຊͷํ޲ੑͱཱͪҐஔʯΛ཈͑
    w ಡΈ͍ͨͱ͜Ζ͚ͩΛಡΉ
    w ʢӳޠ͕ۤखͳͻͱʹͱͬͯʣ௨ಡ͕೉͍͔͠Βͦ͜ɺ
    ͜ͷಡΈํΛੵۃతʹ͢΂͖Ͱ͸

    View Slide

  44. 8IPBN*
    DIBTQZ
    DIBTQZ@
    4JUF3FMJBCJMJUZ&OHJOFFS
    BU2VJQQFS
    5BLFTIJ,POEP

    View Slide