SREチームができるまでと、これから

by Takuya Nishigori

Slide 1

Slide 1 text

SREνʔϜ͕Ͱ͖Δ·Ͱͱɺ͜Ε͔Β ɹ VOYAGE GROUP, Inc. @_nishigori SRE LifeCycle in my team - @_nishigori #shinjukugl 1

Slide 2

Slide 2 text

Who are you? • Twi%er @_nishigori • ﬂuct SREs Manager • I love Makeﬁle SRE LifeCycle in my team - @_nishigori #shinjukugl 2

Slide 3

Slide 3 text

ڈ೥΍ͬͯͨ͜ͱ "ࣄۀΛࢧ͑ΔPHP" VOYAGE GROUP ͑Μ͡ʹΌʔͣͰ࿈ࡌ WEB+DB PRESS - ٕज़ධ࿦༷͔ࣾΒग़൛ • Vol.104 PHPͷܧଓతόʔδϣϯΞοϓ • Vol.107 ຊ౰ʹ஌ͬͯΔʁ php.ini ʢPHPҎ֎΋ͨ͘͞Μݴޠ৮͍ͬͯΔ SRE LifeCycle in my team - @_nishigori #shinjukugl 3

Slide 4

Slide 4 text

VOYAGE GROUPͷࣄۀ (2018ݱࡏ) h"ps:/ /voyagegroup.com/business/ SRE LifeCycle in my team - @_nishigori #shinjukugl 4

Slide 5

Slide 5 text

SRE LifeCycle in my team - @_nishigori #shinjukugl 5

Slide 6

Slide 6 text

Agenda • What is SRE? • SRE Book • case ﬂuct • class SRE implements DevOps • Division is not a ﬁrewall SRE LifeCycle in my team - @_nishigori #shinjukugl 6

Slide 7

Slide 7 text

What is SRE? SRE LifeCycle in my team - @_nishigori #shinjukugl 7

Slide 8

Slide 8 text

SRE LifeCycle in my team - @_nishigori #shinjukugl 8

Slide 9

Slide 9 text

ಡ΋͏ʂʂʢ਎΋֖΋ͳ͍ SRE LifeCycle in my team - @_nishigori #shinjukugl 9

Slide 10

Slide 10 text

Case ﬂuct SSP SRE LifeCycle in my team - @_nishigori #shinjukugl 10

Slide 11

Slide 11 text

SRE LifeCycle in my team - @_nishigori #shinjukugl 11

Slide 12

Slide 12 text

࣮ʹෳࡶ >< SRE LifeCycle in my team - @_nishigori #shinjukugl 12

Slide 13

Slide 13 text

͜ͷதͰɺकΓ͍ͨ΋ͷ͸Ͳ͔͜… SRE LifeCycle in my team - @_nishigori #shinjukugl 13

Slide 14

Slide 14 text

SLO (αʔϏεϨϕϧ໨ඪ) ޿ࠂ഑৴෦෼ͷγεςϜతͳ౎߹Ͱʮ޿ࠂ͕ݟΕͳ͍ʯΛͳ͘͢ SRE LifeCycle in my team - @_nishigori #shinjukugl 14

Slide 15

Slide 15 text

SLO (αʔϏεϨϕϧ໨ඪ) SLOʹؚΊ͍ͯͳ͍΋ͷ: • αΠτͷjsࢯࢮ๢ • NoAd (has many many reason) ޿ࠂ͕ݟΕͳ͍έʔε͸࣮ʹ༷ʑɻ ࢪࡦ͸औ͍ͬͯͨΓ͢Δ͕SLOʹ͸͋͘·ͰؚΊ͍ͯͳ͍ɻ ʢԿඦɾԿઍഔମ΋͋ΔͱҰ݄Ͱ··͋Δʣ SRE LifeCycle in my team - @_nishigori #shinjukugl 15

Slide 16

Slide 16 text

SLO (αʔϏεϨϕϧ໨ඪ) ӈਤͷதͰ: • ഑৴αʔό͕௚઀བྷΉϦΫΤετ -> Ϩ εϙϯεΛࢭΊͳ͍ • ٯʹͦ͜Ҏ֎͸ͨ·ʹࢭ·ͬͯ΋͝Ί ΜͪΌ͍ • ؅ཧը໘, etc ... • ʢળॲ͸ͯ͠ΔYoʣ SRE LifeCycle in my team - @_nishigori #shinjukugl 16

Slide 17

Slide 17 text

SLI (αʔϏεϨϕϧࢦඪ) • Τϥʔ཰ • ϨΠςϯγ (˚˚ms Ҏ಺ʹฦͧ͢ʙ) SRE LifeCycle in my team - @_nishigori #shinjukugl 17

Slide 18

Slide 18 text

ʮਪଌ͢ΔͳɹܭଌͤΑʯ — Rob Pike - Notes on Programming in C ϞχλϦϯάେࣄ SLI͕ةͿ·Εͨ࣌ʹɺͦΕҎ֎΋ՄࢹԽ ͯ͠ͳ͍ͱ݁ہԿ͕ݪҼ͔෼͔Βͳ͍ɻ e.g. process͝ͱͷϚγϯϦιʔε࢖༻཰ ӈਤ sre-book Simplicity Part III - Prac4ces ΑΓ SRE LifeCycle in my team - @_nishigori #shinjukugl 18

Slide 19

Slide 19 text

ﬂuct SREνʔϜ͸ SLO / SLA Λҡ࣋͠ଓ͚ΔͨΊͷूஂ SRE LifeCycle in my team - @_nishigori #shinjukugl 19

Slide 20

Slide 20 text

SRE book 5.3 ΤϯδχΞϦϯάͰ͋ΔͨΊͷ৚݅ ͪΐͬͱﬂuct SREνʔϜͰԿͯ͠Δ͔౰ͯ͸Ίͯൈਮͯ͠Έͨ SRE LifeCycle in my team - @_nishigori #shinjukugl 20

Slide 21

Slide 21 text

ιϑτ΢ΣΞΤϯδχΞϦϯά ͍ͩͿίʔυॻ͘ूஂͰ͸͋Δ • terraform • consul and related anything • puppet manifests • bot࡞੒ • etc... SRE LifeCycle in my team - @_nishigori #shinjukugl 21

Slide 22

Slide 22 text

γεςϜΤϯδχΞϦϯά • ૉૣ͍ϏϧυγεςϜͷߏஙɾӡ༻ (Makeﬁle) • σϓϩΠ଎౓ͷ޲্ • Grafana Monitoring͝ʹΐ͝ʹΐ • OSΞοϓάϨʔυɾKernelύϥϝʔλ͍͍ͫͫ • Container Orchestra8on SRE LifeCycle in my team - @_nishigori #shinjukugl 22

Slide 23

Slide 23 text

τΠϧ ๾໓͢Δͧʙ • redashɺର৅ςʔϒϧͷ௥Ճ • ECSͷEC2 Instance drainingϙνοͱͳ Φʔόʔϔου ʮ҆○ઌੜɺਓ͕ཉ͍͠Ͱ͢…ʯ SRE LifeCycle in my team - @_nishigori #shinjukugl 23

Slide 24

Slide 24 text

Χϯόϯ͸͍ͩͿΏΔ;Θ SRE LifeCycle in my team - @_nishigori #shinjukugl 24

Slide 25

Slide 25 text

ͳΜͯ͜ͱΛͯͨ͠Β͜ͷ਺೥Ͱɺ • ΊͬͪΌϞχλϦϯά૿͑ͨ • ϓϩμΫγϣϯϛʔςΟϯάͷස౓͕૿͑ͨʢSREຊ 31ষ SRE ʹ͓͚ΔίϛϡχέʔγϣϯͱίϥϘϨʔγϣϯʣ • blue-green deployment / canary release SRE LifeCycle in my team - @_nishigori #shinjukugl 25

Slide 26

Slide 26 text

ﬂuct SREνʔϜ is Not ઐ೚ But ઐ໳ SRE LifeCycle in my team - @_nishigori #shinjukugl 26

Slide 27

Slide 27 text

SRE LifeCycle in my team - @_nishigori #shinjukugl 27

Slide 28

Slide 28 text

Devision is not a ﬁrewall SRE LifeCycle in my team - @_nishigori #shinjukugl 28

Slide 29

Slide 29 text

class SRE implements DevOps DevOps as ఩ֶ / จԽ SRE is prescrip+ve (نൣ) ~ SRE vs. DevOps: compe1ng standards or close friends? ~ SRE LifeCycle in my team - @_nishigori #shinjukugl 29

Slide 30

Slide 30 text

class SRE implements DevOps ﬂuct SREνʔϜʹͱͬͯ - • developers͸͓٬༷Ͱ͸ͳ͍ʢͳ͔ʔ·ʣ • ݏ͍ͳݴ༿: ʮґཔʯ • ޷͖ͳݴ༿: ʮ૬ஊʯ ʢ͔Βͷʣ ʮϖΞϓϩʯ SRE LifeCycle in my team - @_nishigori #shinjukugl 30

Slide 31

Slide 31 text

ϖΞϓϩάϥϛϯάศར!!! SRE LifeCycle in my team - @_nishigori #shinjukugl 31

Slide 32

Slide 32 text

͋͘·Ͱઐ໳ͳͷͰ ͿͬͪΌ͚୭͕΍ͬͯ΋͍͍ ͦΕͰ΋ɺSREνʔϜ͕ಘҙͳྖҬ͸ͨ͘͞Μ͋Δ • AWSपΓͷ஌ࣝ • OSಛ༗ͷ໰୊ • CI/CD ࢥߟ ͜͜Β΁Μɺ ϖΞϓϩΛܦͯ΍ΔͱΊͬͪΌֶͼ͕͋Δͳͱ SRE LifeCycle in my team - @_nishigori #shinjukugl 32

Slide 33

Slide 33 text

ϖΞϓϩͯ͠ΔͱΑ͘Ϳͪ౰ͨΔ՝୊ ʂݖݶ͕ͳ͍ʂ ʂʂݖݶ͕ͳ͍ʂʂ ʂʂʂݖݶ͕ͳ͍ʂʂʂ SRE LifeCycle in my team - @_nishigori #shinjukugl 33

Slide 34

Slide 34 text

ద੾ͳݖݶҕৡ "You are administrator" AWSͰͳΜͰ΋Ͱ͖ΔIAM User͸ੵۃతʹ౉͢ελΠϧ SRE LifeCycle in my team - @_nishigori #shinjukugl 34

Slide 35

Slide 35 text

ʮadmin౉ͯ͠ΦγϚΠʯ ͡Όͳ͍ ނʹϖΞϓϩάϥϛϯάͯͨ͠Γ SRE LifeCycle in my team - @_nishigori #shinjukugl 35

Slide 36

Slide 36 text

ٯʹDevelopersʹूதͯ͠΄͍͠ྖҬ΋͋Δɻ ͷͰωοτϫʔΫपΓͱ͔͸ͪ͜Β͕΄΅ઌճΓ͍ͨ͠ • AWS Transit Gateway • VPN Connec6ons • EL8 ରԠ • etc ...** SRE LifeCycle in my team - @_nishigori #shinjukugl 36

Slide 37

Slide 37 text

นͳͲͳ͍ʢ͋ͬͨΒյͦ͏ SRE LifeCycle in my team - @_nishigori #shinjukugl 37

Slide 38

Slide 38 text

"ͳΜͱͳ͘఻͔͑ͨͬͨ͜ͱ" SREΛಋೖ͍ͨ͠ ·ͣ SLO / SLI ΛܾΊΑ͏ɺ࿩͸ͦΕ͔Βͩ ʢࠓճ࿩ͯ͠ͳ͍͚ͲʣΠϯγσϯτपΓͱ͔ Φψψϝ: SRE νʔϜͷධՁʹ໾ཱͭϨϕϧผνΣοΫ Ϧετ SRE LifeCycle in my team - @_nishigori #shinjukugl 38

Slide 39

Slide 39 text

"ͳΜͱͳ͘఻͔͑ͨͬͨ͜ͱ" ΋͠SREνʔϜΛͭ͘Γͨ͘ͳͬͨΒ ʮνʔϜͰ͋Δඞཁ͋Δʁʯ΋ߟྀͯ͠ɺɺɺ ෦ॺɾνʔϜ͸น͡Όͳ͍Α ͦͷͨΊͷ૬ޓ౒ྗ͸੯͠·ͣ SRE LifeCycle in my team - @_nishigori #shinjukugl 39

Slide 40

Slide 40 text

[nits] Q. ৽نαʔϏε΍Δͱͨ͠Β SREνʔϜͭ͘Δʁ SRE LifeCycle in my team - @_nishigori #shinjukugl 40

Slide 41

Slide 41 text

[nits] Q. ৽نαʔϏε΍Δͱͨ͠Β SREνʔϜͭ͘Δʁ A. No SRE LifeCycle in my team - @_nishigori #shinjukugl 41

Slide 42

Slide 42 text

[nits] Q. ৽نαʔϏε΍Δͱͨ͠Β SREνʔϜͭ͘Δʁ A. No • ·ͩͦͷن໛͡Όͳ͍ͩΖ͏ͱ͔ࢥ͏ • νʔϜɺ͸ͭ͘Βͳ͍͔ͳ SRE LifeCycle in my team - @_nishigori #shinjukugl 42

Slide 43

Slide 43 text

[nits] Q. ৽نαʔϏε΍Δͱͨ͠Β SREνʔϜͭ͘Δʁ ΍Δͱͨ͠ΒɺSLO / SLA ΛܾΊͯ Full Cycle DevelopersͷूஂΛ໨ࢦ͢ɺ ͔ͳ • Full Cycle Developers at Ne3lix — Operate What You Build • Ne3lixʹ͓͚ΔϑϧαΠΫϧ։ൃऀ―։ ൃͨ͠΋ͷ͕ӡ༻͢Δ SRE LifeCycle in my team - @_nishigori #shinjukugl 43

Slide 44

Slide 44 text

ଞ࿩͔ͨͬͨ͜͠ͱ • Releasement Cycle • Reliable V.S. Speed • Infrastructure as Code ͷढ͍ • αΠτ৴པੑͷϋϯυϦϯά Ͳ͔͜Ͱ࿩ͤΔͱخ͍͠ͳ SRE LifeCycle in my team - @_nishigori #shinjukugl 44

Slide 45

Slide 45 text

thx. ! SRE LifeCycle in my team - @_nishigori #shinjukugl 45