Site Reliability Engineering for GMO

Slide 1

Slide 1 text

Site Reliability Engineering ʙفΔͳ!!1શ෦͓લ͕௚͢Μͩฤʙ

Slide 2

Slide 2 text

ࢁԼ࿨඙!QZBNB (.0ϖύϘٕज़ج൫νʔϜ γχΞɾϓϦϯγύϧ ɹΩϟϯϓɺཱྀߦɺώϧτϯ८ΓɺιϫχΤ८Γ 1SPYZαʔό։ൃɺ0QFO5FMFNFUSZ

Slide 3

Slide 3 text

෱རްੜͱਓੜͷςί

Slide 4

Slide 4 text

෱རްੜͱਓੜͷςί JTBΛར༻ͨ॓͠ധ͕೥ؒ3ധ·Ͱ5,000ԁׂҾ ң͕ΜϦεΫݕ਍(ϐϩϦە༗ແ)͕ແྉ Yours࠷ߴͰ͢Ͷʂʂʂʂ ߤۭܥͷձࣾͷ্ڃձһࢿ֨ΛಘΔ ࣗ෼ʹ͋ͬͨΫϨδοτΧʔυΛݟ͚ͭΔ ※શͯIT݈อΛલఏͱ͍ͯ͠·͢

Slide 5

Slide 5 text

ࠓ೔࿩͢͜ͱ SRE (Site Reliability Engineering) ͷجૅ ࣮ફతͳ՝୊ղܾ: GMOϖύϘฤ

Slide 6

Slide 6 text

Site Reliability Engineering

Slide 7

Slide 7 text

Site Reliability Engineering GoogleͰഓΘΕͨγεςϜ؅ཧ ͱαʔϏεӡ༻ͷํ๏࿦ ࠃ಺ʹ͓͍ͯ͸຋༁ຊͷ ొ৔ʹΑͬͯҰؾʹ޿·ͬͨ https://www.oreilly.co.jp/books/9784873117911/

Slide 8

Slide 8 text

–O’REILLY SRE αΠτϦϥΠΞϏϦςΟΤϯδχΞϦϯά 1.2 αʔϏε؅ཧऀ΁ͷGoogleͷΞϓϩʔν “SREͱ͸ɺιϑτ΢ΣΞΤϯδχΞʹ ӡ༻νʔϜͷઃܭΛґཔͨ͠ͱ͖ʹͰ͖͕͋Δ΋ͷͰ͢”

Slide 9

Slide 9 text

DevͱOps ։ൃऀͱӡ༻ऀ͸෼͔Ε͍ͯΔ͜ͱ͕ଟ͍ ։ൃνʔϜ ӡ༻νʔϜ αʔϏε։ൃ ػೳ։ൃ ϦϦʔε αʔϏεӡ༻ Πϯϑϥߏங ো֐ରԠ ϞχλϦϯά

Slide 10

Slide 10 text

Site Reailibility Engineer ιϑτ΢ΣΞͰӡ༻՝୊Λղܾ͢Δ ։ൃνʔϜ ӡ༻νʔϜ αʔϏε։ൃ ػೳ։ൃ ϦϦʔε αʔϏεӡ༻ Πϯϑϥߏங ো֐ରԠ ϞχλϦϯά ιϑτ΢ΣΞ։ൃΛ ࣠ʹӡ༻Λߦ͏

Slide 11

Slide 11 text

Site Reliability Engineer(ing)? Site Reliability Engineer Site Reliability Engineering

Slide 12

Slide 12 text

Reliability = Մ༻ੑ

Slide 13

Slide 13 text

γεςϜʹ͓͚ΔՄ༻ੑͱ͸ γεςϜ͕ར༻Ͱ͖Δঢ়ଶͰ͋Δ͜ͱ γεςϜ͕ܧଓՔಇͰ͖Δ͜ͱ

Slide 14

Slide 14 text

ͳͥWEBαʔϏεͷՄ༻ੑ͕େࣄͳͷ͔ αʔϏεΛར༻Ͱ͖Δ͜ͱ͕ച্ɺརӹͷݯઘ αʔϏε͕࢖͑ͳ͍ˠച্ɺརӹ͕ݮΔˠձࣾͷച্͕ݮΔˠ  ๻ͷڅྉ͕ݮΔˠ ώϧτϯߦ͚ͳ͍

Slide 15

Slide 15 text

ഁ໓ ϩυϦήε!!1

Slide 16

Slide 16 text

͜ΕΒΛղܾ͢Δ=SRE SRE͸ϩʔϧͰ͋Γɺٕज़Ͱ͋Γɺ֓೦Ͱ͋Δɻ  ։ൃऀ΋SREΛ༻͍Δ΂͖ͩ͠ɺ؅ཧऀ΋SREΛ  ཧղ͢΂͖Ͱ͋Δɻ  ͳͥͳΒ͹ɺՄ༻ੑ͕୲อ͞Εͯ  ॳΊͯϢʔβʔ͸αʔϏεΛར༻Ͱ͖Δ͔ΒͰ͋Δ

Slide 17

Slide 17 text

ϩʔϧͱͯ͠ͷSRE

Slide 18

Slide 18 text

SREʹٻΊΒΕΔཁૉ OSʹؔ͢Δ஌ࣝ ιϑτ΢ΣΞ։ൃʹؔ͢Δ஌ࣝ ιϑτ΢ΣΞͰ࣌ؒΛ࡞Δࢥߟ Φϯίʔϧʹର͢Δ੹೚

Slide 19

Slide 19 text

OSʹؔ͢Δ஌ࣝ ϩά͕Ͳ͜ʹ͋Δ͔ʁ(ྫ: /var/log/syslog) ֤छϝτϦΫεͷ֬ೝํ๏͸ʁ(ྫ: cat /proc/meminfo) Ͳ͏͍ͬͨόοϑΝʔ͕͋Δ͔ʁ(ྫ: sysctl xxx) ωοτϫʔΫͷ੍ޚ͸Ͳ͏͔ʁ(ྫ: netstat) ϓϩηεͷঢ়گ͸Ͳ͏͔ʁ(ྫ: ps)

Slide 20

Slide 20 text

ιϑτ΢ΣΞ։ൃʹؔ͢Δ஌ࣝ ൃੜͨ͠՝୊ΛղܾͰ͖͏Δ։ൃྗ ӡ༻͢Διϑτ΢ΣΞʹؔ͢Δ஌ࣝ ύϑΥʔϚϯενϡʔχϯάʹؔ͢Δ஌ࣝ ιʔεɾ࣮ߦঢ়ଶ͔ΒಡΈղ͘ϦόʔεɾΤϯδχΞϦϯάྗ

Slide 21

Slide 21 text

ιϑτ΢ΣΞͰ࣌ؒΛ࡞Δࢥߟ ιϑτ΢ΣΞ͸௕͘ಈ͚͹ಈ͘΄ͲෳࡶʹͳΓɺେن໛ʹͳΔ ɹࠓ೔10݅࡞ۀͨ͠಺༰͸̍೥ޙʹ͸1,000݅ʹͳΔ

Slide 22

Slide 22 text

Φϯίʔϧʹର͢Δ੹೚ Φϯίʔϧ = γεςϜো֐౳Ͱൃੜ͢ΔۓٸରԠ ɹେମɺਂ໷ʹ͓͚Δۓٸݺͼग़͠ͷ͜ͱΛࢦ͢

Slide 23

Slide 23 text

Մ༻ੑ

Slide 24

Slide 24 text

WEBαʔϏεͷՄ༻ੑͷࢦඪ αΠτ͕ར༻Մೳ͔ʁ ϨΠςϯγ͸े෼଎͍͔ʁ ίΞػೳ͕ఏڙͰ͖͍ͯΔ͔ʁ ྫ

Slide 25

Slide 25 text

SLIͱSLO SLI = Service Level Indicator αʔϏεϨϕϧࢦඪ SLO = Service Level Objective αʔϏεϨϕϧ໨ඪ

Slide 26

Slide 26 text

Service Level Indicator αʔϏεϨϕϧࢦඪ ϦΫΤετͷ੒ޭ཰ ϨΠςϯγ όονͷ੒ޭ཰ σʔλύΠϓϥΠϯͷॲཧ࣌ؒ Մ༻ੑͷঢ়ଶΛ൑அ͢ΔࢦඪΛఆΊΔ

Slide 27

Slide 27 text

SLO = Service Level Objective αʔϏεϨϕϧ໨ඪ ϦΫΤετͷ੒ޭ਺͕99.999%Ҏ্ ϨΠςϯγͷ99% Tile͕1.5ඵҎԼͷ֬཰͕99.999Ҏ্ ͲΕ͘Β͍ͷՄ༻ੑΛ୲อ͢Δ͔

Slide 28

Slide 28 text

SLO = Service Level Objective αʔϏεϨϕϧ໨ඪ ͜ͷੈʹ࣮֬ͳ͜ͱ͸΄ͱΜͲͳ͍ 100%ͱ99.999%ͷՄ༻ੑͷؒʹܦࡁ߹ཧੑ͕͋Δ͔ʁ 100%ͱ99.999%ͷҧ͍͸Θ͔Δ͔ʁ ͳͥ100%Λ໨ࢦ͞ͳ͍ͷ͔

Slide 29

Slide 29 text

“SRE͸ɺ͓Αͦ70%ͷαʔϏεো֐͸ ಈ࡞தͷγεςϜͷมߋʹΑͬͯੜ͍ͯ͡Δ͜ͱΛൃݟ͠·ͨ͠” –O’REILLY SRE αΠτϦϥΠΞϏϦςΟΤϯδχΞϦϯά 1.3.5 มߋ؅ཧ

Slide 30

Slide 30 text

WEBαʔϏεʹ͓͚Δมߋ ϦϦʔεɺσϓϩΠ ύϥϝʔλʔมߋ ࣗಈԽ͞ΕͨϓϩηεʹΑΔมߋ ߏ੒มߋ มߋ͢Ε͹͢Δ΄Ͳো֐͕ى͖ΔՄೳੑ͸ߴ·Δ

Slide 31

Slide 31 text

ઈରϦϦʔε͍ͨ͠Ϛϯ vs ઈର໷தʹى͜͞Εͨ͘ͳ͍Ϛϯ

Slide 32

Slide 32 text

ΤϥʔόδΣοτ

Slide 33

Slide 33 text

ΤϥʔόδΣοτ SLO - SLI = ΤϥʔόδΣοτ ΤϥʔόδΣοτͷൣғͰϦϦʔεΛͨ͠Γɺ՝୊ղܾΛ͢Δ ΤϥʔόδΣοτ͕ͳ͘ͳΕ͹ɺγεςϜͷՄ༻ੑΛվળ͢Δඞཁ͕͋Δ

Slide 34

Slide 34 text

ར༻Ͱ͖ͳ͍ ৽ػೳ͸Ձ஋Λ ࢈·ͳ͍

Slide 35

Slide 35 text

ΤϥʔόδΣοτ͕ރׇͨ͠Β΍Δ͜ͱ ൃੜ͍ͯ͠Δ՝୊Λ։ൃऀͱSREͰڠྗͯ͠ղܾ͢Δ ϦεΫͷߴ͍ϦϦʔεΛԆظ͢Δ SREͷ΃ʔδϟʔΛ։ൃνʔϜʹฦ٫͢Δ

Slide 36

Slide 36 text

Ͱ΋େମ͏·͍͔͘ͳ͍͔Β શ෦ɺ͓લ͕௚͢Μͩ!!1

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

τΠϧ

Slide 39

Slide 39 text

τΠϧͱ͸ γεςϜͷӡ༻ʹඞཁ͕ͩɺ௕ظతʹՁ஋ͷͳ͍͜ͱ खಈʹΑΔσϓϩΠ ա৒ϦιʔεΛར༻͢ΔΫϥΠΞϯτͷ੍ݶ Զͷߟ͑ͨ࠷ڧͷεΫϦϓτͷखಈ࣮ߦ αʔϏεͷ੒௕ʹൺྫͯ͠૿͑Δ΋ͷ

Slide 40

Slide 40 text

ͳͥτΠϧΛ๾໓͢Δඞཁ͕͋Δ͔ʁ αʔϏεͷ੒௕ʹൺྫͯ͋͠ͳͨͷ࣌ؒ͸ͳ͘ͳΔ खಈʹΑΔσϓϩΠ → 1೔10ճ͔Β1೔1,000ճ΁ ա৒ϦιʔεΛར༻͢ΔΫϥΠΞϯτͷ੍ݶ → ഭΓ͘ΔDDoS Զͷߟ͑ͨ࠷ڧͷεΫϦϓτͷखಈ࣮ߦ → ࠷ڧͷεΫϦϓτϕετ1000 αʔϏεͷ੒௕ʹൺྫͯ͠૿͑Δ΋ͷ → ͓͚ۚͩ૿͑Ε͹͍͍ͷʹͶ

Slide 41

Slide 41 text

ԿͰ΋๾໓͢Ε͹͍͍ͬͯ࿩Ͱ΋ͳ͍ ൃੜස౓ͱॴཁ࣌ؒ ๾໓͢Δͷʹ͔͔ΔίετͱϕωϑΟοτ τΠϧͷมԽʹର͢Δϝϯςφϯεɾίετ

Slide 42

Slide 42 text

໦͜ΓͷδϨϯϚ ੾Εͳ͍佁Ͱ ໦Λ੾Γଓ͚ͯ͸ͳΒͳ͍

Slide 43

Slide 43 text

1೔24࣌ؒ͸͋·Γʹ୹͍ ຖ೔ɺखಈରԠͰ30෼͢Δ࡞ۀ͕ɺࣗಈԽ͢Δͷʹ3͔͔࣌ؒΔͳΒɺ  1िؒͰճऩͰ͖Δ͔Β͙͢ʹࣗಈԽ͢Δɻ ͦ͏͢Ε͹໌೔͔Β͋ͳͨʹ͸30෼ͷ͕࣌ؒੜ·ΕΔɻ ໌೔ͷࣗ෼ͷ࣌ؒΛ࡞Δͷ͸ࠓ೔ͷࣗ෼Ͱ͋Δ

Slide 44

Slide 44 text

ϞχλϦϯάͱ ΦϒβʔόϏϦςΟ

Slide 45

Slide 45 text

ϞχλϦϯά ࢮ׆؂ࢹ Ϧιʔε؂ࢹ ֎ܗ؂ࢹ γεςϜͷ݈શੑΛ؂ࢹ͢Δ ྫɿαʔόɺαʔϏε͕ىಈ͍ͯ͠Δ͔ʁ ྫɿCPU΍ϝϞϦɺωοτϫʔΫͷঢ়گ͸Ͳ͏͔ʁ ྫɿhttps://pepabo.comΛ֎෦͔ΒΞΫηεͯ͠ར༻Մೳ͔ʁ

Slide 46

Slide 46 text

Ξϥʔτ αʔϏεɺϊʔυμ΢ϯ Ϧιʔεͷա৒ར༻ େྔΞΫηε ؂ࢹɺ؍ଌͨ͠ςϨϝτϦʔσʔλΛ΋ͱʹɺΞϥʔτ͢Δ

Slide 47

Slide 47 text

ΦϒβʔόϏϦςΟ γεςϜͷঢ়ଶΛ֎෦͔Βཧղ͢Δ͜ͱ͕Ͱ͖Δ=ΦϒβʔόϏϦςΟ͕ߴ͍ ͋ΔϦΫΤετͰར༻͞ΕͨϦιʔε͕୳ࡧՄೳͰ͋Δ ͋Δϓϩηεͷॲཧঢ়گ͕֎෦͔ΒΘ͔Δ ϩά͔ΒͲ͏͍ͬͨॲཧ͕ߦΘΕ͍ͯΔͷ͔Θ͔Δ Մ؍ଌੑ=γεςϜͷঢ়ଶ͕֎෦͔ΒखʹऔΔΑ͏ʹΘ͔Δ

Slide 48

Slide 48 text

OpenTelemetryͷΠϕϯτ΍Γ·͢ ࠓɺ׬શʹΩςΔ!!! Telemetryσʔλͷඪ४Խ

Slide 49

Slide 49 text

γεςϜ ো֐ରԠ

Slide 50

Slide 50 text

WEBαʔϏεͰൃੜ͢Δো֐ ਓҝతͳมߋɺϛεʹΑΔো֐ ա৒ͳϦιʔεফඅʹΑͬͯى͜Δো֐ ෺ཧো֐ ఱࡂ ਆͷౖΓ

Slide 51

Slide 51 text

τϥϒϧγϡʔςΟϯάͷϑϨʔϜϫʔΫ ؍ଌ Ծઆ ݕূ ϑΟʔυόοΫ γεςϜ͕ࠓͲ͏͍͏ঢ়گͳͷ͔ɺਪଌͤͣɺ؍ଌ͢Δ ؍ଌͨ͠಺༰͔Β٬؍తʹਖ਼͍͠ͱࢥ͑ΔԾઆΛཱͯΔ ԾઆΛ΋ͱʹΞϓϩʔν͢Δ ݕূͨ݁͠ՌΛ֬ೝ͠ɺ࠶౓؍ଌ͢Δ

Slide 52

Slide 52 text

ো֐ରԠ͸ۃΊͯߴ౓ͳࣄ͕ଟ͍ ։࢝࣌఺ͰϢʔβʔӨڹ͕ग़͍ͯΔ͜ͱ͕ଟ͍ ௕ظԽ=ܦࡁଛࣦʹ௚݁ ؍ଌɺԾઆݕূΛߦ͏ʹ΋ϕʔεͱͳΔ஌͕ࣝඞཁ

Slide 53

Slide 53 text

ݟ͑ͳ͍ఢͱ ઓΘͳ͍

Slide 54

Slide 54 text

Known-Unknowns Known - Knonwns Known - Unknowns Unknown - Unknown –O’REILLY ৄղ γεςϜɾύϑΥʔϚϯε 2.3.8 Known - Unknowns ʮΧϨʔ͸ඒຯ͍͠ʯ ʮΧϨʔͬͯ৯͍෺͸ඒຯ͍Β͍͠ͳʁʯ ʮനถ͕Ұ൪ʂʂ͜ΕҎ֎ߟ͑ΒΕͳ͍ʂʂʂʯ ແ֮ࣗʹ஌Βͳ͍··ຊ൪αʔόʹΞϓϩʔν͠ͳ͍

Slide 55

Slide 55 text

Πϯγσϯτ؅ཧ

Slide 56

Slide 56 text

責任の所在は「このパラメーター変えてみたら？」「サーバ落ちてね？」この障害いつ頃に収束しますか？影響範囲を教えて下さい何もしてないのに壊れた

Slide 57

Slide 57 text

Πϯγσϯτ؅ཧͷৄࡉ https://speakerdeck.com/hiboma

Slide 58

Slide 58 text

ϙετϞʔςϜ ൃੜͨ͠γεςϜো֐΍Πϯγσϯτʹରͯ͠ɺৼΓฦΓɺจॻԽ͠ɺ  ࠜຊରԠͷ؅ཧ΍ɺະདྷ΁ͷφϨοδͱ͢Δ ൷൑ɺඇ೉Λආ͚ɺࣄ࣮Λ΋ͱʹͲ͏͢Ε͹վળͰ͖Δ͔Λߟ͑Δ

Slide 59

Slide 59 text

ࠜຊରԠ͸ ৸͔ͤͯ͸ͳΒͳ͍

Slide 60

Slide 60 text

ো֐͸ࠓ໷΋ى͖Δ DDoS߈ܸ ಛఆͷϢʔβʔʹىҼ͢ΔύϑΥʔϚϯεͷ໰୊ σʔλϕʔεͷϩοΫʹΑΔτϥϯβΫγϣϯͷ஗Ԇ ۮൃతʹൃੜͨ͠ࣄ৅͸ɺ࣍ͷॠؒʹ΋ى͖ΔՄೳੑ͕͋Δ

Slide 61

Slide 61 text

࣮ફSRE

Slide 62

Slide 62 text

ύϑΥʔϚϯεɾνϡʔχϯά

Slide 63

Slide 63 text

ͳͥύϑΥʔϚϯεɾνϡʔχϯά͕େࣄ͔ Ϣʔβʔମݧͷ޲্ ίετμ΢ϯ ϨεϙϯελΠϜ͸ച্ʹ௚݁͢Δ CPU࣌ؒ = ίετ

Slide 64

Slide 64 text

N+1໰୊ ID Name Department ID 1 A 1 2 B 2 3 C 3 4 D 4 ID Name 1 ૯຿෦ 2 ܦཧ෦ 3 ٕज़෦ 4 ୎ٿ෦ ैۀһςʔϒϧ ෦ॺςʔϒϧ ͢΂ͯͷैۀһͷ෦ॺ໊Λऔಘ͍ͨ͠

Slide 65

Slide 65 text

N+1໰୊ result = query(“SELECT * FROM ैۀһςʔϒϧ”) result.each do |r| dr = query(“SELECT * FROM ෦ॺςʔϒϧ WHERE ID = #{result[‘Department ID’]}”) end O(n²)Ͱޮ཰͕ѱ͍

Slide 66

Slide 66 text

N+1໰୊ result = query(“SELECT ɹɹɹɹɹɹɹ * ɹɹɹɹɹɹɹFROM ैۀһςʔϒϧ INNER JOIN ෦ॺςʔϒϧ ON ैۀһςʔϒϧ.ID = ෦ॺςʔϒϧ.ID”) ݁߹͢Δ͜ͱͰܭࢉྔΛ࡟ݮ

Slide 67

Slide 67 text

ύϑΥʔϚϯεͷ՝୊ΛͲ͏ݟ͚ͭΔ͔ Application Performance Management Tool DataDog Grafana Sentry

Slide 68

Slide 68 text

ա৒ͳτϥϑΟοΫ΁ͷରԠ

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

ECαʔϏεʹେྔΞΫηε͸͖ͭ΋ͷ ͓٬༷ͷݶఆ঎඼ൢചͳͲʹΑΔେྔΞΫηε ΧʔτϘοτʹΑΔෆਖ਼ͳେྔΞΫηε DDoS߈ܸ

Slide 71

Slide 71 text

Slack͔ΒIP੍ݶ Slack͕UIʹͳΔͱ ୭Ͱ΋ରԠ͕ՄೳʹͳΔ͠ ରԠ಺༰͕ཤྺʹ࢒Δ

Slide 72

Slide 72 text

ΞΫηεूத

Slide 73

Slide 73 text

ʮϨʔτϦϛοτΛ ͔͚ΔͱͲ͏ͳΔʁʯ

Slide 74

Slide 74 text

஌ΒΜͷ͔ʁ

Slide 75

Slide 75 text

ΤΠοʂʂʂ ঎඼͕ ചΕͳ͘ͳΔͷͩ

Slide 76

Slide 76 text

ৄղϨʔτϦϛοτʹΑΔܦࡁଛࣦ 1. େྔΞΫηε͕ൃੜ͢Δ 2. ϨʔτϦϛοτΛ͔͚Δ 3. ঎඼ʹΞΫηε͕Ͱ͖ͳ͍ͷͰചΕͳ͘ͳΔ 4. ঎඼͕ചΕͳ͍ͱ͓٬༷γϣοϓͷച্্͕͕Βͳ͍ 5. ͓٬༷ͷച্্͕͕Βͳ͍ͱΧϥʔϛʔͷച্্͕͕Βͳ͍ 6. ձࣾͷച্্͕͕Βͳ͍ 7.Զͷڅྉ্͕͕Βͳ͍!!!ώϧτϯʹߦ͚ͳ͍!!1

Slide 77

Slide 77 text

Ծ૝଴߹ࣨ ઌணॱʹྲྀྔ੍ޚ͠ͳ͕Β  αΠτ΁Ҋ಺͢Δ͜ͱͰ ߴෛՙ࣌Ͱ΋҆ఆͨ͠ΞΫηε͕Մೳ

Slide 78

Slide 78 text

Ծ૝଴߹ࣨ֓ཁ proxy role nginx waiting room proxy role nginx waiting room proxy role nginx waiting room proxy role nginx waiting room proxy role nginx waiting room proxy role nginx waiting room proxy role nginx waiting room ڐՄϦετ Site A 200 Site B ... Site C ... ΫϥΠΞϯτʹ଴ͪ൪߸Λ࠾൪ͯ͠ɺڐՄϦετͷ൪߸ΑΓ খ͍͞൪߸Λ͍࣋ͬͯΕ͹Ұఆ࣌ؒΞΫηε͕Մೳ Client A ΫϥΠΞϯτ൪߸:100 ΞΫηεڐՄ Client B ΫϥΠΞϯτ൪߸:300 ଴߹ࣨ΁

Slide 79

Slide 79 text

https://speakerdeck.com/pyama86/godeshi-zhuang-saretagao-su-na-jia-xiang-dai-he-shi-sabanoshi-zhuang-toxiang-jie

Slide 80

Slide 80 text

σϦόϦʔ

Slide 81

Slide 81 text

σϓϩΠͷ ޫͱҋ

Slide 82

Slide 82 text

届ける価値ステージングと異なるマスター「検証環境では動いたんですけど」環境変数このCI、パスしてるけどテストされてないじゃん急ぎなので、緊急リリースします fi x typお

Slide 83

Slide 83 text

“SRE͸ɺ͓Αͦ70%ͷαʔϏεো֐͸ ಈ࡞தͷγεςϜͷมߋʹΑͬͯੜ͍ͯ͡Δ͜ͱΛൃݟ͠·ͨ͠” –O’REILLY SRE αΠτϦϥΠΞϏϦςΟΤϯδχΞϦϯά 1.3.5 มߋ؅ཧ ࠶ܝ

Slide 84

Slide 84 text

ϦϦʔε ϑϥά

Slide 85

Slide 85 text

ϦϦʔεϑϥάͷجຊ ϦϦʔεϑϥά ɹ֎෦ͷαʔόͱ࿈ܞͯ͠ɺιʔεΛมߋ͢Δ ɹ͜ͱͳ͘ɺΞϓϦέʔγϣϯͷڍಈΛ ɹมߋͰ͖Δ

Slide 86

Slide 86 text

ಛఆͷϢʔβʔͷΈϦϦʔε N%ͷϢʔβʔͷΈϦϦʔε ஈ֊తͳϦϦʔε͕ՄೳͳͷͰ ӨڹൣғΛݶఆͯ͠ϦϦʔε͕Ͱ͖Δ

Slide 87

Slide 87 text

ϓϩάϨογϒ σϦόϦʔ

Slide 88

Slide 88 text

ϓϩάϨογϒσϦόϦʔ ϓϩάϨογϒ ʦ໊ɾܗಈʧਐาతͳ͞·ɻֵ৽తͳ͞·ɻ·ͨɺਐาओٛऀɻ※1 ΧφϦΞϦϦʔεͳͲΛར༻ͯ͠ঃʑʹΞϓϦέʔγϣϯΛϦϦʔε͠ɺ ϝτϦΫεΛ௨ͯ͡ΞϓϦέʔγϣϯ͔Β৘ใΛऩूͯ͠ɺ ϑΟʔυόοΫ੍ޚ͠ͳ͕Β҆શʹιϑτ΢ΣΞΛϦϦʔε͢Δख๏ ※1 https://kotobank.jp/ΑΓ

Slide 89

Slide 89 text

ϓϩάϨογϒσϦόϦʔ ܧଓతσϦόϦʔͱ ϓϩάϨογϒσϦόϦʔ ϓϧϦΫΤετ ςετ ΧφϦΞϦϦʔε ϞχλϦϯά ϦϦʔε ϩʔϧόοΫ ܧଓతσϦόϦʔ ϓϩάϨογϒɾσϦόϦʔ ϓϧϦΫΤετ ϦϦʔε ςετ

Slide 90

Slide 90 text

ϦϦʔεํࣜ ୅දతͳϦϦʔεํࣜ ΧφϦΞϦϦʔε App App App App 25%͚ͩϦϦʔε͢Δ ϩʔϦϯάϦϦʔε App App App App ஈ֊తʹ৽͘͠ϦϦʔε͢Δ BlueGreenϦϦʔε App App ৽͍͠ϦϦʔεͷάϧʔϓΛ࡞Δ App App Ұ؏ੑ͕͋Γɺ੾Γ໭͠΋༰қ͕ͩ Ϧιʔε͕ഒඞཁʹͳΔ Өڹൣғ͕ݶఆ͞ΕΔ͕ɺ ΞϓϦέʔγϣϯͷ Ұ؏ੑ͸ଛͳΘΕΔ ΧφϦΞϦϦʔεͱ Ұॹʹར༻͞ΕΔ͜ͱ͕ଟ͍

Slide 91

Slide 91 text

ϞχλϦϯά • ΞϓϦέʔγϣϯΤϥʔͷൃੜ਺ • HTTPεςʔλείʔυͷมԽ • HTTPϨΠςϯγʔͷมԽ ϝτϦΫεͷѱԽͰࣗಈͰϩʔϧόοΫ͢Δ

Slide 92

Slide 92 text

GMOϖύϘ͸Kubernetes͸Argo RolloutsΛར༻ Argo Rollouts OpenSource GitOps Istio/Nginx Ingress

Slide 93

Slide 93 text

GMOϖύϘ͸Kubernetes͸Argo RolloutsΛར༻

Slide 94

Slide 94 text

VM͸OSSΛ։ൃ 1. github.com(or GHES)ΛϙʔϦϯά 2. ϦϦʔε͕͋Ε͹μ΢ϯϩʔυ 3. 1୆σϓϩΠ͢Δ 4. ࢦఆ࣌ؒϔϧενΣοΫ͕௨Ε͹ɺ શ୆ϦϦʔεɺ௨Βͳ͚Ε͹ϩʔϧ όοΫ https://ten-snapon.com/archives/2972

Slide 95

Slide 95 text

ࠓ೔࿩ͨ͜͠ͱ • Մ༻ੑ͕୲อͰ͖ͯॳΊͯച্্͕͕Δ • SRE͸SREngneer͚͕࣮ͩફ͢Δ΋ͷͰ͸ͳ͘ɺ؅ཧऀɺ։ൃऀ΋  ࣮ફ͢Δඞཁ͕͋Δ • SREͷεΩϧཁ݅͸ଟذʹΘͨΓɺਂ͍ • Մ༻ੑͷ޲্ʹඞཁͳ͜ͱͰ͋Ε͹ͳΜͩͬͯ΍͍͍ͬͯ