The Fourth Age of SRE

Cloud Operator Days Tokyo 2021の登壇資料です

Transcript

1. 1 The Fourth Age of SRE No SREΛ໨ࢦͯ͠ ɹɹʛɹɹ© PLAID Inc. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ גࣜձࣾϓϨΠυ দҪେٍ
2. 2 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ࣗݾ঺հ দҪ େٍ גࣜձࣾϓϨΠυ Lead SRE SREจԽΛͻΖΊΔ 2015/11ೖࣾ GCP/AWS twitter : @ikemonn 31 α΢φ͕޷͖
3. 3 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. KARTEʹ͍ͭͯ $9 ސ٬ମݧ ϓϥοτϑΥʔϜ
4. 4 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. KARTEʹ͍ͭͯ ֬ೝ͢Δ جຊαʔϏε΍͝ར༻ํ๏Λ͝঺հ͠·͢ɻ ͓ಘͳಛయ΍໾ཱͭ৘ใ͕ຬࡌͰ͢ɻ ॳΊͯͷํ΁ ְתּׅ然钠ׅ׷ 6*رؠ؎ش٦׌ֽוזַזַ♳麦׃זְծ➙ ״׶إٔؗ،حف׃׋ְהְֲ倯䗳铣דׅկ UIرؠ؎ش٦ָ濼׏גֶֻץֹ 7אךرؠ؎ٕٝ٦ٕ DESIGN RULES » CHECK ׆׏ה⢪ִ׷،؎ذي׌ֽ׾䲧ִת׃׋կ ֿך堣⠓׾ֶ鋅鷕׃זֻկ 窫㼎ծ 妜׃ְ ౙͷओ໾ɺ Ξ΢λʔ COLLECTION OUTER 嗚稊勴⟝׾㼰׃㢌刿ׅ׷׌ֽדծ֮ז׋ך椚 䟝ך暟⟝ח⳿⠓ִ׷〳腉䚍ָ넝ֻז׶תׅկ 勴⟝׾㢌ִג嗚稊׃ג׫גֻ׌ְׁկ ׀䋞劄ך暟⟝כ 鋅אַ׶תׇ׿ד׃׋ַ ꟗׄ׷ ҰਓҰਓʹ߹Θͤͨ ސ٬ମݧΛఏڙ WebαΠτͷ๚໰ऀͷߦಈΛ
 ސ٬͝ͱʹϦΞϧλΠϜʹղੳ $9 ސ٬ମݧ ϓϥοτϑΥʔϜ
5. 5 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. Stats of KARTE Real-time, Large-Scale, Global to real-time actions 0.x sec stored 8+ PB events / sec 105,000 inserted / month 1 PB analyzed / month 180+ PB
6. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ຊ೔ͷςʔϚ 6 ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͯ͠ ϓϨΠυ͕ߦ͍ͬͯΔ௅ઓ💪 • The Fourth Age of SREͱ͸ • No SREΛ໨ࢦͨ͠ܦҢ • No SREΛ࣮ݱ͢ΔϙΠϯτ • ·ͱΊ ࿩͢͜ͱ
7. 7 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of SREͱ͸
8. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ The Fourth Age of SREͱ͸ 8 ɹɹʛɹɹ© PLAID Inc. 97 Things Every SRE Should Know( O’Reilly) The Third Age of SRE 1⃣ The First Age SRE͸Googleಠࣗͷ΋ͷͩͬͨ 2⃣ The Second Age SREຊ͕ൃച͞Εɺόζϫʔυʹͳͬͨ 3⃣ The Third Age ઐ೚ͷSRE͕࠷దԽ͞ΕɺSRE͚ͩͰͳ͘ ͢΂ͯͷΤϯδχΞ͕SREͷ໾ׂΛ୲͏
9. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ The Fourth Age of SREͱ͸ 9 ɹɹʛɹɹ© PLAID Inc. 97 Things Every SRE Should Know( O’Reilly) The Third Age of SRE The First Age SRE͸Googleಠࣗͷ΋ͷͩͬͨ The Second Age SREຊ͕ൃച͞Εɺόζϫʔυʹͳͬͨ 3⃣ The Third Age ઐ೚ͷSRE͕ݮΓɺ͢΂ͯͷΤϯδχΞ͕ ࢓ࣄͷҰ෦ͱͯ͠SREͷ໾ׂΛ୲͏ ϓϨΠυ͸ࠓίί
10. 10 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of SREͱ͸ SREͷจԽΛ ΤϯδχΞʹ޿Ί͍ͯΔஈ֊
11. 11 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of SREͱ͸ ࠷ऴతʹ͸ɺ΋͏ҰาਐΜͩ The Fourth Age of SRE ΛΊ͍ͯ͟͠Δ🎯
12. 12 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of SREͱ͸ The Fourth Age of SRE = No SRE
13. 13 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. The Fourth Age of SREͱ͸ SREͷจԽ͕͖ࠜͮɺ SREͳ͠Ͱ΋ྑ͍΋ͷ͕ͭ͘ΕΔ࣌୅
14. 14 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ
15. 15 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ αʔϏεͱ૊৫ͷ੒௕Λ SRE͕Χόʔ͖͠Εͳ͘ͳΓͦ͏ͩͬͨ
16. 16 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • 2018೥͔Β։ൃऀ͕ٸ૿ • SRE͸2,3໊ • ࿈೔ͷো֐ରԠʹΑΔർ ฐ • ։ൃͱӡ༻͕෼཭ͨ͜͠ ͱʹΑΔαΠϩԽ No SREΛ໨ࢦͨ͠ܦҢ ΤϯδχΞ਺ͷਪҠ
17. 17 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ؤுͬͯSREΛ΋ͬͱ࠾༻͢Δʁ
18. 18 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ SREͷ࠾༻͸೉͍͠ Ծʹ࠾༻Ͱ͖ͯ΋ɺ͍͔ͭ͸ಉ͡ঢ়گ͕ى͜Δ
19. 19 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ࣾ಺ͷΤϯδχΞʹ SREͷจԽΛ޿ΊΕ͹Α͍ͷͰ͸ʁ
20. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 20 ɹɹʛɹɹ© PLAID Inc. ϓϨΠυͷ SREʹٻΊΒΕΔ͜ͱ • ϓϩμΫτͷج൫ઃܭ/ߏங • ϓϩμΫτͷ҆ఆӡ༻
21. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 21 ɹɹʛɹɹ© PLAID Inc. ϓϩμΫτͷ ج൫ઃܭ/ߏங • Ϋϥ΢υΠϯϑϥͷઃܭ/ߏங • Microservicesͷج൫੔උ • etc …
22. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 22 ɹɹʛɹɹ© PLAID Inc. ϓϩμΫτͷ ҆ఆӡ༻ • ΞϓϦέʔγϣϯͷ҆ఆੑ޲্ • ϞχλϦϯά؀ڥͷ੔උ • ো֐ରԠ • ΦϖϨʔγϣϯͷࣗಈԽ • etc …
23. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 23 ɹɹʛɹɹ© PLAID Inc. ͜ͷ2ͭΛ No SREͰ࣮ݱͰ͖ΔΑ͏ʹ • ϓϩμΫτͷج൫ઃܭ/ߏங • ϓϩμΫτͷ҆ఆӡ༻
24. 24 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020 2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team 2019: ϓϩμΫτͷج൫ઃܭ/ߏங໘Λղܾ 2020: ϓϩμΫτͷ҆ఆӡ༻໘Λվળ 2021
25. 25 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020 2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
26. 26 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • VM based MultiCloud • ϚωδϝϯταʔϏεΛੵ ۃར༻ • ηϧϑϗετͷDB΋ӡ༻ • Πϯϑϥ͸terraformԽ • ։ൃऀ͕ٸ૿ • ϞϊϦγοΫͳΞʔΩςΫ νϟ͕ݪҼͰ։ൃ͕εέʔ ϧ͠ͳ͍ ը૾͕ೖΓ·͢ 2018 No SREΛ໨ࢦͨ͠ܦҢ
27. 27 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020 2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
28. 28 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. • GKEΛ׆༻ͨ͠ϚΠΫϩ αʔϏε • ֤νʔϜ͕1ػೳΛ୲͏ • νʔϜ͝ͱʹαʔό΍ DBΛߏஙɾӡ༻Ͱ͖Δ ٕज़ج൫͸੔ͬͨ 2019 No SREΛ໨ࢦͨ͠ܦҢ
29. 29 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ 2019 2020 2021 Management services VM based MultiCloud Microservices VM based MultiCloud GKE ops team ϓϩμΫτͷج൫ઃܭ/ߏங ϓϩμΫτͷ҆ఆӡ༻ 2021
30. 30 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ MicroservicesԽ͕ͨ͠ɺ SREͷෛ୲͸ݮΒͳ͍
31. 31 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ٕज़ & ΞʔΩςΫνϟ໘͚ͩͰ͸ͳ͘ɺ ϚΠϯυηοτ໘΋ඞཁ
32. 32 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ops teamͱ͍͏ ։ൃऀͰߏ੒͞ΕͨνʔϜΛͭ͘Γ SREͷจԽͷ఻ಓࢣʹͳͬͯ΋Β͏🧙
33. 33 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ໨ࢦͨ͠ܦҢ ops teamͱ͸ʁ • ໨త: SREͷจԽΛࣾ಺ʹ޿Ίͯ΋Β͏ • ϝϯόʔ: KARTEͷશମ૾Λ೺Ѳ͍ͯ͠Δ/ͯ͠΄͠ ͍ or Πϯϑϥͷ஌ݟ͕͋ΔΤϯδχΞ10໊΄Ͳ • ΍Δ͜ͱ: ো֐ͷҰ࣍ରԠΛߦ͍ɺSREͷ஌ݟΛಘΔ
34. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 34 ɹɹʛɹɹ© PLAID Inc. ops teamͷӡ༻Ͱ ஫ҙͨ͜͠ͱ • ΦϯϘʔσΟϯά • ৼΓฦΓ • εϞʔϧελʔτ
35. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 35 ɹɹʛɹɹ© PLAID Inc. εϞʔϧελʔτ • ཧ૝͸֤νʔϜͷো֐ରԠΛ೚ͤΔ • ؀ڥͷ੔උʹ޻਺͕͔͔ΔͷͰແཧ • ؂ࢹମ੍͕͔ͬ͠Γ੔͍ͬͯΔαʔ Ϗεͷো֐ରԠ͔Β࢝ΊΔ • ௚ۙ3ϲ݄Ͱසൃ͓ͯ͠ΓɺରԠํ ๏͕͋Δఔ౓ܾ·͍ͬͯΔ͕ɺਓؒ ͷ൑அ͕ඞཁͳো֐Λ5ݸ΄Ͳબ୒
36. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 36 ɹɹʛɹɹ© PLAID Inc. ΦϯϘʔσΟϯά ‣ Πϯϑϥߏ੒ ‣ ো֐ͷӨڹൣғ ‣ Dashboardͷݟํ ‣ ໰୊ͷ੾Γ෼͚ํ ‣ ରԠͷ࢓ํ • ର৅ো֐͝ͱʹɺԼهΛυΩϡϝϯ τʹ·ͱΊΔ • ࣮ࡍʹো֐࣌ͷάϥϑΛݟͳ͕Βର Ԡํ๏Λઆ໌ • ͠͹Β͘͸ো֐ൃੜ࣌ʹSREͷ୭͔ ͕ݟकΓͭͭରԠ
37. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 37 ɹɹʛɹɹ© PLAID Inc. ৼΓฦΓ • ຖिɺͲΜͳো֐͕ଟ͔ͬͨͷ͔ɺ ୭͕ରԠͯ͘͠Ε͔ͨΛ֬ೝ • ॏେͳΠϯγσϯτʹؔͯ͠͸ɺ Github౳ʹৼΓฦΓ಺༰Λ·ͱΊΔ
38. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ໨ࢦͨ͠ܦҢ 38 ɹɹʛɹɹ© PLAID Inc. ӡ༻ͯ͠Ͳ͏͔ͩͬͨʁ • SREνʔϜ͸ղࢄ🎊 • ϓϩμΫτͷߏங/ӡ༻͸֤νʔϜʹ͓ ೚ͤ • SLO/SLI/ΤϥʔόδΣοτ౳΋ঃʑʹ ઃఆ͞Ε͖ͯͨ • ো֐ରԠ͢ΔਓʹภΓ͕ͰΔ 😄 ྑ͔ͬͨ 😥 վળ͢΂͖
39. 39 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ࣮ݱ͢ΔϙΠϯτ
40. 40 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. No SREΛ࣮ݱ͢ΔϙΠϯτ • ٕज़ & ΞʔΩςΫνϟ • ϚΠϯυηοτ ͷ͕྆࣠ඞཁ
41. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ࣮ݱ͢ΔϙΠϯτ 41 ɹɹʛɹɹ© PLAID Inc. ٕज़ & ΞʔΩςΫνϟ • MicroservicesԽ • ػೳ͝ͱʹνʔϜΛ෼͚Δ͜ͱͰɺ੹೚ൣғ ͕໌֬ʹͳͬͨ • νʔϜ͝ͱʹGKEͷઃఆ΍ඞཁͳϦιʔεͷ ߏங/ӡ༻ΛͰ͖ΔΑ͏ʹͨ͠ • ϚωδϝϯταʔϏεͷਐԽ&ੵۃར༻ • Πϯϑϥͷந৅Խ͕ਐΈɺѻ͍΍͘͢ͳͬͨ • جຊతʹϚωδϝϯταʔϏεΛ࢖͏ • terraformͰ؆୯ʹߏஙͰ͖ΔΑ͏ʹ͢Δ
42. 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ No SREΛ࣮ݱ͢ΔϙΠϯτ 42 ɹɹʛɹɹ© PLAID Inc. ϚΠϯυηοτ • ٕज़͚ͩͰ͸ແཧ • ֤νʔϜ͕ΠϯϑϥΛͦΕͧΕͰӡ༻Ͱ͖Δ ج൫͕੔͍ͬͯΔ͚ͩͰ͸ɺࣗ෼ͨͪͰ໘౗ ΛΈΔํ޲ʹ͸͢͢·ͳ͍ • ো֐ରԠΛ௨ֶͯ͠ͼΛਂΊΔ • ؂ࢹମ੍͕੔ͬͨ؀ڥͰɺো֐ରԠΛ࣮ફ͠ ͯ΋Β͏͜ͱͰϕετϓϥΫςΟεֶ͕΂Δ • ࣗ෼ͰखΛಈ͔͢ͷͰɺࣗνʔϜͰ͸Ͳ͏׆ ͔ͤΔ͔Λߟ͑Δ͖͔͚ͬʹͳΔ • SREͷจԽΛ࣮ફ͠ɺ఻͍͑ͯ͘ਓ͕ඞཁ
43. 43 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ
44. 44 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ SREͷจԽΛ ΤϯδχΞʹ޿Ί͍ͯΔஈ֊
45. 45 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ SREͷจԽ͕͖ࠜͮɺ SREͳ͠Ͱ΋ྑ͍΋ͷ͕ͭ͘ΕΔঢ়ଶ Λ໨ࢦ͍ͯ͠Δ🎯
46. 46 2021.07ɹɹʛɹɹCODT 2021ɹɹʛɹ ɹɹʛɹɹ© PLAID Inc. ·ͱΊ • ٕज़ & ΞʔΩςΫνϟ • ϚΠϯυηοτ ͷ͕྆࣠ඞཁ