Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Who owns the Service Level?
Search
Takeshi Kondo
May 15, 2022
Technology
12k
5
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Who owns the Service Level?
SRE NEXT 2022
https://sre-next.dev/2022/
Takeshi Kondo
May 15, 2022
More Decks by Takeshi Kondo
See All by Takeshi Kondo
SREの知識地図 - 第2章の紹介 - / Knowledge Map of SRE – Introduction to Chapter 2 –
chaspy
0
120
SRE NEXT CfP チームが語る 聞きたくなるプロポーザルとは / Proposals by the SRE NEXT CfP Team that are sure to be accepted
chaspy
2
1.9k
Slack Platform(Deno) での RAG 実装 - LangChain(js) を使ってみた / rag-implementation-on-slack-platform-deno-experimenting-with-langchain-js
chaspy
0
340
SRE の考えをマネジメントに活かす / applying SRE ideas to management
chaspy
7
8.5k
RAGの簡易評価によるフィードバックサイクル実践 / Feedback cycle practice through simplified assessment of RAGs
chaspy
2
6.2k
定量データと定性評価を用いた技術戦略の組織的実践 / Systematic implementation of technology strategies using quantitative data and qualitative evaluation
chaspy
9
2.4k
エンジニアブランディングチームの KPI / KPI's of engineer branding team
chaspy
2
2.6k
「SLO Review」今やるならこうする / If I had to do the "SLO Review" again
chaspy
3
2.5k
開発者とともに作る Site Reliability Engineering / SREing with Developers
chaspy
10
9.3k
Other Decks in Technology
See All in Technology
「私が一番顧客に詳しい」を捨てる~ 全員が一次情報に精通する開発チームをつくり、PMが次のボトルネックへ染み出すまで
kaminashi
3
1.2k
From Developer Experience to Agentic Experience: The Next Evolution of APIs
benjagm
0
100
AI コーディングと人間の認知スループット / CTO Night & Day 2026 Morning Session / AI Coding and Human Cognitive Throughput
prog893
5
1.5k
アカウント統制の健康診断
kenichinakamura
1
100
DBについての基礎説明(MySQL編) - コロプラ 2026年度 新卒研修
colopl
0
160
Active Record Connection Adaptersにおける長年の課題に再び取り組む
yahonda
0
410
DGX Spark でどこまでハッキングができるのか?
kash1064
3
1.4k
[JAWS-UG彩の国埼玉#12]そろそろ準備する?re:Invent2026をやさしく解説 / reinvent2026-sainokuni
sh_fk2
1
380
AWSDirectConnectの構築の進め方を整理してみた
masa_my
1
120
AI導入は済んだ。次は統制を作り替える番だ_AI×開発組織 Summit 2026
techtekt
PRO
0
500
Temporal 最高!でも改めて「日付」について調べたら人類の負債すぎる件
riyaamemiya
0
220
Persisting Massive Data with Rails ~Designing for 60 billion rows, and fighting the bottlenecks~ Railsで挑む大量データ永続化 〜600億件を永続化するための設計とボトルネックとの戦い〜
occhi
0
770
Featured
See All Featured
Code Reviewing Like a Champion
maltzj
529
40k
Code Review Best Practice
trishagee
74
20k
Side Projects
sachag
456
43k
Large-scale JavaScript Application Architecture
addyosmani
516
110k
Visualization
eitanlees
153
17k
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
33
5.3k
Gemini Prompt Engineering: Practical Techniques for Tangible AI Outcomes
mfonobong
2
590
Discover your Explorer Soul
emna__ayadi
2
1.3k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
360
31k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
133
20k
YesSQL, Process and Tooling at Scale
rocio
174
15k
Agile Leadership in an Agile Organization
kimpetersen
PRO
0
290
Transcript
Who owns the Service Level? Takeshi Kondo / @chaspy 2022/05/15
SRE NEXT 2022
Who am I chaspy chaspy_ Engineering Manager, Site Reliability at
Recruit Co., Ltd. Takeshi Kondo https://chaspy.me
લఏɿϓϩμΫτհ - ελσΟα ϓϦ
Hello, again! SRE NEXT
ࠓ͢͜ͱ SRE Λ࣮ݱ͢ΔͨΊʹඞཁͳ͜ͱ
Tl;dr / SRE Λ࣮ݱ͢ΔͨΊʹඞཁͳ͜ͱ • ࢦඪΛݩʹվળ͠ଓ͚ΔจԽ • ࣗݾ݁ԽΛࢧ͑Δ Platform •
༏ઌॱҐΛมߋ͢ΔͨΊͷٕज़ઓུ
Agenda • લఏɿSRE Λ࣮ݱ͢ΔͱͲ͏͍͏͜ͱ͔ • ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱࣄۀʹ͓͚Δ SRE • 2020ʮSLO ReviewʯͰͷֶͼͱࣦഊ
• SRE ͱٕज़ઓུ • ·ͱΊͱࠓޙ
Agenda • લఏɿSRE Λ࣮ݱ͢ΔͱͲ͏͍͏͜ͱ͔ • ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱࣄۀʹ͓͚Δ SRE • 2020ʮSLO ReviewʯͰͷֶͼͱࣦഊ
• SRE ͱٕज़ઓུ • ·ͱΊͱࠓޙ
લఏ෦3݄ͷ6ࣾ߹ಉษڧձͱಉ͡༰Ͱ͓͞Β͍͠·͢
SRE Λ࣮ݱ͢Δͱ
։ൃνʔϜ͕৴པੑΛ ίϯτϩʔϧ͢Δ Capability Λ ʹ͚͍ͭͯΔ͜ͱ
ͦͦ Site Reliability Engineering ͱ: Not like this • αʔϏε͕ʮߴ͍৴པੑ
(ʹ100%)ʯΛอ͍ͬͯΔ͜ͱ • SLI/SLO ΛकΕ͍ͯΔ͜ͱ • ΦϯίʔϧϩʔςʔγϣϯΛ։ൃνʔϜͰߦ͏͜ͱ https://github.com/twitter/twemoji
ͦͦ Site Reliability Engineering ͱ: Like this! • αʔϏε͕ʮϢʔβ͕ظ͢Δ৴པੑʯΛอ͍ͬͯΔ͜ͱ •
SLI/SLO Λઃఆ͠ɺඇػೳཁ݅ͱػೳཁ݅ͷ༏ઌܾఆͷ ࢦඪͱͯ͠׆༻͍ͯ͠Δ • SLO ҧ͕ൃੜͨ͠ͱ͖ʹదʹରॲͰ͖ΔΑ͏ͳϞχλ Ϧϯάํ๏ͱϙϦγʔ͕νʔϜͰಉҙ͞Ε͍ͯΔ • ্ه͕ఆظతʹݟ͞Ε͍ͯΔ https://github.com/twitter/twemoji
։ൃνʔϜ͕৴པੑΛίϯτϩʔϧ͢Δ Capability Λʹ͚ͭΔ: Like this! SRE ։ൃ νʔϜ ։ൃνʔϜͷ৴པੑʹ ؔ͢Δ
Capability औಘ Λࢧԉ͢Δ ࣗͨͪͷαʔϏεͷ ৴པੑΛࣗͨͪͰί ϯτϩʔϧͰ͖͍ͯΔ
Team Topologies • 4ͭͷνʔϜύλʔϯ • Stream Aligned • Platform •
Enabling • Complicated Subsystem • 3ͭͷίϛϡχέʔγϣϯύλʔϯ • Collaboration • X as a Service • Facilitation https://pub.jmam.co.jp/book/b593881.html
։ൃνʔϜ͕৴པੑΛίϯτϩʔϧ͢Δ Capability Λʹ͚ͭΔ: Like this! SRE ։ൃ νʔϜ ։ൃνʔϜͷࣗݾ݁ԽΛ ࢧ͑ΔϓϥοτϑΥʔϜͱ
จԽΛ࡞Δ Platform Team Enabling Team Stream Aligned Team ࣗͨͪͰඞཁͳͷΛ ࣗͨͪͰ༻ҙͰ͖Δ = self-contained / ࣗݾ݁Խ
• ࢧԉҰ࣌తͰɺظԽͤͯ͞ͳΒͳ͍ ༨ஊ: Enabling ͱ͍͏୯ޠʹϙδςΟϒɾωΨςΟϒ྆໘͋Δ https://ja.wikipedia.org/wiki/%E3%82%A4%E3%83%8D%E3%83%BC%E3%83%96%E3%83%AA%E3%83%B3%E3%82%B0
SRE Team ͷ Vision / Mission / Values https://blog.studysapuri.jp/entry/sre-vision-mission-values
Mission ࣗݾ݁νʔϜ͕ϓϩμΫ τΛૉૣ҆͘શʹಧ͚ଓ͚ ΔͨΊͷϓϥοτϑΥʔϜ ͱจԽΛ࡞Δ
ͳͥࣗݾ݁Խ͕ॏཁ͔ SRE ։ൃ νʔϜ ։ൃνʔϜͷࣗݾ݁ԽΛ ࢧ͑ΔϓϥοτϑΥʔϜͱ จԽΛ࡞Δ Platform Team Enabling
Team Stream Aligned Team ࣗͨͪͰඞཁͳͷΛ ࣗͨͪͰ༻ҙͰ͖Δ = self-contained / ࣗݾ݁Խ
ͳͥࣗݾ݁Խ͕ॏཁ͔: Not “VS”, but “And” • Dev vs and Ops
• Ϣʔβ͔ΒߴʹϑΟʔυόοΫΛಘΔ (DevOps) • Dev vs and Infrastructure • ηϧϑαʔϏεͰߏஙͯ͠ϦʔυλΠϜॖ • Productivity vs and Reliability • ੜ࢈ੑͱ৴པੑ૬ޓʹґଘ͢Δ
ͳͥࣗݾ݁Խ͕ॏཁ͔ • ࣗݾ݁νʔϜͳΒ • ຊ൪ڥ͔ΒϑΟʔυόοΫΛಘଓ͚ΒΕΔ • ࠷খݶͷίϛϡχέʔγϣϯͰߴͳҙࢥܾఆ͕Ͱ͖Δ • ػೳཁٻ͚ͩͰͳ͘ඇػೳཁٻʹԠ͑Δ͜ͱ͕Ͱ͖Δ
• ։ൃνʔϜ͕৴པੑΛίϯτϩʔϧ͢Δ Capability Λʹ͚ͭͯ ͍Δ͜ͱ • ։ൃνʔϜ͕”ࣗݾ݁Խ”͍ͯ͠Δঢ়ଶ • SRE νʔϜ͜ΕΛϓϥοτϑΥʔϜͱจԽৢͰࢧ͑Δ
• ͜ΕΛ࣮ݱ͢ΔʹϓϩμΫτ։ൃʹด͡ͳ͍ଟ༷ͳࢹ͕ඞཁ • ϢʔβͷظΛΔ / Product Management • ߴ͍։ൃੜ࢈ੑ / Development Skills • ඇػೳཁٻʹͲΕ͚ͩίετΛ͔͚Δ͔ / Business Development ·ͱΊɿSRE Λ࣮ݱ͢Δͱ
Agenda • લఏɿSRE Λ࣮ݱ͢ΔͱͲ͏͍͏͜ͱ͔ • ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱࣄۀʹ͓͚Δ SRE • 2020ʮSLO ReviewʯͰͷֶͼͱࣦഊ
• SRE ͱٕज़ઓུ • ·ͱΊͱࠓޙ
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE ~ @chaspy ೖࣾޙ • 2018: @chaspy ೖࣾ • 2019:
Application Platform Λ Kubernetes Ҡ • 2020: Microservices Readiness ͷඋ • αʔϏεΦʔφʔγοϓͷࡦఆ • Design Doc / Production Readiness Checklist • Self-services Infrastructure (terraform monorepo) • SLI/SLO • 2021: SLI/SLO ӡ༻Λ։ൃνʔϜʹશҠৡ Platform Team ͱͯ͠ Platform Λ࡞͍ͬͯΔ Enabling Team ͱͯ͠ ։ൃ৫ʹ SLI/SLO ͳͲͷΧϧνϟʔৢ
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE ~ 2021 • COVID-19 ྲྀߦɺΞΫηε૿େ • Platform ͷਐԽ •
Terraform monorepo • Loadtest Platform • GitHub Actions ʹΑΔ monorepo CI • ৫ͷมԽ • ٕज़ઓུάϧʔϓൃ • ࣄۀҠʹΑΓϦΫϧʔτస੶ɺQuipper ຊࢧళਫ਼ࢉ • chaspy EM ༻
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE ~ 2022 • ৽͍͠ SRE ࣮ફͷܗ • Partially Embedded
SRE • Partially Enabling SRE in development team
৫نͷਪҠ ։ൃऀ 35 53 54
73 114 43& 4 5 7 7 7 ։ൃऀελσΟαϓϦɾQuipper ྆ํͷɺWeb Engineer (frontend&backend) ͷɻNative আ֎͍ͯ͠Δɻ 2022͔ΒۀҕୗͷํΧϯτ͍ͯ͠Δɻ2021Ҏલۀҕୗͷํͱࣄ͍ͯͨ͠ɻ
ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱSRE • ͍ͣΕͷ࣌ Platform Team ͱ Enabling Team ͷৼΔ ͍Λ͍ͯ͠Δ
• ಛʹ2019͔Βʮࣗݾ݁ԽʯΛςʔϚʹɺ͓ئ͍͞Ε Δ͜ͱΛۃྗݮΒͤΔ Platform Λ࡞͖ͬͯͨ • ಉ࣌ʹ։ൃνʔϜͷʮจԽΛͭ͘Δʯ͜ͱʹ౿ΈࠐΈɺSLI/ SLO Λݟ͍ͯ͘จԽΛ৫ʹৢͨ͠ • →ʮSLO Reviewʯat SRE NEXT 2020
Agenda • લఏɿSRE Λ࣮ݱ͢ΔͱͲ͏͍͏͜ͱ͔ • ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱࣄۀʹ͓͚Δ SRE • 2020ʮSLO ReviewʯͰͷֶͼͱࣦഊ
• SRE ͱٕज़ઓུ • ·ͱΊͱࠓޙ
ʮSLO Reviewʯat SRE NEXT 2020 • ։ൃ৫ʹ SLO Λ Review
͍ͯ͘͠จԽΛ࡞ͬͨऔΓΈ • 2 Product, 15 Team ʹϘτϜΞοϓͰಋೖ • ొΓํͷ4εςοϓ • γεςϜͱ৫ͷΦʔφʔγοϓΛܾΊΔ • 1ਓͰϓϩηεΛ·Θ͠ɺ࣮ݱํ๏Λཱ֬͢Δ • Developer ͱҰॹʹ SLI/SLO Λఆٛ͠ɺϓϩηεΛ·Θ͢ • Error Budget Policy Λఆٛͯ͠ߦಈ͢Δ(ະ࣮ݱ) • ಘֶͨͼ • ඪ४Խ͞Εͨ SLI Λఏڙ͢Δ • ઃఆૣ͍ஈ֊ͰίʔυԽ͢Δ • ֶशۂઢΛٸޯʹ͢Δ https://blog.studysapuri.jp/entry/2020/01/30/slo-review
Α͔ͬͨ • ։ൃ৫ʹ SLO Λ Review ͍ͯ͘͠จԽΛ࡞ͬͨऔΓΈ • 2 Product,
15 Team ʹϘτϜΞοϓͰಋೖ • ొΓํͷεςοϓ • γεςϜͱ৫ͷΦʔφʔγοϓΛܾΊΔ • 1ਓͰϓϩηεΛ·Θ͠ɺ࣮ݱํ๏Λཱ֬͢Δ • Developer ͱҰॹʹ SLI/SLO Λఆٛ͠ɺϓϩηεΛ·Θ͢ • Error Budget Policy Λఆٛͯ͠ߦಈ͢Δ(ະ࣮ݱ) • ಘֶͨͼ • ඪ४Խ͞Εͨ SLI Λఏڙ͢Δ • ઃఆૣ͍ஈ֊ͰίʔυԽ͢Δ • ֶशۂઢΛٸޯʹ͢Δ https://blog.studysapuri.jp/entry/2020/01/30/slo-review ։ൃνʔϜͷೝෛՙΛపఈత ʹԼ͛Δ͜ͱʹͩ͜Θͬͨ తෆ࣮֬ੑͷݮͷͨΊ ϑΟʔυόοΫαΠΫϧΛճͨ͠
Α͘ͳ͔ͬͨʁ • ։ൃ৫ʹ SLO Λ Review ͍ͯ͘͠จԽΛ࡞ͬͨऔΓΈ • 2 Product,
15 Team ʹϘτϜΞοϓͰಋೖ • ొΓํͷεςοϓ • γεςϜͱ৫ͷΦʔφʔγοϓΛܾΊΔ • 1ਓͰϓϩηεΛ·Θ͠ɺ࣮ݱํ๏Λཱ֬͢Δ • Developer ͱҰॹʹ SLI/SLO Λఆٛ͠ɺϓϩηεΛ·Θ͢ • Error Budget Policy Λఆٛͯ͠ߦಈ͢Δ(ະ࣮ݱ) • ಘֶͨͼ • ඪ४Խ͞Εͨ SLI Λఏڙ͢Δ • ઃఆૣ͍ஈ֊ͰίʔυԽ͢Δ • ֶशۂઢΛٸޯʹ͢Δ https://blog.studysapuri.jp/entry/2020/01/30/slo-review ͳͥ͏·͍͔͘ͳ ͔ͬͨͷ͔ʁ
ʢ࠶ܝʣSRE Λ࣮ݱ͢ΔͱԿ͔ • ։ൃνʔϜ͕৴པੑΛίϯτϩʔϧ͢Δ Capability Λʹͭ ͚͍ͯΔ͜ͱ • ʮࢦඪΛݩʹߦಈ͕Ͱ͖Δʯ͜ͱ https://blog.studysapuri.jp/entry/2020/01/30/slo-review
4-*4-0 &SSPS#VEHFU 1PMJDZ ։ൃνʔϜ ఆٛ͢Δ / ఆظతʹݟ͢ Policy ʹैͬͯߦಈ͢Δ
ʢ࠶ܝʣSRE Λ࣮ݱ͢ΔͱԿ͔ • ։ൃνʔϜ͕৴པੑΛίϯτϩʔϧ͢Δ Capability Λʹͭ ͚͍ͯΔ͜ͱ • ʮࢦඪΛݩʹߦಈ͕Ͱ͖Δʯ͜ͱ https://blog.studysapuri.jp/entry/2020/01/30/slo-review
4-*4-0 &SSPS#VEHFU 1PMJDZ ։ൃνʔϜ ఆٛ͢Δ / ఆظతʹݟ͢ Policy ʹैͬͯߦಈ͢Δ Ͱ͖ͨ
ʢ࠶ܝʣSRE Λ࣮ݱ͢ΔͱԿ͔ • ։ൃνʔϜ͕৴པੑΛίϯτϩʔϧ͢Δ Capability Λʹͭ ͚͍ͯΔ͜ͱ • ʮࢦඪΛݩʹߦಈ͕Ͱ͖Δʯ͜ͱ https://blog.studysapuri.jp/entry/2020/01/30/slo-review
4-*4-0 &SSPS#VEHFU 1PMJDZ ։ൃνʔϜ ఆٛ͢Δ / ఆظతʹݟ͢ Policy ʹैͬͯߦಈ͢Δ ͏·͍͔͘ͳ͔ͬͨ
ͳͥ”ߦಈ͢Δ”·ͰࢸΒͳ͔ͬͨͷ͔ • ࣌ɺSLO ҧ࣌ͷΞΫγϣϯ Product Manager / Team ʹҠৡ͍ͯͨ͠ •
·ͬͨ͘ԿͰ͖ͳ͔ͬͨΘ͚Ͱͳ͍ • ͱͱνʔϜʹ༧ࢉͷ͋ΔɺվળͷͨΊͷ࣌ؒͰͰ͖Δ͜ͱ͔͠Ͱ͖ͳ͔ͬͨʢִि1ʣ • QB Day ͱݺΕΔ • Τϥʔʹର͢Δతͳରॲɺܰඍͳ Performance վળͳͲ • ΞʔΩςΫνϟมߋɺΠϯϑϥͳͲɺظతɾࠜຊతରॲ͔ͬͨ͠ • ʮࢦඪΛݩʹػೳཁٻͱඇػೳཁٻͷ༏ઌஅ͕Ͱ͖Δʯ·Ͱ౸ୡ͠ͳ͔ͬͨ • ༏ઌஅʹʹཱͨͳ͍ͷͰ͋Εɺ։ൃνʔϜʹͱͬͯΔ͜ͱ͕૿͚͑ͨͩͱݴ͑Δ
ͳͥ”ߦಈ͢Δ”·ͰࢸΒͳ͔ͬͨͷ͔ • ৴པੑͷ(ඇػೳཁٻ)ʹରॲ͢Δݖݶɾ༧ࢉෆ • ػೳ։ൃͷ༏ઌ͕ߴ͔ͬͨ
SLO ҧͨ͠Β“ϦϦʔεετοϓ”ͱ͍͏ݬ • SLO ҧͨ͠Βશͯͷ։ൃΛࢭΊͯͦͷमਖ਼Λߦ͏ͷݱ࣮ తͰͳ͍ • ͦΕΛߦ͏ͨΊʹࣄۀऀ͕͜ͷӡ༻ʹ߹ҙ͢Δඞཁ͕͋Δ • ͦΕΛߦ͏ͨΊʹ”ᘳͳSLI/SLO”͕ඞཁ͕ͩɺ”ᘳͳSLI/SLO”
ଘࡏ͠ͳ͍ • ৫͕֦େ͠ɺBiz Dev / Product / CS ͱΛ୲͍ͯ͠Δঢ়گͰɺ ݱஅͰ͜ΕΛΔͷίϛϡχέʔγϣϯίετ͕ߴ͗͢Δ
SLI/SLO ӡ༻ͷཧ: ͱ͋Δ Developer ͷίϝϯτ
ʮSLO Reviewʯat SRE NEXT 2020 ͦͷޙͷ·ͱΊ • ʮ৴པੑࢦඪΛఆΊɺ؍͢ΔʯจԽΛ࡞ͬͨ͜ͱʹՁ͕͋ͬͨ • ࣄۀઓ্ུͷҙࢥܾఆʹʹཱͭࢦඪʹҭͭ·ͰʹࢸΒͳ͔ͬͨ
• ཧ༝1. ඇػೳཁٻͱػೳཁٻͷόϥϯεΛม͑Δҙࢥܾఆݖݶɾ༧ࢉ͕ϓϩμΫτ ։ൃνʔϜʹͳ͔ͬͨ • ৽نػೳ։ൃͷΠϯηϯςΟϒ͕େ͖͍ঢ়گ • ͦͷΑ͏ͳٕज़ઓུ/ٕज़ࢿΛϓϩμΫτ All Ͱߦ͑ΔΈ͕ͳ͔ͬͨ • ཧ༝2. ৴པੑࢦඪ͕ Biz/Dev/SRE શһ͕ཧղ͍͢͠ࢦඪͰͳ͔ͬͨ • backend API ͷ SLI ϢʔβମݧΛද͓ͯ͠ΒͣɺLatency ʹؔ͢Δରॲ TPM ͷઆ໌͍͠
Agenda • લఏɿSRE Λ࣮ݱ͢ΔͱͲ͏͍͏͜ͱ͔ • ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱࣄۀʹ͓͚Δ SRE • 2020ʮSLO ReviewʯͰͷֶͼͱࣦഊ
• SRE ͱٕज़ઓུ • ·ͱΊͱࠓޙ
2021 ελσΟαϓϦ খதߴϓϩμΫτ։ൃ෦ ٕज़ઓུάϧʔϓൃ
ελσΟαϓϦখֶɾதֶɾߴߍɾେֶडݧߨ࠲ ελσΟαϓϦ For TEACHERS ελσΟαϓϦ For SCHOOL ݱঢ়ͷ৫ਤ: খதߴϓϩμΫτ։ൃ෦ ҎԼ17άϧʔϓ
TPM BtoB TPM BtoC TPM ForSCHOOL TPM ԣஅ BtoC BtoB QA ։ൃࢧԉ SRE ٕज़ઓུ ίʔνϯά ৽ن։ൃ1 Τϯϋϯε ֶशࢧԉ Native iOS Android ৽ن։ൃ2 ਐ࿏ओମੑ ίϛϡχέʔγϣϯࢧԉ ForSCHOOLϞόΠϧ
Disclaimer • ٕज़ઓུάϧʔϓͷ্ཱͪ͛લϚωʔδϟ͕ߦͬͨͷ • ࡢ࣌Ͱ @chaspy DevOps WG ͷ
Lead -> EM/Lead • લͷୀ৬ʹ͍ٕज़ઓུάϧʔϓͷ EM ෦͕݉ͭ͠ ͭɺଞ໊ͷ EM ͱҰॹʹӡӦ͍ͯ͠Δ • SLO ҧͷରॲ͕Ͱ͖ͳ͍͜ͱ͕ཧ༝Ͱ্ཱ͕ͪͬͨΘ͚Ͱ ͳ͍
ٕज़ઓུͱ ϓϩμΫτ։ൃ৫ʹ͓͍ͯ ԿΛΔ͔ɺԿΛΒͳ͍͔
ͳٕͥज़ઓུ͕ඞཁ͔ • Ϗδωεͷʹਵ͢ΔͨΊ • มߋΛ͍͢͠γεςϜɾίʔυɾ৫ʹ͢Δ • มߋΛ͛ΔཁҼઌճΓͯ͠ରॲ͢Δ(ٕज़తෛ࠴ͷฦࡁ) • ͦͷঢ়ଶΛظతʹҡ࣋Ͱ͖ΔೳྗΛಘΔ
ͳٕͥज़ઓུ”άϧʔϓ”͕ඞཁ͔ • ٕज़ઓུͷܾΊํ৫ʹΑͬͯҟͳΔ • 1ਓͷ CTO ͕τοϓμϯͰܾΊ͍͍ͯ • ϘτϜΞοϓͰશһ߹ٞͰܾΊ͍͍ͯ •
ͦͷதؒͰ͍͍ • ελσΟαϓϦখதߴ։ൃ৫ٕज़ઓུΛ1ਓʹґଘ͠ͳ͍ΈΛ ࡞Δ͜ͱʹઓ͍ͯ͠Δ
ٕज़ઓུάϧʔϓͱ • త • ϓϩμΫτ։ൃ৫ͱͦͷγεςϜΛΑΓมԽʹڧ͘͢Δ • ඪ • ٕज़తͳϏδϣϯͱํͷࡦఆ •
ٕज़త՝ɾෛ࠴ΛίϯτϩʔϧԼʹஔ͘ • վળαΠΫϧͷཱ֬ͱࣗݾஅೳྗͷ֫ಘ
࠷ॳͷ1Ͱͬͨ͜ͱ • ৽ن։ൃɿΤϯϋϯεɿٕज़తෛ࠴ղফ = 1:1:1 ͷ༧ࢉએݴ • ٕज़తෛ࠴ղফҊ݅ΛܾΊͯ։ൃϩʔυϚοϓʹΈࠐΉ • ٕज़՝ͷཧํ๏ͱൃੜ࣌ͷτϦΞʔδํ๏ͷܾఆ
SLO ҧ࣌ͷΞΫγϣϯ͕औΕΔΑ͏ʹΈ͕Ͱ͖ͨ • Ұఆඇػೳཁٻʹ༧ࢉ͕ͯΒΕɺൃੜٕͨ͠ज़՝Λཧ ͠ɺ࣮ߦͰ͖Δମ੍͕ͬͨ • SLO ҧͨ͠ࡍʹ࣮ࢪ͢Δ͜ͱ͕͔ٕͬͨ͠ज़՝Λܭը ʹΈࠐΊΔΑ͏ʹͳͬͨ
׆ಈମ • త • ϓϩμΫτ։ൃ৫ͱͦͷγεςϜΛΑΓมԽʹڧ͘͢Δ • ඪ • ٕज़తͳϏδϣϯͱํͷࡦఆ •
ٕज़త՝ɾෛ࠴ΛίϯτϩʔϧԼʹஔ͘ • վળαΠΫϧͷཱ֬ͱࣗݾஅೳྗͷ֫ಘ DevOps WG ԣஅWG Backend WG Frontend WG
׆ಈମ • త • ϓϩμΫτ։ൃ৫ͱͦͷγεςϜΛΑΓมԽʹڧ͘͢Δ • ඪ • ٕज़తͳϏδϣϯͱํͷࡦఆ •
ٕज़త՝ɾෛ࠴ΛίϯτϩʔϧԼʹஔ͘ • վળαΠΫϧͷཱ֬ͱࣗݾஅೳྗͷ֫ಘ DevOps WG ԣஅWG Backend WG Frontend WG
ٕज़త՝ɾෛ࠴ΛίϯτϩʔϧԼʹஔ͘ • ٕज़՝Λ͛ࠐΊΔॴɾΈΛ࡞Δ • ٕज़՝ͷ༏ઌΛ͚ͭΔ • དྷظͷ։ൃϩʔυϚοϓʹଓ͢Δ
ٕज़՝Λ͛ࠐΊΔॴɾΈΛ࡞Δ ୭Ͱٕज़՝Λ͛ࠐΊΔ spreadsheet reacji Ͱؾܰʹ͛ࠐΊΔ
͛ࠐ·Εͨ՝ͷ re fi nement ͭΒ͞ = ϖΠϯΛݴޠԽ ݱ࣌Ͱු͔Ϳ How Λهࡌ
(༏ઌ͕͚ͭΒΕΕ͍͍ͷͰɺ͜ͷཝ optional)
՝ͷ༏ઌॱҐ͚ ϖΠϯͷසͱڧͰϚοϐϯά ٕज़՝ͷ૬ରධՁΛ͚ͭΔ
དྷظͷ։ൃϩʔυϚοϓʹଓ͢Δ ϖΠϯͷԽ ։ൃϩʔυϚοϓଓঢ়گ
ͪΖΜɺᘳͰͳ͍ • ٕज़՝සͱڧͰଌΕΔͷͰͳ͍ • ఆੑతͰ͋Δ • ࢀՃϝϯόʔͷภΓ͕͋Δ͔ • ෳ member
ͷ vote ݁Ռͷॏ৺ʹஔ͍͍ͯΔͷͰਫ਼ʹٙ • ։ൃϦιʔεɺٕज़తқɺϦεΫʹΑ͙ͬͯ͢ʹऔΓ͔͔Εͳ ͍՝͋Δ • ՝ͷ༏ઌ͚ʹ͕͔͔࣌ؒΔ • etc…
͔͠͠ɺಛఆͷਓʹґଘ͠ͳ͍ελʔτϥΠϯʹཱͬͨ • ୭Ͱٕज़՝Λද໌͢Δ͜ͱ͕Ͱ͖Δ • ද໌ͨ͠՝ඞͣऔΓѻΘΕΔ • શһͰʹ͖߹͑Δ - vs
ࢲͨͪ • ظతͳ৴པੑͷ Controlability ୲อʹཱͭ
׆ಈମ • త • ϓϩμΫτ։ൃ৫ͱͦͷγεςϜΛΑΓมԽʹڧ͘͢Δ • ඪ • ٕज़తͳϏδϣϯͱํͷࡦఆ •
ٕज़త՝ɾෛ࠴ΛίϯτϩʔϧԼʹஔ͘ • վળαΠΫϧͷཱ֬ͱࣗݾஅೳྗͷ֫ಘ DevOps WG ԣஅWG Backend WG Frontend WG
DevOps WG ͷతͱ׆ಈ • తɿʢ։ൃνʔϜͷʣࣗݾஅೳྗͷ֫ಘͷͨΊʹઃஔ • ϝϯόʔ ྖҬ͝ͱͷ WebDev /
QA / SRE • ׆ಈ༰ • όϦϡʔετϦʔϜϚοϐϯάͷ࣮ࢪ • ީิͱͳΔ Metrics / Indicator ͷચ͍ग़͠ͱܭଌ • DX Criteria ͷ࣮ࢪ • όϦϡʔετϦʔϜΛ્͢ΔཁҼͷղܾ(e.g. E2E Automation) • ϓϩμΫτ։ൃ෦֎ͷใ׆ಈ • ·ͣ༗ޮͦ͏ͳ metrics ΞηεϝϯτΛݕূͨ͠
όϦϡʔετϦʔϜϚοϐϯά Meta Issue (Epic) ࡞~ ։ൃ
όϦϡʔετϦʔϜϚοϐϯά ։ൃྃ ~ εϓϦϯτϨϏϡʔ QA ~ ຊ൪ϦϦʔε
ީิͱͳΔ Metrics / Indicator ͷચ͍ग़͠ͱܭଌ όϦϡʔετϦʔϜϚοϐ ϯάΛΠϯϓοτʹɺվ ળͨ͠߹ͷϝϦοτͷ ԾઆɺͦΕΛ્ͯ͠ ͍ΔΛϐοΫΞοϓ
Q https://blog.studysapuri.jp/entry/2020/08/17/dx-criteria-system
ϓϩμΫτ։ൃ෦֎Ͱͷใ׆ಈ: BtoC All Hands Ͱͷൃද https://blog.studysapuri.jp/entry/2020/08/17/dx-criteria-system • ॴଐάϧʔϓΛ͑ͨࣄۀঢ়گΛΔ • Ϛʔέοτχϡʔε
• ࣄۀঢ়گ • ϓϩμΫτ KPI • SLI / ։ൃऀੜ࢈ੑ • ͦͷଞτϐοΫ͞·͟· • SRE ͱͳʹʁ • ϚΠΫϩαʔϏεͬͯͳʹʁ͏Ε͍͠ͷʁ
ϓϩμΫτ։ൃ෦֎Ͱͷใ׆ಈ: BtoC All Hands Ͱͷൃද • “͡Ίͯͷ SRE” ͱͯ͠։ൃ෦Ҏ֎ͷํ͚ʹൃද
ϓϩμΫτ։ൃ෦֎Ͱͷใ׆ಈ: BtoC All Hands Ͱͷൃද • “͡Ίͯͷ SRE” ͱͯ͠։ൃ෦Ҏ֎ͷํ͚ʹൃද
DevOps WG ͷతͱ׆ಈ • తɿʢ։ൃνʔϜͷʣࣗݾஅೳྗͷ֫ಘͷͨΊʹઃஔ • ʢ։ൃνʔϜ͕ʣࢦඪΛݩʹߦಈ͢ΔจԽΛ࡞Ζ͏ͱ͍ͯ͠Δ • SLI/SLO Λݟͯ৴པੑΛ؍͍ͯ͘͜͠ͱͱಉ͡Ͱ…?
SRE ͱٕज़ઓུ • DevOps WG ͷ׆ಈ ʮSRE ͷ࣮ݱʯͷจԽ໘Ͱͷ֦ு • զʑ͕ݟΔ͖ࢦඪγεςϜͷ৴པੑࢦඪ͚ͩͰͳ͍
• ͋ΒΏΔͷΛࢦඪΛݟͯɺҙࢥܾఆ͢Δ • ࠓޙ͜ͷจԽৢͦͷͷͷվળαΠΫϧΛճ͢ • 1. ީิͱͳΔ metrics ͷ༗ޮੑ͕໌Β͔ʹͳΓɺԽ͢Δ • 2. ։ൃνʔϜ͕ͦΕΛݟͯɺΞΫγϣϯΛߟ͑Δ͜ͱ͕Ͱ͖Δ • 3. ։ൃνʔϜ͕ΞΫγϣϯ->վળͷαΠΫϧΛճ͢ • 4. 1-3 ͦΕࣗମ͕͏·͍͍ͬͯ͘Δ͔ΛධՁ͢Δ
SRE ͱٕज़ઓུ: ·ͱΊ • SRE Λ࣮ݱ͢ΔͨΊʹɺSLO ҧΛͨ࣌͠ʹߦಈͰ͖Δ༧ࢉͱݖݶ ͕ඞཁ • ͦͷ্Ͱɺٕज़՝Λղܾ͢Δ༏ઌॱҐΛ͚ͭΒΕΔٕज़ઓུ͕ඞཁ
• ʰελσΟαϓϦʱখதߴϓϩμΫτ։ൃ෦Ͱ͜ͷٕज़ઓུΛ1ਓ ʹґଘͤͣɺάϧʔϓͰ࣮ݱ͢Δ͜ͱʹઓ͍ͯ͠Δ • ͋ΒΏΔͷΛࢦඪͰݟ͍ͯ͘จԽ͕৴པੑͷͨΊʹॏཁ
Agenda • લఏɿSRE Λ࣮ݱ͢ΔͱͲ͏͍͏͜ͱ͔ • ྺ࢙͔ΒৼΓฦΔʰελσΟαϓϦʱࣄۀʹ͓͚Δ SRE • 2020ʮSLO ReviewʯͰͷֶͼͱࣦഊ
• SRE ͱٕज़ઓུ • ·ͱΊͱࠓޙ
·ͱΊ / SRE Λ࣮ݱ͢ΔͨΊʹඞཁͳ͜ͱ • ࢦඪΛݩʹվળ͠ଓ͚ΔจԽ • ࣗݾ݁ԽΛࢧ͑Δ Platform •
༏ઌॱҐΛมߋ͢ΔͨΊͷٕज़ઓུ
SRE Λ࣮ݱ͢Δͱ
։ൃνʔϜ͕৴པੑΛ ίϯτϩʔϧ͢Δ Capability Λ ʹ͚͍ͭͯΔ͜ͱ
SRE “NEXT” in ʰελσΟαϓϦʱ • “৴པੑ” ʹؔͯ͠ Enabling Team ͱͯ͠ͷ
SRE Team ׂΛՌͨͭͭ͋͠Δ • SRE Team ͷࠓޙ • ΑΓ৫Λ Sustainable / Scalable ʹ͢ΔͨΊʹɺPlatform ʹؔ͢Δ ΦϯϘʔσΟϯάͷ֦ॆɺ։ൃνʔϜ͕ࣗతʹ৴པੑʹؔ͢Δ Capability शಘΛஅͰ͖ΔΞηεϝϯτΛఏڙ͢Δ • ৴པੑ͚ͩͰͳ͍ɺ։ൃੜ࢈ੑΛՌͨͤΔ Platform ։ൃʹྗ͢Δ
Who owns the Service Level? • Service Level ϓϩμΫτʹؔΘΔશһͷͷ •
શһ͕ؔ৺Λ࣋ͯΔΑ͏ͳ৴པੑࢦඪʹਐԽͤ͞·͠ΐ͏ • ϢʔβମݧΛతʹද͢ Client-side(WebFrontend/Native) Ͱͷ SLI/SLO Λ͏ • ʮࢦඪΛݟͯߦಈ͢ΔʯͦͷͷͷվળαΠΫϧΛճ͠·͠ΐ͏
Thank you! chaspy chaspy_ Engineering Manager, Site Reliability at Recruit
Co., Ltd. Takeshi Kondo https://chaspy.me