Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Instrumenting The Rest Of The Company: Hunting ...
Search
Eric Sigler
May 23, 2017
Technology
0
370
Instrumenting The Rest Of The Company: Hunting For Metrics
Presented at Monitorama 2017, video at:
https://youtu.be/wnjCNBfH3kg?t=3h3m35s
Eric Sigler
May 23, 2017
Tweet
Share
More Decks by Eric Sigler
See All by Eric Sigler
Four years of breaking things in production, on purpose.
esigler
0
58
A Brief Introduction To DevOps
esigler
0
110
Humans are terrible compilers: A User's Guide
esigler
0
120
Do You Know If Your Service Is Working Properly? A Guide To Being Paranoid.
esigler
0
170
"Is there any strong objection?"
esigler
0
220
Fear, Uncertainty, and Continuous Deployment
esigler
1
120
3AM, a survey.
esigler
0
230
Strategies For Being On Call & Keeping Your Sanity At The Same Time
esigler
0
160
Engineering for Engineers
esigler
0
89
Other Decks in Technology
See All in Technology
Contract One Engineering Unit 紹介資料
sansan33
PRO
0
6.9k
伴走から自律へ: 形式知へと導くSREイネーブリングによる プロダクトチームの信頼性オーナーシップ向上 / SRE NEXT 2025
visional_engineering_and_design
3
460
Introduction to Bill One Development Engineer
sansan33
PRO
0
260
Bill One 開発エンジニア 紹介資料
sansan33
PRO
4
13k
名刺メーカーDevグループ 紹介資料
sansan33
PRO
0
820
SRE不在の開発チームが障害対応と 向き合った100日間 / 100 days dealing with issues without SREs
shin1988
2
2.1k
全部AI、全員Cursor、ドキュメント駆動開発 〜DevinやGeminiも添えて〜
rinchsan
10
5.1k
衛星運用をソフトウェアエンジニアに依頼したときにできあがるもの
sankichi92
1
1k
【あのMCPって、どんな処理してるの?】 AWS CDKでの開発で便利なAWS MCP Servers特集
yoshimi0227
6
950
[SRE NEXT] ARR150億円_エンジニア140名_27チーム_17プロダクトから始めるSLO.pdf
satos
5
3k
Snowflake Intelligenceという名のAI Agentが切り開くデータ活用の未来とその実現に必要なこと@SnowVillage『Data Management #1 Summit 2025 Recap!!』
ryo_suzuki
1
160
ゼロから始めるSREの事業貢献 - 生成AI時代のSRE成長戦略と実践 / Starting SRE from Day One
shinyorke
PRO
0
110
Featured
See All Featured
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Navigating Team Friction
lara
187
15k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.4k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
Practical Orchestrator
shlominoach
189
11k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
54k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Done Done
chrislema
184
16k
For a Future-Friendly Web
brad_frost
179
9.8k
Java REST API Framework Comparison - PWX 2021
mraible
31
8.7k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Transcript
@esigler Instrumenting The Rest Of The Company: Hunting For Useful
Metrics Eric Sigler, Head of DevOps, PagerDuty
@esigler Alternatively: ”Lies, Damn Lies, and Hacky Scripts"
@esigler
@esigler Engineer Eng Engineer Eng? Manager Mgr Manager
@esigler (No stock photos harmed in the making of this
talk.)
@esigler "We have problem $foo, so we're going to do
$bar."
@esigler "What data did you use to understand $foo? And
how will we know if $bar improved anything?”
@esigler “We can’t really measure either $foo and/or $bar.”
@esigler “Without data, you're just another person with an opinion.”
- W. Edwards Deming
@esigler
@esigler
@esigler
@esigler (Turns out other managers do this too.)
@esigler
@esigler "We have a problem with people not knowing what
the chatbot does, so we're going to write better documentation."
@esigler
@esigler ?
@esigler
@esigler “If only there was some way we could track
events, and show them over time.”
@esigler
@esigler
@esigler
@esigler Outcome: Writing a smarter help function in the chat
bot. (And simplifying some commands).
@esigler
@esigler Takeaway: Reuse existing tools when it makes sense.
@esigler
@esigler "We have slow tests in CI, so we're going
to complain a lot about it.”
@esigler “Define slow.”
@esigler Local != CI
@esigler
@esigler
@esigler
@esigler
@esigler “Tests take forever to start.”
@esigler
@esigler ?
@esigler
@esigler Outcome: More workers. (And, knowing how many to budget
for.)
@esigler Takeaway: Look for ways to reverse engineer existing metrics.
@esigler
@esigler "We have to ship code faster, so we're going
to reorganize."
@esigler
@esigler
@esigler But it doesn’t show where the bottlenecks are.
@esigler Pipe GitHub metrics into &
@esigler
@esigler
@esigler
@esigler Then start making changes.
@esigler
@esigler
@esigler Outcome: Productivity success! (With massive organizational change to enable
it.)
@esigler Takeaway: Look for proxy metrics
@esigler Potpourri: Data collection (chat, email, calendars) Cross-validation of metrics
(“Sniff test”) Cognitive biases around metrics Plotting against organization events
@esigler Takeaways: Useful metrics are everywhere You aren’t alone in
digging for metrics Existing tools can be repurposed Look to reverse engineer your way to a metric Look for proxy metrics (but choose wisely)
@esigler Thank you!
@esigler Image credits: https://commons.wikimedia.org/wiki/File:Staff_meeting.jpg https://blogs-images.forbes.com/kellyallan/files/2015/06/Deming-in-Tuxedo-DEM-1078-Dr.-Deming2-1940x1130.jpg (Wherever I grabbed that
screenshot from Pulp Fiction, my apologies I am a terrible person for not capturing the URL)