Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Instrumenting The Rest Of The Company: Hunting ...
Search
Eric Sigler
May 23, 2017
Technology
0
370
Instrumenting The Rest Of The Company: Hunting For Metrics
Presented at Monitorama 2017, video at:
https://youtu.be/wnjCNBfH3kg?t=3h3m35s
Eric Sigler
May 23, 2017
Tweet
Share
More Decks by Eric Sigler
See All by Eric Sigler
Four years of breaking things in production, on purpose.
esigler
0
58
A Brief Introduction To DevOps
esigler
0
110
Humans are terrible compilers: A User's Guide
esigler
0
120
Do You Know If Your Service Is Working Properly? A Guide To Being Paranoid.
esigler
0
180
"Is there any strong objection?"
esigler
0
230
Fear, Uncertainty, and Continuous Deployment
esigler
1
120
3AM, a survey.
esigler
0
230
Strategies For Being On Call & Keeping Your Sanity At The Same Time
esigler
0
160
Engineering for Engineers
esigler
0
90
Other Decks in Technology
See All in Technology
[OCI Skill Mapping] AWSユーザーのためのOCI(2025年8月20日開催)
oracle4engineer
PRO
2
140
実践データベース設計 ①データベース設計概論
recruitengineers
PRO
2
200
アジャイルテストで高品質のスプリントレビューを
takesection
0
110
Devinを使ったモバイルアプリ開発 / Mobile app development with Devin
yanzm
0
180
Postman MCP 関連機能アップデート / Postman MCP feature updates
yokawasa
0
150
実践アプリケーション設計 ③ドメイン駆動設計
recruitengineers
PRO
1
160
我々は雰囲気で仕事をしている / How can we do vibe coding as well
naospon
2
220
株式会社ARAV 採用案内
maqui
0
340
マイクロモビリティシェアサービスを支える プラットフォームアーキテクチャ
grimoh
1
200
そのコンポーネント、サーバー?クライアント?App Router開発のモヤモヤを可視化する補助輪
makotot
3
280
kintone開発チームの紹介
cybozuinsideout
PRO
0
73k
「守る」から「進化させる」セキュリティへ ~AWS re:Inforce 2025参加報告~ / AWS re:Inforce 2025 Participation Report
yuj1osm
1
110
Featured
See All Featured
StorybookのUI Testing Handbookを読んだ
zakiyama
30
6k
Writing Fast Ruby
sferik
628
62k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
The Language of Interfaces
destraynor
160
25k
Speed Design
sergeychernyshev
32
1.1k
How STYLIGHT went responsive
nonsquared
100
5.7k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.6k
Automating Front-end Workflow
addyosmani
1370
200k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.8k
Imperfection Machines: The Place of Print at Facebook
scottboms
268
13k
jQuery: Nuts, Bolts and Bling
dougneiner
64
7.9k
Transcript
@esigler Instrumenting The Rest Of The Company: Hunting For Useful
Metrics Eric Sigler, Head of DevOps, PagerDuty
@esigler Alternatively: ”Lies, Damn Lies, and Hacky Scripts"
@esigler
@esigler Engineer Eng Engineer Eng? Manager Mgr Manager
@esigler (No stock photos harmed in the making of this
talk.)
@esigler "We have problem $foo, so we're going to do
$bar."
@esigler "What data did you use to understand $foo? And
how will we know if $bar improved anything?”
@esigler “We can’t really measure either $foo and/or $bar.”
@esigler “Without data, you're just another person with an opinion.”
- W. Edwards Deming
@esigler
@esigler
@esigler
@esigler (Turns out other managers do this too.)
@esigler
@esigler "We have a problem with people not knowing what
the chatbot does, so we're going to write better documentation."
@esigler
@esigler ?
@esigler
@esigler “If only there was some way we could track
events, and show them over time.”
@esigler
@esigler
@esigler
@esigler Outcome: Writing a smarter help function in the chat
bot. (And simplifying some commands).
@esigler
@esigler Takeaway: Reuse existing tools when it makes sense.
@esigler
@esigler "We have slow tests in CI, so we're going
to complain a lot about it.”
@esigler “Define slow.”
@esigler Local != CI
@esigler
@esigler
@esigler
@esigler
@esigler “Tests take forever to start.”
@esigler
@esigler ?
@esigler
@esigler Outcome: More workers. (And, knowing how many to budget
for.)
@esigler Takeaway: Look for ways to reverse engineer existing metrics.
@esigler
@esigler "We have to ship code faster, so we're going
to reorganize."
@esigler
@esigler
@esigler But it doesn’t show where the bottlenecks are.
@esigler Pipe GitHub metrics into &
@esigler
@esigler
@esigler
@esigler Then start making changes.
@esigler
@esigler
@esigler Outcome: Productivity success! (With massive organizational change to enable
it.)
@esigler Takeaway: Look for proxy metrics
@esigler Potpourri: Data collection (chat, email, calendars) Cross-validation of metrics
(“Sniff test”) Cognitive biases around metrics Plotting against organization events
@esigler Takeaways: Useful metrics are everywhere You aren’t alone in
digging for metrics Existing tools can be repurposed Look to reverse engineer your way to a metric Look for proxy metrics (but choose wisely)
@esigler Thank you!
@esigler Image credits: https://commons.wikimedia.org/wiki/File:Staff_meeting.jpg https://blogs-images.forbes.com/kellyallan/files/2015/06/Deming-in-Tuxedo-DEM-1078-Dr.-Deming2-1940x1130.jpg (Wherever I grabbed that
screenshot from Pulp Fiction, my apologies I am a terrible person for not capturing the URL)