Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Instrumenting The Rest Of The Company: Hunting ...
Search
Eric Sigler
May 23, 2017
Technology
0
330
Instrumenting The Rest Of The Company: Hunting For Metrics
Presented at Monitorama 2017, video at:
https://youtu.be/wnjCNBfH3kg?t=3h3m35s
Eric Sigler
May 23, 2017
Tweet
Share
More Decks by Eric Sigler
See All by Eric Sigler
Four years of breaking things in production, on purpose.
esigler
0
52
A Brief Introduction To DevOps
esigler
0
99
Humans are terrible compilers: A User's Guide
esigler
0
110
Do You Know If Your Service Is Working Properly? A Guide To Being Paranoid.
esigler
0
160
"Is there any strong objection?"
esigler
0
210
Fear, Uncertainty, and Continuous Deployment
esigler
1
110
3AM, a survey.
esigler
0
210
Strategies For Being On Call & Keeping Your Sanity At The Same Time
esigler
0
160
Engineering for Engineers
esigler
0
84
Other Decks in Technology
See All in Technology
Microsoft Intune アプリのトラブルシューティング
sophiakunii
1
400
Microsoft Fabric OneLake の実体について
ryomaru0825
0
190
Redmine 6.0 新機能評価ガイド
vividtone
0
260
RAGのためのビジネス文書解析技術
eida
3
660
ジョブマッチングサービスにおける相互推薦システムの応用事例と課題
hakubishin3
3
620
地理情報データをデータベースに格納しよう~ GPUを活用した爆速データベース PG-Stromの紹介 ~
sakaik
1
110
AWS Lambdaと歩んだ“サーバーレス”と今後 #lambda_10years
yoshidashingo
1
110
国土交通省 データコンペ参加者向け勉強会
takehikohashimoto
0
400
OCI Data Integration技術情報 / ocidi_technical_jp
oracle4engineer
PRO
1
2.6k
インフラとバックエンドとフロントエンドをくまなく調べて遅いアプリを早くした件
tubone24
0
110
マルチモーダル / AI Agent / LLMOps 3つの技術トレンドで理解するLLMの今後の展望
hirosatogamo
1
180
AI長期記憶システム構築のための LLMマルチエージェントの取り組み / Awarefy-LLM-Multi-Agent
iktakahiro
2
350
Featured
See All Featured
Fashionably flexible responsive web design (full day workshop)
malarkey
404
65k
Building Flexible Design Systems
yeseniaperezcruz
327
38k
GraphQLの誤解/rethinking-graphql
sonatard
67
10k
Statistics for Hackers
jakevdp
796
220k
Keith and Marios Guide to Fast Websites
keithpitt
409
22k
Writing Fast Ruby
sferik
627
61k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
42
2.2k
Building Adaptive Systems
keathley
38
2.3k
Code Review Best Practice
trishagee
64
17k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
33k
Testing 201, or: Great Expectations
jmmastey
38
7.1k
What's in a price? How to price your products and services
michaelherold
243
12k
Transcript
@esigler Instrumenting The Rest Of The Company: Hunting For Useful
Metrics Eric Sigler, Head of DevOps, PagerDuty
@esigler Alternatively: ”Lies, Damn Lies, and Hacky Scripts"
@esigler
@esigler Engineer Eng Engineer Eng? Manager Mgr Manager
@esigler (No stock photos harmed in the making of this
talk.)
@esigler "We have problem $foo, so we're going to do
$bar."
@esigler "What data did you use to understand $foo? And
how will we know if $bar improved anything?”
@esigler “We can’t really measure either $foo and/or $bar.”
@esigler “Without data, you're just another person with an opinion.”
- W. Edwards Deming
@esigler
@esigler
@esigler
@esigler (Turns out other managers do this too.)
@esigler
@esigler "We have a problem with people not knowing what
the chatbot does, so we're going to write better documentation."
@esigler
@esigler ?
@esigler
@esigler “If only there was some way we could track
events, and show them over time.”
@esigler
@esigler
@esigler
@esigler Outcome: Writing a smarter help function in the chat
bot. (And simplifying some commands).
@esigler
@esigler Takeaway: Reuse existing tools when it makes sense.
@esigler
@esigler "We have slow tests in CI, so we're going
to complain a lot about it.”
@esigler “Define slow.”
@esigler Local != CI
@esigler
@esigler
@esigler
@esigler
@esigler “Tests take forever to start.”
@esigler
@esigler ?
@esigler
@esigler Outcome: More workers. (And, knowing how many to budget
for.)
@esigler Takeaway: Look for ways to reverse engineer existing metrics.
@esigler
@esigler "We have to ship code faster, so we're going
to reorganize."
@esigler
@esigler
@esigler But it doesn’t show where the bottlenecks are.
@esigler Pipe GitHub metrics into &
@esigler
@esigler
@esigler
@esigler Then start making changes.
@esigler
@esigler
@esigler Outcome: Productivity success! (With massive organizational change to enable
it.)
@esigler Takeaway: Look for proxy metrics
@esigler Potpourri: Data collection (chat, email, calendars) Cross-validation of metrics
(“Sniff test”) Cognitive biases around metrics Plotting against organization events
@esigler Takeaways: Useful metrics are everywhere You aren’t alone in
digging for metrics Existing tools can be repurposed Look to reverse engineer your way to a metric Look for proxy metrics (but choose wisely)
@esigler Thank you!
@esigler Image credits: https://commons.wikimedia.org/wiki/File:Staff_meeting.jpg https://blogs-images.forbes.com/kellyallan/files/2015/06/Deming-in-Tuxedo-DEM-1078-Dr.-Deming2-1940x1130.jpg (Wherever I grabbed that
screenshot from Pulp Fiction, my apologies I am a terrible person for not capturing the URL)