Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Servers are doomed to fail
Search
JBD
May 17, 2019
Technology
1.6k
3
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Servers are doomed to fail
JBD
May 17, 2019
Transcript
30 slides
Servers are doomed to fail Jaana B. Dogan
[email protected]
@rakyll
Serverless is also doomed to fail Jaana B. Dogan
[email protected]
@rakyll
Systems are doomed to fail Jaana B. Dogan
[email protected]
@rakyll
Is failure OK? Is failure an unexpected case?
Failure is not an exception. Systems change all the time.
“I haven’t touched the code for a century, it should
just work.” Said no one ever.
Failure is expected. Yes, it is.
None
@rakyll monitoring debugging postmortem
Monitoring is about saying if something is broken.
“99.99% of the requests should return in 100ms.”
@rakyll
@rakyll
Debugging
Debugging is collaborative.
Debugging comes in flavors. Logs Traces Metrics ...
Postmortems
Postmortems
Postmortems
Blameless? Focus on identifying problems.
Collaboration Design for collaboration.
Design for failure Set SLOs, plan for instrumentation, plan for
debugging.
Cross-stack debugging Accountability across stack with high cardinality data. speakerdeck.com/rakyll/rpc-metrics-at-google
Correlation Jump from monitoring/debugging data to data.
On-call debugging Jump from distributed tracing data to on-call information.
who to page?
Dynamic collection Capability to enable more collection in production when
needed.
Continuous collection Continuously collect signals, generate fleet-wide analysis reports.
Introspection Introspection pages provided from the services.
@rakyll monitoring debugging postmortem
Thank you Jaana B. Dogan Google
[email protected]
More Decks by JBD
See All by JBD
eBPF in Microservices Observability at eBPF Day
rakyll
1
2.3k
eBPF in Microservices Observability
rakyll
1
1.8k
OpenTelemetry at AWS
rakyll
1
1.9k
Debugging Code Generation in Go
rakyll
5
1.7k
Are you ready for production?
rakyll
8
3.1k
Serverless Containers
rakyll
1
310
Critical Path Analysis
rakyll
0
730
Monitoring and Debugging Containers
rakyll
2
1.2k
CPDD
rakyll
0
4.3k
Other Decks in Technology
See All in Technology
コーディングできない僕にBobが舞い降りた
shumpei3
0
190
[関西Kaggler会 20261113] 続・コミュニティコンペのススメ
nejumi
0
200
IP重複は"ランディングVPC"で受け止める ― PrivateLink と Transit Gatewayによるマルチテナント接続基盤の設計
hatahata021
3
260
Well, that’s weird…
csswizardry
1
200
OpenTelemetryで変わった 新時代のCloudWatchメトリクス監視
tsukuboshi
0
500
Kansai WordPress Meetup@大阪『ブロックテーマを作る上で知っておくべきこと
torounit
0
300
こんなに変わった! AWSネットワークの常識
shotashiratori
16
11k
自動計装なしでECSサービスをApplication Signalsに載せるまで
ota1022
2
470
メール到達性を支える、リンク先URLのドメイン評価の扱い方
wind111
0
200
パスワードのない鍵で、 データを手元に置く
ginjihayashi
0
400
Terraform 0 to 100:100個以上のリポジトリを常に最新にし続けるための自動化
sue445
3
1.4k
Del vibe coding a un Desarrollo (agéntico) fiable
jcabot
0
210
Featured
See All Featured
How To Speak Unicorn (iThemes Webinar)
marktimemedia
1
640
Optimising Largest Contentful Paint
csswizardry
37
4k
The Invisible Side of Design
smashingmag
301
52k
The SEO Collaboration Effect
kristinabergwall1
1
630
Highjacked: Video Game Concept Design
rkendrick25
PRO
2
590
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
10k
職位にかかわらず全員がリーダーシップを発揮するチーム作り / Building a team where everyone can demonstrate leadership regardless of position
madoxten
70
69k
Jess Joyce - The Pitfalls of Following Frameworks
techseoconnect
PRO
1
480
How to Build an AI Search Optimization Roadmap - Criteria and Steps to Take #SEOIRL
aleyda
1
2.3k
The Hidden Cost of Media on the Web [PixelPalooza 2025]
tammyeverts
2
580
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
130k
Darren the Foodie - Storyboard
khoart
PRO
5
4.1k