Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Servers are doomed to fail

E7526ec3e801f8ba99f6746498a154a6?s=47 JBD
May 17, 2019
Technology
3
1.2k

Servers are doomed to fail

E7526ec3e801f8ba99f6746498a154a6?s=128

JBD

May 17, 2019
Tweet

More Decks by JBD

See All by JBD
eBPF in Microservices Observability at eBPF Day
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
1
1.6k
eBPF in Microservices Observability
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
1
1.3k
OpenTelemetry at AWS
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
1
1.6k
Debugging Code Generation in Go
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
5
1.3k
Are you ready for production?
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
8
2k
Serverless Containers
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
1
160
Critical Path Analysis
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
0
230
Monitoring and Debugging Containers
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
2
970
CPDD
E7526ec3e801f8ba99f6746498a154a6?s=48 rakyll
0
3.8k

Other Decks in Technology

See All in Technology
個人的に使って良かったUiPathアクティビティ
0f4a27389318ca7087ae12b1593171c1?s=48 saoritakita
0
280
SoftbankとBlockchain
61c616e83bef28a1cd2666439ebf334b?s=48 sbtechnight
0
100
プロダクション環境の信頼性を損ねず観測する技術
3baceb46dfe36422370df30b8eab61d1?s=48 egmc
4
950
Remixの凄みを紹介したい
56f81d479d44a1e00a577424bb45fd4a?s=48 aiji42
5
3.4k
Declarative Clients in Spring
2559c50bf24b65eaba52fd7c75260c93?s=48 olgamaciaszek
0
120
Dagu | オンプレ向けワークフローエンジン（WebUI 同梱）
2a80a0a6f834a75cc418ef59175ecbb0?s=48 yohamta
1
210
CAMのサービス開発の歴史と共通基盤を使った 開発スタイルへの変遷について
62e3beb467ce4d447d1491263dcf8d46?s=48 ishikawa_pro
0
110
統計学と機械学習の違いと接点とは？
76020f2b85928745e24f9b1e017c7c06?s=48 ueniki
0
250
NestJS + Prisma2 で歩む RLS の世界
3c6726d117aabeca68ccd7340c9204df?s=48 ynaka81
1
110
E2E自動テスト導入・運用をめぐる先入観と実際に起きたこと / Preconceptions and What Happened with E2E Testing
77fa3ebbf229a884c74c905034b1f4c2?s=48 ak1210
6
1.4k
モダンデータスタックとかの話(データエンジニアのお仕事とは)
95be3c1812a3ae53c2d81cc7ea2eeb3d?s=48 foursue
0
500
一人から始めるプロダクトSRE / How to start SRE in a product team, all by yourself
12a5cec7a54018170b1a009731acf477?s=48 vtryo
4
3.1k

Featured

See All Featured
Large-scale JavaScript Application Architecture
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
499
110k
Building an army of robots
5f2da528927a2ec9ba4fec2069cbc958?s=48 kneath
299
40k
Web development in the modern age
465724d73fe3a92c0879fdfb43a3a6f3?s=48 philhawksworth
197
9.3k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
3d4519fd34b76fb265fc0237f3792bd4?s=48 danielanewman
212
20k
Side Projects
027d1ebf66cc039a0bd3b55eeadbe75d?s=48 sachag
449
37k
WebSockets: Embracing the real-time Web
E76911cbe088e5b850d966de3fc7435b?s=48 robhawkes
57
5k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
E195ae45320d9202eaa01c9f1d31a416?s=48 marktimemedia
15
920
Art Directing for the Web. Five minutes with CSS Template Areas
433acaea1012b25d97ae66da9b998dad?s=48 malarkey
196
9.4k
Practical Orchestrator
168ccec72eee0530b818d44f3fedaacf?s=48 shlominoach
178
8.6k
The Most Common Mistakes in Cover Letters
1ce0eb06fd6a9e9566a0cb310cfee635?s=48 jrick
PRO
4
24k
The Language of Interfaces
Fde6c3a50ca518a909ef35c22c0ee0e4?s=48 destraynor
148
20k
GraphQLの誤解/rethinking-graphql
3cb4e761dbdb7aebd1ffd85f752e1e3e?s=48 sonatard
24
6.2k

Transcript

  1. Servers are doomed to fail Jaana B. Dogan jbd@google.com @rakyll

  2. Serverless is also doomed to fail Jaana B. Dogan jbd@google.com

    @rakyll

  3. Systems are doomed to fail Jaana B. Dogan jbd@google.com @rakyll

  4. Is failure OK? Is failure an unexpected case?

  5. Failure is not an exception. Systems change all the time.

  6. “I haven’t touched the code for a century, it should

    just work.” Said no one ever.

  7. Failure is expected. Yes, it is.

  8. None

  9. @rakyll monitoring debugging postmortem

  10. Monitoring is about saying if something is broken.

  11. “99.99% of the requests should return in 100ms.”

  12. @rakyll

  13. @rakyll

  14. Debugging

  15. Debugging is collaborative.

  16. Debugging comes in flavors. Logs Traces Metrics ...

  17. Postmortems

  18. Postmortems

  19. Postmortems

  20. Blameless? Focus on identifying problems.

  21. Collaboration Design for collaboration.

  22. Design for failure Set SLOs, plan for instrumentation, plan for

    debugging.

  23. Cross-stack debugging Accountability across stack with high cardinality data. speakerdeck.com/rakyll/rpc-metrics-at-google

  24. Correlation Jump from monitoring/debugging data to data.

  25. On-call debugging Jump from distributed tracing data to on-call information.

    who to page?

  26. Dynamic collection Capability to enable more collection in production when

    needed.

  27. Continuous collection Continuously collect signals, generate fleet-wide analysis reports.

  28. Introspection Introspection pages provided from the services.

  29. @rakyll monitoring debugging postmortem

  30. Thank you Jaana B. Dogan Google jbd@google.com