Upgrade to Pro — share decks privately, control downloads, hide ads and more …
Sign up for free

RPC Metrics at Google

Sponsored · Your Podcast. Everywhere. Effortlessly. Share. Educate. Inspire. Entertain. You do you. We'll handle the rest. →
Avatar for JBD JBD
August 09, 2018
Programming
650
2

RPC Metrics at Google

Avatar for JBD

JBD

August 09, 2018

More Decks by JBD

See All by JBD
eBPF in Microservices Observability at eBPF Day
Avatar for JBD rakyll
1
2.3k
eBPF in Microservices Observability
Avatar for JBD rakyll
1
1.8k
OpenTelemetry at AWS
Avatar for JBD rakyll
1
1.9k
Debugging Code Generation in Go
Avatar for JBD rakyll
5
1.7k
Are you ready for production?
Avatar for JBD rakyll
8
3k
Servers are doomed to fail
Avatar for JBD rakyll
3
1.6k
Serverless Containers
Avatar for JBD rakyll
1
310
Critical Path Analysis
Avatar for JBD rakyll
0
720
Monitoring and Debugging Containers
Avatar for JBD rakyll
2
1.2k

Other Decks in Programming

See All in Programming
自作Discord音声botをLambda MicroVMsに移行した話 / Migrating My Discord Voice Bot to Lambda MicroVMs
Avatar for アキキー | Akihisa Ikeda akihisaikeda
0
150
Kotlinの15年とServer-Side Kotlin
Avatar for Takehata Naoto n_takehata
1
170
Next.js: The final boss of Web Performance
Avatar for Estela Franco guaca
0
340
Python Lambdaにおけるテストしやすい設計とAI支援の活用 / Designing Testable Python Lambda Functions with AI
Avatar for Kitano Yuichi kitano_yuichi
1
200
Performance Optimization With LLMs
Avatar for xLeitix xleitix
0
210
個人開発者よ、Honoを使え（おじさんver）
Avatar for tanahiro2010 tanahiro2010
2
280
Atomic Design, Enforced: Scaling Mobile Design Systems (next.app devCon / droidCon Berlin 2026)
Avatar for Stelios Frantzeskakis steliosf
PRO
0
180
作れる、でも消せるか？──安全に「枯らして消す」機能削除術 / Drain, Then Delete
Avatar for chobishiba chobishiba
2
1k
No More NoSuchMethodError
Avatar for TATSUNO Yasuhiro exoego
1
110
Findy - エンジニア向け会社紹介/Findy Company Deck
Avatar for Findy findyinc
6
400k
R8 の設定見直しでアプリサイズ削減
Avatar for ANDPAD inc andpad
0
230
AIが生成したコードは読むべきか？抽象化設計とコードの品質保証 / abstraction-design-and-quality
Avatar for MinoDriven minodriven
4
1.2k

Featured

See All Featured
Mind Mapping
Avatar for Hélio Medeiros helmedeiros
1
410
A brief & incomplete history of 
UX Design for the World Wide Web: 1989–2019
Avatar for Jason CranfordTeague jct
2
540
Building a A Zero-Code AI SEO Workflow
Avatar for Ian Lurie portentint
PRO
0
790
Building an army of robots
Avatar for Kyle Neath kneath
307
47k
Design in an AI World
Avatar for tapps tapps
1
360
SEO in 2025: How to Prepare for the Future of Search
Avatar for Michael King ipullrank
3
3.9k
The Pragmatic Product Professional
Avatar for Laura Van Doore lauravandoore
37
7.5k
Stop Working from a Prison Cell
Avatar for Chris Michel hatefulcrawdad
274
21k
Paper Plane
Avatar for Katie Cordina Lindsey katiecoart
PRO
5
54k
GraphQLとの向き合い方2022年版
Avatar for Yosuke Kurami quramy
50
15k
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
Avatar for Sophie Logan marketingsoph
0
440
Heart Work Chapter 1 - Part 1
Avatar for Lake Fama lfama
PRO
10
37k

Transcript

  1. RPC Metrics at Google JBD, Google (@rakyll)

  2. gRPC Metrics at Google JBD, Google (@rakyll)

  3. Request Metrics at Google JBD, Google (@rakyll)

  4. @rakyll "100% is the wrong reliability target for basically everything."

    -- Benjamin Treynor Sloss, VP of Engineering, Google

  5. @rakyll "A service is available if users cannot tell that

    there was an outage."

  6. @rakyll Principled way of saying what level of downtime is

    acceptable. • Error rate • Latency expectations SLOs

  7. @rakyll Analytics frontend server Authentication Reporting Users ... Spanner Blob

    Store

  8. @rakyll Questions infra teams want to ask: • Are we

    meeting the SLO for the other team? • What’s the impact of a product on infra? • How much do we need to scale up if product grows 10%?

  9. @rakyll High-Cardinality Breaking down the metrics data...

  10. @rakyll Query the collected data in various ways: • Latency

    distribution for RPCs originated at Google Analytics. • Requests take took more than 100ms for the customer #123. • Compare the request latency initiated at web vs mobile frontend.

  11. @rakyll Analytics frontend server Authentication Reporting Users ... Spanner Blob

    Store originator=analytics; ...

  12. @rakyll Blob store read errors by originator

  13. @rakyll Dynamically choose aggregation (split between recording and aggregation)

  14. @rakyll Exemplars

  15. @rakyll /rpz and /statz

  16. @rakyll http://server:7777/debug/rpcz

  17. @rakyll Export? Monarch, Prometheus, and more.

  18. @rakyll import “cloud.google.com/go/pubsub”

  19. @rakyll +

  20. Thank you! JBD, Google [email protected] @rakyll