Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Critical Path Analysis
JBD
February 06, 2019
0
230
Critical Path Analysis
JBD
February 06, 2019
Tweet
Share
More Decks by JBD
See All by JBD
eBPF in Microservices Observability at eBPF Day
rakyll
1
1.6k
eBPF in Microservices Observability
rakyll
1
1.3k
OpenTelemetry at AWS
rakyll
1
1.6k
Debugging Code Generation in Go
rakyll
5
1.3k
Are you ready for production?
rakyll
8
2k
Servers are doomed to fail
rakyll
3
1.2k
Serverless Containers
rakyll
1
160
Monitoring and Debugging Containers
rakyll
2
970
CPDD
rakyll
0
3.8k
Featured
See All Featured
Intergalactic Javascript Robots from Outer Space
tanoku
261
25k
Fontdeck: Realign not Redesign
paulrobertlloyd
73
4.1k
We Have a Design System, Now What?
morganepeng
35
2.9k
From Idea to $5000 a Month in 5 Months
shpigford
372
44k
Streamline your AJAX requests with AmplifyJS and jQuery
dougneiner
125
8.5k
KATA
mclloyd
7
8.6k
Docker and Python
trallard
27
1.5k
Optimizing for Happiness
mojombo
365
63k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
38
12k
Keith and Marios Guide to Fast Websites
keithpitt
404
21k
Principles of Awesome APIs and How to Build Them.
keavy
113
15k
4 Signs Your Business is Dying
shpigford
169
20k
Transcript
@rakyll
@rakyll
@rakyll We use facilities that are not reserved for our
flight but are essential in making it happen.
@rakyll We use facilities that are not reserved for our
flight but are essential in making it happen.
@rakyll
@rakyll Systems? who understands them?
@rakyll Jaana B. Dogan jbd@google.com
@rakyll
@rakyll Growing...
@rakyll Tech stack gets fragmented, monoliths are gone...
@rakyll one becomes many failure in isolation escalation in failure?
@rakyll and it may get larger...
@rakyll good guy jeff
@rakyll
@rakyll
@rakyll code search
@rakyll go_library( name = "logs", srcs = ["logs.go"], visibility =
["//visibility:public"], deps = [ …. ], ) References (641 occurrences) - //source/ads/monitoring/BUILD - //source/ads/analysis/BUILD - //source/ads/mobile/BUILD ...
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer critical path
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer critical path
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer critical path
@rakyll CPA (critical path analysis)
@rakyll discover the critical paths make them reliable and fast
make them debuggable
@rakyll How do we get there? (tracing, events, logs)
@rakyll Why? Why? Why? Why? Why?
@rakyll GET /timeline edge-lb sched api-server auth.Auth cache.Get mysql.Query user.Profile
cache.Get mysql.Query images.Filter blobstore.Get
@rakyll bare metal kernel network stack cloud stack libraries frameworks
your code
@rakyll GET /timeline edge-lb sched api-server auth.Auth cache.Get mysql.Query user.Profile
cache.Get mysql.Query images.Filter blobstore.Get not my fault
@rakyll GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter
blobstore.Get cache.Get mysql.Query blob.Get where is the source code?
@rakyll GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter
blobstore.Get cache.Get mysql.Query blob.Get who to call?
@rakyll GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter
blobstore.Get cache.Get mysql.Query blob.Get give me the logs, runtime events, profiles...
@rakyll Challenges...
@rakyll This is an organizational problem. CHALLENGE #1:
@rakyll Engineers don’t know where to start. CHALLENGE #2:
@rakyll Infra is still a blackbox. CHALLENGE #3:
@rakyll Instrumentation is expensive. CHALLENGE #4:
@rakyll Dynamic capabilities are underestimated. CHALLENGE #5:
@rakyll
@rakyll A tool to close knowledge gaps (which we don’t
talk about)
@rakyll Fin jbd@google.com