Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Critical Path Analysis
JBD
February 06, 2019
0
110
Critical Path Analysis
JBD
February 06, 2019
Tweet
Share
More Decks by JBD
See All by JBD
rakyll
5
1.2k
rakyll
7
1.5k
rakyll
2
1.1k
rakyll
0
120
rakyll
2
870
rakyll
0
3.6k
rakyll
1
190
rakyll
0
100
rakyll
2
1.5k
Featured
See All Featured
bryan
30
3.3k
marcelosomers
220
15k
mthomps
39
2.3k
brad_frost
156
6.4k
scottboms
251
11k
tenderlove
52
3.4k
keavy
106
14k
thoeni
4
550
jrom
114
7.1k
rasmusluckow
318
18k
stephaniewalter
260
11k
hatefulcrawdad
257
17k
Transcript
@rakyll
@rakyll
@rakyll We use facilities that are not reserved for our
flight but are essential in making it happen.
@rakyll We use facilities that are not reserved for our
flight but are essential in making it happen.
@rakyll
@rakyll Systems? who understands them?
@rakyll Jaana B. Dogan jbd@google.com
@rakyll
@rakyll Growing...
@rakyll Tech stack gets fragmented, monoliths are gone...
@rakyll one becomes many failure in isolation escalation in failure?
@rakyll and it may get larger...
@rakyll good guy jeff
@rakyll
@rakyll
@rakyll code search
@rakyll go_library( name = "logs", srcs = ["logs.go"], visibility =
["//visibility:public"], deps = [ …. ], ) References (641 occurrences) - //source/ads/monitoring/BUILD - //source/ads/analysis/BUILD - //source/ads/mobile/BUILD ...
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer critical path
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer critical path
@rakyll frontend server authentication users images memcache blobservice memcache memcache
(metadata) (disks) load balancer critical path
@rakyll CPA (critical path analysis)
@rakyll discover the critical paths make them reliable and fast
make them debuggable
@rakyll How do we get there? (tracing, events, logs)
@rakyll Why? Why? Why? Why? Why?
@rakyll GET /timeline edge-lb sched api-server auth.Auth cache.Get mysql.Query user.Profile
cache.Get mysql.Query images.Filter blobstore.Get
@rakyll bare metal kernel network stack cloud stack libraries frameworks
your code
@rakyll GET /timeline edge-lb sched api-server auth.Auth cache.Get mysql.Query user.Profile
cache.Get mysql.Query images.Filter blobstore.Get not my fault
@rakyll GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter
blobstore.Get cache.Get mysql.Query blob.Get where is the source code?
@rakyll GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter
blobstore.Get cache.Get mysql.Query blob.Get who to call?
@rakyll GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter
blobstore.Get cache.Get mysql.Query blob.Get give me the logs, runtime events, profiles...
@rakyll Challenges...
@rakyll This is an organizational problem. CHALLENGE #1:
@rakyll Engineers don’t know where to start. CHALLENGE #2:
@rakyll Infra is still a blackbox. CHALLENGE #3:
@rakyll Instrumentation is expensive. CHALLENGE #4:
@rakyll Dynamic capabilities are underestimated. CHALLENGE #5:
@rakyll
@rakyll A tool to close knowledge gaps (which we don’t
talk about)
@rakyll Fin jbd@google.com