CPDD

Transcript

1. None
2. None
3. None

4. systems? who does that?

5. jaana b. dogan 6+ years at Google, touched many projects

6. early days (of a company)

7. None

8. growing...

9. what growth looks like service A service B service C

   service D service E service auth email

10. one becomes many failure in isolation who to ping in

    failure?

11. and it goes larger...

12. good guy jeff

13. None
14. None

15. code search

16. go_library( name = "logs", srcs = ["logs.go"], visibility = ["//visibility:public"],

    deps = [ …. ], ) References (641 occurrences) - //source/ads/monitoring/BUILD - //source/ads/analysis/BUILD - //source/ads/mobile/BUILD ...

17. frontend server authentication users images memcache blobservice memcache memcache (metadata)

    (disks) load balancer

18. frontend server authentication users images memcache blobservice memcache memcache (metadata)

    (disks) load balancer critical path

19. cpdd (critical path driven development)

20. discover the critical paths make them reliable and fast make

    them debuggable

21. how do we get there? events or tracing

22. why? why? why? why? why?

23. GET /timeline edge-lb sched api-server auth.Auth cache.Get mysql.Query user.Profile cache.Get

    mysql.Query images.Filter blobstore.Get

24. bare metal kernel process scheduler network stack cloud stack user

    process frameworks your code

25. GET /timeline edge-lb sched api-server auth.Auth cache.Get mysql.Query user.Profile cache.Get

    mysql.Query images.Filter blobstore.Get not my fault

26. GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter blobstore.Get

    cache.Get mysql.Query blob.Get where is the source code?

27. GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter blobstore.Get

    cache.Get mysql.Query blob.Get who to call?

28. GET /timeline auth.Auth cache.Get mysql.Query user.Profile cache.Get mysql.Query images.Filter blobstore.Get

    cache.Get mysql.Query blob.Get give me the logs, runtime events, profiles...

29. challenges...

30. this is an organizational problem CPDD CHALLENGE #1:

31. github.com/w3c/distributed-tracing

32. engineers don’t know where to start CPDD CHALLENGE #2:

33. infra is still a blackbox CPDD CHALLENGE #3:

34. instrumentation is expensive CPDD CHALLENGE #4:

35. dynamic capabilities are underestimated CPDD CHALLENGE #5:

36. None

37. cpdd: a tool to close knowledge gaps (which we don’t

    talk about)

38. fin jbd@google.com