$30 off During Our Annual Pro Sale. View Details »

Critical Path Analysis

JBD
February 06, 2019
350

Critical Path Analysis

JBD

February 06, 2019
Tweet

Transcript

  1. @rakyll

    View Slide

  2. @rakyll

    View Slide

  3. @rakyll
    We use facilities that are not
    reserved for our flight but are
    essential in making it happen.

    View Slide

  4. @rakyll
    We use facilities that are not
    reserved for our flight but are
    essential in making it happen.

    View Slide

  5. @rakyll

    View Slide

  6. @rakyll
    Systems?
    who understands them?

    View Slide

  7. @rakyll
    Jaana B. Dogan
    [email protected]

    View Slide

  8. @rakyll

    View Slide

  9. @rakyll
    Growing...

    View Slide

  10. @rakyll
    Tech stack gets fragmented,
    monoliths are gone...

    View Slide

  11. @rakyll
    one becomes many
    failure in isolation
    escalation in failure?

    View Slide

  12. @rakyll
    and it may get larger...

    View Slide

  13. @rakyll
    good guy jeff

    View Slide

  14. @rakyll

    View Slide

  15. @rakyll

    View Slide

  16. @rakyll
    code search

    View Slide

  17. @rakyll
    go_library(
    name = "logs",
    srcs = ["logs.go"],
    visibility = ["//visibility:public"],
    deps = [
    ….
    ],
    )
    References (641 occurrences)
    - //source/ads/monitoring/BUILD
    - //source/ads/analysis/BUILD
    - //source/ads/mobile/BUILD
    ...

    View Slide

  18. @rakyll
    frontend server
    authentication
    users images
    memcache
    blobservice
    memcache
    memcache
    (metadata) (disks)
    load balancer

    View Slide

  19. @rakyll
    frontend server
    authentication
    users images
    memcache
    blobservice
    memcache
    memcache
    (metadata) (disks)
    load balancer
    critical path

    View Slide

  20. @rakyll
    frontend server
    authentication
    users images
    memcache
    blobservice
    memcache
    memcache
    (metadata) (disks)
    load balancer
    critical path

    View Slide

  21. @rakyll
    frontend server
    authentication
    users images
    memcache
    blobservice
    memcache
    memcache
    (metadata) (disks)
    load balancer
    critical path

    View Slide

  22. @rakyll
    CPA
    (critical path analysis)

    View Slide

  23. @rakyll
    discover the critical paths
    make them reliable and fast
    make them debuggable

    View Slide

  24. @rakyll
    How do we get there?
    (tracing, events, logs)

    View Slide

  25. @rakyll
    Why? Why?
    Why? Why? Why?

    View Slide

  26. @rakyll
    GET /timeline
    edge-lb
    sched
    api-server
    auth.Auth
    cache.Get
    mysql.Query
    user.Profile
    cache.Get
    mysql.Query
    images.Filter
    blobstore.Get

    View Slide

  27. @rakyll
    bare metal
    kernel
    network stack
    cloud stack
    libraries
    frameworks
    your code

    View Slide

  28. @rakyll
    GET /timeline
    edge-lb
    sched
    api-server
    auth.Auth
    cache.Get
    mysql.Query
    user.Profile
    cache.Get
    mysql.Query
    images.Filter
    blobstore.Get
    not my fault

    View Slide

  29. @rakyll
    GET /timeline
    auth.Auth
    cache.Get
    mysql.Query
    user.Profile
    cache.Get
    mysql.Query
    images.Filter
    blobstore.Get
    cache.Get
    mysql.Query
    blob.Get
    where is the source code?

    View Slide

  30. @rakyll
    GET /timeline
    auth.Auth
    cache.Get
    mysql.Query
    user.Profile
    cache.Get
    mysql.Query
    images.Filter
    blobstore.Get
    cache.Get
    mysql.Query
    blob.Get
    who to call?

    View Slide

  31. @rakyll
    GET /timeline
    auth.Auth
    cache.Get
    mysql.Query
    user.Profile
    cache.Get
    mysql.Query
    images.Filter
    blobstore.Get
    cache.Get
    mysql.Query
    blob.Get
    give me the logs, runtime
    events, profiles...

    View Slide

  32. @rakyll
    Challenges...

    View Slide

  33. @rakyll
    This is an organizational
    problem.
    CHALLENGE #1:

    View Slide

  34. @rakyll
    Engineers don’t know
    where to start.
    CHALLENGE #2:

    View Slide

  35. @rakyll
    Infra is still
    a blackbox.
    CHALLENGE #3:

    View Slide

  36. @rakyll
    Instrumentation is
    expensive.
    CHALLENGE #4:

    View Slide

  37. @rakyll
    Dynamic capabilities are
    underestimated.
    CHALLENGE #5:

    View Slide

  38. @rakyll

    View Slide

  39. @rakyll
    A tool to close
    knowledge gaps
    (which we don’t talk about)

    View Slide

  40. @rakyll
    Fin
    [email protected]

    View Slide