Upgrade to Pro — share decks privately, control downloads, hide ads and more …

When Worst is Best in Distributed Systems Design

pbailis
September 25, 2015

When Worst is Best in Distributed Systems Design

StrangeLoop 2015
25 September 2015
St. Louis, MO

Talk video: https://www.youtube.com/watch?v=ZGIAypUUwoQ
More information: http://bailis.org/

n many areas of systems design, provisioning for worst-case behavior (e.g., load spikes and anomalous user activity) incurs sizable penalties (e.g., performance and operational overheads) in the typical and best cases. However, in distributed systems, building software that is resilient to worse-case network behavior can -- perhaps paradoxically -- lead to improved behavior in typical and best-case scenarios. That is, systems that don't rely on synchronous communication (or coordination) in the worst case frequently aren't forced to wait in any case -- improving latency, scalability, and performance via increased concurrency.

In this talk, we'll explore how to use this worst-case analysis as a more general design principle for scalable systems design. As developers increasingly interacting with and building our own distributed systems, we tend to fixate only on failure scenarios (e.g., "partition tolerance" in the CAP Theorem); this is an important first step, but it's not the whole story. To illustrate why, I'll present practical lessons learned from applying this principle to both web and transaction processing applications as well as database internals such as integrity constraints and indexes. We've found considerable evidence that many of these common tasks and workloads can benefit substantially (e.g., regular order-of-magnitude speedups) from this analysis. In all likelihood, you can too.

pbailis

September 25, 2015
Tweet

More Decks by pbailis

Other Decks in Technology

Transcript

  1. WHEN
    WORST
    IS BEST Peter Bailis
    Stanford CS

    @pbailis

    in distributed
    systems
    design StrangeLoop 2015
    25 September, St. Louis

    View full-size slide

  2. What if we designed computer
    systems for worst case scenarios?

    View full-size slide

  3. Cluster provisioning: 7.3B simultaneous users
    many idle resources!
    What if we designed computer
    systems for worst case scenarios?

    View full-size slide

  4. Cluster provisioning: 7.3B simultaneous users
    many idle resources!
    Hardware: chips for the next Mars rover
    hugely expensive packaging!
    What if we designed computer
    systems for worst case scenarios?

    View full-size slide

  5. Cluster provisioning: 7.3B simultaneous users
    many idle resources!
    Hardware: chips for the next Mars rover
    hugely expensive packaging!
    Security: all our developers are malicious
    expensive code deployment!
    What if we designed computer
    systems for worst case scenarios?

    View full-size slide

  6. Designing for
    the worst case

    often penalizes
    the average case

    View full-size slide

  7. Designing for the worst case
    often penalizes the average case
    Average
    case

    performance
    Worst case performance

    View full-size slide

  8. Designing for the worst case
    often penalizes the average case
    Average
    case

    performance
    Worst case performance
    ???
    this talk

    View full-size slide

  9. This talk: When can designing for
    the worst case improve the
    average case?
    Structure
    Distributed systems and the network

    Beyond the network

    Lessons

    View full-size slide

  10. This talk: When can designing for
    the worst case improve the
    average case?
    Structure
    Distributed systems and the network

    Beyond the network

    Lessons

    View full-size slide

  11. Almost every non-trivial application today is (or is
    becoming) distributed
    Distributed Systems Matter
    Distribution happens over a network

    View full-size slide

  12. Almost every non-trivial application today is (or is
    becoming) distributed
    Corollary:
    Almost every non-trivial application today needs to
    worry about the network
    Distributed Systems Matter
    Distribution happens over a network

    View full-size slide

  13. Networks make design hard
    Many things can go wrong:

    View full-size slide

  14. Networks make design hard
    Many things can go wrong:
    Packets may be delayed
    Packets may be dropped
    Sometimes called an asynchronous network

    View full-size slide

  15. any replica can respond to any request
    Handling Worst-Case Net Behavior
    availability addresses delays, drops:

    View full-size slide

  16. any replica can respond to any request
    Handling Worst-Case Net Behavior
    availability addresses delays, drops:

    View full-size slide

  17. any replica can respond to any request
    Handling Worst-Case Net Behavior
    availability addresses delays, drops:

    View full-size slide

  18. any replica can respond to any request
    Handling Worst-Case Net Behavior
    availability addresses delays, drops:

    View full-size slide

  19. any replica can respond to any request
    Handling Worst-Case Net Behavior
    availability addresses delays, drops:

    View full-size slide

  20. any replica can respond to any request
    Handling Worst-Case Net Behavior
    availability addresses delays, drops:

    View full-size slide

  21. any replica can respond to any request
    Handling Worst-Case Net Behavior
    if our system is available,
    then even when network is fine,

    we still don’t have to talk!
    availability addresses delays, drops:

    View full-size slide

  22. any replica can respond to any request
    Handling Worst-Case Net Behavior
    if our system is available,
    then even when network is fine,

    we still don’t have to talk!
    NO
    COORDINATION
    availability addresses delays, drops:

    View full-size slide

  23. Coordination-free systems
    What if we don’t have to talk?

    View full-size slide

  24. Coordination-free systems:

    1.) Enable infinite scale-out
    What if we don’t have to talk?

    View full-size slide

  25. Coordination-free systems:

    1.) Enable infinite scale-out
    What if we don’t have to talk?

    View full-size slide

  26. Coordination-free systems:

    1.) Enable infinite scale-out
    What if we don’t have to talk?

    View full-size slide

  27. Coordination-free systems:

    1.) Enable infinite scale-out
    What if we don’t have to talk?

    View full-size slide

  28. Coordination-free systems:

    1.) Enable infinite scale-out
    What if we don’t have to talk?

    View full-size slide

  29. A B C D E F G H
    DISTRIBUTED TRANSACTIONS (EC2)
    1 2 3 4 5 6 7
    Number of Items per Transaction
    Throughput (txns/s)
    Number of Servers (Items) Accessed per Transaction
    Number of Servers (Items) Accessed per Transaction

    View full-size slide

  30. A B C D E F G H
    IN-MEMORY
    LOCKING
    COORDINATED
    1 2 3 4 5 6 7
    Number of Items per Transaction
    Throughput (txns/s)
    DISTRIBUTED TRANSACTIONS (EC2)
    Number of Servers (Items) Accessed per Transaction
    Number of Servers (Items) Accessed per Transaction

    View full-size slide

  31. A B C D E F G H
    IN-MEMORY
    LOCKING
    COORDINATED
    1 2 3 4 5 6 7
    Number of Items per Transaction
    Throughput (txns/s)
    DISTRIBUTED TRANSACTIONS (EC2)
    LOG SCALE!
    -398x
    Number of Servers (Items) Accessed per Transaction
    Number of Servers (Items) Accessed per Transaction

    View full-size slide

  32. A B C D E F G H
    IN-MEMORY
    LOCKING
    1 2 3 4 5 6 7
    Number of Items per Transaction
    Throughput (txns/s)
    COORDINATED
    COORDINATION-FREE
    DISTRIBUTED TRANSACTIONS (EC2)
    -398x
    Number of Servers (Items) Accessed per Transaction

    View full-size slide

  33. Coordination-free systems:

    1.) Enable infinite scale-out

    2.) Improve throughput

    What if we don’t have to talk?

    View full-size slide

  34. 133.7+ ms
    RTT

    View full-size slide

  35. 133.7+ ms
    RTT

    View full-size slide

  36. 133.7+ ms
    RTT
    85.1+ ms
    RTT

    View full-size slide

  37. What if we don’t have to talk?
    Coordination-free systems:

    1.) Enable infinite scale-out

    2.) Improve throughput

    3.) Ensure low latency

    4.) Guarantee “always on" response

    View full-size slide

  38. What if we don’t have to talk?
    Coordination-free systems:

    1.) Enable infinite scale-out

    2.) Improve throughput

    3.) Ensure low latency

    4.) Guarantee “always on" response

    View full-size slide

  39. Coordination-free systems:

    1.) Enable infinite scale-out

    2.) Improve throughput

    3.) Ensure low latency

    4.) Guarantee “always on" response
    What if we don’t have to talk?

    View full-size slide

  40. But wait! What about CAP?!?!
    • CAP Thm.: Famous result from Eric Brewer, Inktomi
    • Takeaway (+ related results): properties like serializability
    require unavailability (or require coordination)
    • Common (incorrect) conclusion: availability is too
    expensive, only matters during failures, so forget about it

    View full-size slide

  41. But wait! What about CAP?!?!
    • CAP Thm.: Famous result from Eric Brewer, Inktomi
    • Takeaway (+ related results): properties like serializability
    require unavailability (or require coordination)
    • Common (incorrect) conclusion: availability is too
    expensive, only matters during failures, so forget about it
    surprise: many useful guarantees don’t
    require coordination (or unavailability)!

    View full-size slide

  42. “Worst” is a Design Tool
    legacy implementations: designed for single-
    node context, use coordination
    research question: what if we built systems that
    didn’t have to coordinate?
    result: new designs that avoid coordination unless
    strictly necessary
    Example: Coordination-Avoiding Databases

    View full-size slide

  43. Simple Example: Read Committed
    legacy implementation: lock records during access
    research question: is coordination necessary?
    goal: never read from uncommitted transactions

    View full-size slide

  44. Simple Example: Read Committed
    legacy implementation: lock records during access
    research question: is coordination necessary?
    result: no! for example, buffer writes until commit

    result: OOM speedups over classic implementations
    goal: never read from uncommitted transactions
    VLDB 2014, SIGMOD 2015

    View full-size slide

  45. What if we don’t have to talk?
    Coordination-free systems:

    1.) Enable infinite scale-out

    2.) Improve throughput

    3.) Ensure low latency

    4.) Guarantee “always on" response

    View full-size slide

  46. Coordination-free systems:

    1.) Enable infinite scale-out

    2.) Improve throughput

    3.) Ensure low latency

    4.) Guarantee “always on" response
    What if we don’t have to talk?

    View full-size slide

  47. Coordination-free systems:

    1.) Enable infinite scale-out

    2.) Improve throughput

    3.) Ensure low latency

    4.) Guarantee “always on" response
    What if we don’t have to talk?
    Accounting for worst case improves average case

    View full-size slide

  48. Punchline: Distributed Systems & Networks
    • Systems that behave well during network faults can
    behave better in non-faulty environments too
    • With good designs, popular guarantees from today’s
    RDBMSs can benefit! (see also Martin’s talk, 11AM Sat)
    • Research on coordination-avoiding systems highlights
    potential for huge speedups (see bailis.org)
    • Keywords: CRDTs, I-confluence, RAMP, HAT, Bloom^L

    View full-size slide

  49. This talk: When can designing for
    the worst case improve the
    average case?
    Structure
    Distributed systems and the network

    Beyond the network

    Lessons

    View full-size slide

  50. Replication for fault tolerance
    can increase request capacity
    Replication helps Capacity

    View full-size slide

  51. Fail-over helps (Dev)Ops

    View full-size slide

  52. Fail-over helps (Dev)Ops

    View full-size slide

  53. If services can
    auto-fail-over…
    can kill processes:

    to perform upgrades

    to manage stragglers

    to revoke resources
    Fail-over helps (Dev)Ops

    View full-size slide

  54. 99.9th %ile latency: 100ms
    avg latency: 1.2ms
    YOUR
    SERVICE
    HERE
    Tail Latency in (Micro)services

    View full-size slide

  55. 99.9th %ile latency: 100ms
    avg latency: 1.2ms
    YOUR
    SERVICE
    HERE
    10ms
    Tail Latency in (Micro)services

    View full-size slide

  56. 99.9th %ile latency: 100ms
    avg latency: 1.2ms
    YOUR
    SERVICE
    HERE
    10ms
    1.09ms
    Tail Latency in (Micro)services

    View full-size slide

  57. 99.9th %ile latency: 100ms
    Tail Latency in (Micro)services

    View full-size slide

  58. front-end avg. latency: 64ms
    at 100x fan-out,
    99.9th %ile latency: 100ms
    Tail Latency in (Micro)services

    View full-size slide

  59. front-end avg. latency: 64ms
    at 100x fan-out,
    99.9th %ile latency: 100ms 10ms
    Tail Latency in (Micro)services

    View full-size slide

  60. front-end avg. latency: 64ms
    at 100x fan-out,
    6.7ms
    99.9th %ile latency: 100ms 10ms
    Tail Latency in (Micro)services

    View full-size slide

  61. YOUR SERVICE’S
    CORNER CASE
    MAY BE ITS
    CONSUMER’S
    AVERAGE CASE

    View full-size slide

  62. Universal Design

    View full-size slide

  63. Universal Design

    View full-size slide

  64. There is also a strong business case for accessibility.
    Accessibility overlaps with other best practices such as mobile web
    design, device independence, multi-modal interaction, usability,
    design for older users, and search engine optimization (SEO).

    Case studies show that accessible websites have better search
    results, reduced maintenance costs, and increased audience reach,
    among other benefits.

    View full-size slide

  65. x
    f(x)
    When “Best” Is Brittle
    Idealized function
    Optimum

    View full-size slide

  66. x
    f(x)
    When “Best” Is Brittle
    Idealized function
    Optimum
    Less well-behaved
    x
    f(x)
    Optimum

    View full-size slide

  67. x
    f(x)
    When “Best” Is Brittle
    Idealized function
    Optimum
    Less well-behaved
    x
    f(x)
    Optimum
    Missed the target

    View full-size slide

  68. x
    f(x)
    When “Best” Is Brittle
    Idealized function
    Optimum
    Less well-behaved
    x
    f(x)
    Optimum
    Missed the target
    “Stable”
    solution

    View full-size slide

  69. x
    f(x)
    When “Best” Is Brittle
    Idealized function
    Optimum
    Less well-behaved
    x
    f(x)
    Optimum
    Missed the target
    “Stable”
    solution
    Robust Optimization studies finding the stable solution

    View full-size slide

  70. This talk: When can designing for
    the worst case improve the
    average case?
    Structure
    Distributed systems and the network

    Beyond the network

    Lessons

    View full-size slide

  71. This talk: When can designing for
    the worst case improve the
    average case?

    View full-size slide

  72. When does this apply?
    When corner cases are common
    When environmental conditions are variable
    When “normal” isn’t what we think
    This talk: When can designing for
    the worst case improve the
    average case?

    View full-size slide

  73. DEFINING
    “NORMAL”
    DEFINES OUR
    DESIGNS

    View full-size slide

  74. “Worst” raises tough questions

    View full-size slide

  75. Cluster provisioning:
    what’s our scale-out strategy?
    “Worst” raises tough questions

    View full-size slide

  76. Cluster provisioning:
    what’s our scale-out strategy?
    Hardware:
    what happens during bit flips? do we need ECC?
    “Worst” raises tough questions

    View full-size slide

  77. Cluster provisioning:
    what’s our scale-out strategy?
    Hardware:
    what happens during bit flips? do we need ECC?
    Security:
    how to do we manage internal data accesses?
    “Worst” raises tough questions

    View full-size slide

  78. EXAMINE
    YOUR
    BIASES

    View full-size slide

  79. Reasoning about worst-case scenarios
    can be a powerful design tool
    Key to coordination avoiding distributed
    systems designs
    Can often improve performance and
    robustness, also combat bias
    @PBAILIS // bailis.org

    View full-size slide

  80. Special thanks to

    David Andersen, Ali Ghodsi, Joe Hellerstein, Eddie Kohler,
    Phil Levis, Alex Miller, Oscar Moll, Barzan Mozafari, Ion
    Stoica, Eugene Wu, Jean Yang, Matei Zaharia

    View full-size slide

  81. Reasoning about worst-case scenarios
    can be a powerful design tool
    Key to coordination avoiding distributed
    systems designs
    Can often improve performance and
    robustness, also combat bias
    @PBAILIS // bailis.org

    View full-size slide