Building Scalable Stateful Services

Building Scalable Stateful Services

Slides from Scaling Stateful Services Talk given at Nike Tech Talk 8/11/16

9128d500301ae51524e887bb680f471d?s=128

Caitie McCaffrey

August 12, 2016
Tweet

Transcript

  1. Building Scalable Stateful Services

  2. Caitie McCaffrey Distributed Systems Engineer @caitie caitiem.com

  3. Stateless Services Service

  4. Stateless Services Service

  5. Stateless Services Service Service

  6. Stateless Services Service Service

  7. Stateless Services Service Service

  8. Stateless Services Service Service Service

  9. Stateless Services Service Service Service

  10. Stateless Services Service Service Service

  11. Stateless Services Service Service Service Service

  12. Stateless Services Service Service Service Service

  13. Stateless Services Service Service Service Service

  14. Stateless Services Service Service Service Service Service

  15. Stateless Services Service Service Service Service Service

  16. Data Shipping Paradigm Service Service Service

  17. Data Shipping Paradigm Service Service Service

  18. Data Shipping Paradigm Service Service Service

  19. Data Shipping Paradigm Service Service Service

  20. Data Shipping Paradigm Service Service Service

  21. Data Shipping Paradigm Service Service Service

  22. Data Shipping Paradigm Service Service Service

  23. Data Shipping Paradigm Service Service Service

  24. Data Shipping Paradigm Service Service Service

  25. Data Shipping Paradigm Service Service Service

  26. Data Shipping Paradigm Service Service Service

  27. Data Shipping Paradigm Service Service Service

  28. Data Shipping Paradigm Service Service Service

  29. Data Shipping Paradigm Service Service Service

  30. Data Shipping Paradigm Service Service Service

  31. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  32. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  33. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  34. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  35. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  36. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  37. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  38. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  39. Overview Benefits Building Real World Caution

  40. Data Locality For Low Latency & Data Intensive Services

  41. Function Shipping Paradigm Service Service Service

  42. Function Shipping Paradigm Service Service Service

  43. Function Shipping Paradigm Service Service Service

  44. Function Shipping Paradigm Service Service Service

  45. Function Shipping Paradigm Service Service Service

  46. Function Shipping Paradigm Service Service Service

  47. Function Shipping Paradigm Service Service Service

  48. Function Shipping Paradigm Service Service Service

  49. Function Shipping Paradigm Service Service Service

  50. Function Shipping Paradigm Service Service Service

  51. Function Shipping Paradigm Service Service Service

  52. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  53. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  54. Service Service Service Service Service Service Service Front End Data

    Shipping Paradigm
  55. Just Put A Cache On It?

  56. Service Service Service Service Service Service Service Front End Cache

  57. Service Service Service Service Service Service Service Front End Cache

  58. Service Service Service Service Service Service Service Front End Cache

  59. Concurrency Issues Greater Operational Burden Problems with Caching

  60. Sticky Connections & Consistency Additional Available Consistency Models

  61. Sticky Connections

  62. Sticky Connections

  63. Linearizable Sequential Causal Pipelined Random Access Memory Read Your Write

    Monotonic Read Monotonic Write Write From Read Consistency Models CP Consistency AP Consistency
  64. Linearizable Sequential Causal Pipelined Random Access Memory Read Your Write

    Monotonic Read Monotonic Write Write From Read CP Consistency AP Consistency AP Consistency w/ Sticky Connections Consistency Models
  65. - Werner Vogel 2007 “Whether or not read-your-write, session and

    monotonic consistency can be achieved depends in general on the "stickiness" of clients to the server that executes the distributed protocol for them… Using sessions, which are sticky, makes this explicit and provides an exposure level that clients can reason about.”
  66. Building Sticky Connections For Low Latency & Data Intensive Services

  67. Building Sticky Connections

  68. Building Sticky Connections

  69. Persistent Connections Load Balancing Problems No Stickiness Once Connection Breaks

    Problems
  70. Persistent Connections Load Balancing Problems No Stickiness Once Connection Breaks

    Problems
  71. Routing Logic • Cluster Membership • Work Distribution Problems to

    Solve
  72. Routing Logic • Cluster Membership • Work Distribution Problems to

    Solve
  73. Routing Logic • Cluster Membership • Work Distribution Problems to

    Solve
  74. Cluster Membership Dynamic Static

  75. Static Cluster Membership Machine Outage means Partial Service Outage Downtime

    to Add Capacity
  76. Static Cluster Membership Machine Outage means Partial Service Outage Downtime

    to Add Capacity
  77. Static Cluster Membership Machine Outage means Partial Service Outage Downtime

    to Add Capacity
  78. Dynamic Cluster Membership Fault Tolerance

  79. Dynamic Cluster Membership Fault Tolerance

  80. Dynamic Cluster Membership Fault Tolerance

  81. Dynamic Cluster Membership Fault Tolerance

  82. Dynamic Cluster Membership Fault Tolerance

  83. Dynamic Cluster Membership Gossip Protocols Consensus Systems Availability vs Consistency

  84. Consensus Systems for Consistency Paxos ETCD Zookeeper Raft

  85. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  86. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  87. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  88. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  89. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  90. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  91. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  92. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  93. Gossip Protocols for Availability • Assumes Non-Reliable Networks • Information

    Dissemination • Pairwise Communication
  94. Work Distribution Consistent Hashing Distributed Hash Tables Random Placement

  95. Random Placement Write Anywhere Read from Everywhere

  96. Random Placement Write Anywhere Read from Everywhere

  97. Random Placement Write Anywhere Read from Everywhere

  98. Random Placement Write Anywhere Read from Everywhere

  99. Random Placement Write Anywhere Read from Everywhere

  100. Random Placement Write Anywhere Read from Everywhere

  101. Random Placement Write Anywhere Read from Everywhere

  102. Consistent Hashing Deterministic Placement Node A Node B Node C

    NodeD Consistent Hashing & Random Trees: Distributed caching protocols for relieving hot spots on the World Wide Web
  103. Consistent Hashing Deterministic Placement Node A Node B NodeD Consistent

    Hashing & Random Trees: Distributed caching protocols for relieving hot spots on the World Wide Web
  104. Consistent Hashing Deterministic Placement Node A Node B NodeD Consistent

    Hashing & Random Trees: Distributed caching protocols for relieving hot spots on the World Wide Web
  105. Distributed Hash Table Non- Deterministic Placement Node B Node A

    Node C
  106. Distributed Hash Table Non- Deterministic Placement Node B Node A

    Node C
  107. Distributed Hash Table Non- Deterministic Placement Node B Node A

    Node C
  108. Distributed Hash Table Non- Deterministic Placement Node B Node A

    Node C
  109. Distributed Hash Table Non- Deterministic Placement Node B Node A

    Node C
  110. Stateful Services In the Real World

  111. Scuba is a fast, scalable, distributed, in-memory database built at

    Facebook. It is the workhorse behind code regression analysis & bug report, revenue, and performance debugging Fan-out request to all machines in the cluster Compose Results Return Results and Completeness
  112. Scuba is a fast, scalable, distributed, in-memory database built at

    Facebook. It is the workhorse behind code regression analysis & bug report, revenue, and performance debugging Fan-out request to all machines in the cluster Compose Results Return Results and Completeness
  113. Scuba is a fast, scalable, distributed, in-memory database built at

    Facebook. It is the workhorse behind code regression analysis & bug report, revenue, and performance debugging Fan-out request to all machines in the cluster Compose Results Return Results and Completeness
  114. Uber Ringpop is an open- source Node.js library that brings

    application-layer sharding to many of their dispatching platform services. Swim Gossip Protocol Consistent Hashing +
  115. Uber Ringpop is an open- source Node.js library that brings

    application-layer sharding to many of their dispatching platform services. Swim Gossip Protocol Consistent Hashing +
  116. Uber Ringpop is an open- source Node.js library that brings

    application-layer sharding to many of their dispatching platform services. Swim Gossip Protocol Consistent Hashing +
  117. Orleans Cluster Orleans is a runtime and Programming model for

    building distributed systems based on the Actor Model from the eXtreme Computing Group at MSR Gossip Protocol Consistent Hashing + + Distributed Hash Table Actor Actor Actor Actor Actor
  118. Orleans Cluster Orleans is a runtime and Programming model for

    building distributed systems based on the Actor Model from the eXtreme Computing Group at MSR Gossip Protocol Consistent Hashing + + Distributed Hash Table Actor Actor Actor Actor Actor
  119. Orleans Cluster Orleans is a runtime and Programming model for

    building distributed systems based on the Actor Model from the eXtreme Computing Group at MSR Gossip Protocol Consistent Hashing + + Distributed Hash Table Actor Actor Actor Actor Actor
  120. Orleans Cluster Orleans Distributed Hash Table

  121. Orleans Cluster Orleans Distributed Hash Table

  122. Orleans Cluster Orleans Distributed Hash Table

  123. Orleans Cluster Orleans Distributed Hash Table

  124. Orleans Cluster Orleans Distributed Hash Table

  125. Orleans Cluster Orleans Distributed Hash Table

  126. Orleans Cluster Orleans Distributed Hash Table

  127. Orleans Cluster Orleans Distributed Hash Table

  128. Caution Stateful Services Ahead

  129. Unbounded Data Structures Implicit Assumptions are the Killer of Distributed

    Systems
  130. Memory Management Get Ready to Make Friends with the Garbage

    Collector Profiler
  131. Reloading State • First Connection • Recovering From Crashes •

    Deploying New Code
  132. Fast Restarts at Facebook “Our Key Observation is that we

    can decouple the memory lifetime from the process lifetime. When we shutdown a server for a planned upgrade.”
  133. Make Assumptions Explicit for Reliable Distributed Systems

  134. Conclusion Data Locality & Available Consistency Cluster Membership & Work

    Distribution Successful Stateful Real World Systems Caution: Some New Challenges
  135. https://github.com/CaitieM20/ScalingStatefulServices @Caitie Questions

  136. None