SWIM: Scalable Weakly Consistent Infection Style Process Group Membership Protocol

Papers We Love, Chicago
July 22, 2015

Transcript

1. SWIM Scalable Weakly Consistent Infection Style Process Group Membership Protocol
2. Paul Hinze phinze
3. Paul Hinze phinze death stare
4. Armon Dadgar armon creator of Serf and Consul
5. ma
8. Process Group Membership Protocol Who is alive
10. Process Group Membership Protocol
11. SWIM Scalable Weakly Consistent Infection Style Process Group Membership Protocol
12. Scalable The SWIM effort is motivated by the unscalability of traditional heartbeating protocols.
13. Heartbeating A B C A A B B Failure Detection + Membership
14. Heartbeating
15. Evaluating Protocols Completeness Speed Accuracy Overhead
16. Evaluating Protocols Completeness Speed Accuracy Overhead heartbeating Yes Limit * Interval High Nodes2 !
17. Key Insight Failure Detection State Updates. Solve separately from
18. Failure Detection ping! ack! A B C D {B,C,D}
19. Failure Detection ping! ack! A B C D {B,C,D}
20. Indirect Ping ping(C)! ack! B C D ping(C)! ping ack fail {B,C,D}
21. Indirect Ping ping(C)! B C D ping(C)! fail ...C is dead! fail
22. Key Insight Failure Detection State Updates. Solve separately from
23. Key Insight State Updates Failure Detection Piggyback onto messages.
24. State Updates ping! (B is dead) ack! (D just joined) A B C D {B,C} {D}
25. Infection Style A B C D {B,C} {A,B,D} {A,B} weakly consistent
26. Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, eventually 1 * Interval High-ish O(N)
27. Improvements Time Bounded Completeness Increased Accuracy
28. Completeness A B {B, C, D, ..., N} N
29. Completeness A B {B, C, D, ..., N} N 1. Shuffle List 2. Iterate
30. Completeness Fixed Time
31. Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time 1 * Interval High-ish O(N)
32. Accuracy B C D ...C is MAYBE dead! A
33. Accuracy B C D I heard C might be dead. A
34. Accuracy B C D I'm not dead! A
35. Accuracy
36. Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time 1 * Interval High O(N)
37. Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time 1 * Interval High O(N)
39. Limitations Update Latency Problem Solution Separate Gossip Timer
40. Limitations Cannot Handle Network Partitions Problem Solution Track and Retry Recently Dead Nodes
41. Limitations No Concept of Graceful Leave (vs Failure) Problem Solution Broadcast and Tracking of "Intents"
42. Limitations New Nodes Take Too Long Materialize Initial State Problem Solution Anti-entropy TCP State Syncs
43. Limitations No built-in facility for user data Problem Solution Implement user payloads (ordering via lamport clocks)
44. Limitations No peer metadata, only IP Addresses Problem Solution Inject versioned peer metadata into state messages
45. Limitations No encryption Problem Solution Implement AES-GCM with key rotation
46. 56 nodes 2K nodes Performance
47. Implementations Memberlist
 https://github.com/hashicorp/memberlist Serf lib
 https://github.com/hashicorp/serf Serf
 http://www.serfdom.io Consul
 http://www.consul.io
48. Implementations events, queries, and scripts serf CLI serf lib memberlist
49. Implementations service discovery, K/V, health checks consul serf lib memberlist
50. Implementations service discovery, K/V, health checks
51. Thanks Cloud icons by Julien Deveaux from the Noun Project