Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
SWIM: Scalable Weakly Consistent Infection Style Process Group Membership Protocol
Search
Paul Hinze
July 22, 2015
Technology
2
280
SWIM: Scalable Weakly Consistent Infection Style Process Group Membership Protocol
Papers We Love, Chicago
July 22, 2015
Paul Hinze
July 22, 2015
Tweet
Share
More Decks by Paul Hinze
See All by Paul Hinze
Getting Good at System Failure Analysis
phinze
0
420
Applying Graph Theory to Infrastructure As Code
phinze
6
1.9k
Infrastructure as Code with Terraform and Friends
phinze
2
270
Smoke & Mirrors: The Primitives of High Availability
phinze
1
410
Git: Everybody's Favorite MMO
phinze
0
100
Shut Up and Pipe! Unix-style Object Collaboration in Rack and Vagrant
phinze
0
110
Freighthop: Vagrant on Rails
phinze
0
250
Puppet Modules Are Our Friends
phinze
0
64
Who Needs Clouds?: HA in Your Datacenter
phinze
1
480
Other Decks in Technology
See All in Technology
Postman v10リリース後を振り返る / Looking back at Postman v10 after release
yokawasa
1
160
TechFeed Experts Night#27 〜 フロントエンドフレームワーク最前線 (Svelte)
baseballyama
1
530
【NW X Security JAWS#3】L3-4:AWS環境のIPv6移行に向けて知っておきたいこと
shotashiratori
0
340
MLOpsの「壁」を乗り越える、LINEヤフーの Data Quality as Code
lycorptech_jp
PRO
5
530
アクセス制御にまつわる改善 / Improving access control
itkq
0
550
VSCodeの拡張機能を作っている話
ebarakazuhiro
1
510
反実仮想機械学習とは何か
usaito
PRO
11
4.7k
どうするコスト最適化のトレードオフ
tetsuyaooooo
1
530
本当のAWS基礎
toru_kubota
0
520
AOAI をきっかけに 社内の Azure 管理を見直した話
recruitengineers
PRO
1
300
プロンプトエンジニアリングでがんばらない-Agentic Workflow へ-近藤憲児
kenjikondobai
3
810
開発生産性大幅アップ!Postman VS Code拡張機能
nagix
2
380
Featured
See All Featured
Scaling GitHub
holman
457
140k
Building Effective Engineering Teams - LeadDev
addyosmani
28
1.8k
A designer walks into a library…
pauljervisheath
200
23k
Atom: Resistance is Futile
akmur
259
25k
Building a Scalable Design System with Sketch
lauravandoore
456
32k
Stop Working from a Prison Cell
hatefulcrawdad
266
19k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
244
20k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
14
1.6k
Navigating Team Friction
lara
178
13k
For a Future-Friendly Web
brad_frost
172
9k
A Tale of Four Properties
chriscoyier
151
22k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
14
1.5k
Transcript
SWIM Scalable Weakly Consistent Infection Style Process Group Membership Protocol
Paul Hinze phinze
Paul Hinze phinze death stare
Armon Dadgar armon creator of Serf and Consul
ma
None
None
Process Group Membership Protocol Who is alive
None
Process Group Membership Protocol
SWIM Scalable Weakly Consistent Infection Style Process Group Membership Protocol
Scalable The SWIM effort is motivated by the unscalability of
traditional heartbeating protocols.
Heartbeating A B C A A B B Failure Detection
+ Membership
Heartbeating
Evaluating Protocols Completeness Speed Accuracy Overhead
Evaluating Protocols Completeness Speed Accuracy Overhead heartbeating Yes Limit *
Interval High Nodes2 !
Key Insight Failure Detection State Updates. Solve separately from
Failure Detection ping! ack! A B C D {B,C,D}
Failure Detection ping! ack! A B C D {B,C,D}
Indirect Ping ping(C)! ack! B C D ping(C)! ping ack
fail {B,C,D}
Indirect Ping ping(C)! B C D ping(C)! fail ...C is
dead! fail
Key Insight Failure Detection State Updates. Solve separately from
Key Insight State Updates Failure Detection Piggyback onto messages.
State Updates ping! (B is dead) ack! (D just joined)
A B C D {B,C} {D}
Infection Style A B C D {B,C} {A,B,D} {A,B} weakly
consistent
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, eventually 1
* Interval High-ish O(N)
Improvements Time Bounded Completeness Increased Accuracy
Completeness A B {B, C, D, ..., N} N
Completeness A B {B, C, D, ..., N} N 1.
Shuffle List 2. Iterate
Completeness Fixed Time
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time
1 * Interval High-ish O(N)
Accuracy B C D ...C is MAYBE dead! A
Accuracy B C D I heard C might be dead.
A
Accuracy B C D I'm not dead! A
Accuracy
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time
1 * Interval High O(N)
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time
1 * Interval High O(N)
None
Limitations Update Latency Problem Solution Separate Gossip Timer
Limitations Cannot Handle Network Partitions Problem Solution Track and Retry
Recently Dead Nodes
Limitations No Concept of Graceful Leave (vs Failure) Problem Solution
Broadcast and Tracking of "Intents"
Limitations New Nodes Take Too Long Materialize Initial State Problem
Solution Anti-entropy TCP State Syncs
Limitations No built-in facility for user data Problem Solution Implement
user payloads (ordering via lamport clocks)
Limitations No peer metadata, only IP Addresses Problem Solution Inject
versioned peer metadata into state messages
Limitations No encryption Problem Solution Implement AES-GCM with key rotation
56 nodes 2K nodes Performance
Implementations Memberlist https://github.com/hashicorp/memberlist Serf lib https://github.com/hashicorp/serf Serf http://www.serfdom.io Consul http://www.consul.io
Implementations events, queries, and scripts serf CLI serf lib memberlist
Implementations service discovery, K/V, health checks consul serf lib memberlist
Implementations service discovery, K/V, health checks
Thanks Cloud icons by Julien Deveaux from the Noun Project