Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
SWIM: Scalable Weakly Consistent Infection Styl...
Search
Paul Hinze
July 22, 2015
Technology
2
340
SWIM: Scalable Weakly Consistent Infection Style Process Group Membership Protocol
Papers We Love, Chicago
July 22, 2015
Paul Hinze
July 22, 2015
Tweet
Share
More Decks by Paul Hinze
See All by Paul Hinze
Getting Good at System Failure Analysis
phinze
0
440
Applying Graph Theory to Infrastructure As Code
phinze
6
2k
Infrastructure as Code with Terraform and Friends
phinze
2
290
Smoke & Mirrors: The Primitives of High Availability
phinze
1
620
Git: Everybody's Favorite MMO
phinze
0
160
Shut Up and Pipe! Unix-style Object Collaboration in Rack and Vagrant
phinze
0
150
Freighthop: Vagrant on Rails
phinze
0
290
Puppet Modules Are Our Friends
phinze
0
90
Who Needs Clouds?: HA in Your Datacenter
phinze
1
510
Other Decks in Technology
See All in Technology
Data Engineering Study#30 LT資料
tetsuroito
1
560
「現場で活躍するAIエージェント」を実現するチームと開発プロセス
tkikuchi1002
6
1k
FAST導入1年間のふりかえり〜現実を直視し、さらなる進化を求めて〜 / Review of the first year of FAST implementation
wooootack
1
120
データエンジニアリング 4年前と変わったこと、 4年前と変わらないこと
tanakarian
2
360
CSPヘッダー導入で実現するWebサイトの多層防御:今すぐ試せる設定例と運用知見
llamakko
1
210
OpenTelemetry の Log を使いこなそう
biwashi
5
990
東京海上日動におけるセキュアな開発プロセスの取り組み
miyabit
0
130
Microsoft Fabric ガバナンス設計の一歩目を考える
ryomaru0825
1
260
Webの技術とガジェットで那須の子ども達にワクワクを! / IoTLT_20250720
you
PRO
0
120
ゼロから始めるSREの事業貢献 - 生成AI時代のSRE成長戦略と実践 / Starting SRE from Day One
shinyorke
PRO
0
230
分散トレーシングによる コネクティッドカーのデータ処理見える化の試み
thatsdone
0
210
Railsの限界を超えろ!「家族アルバム みてね」の画像・動画の大規模アップロードを支えるアーキテクチャの変遷
ojima_h
3
390
Featured
See All Featured
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
126
53k
Rebuilding a faster, lazier Slack
samanthasiow
83
9.1k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.5k
Why Our Code Smells
bkeepers
PRO
337
57k
A Modern Web Designer's Workflow
chriscoyier
695
190k
Building Adaptive Systems
keathley
43
2.7k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
How to Ace a Technical Interview
jacobian
278
23k
What's in a price? How to price your products and services
michaelherold
246
12k
Transcript
SWIM Scalable Weakly Consistent Infection Style Process Group Membership Protocol
Paul Hinze phinze
Paul Hinze phinze death stare
Armon Dadgar armon creator of Serf and Consul
ma
None
None
Process Group Membership Protocol Who is alive
None
Process Group Membership Protocol
SWIM Scalable Weakly Consistent Infection Style Process Group Membership Protocol
Scalable The SWIM effort is motivated by the unscalability of
traditional heartbeating protocols.
Heartbeating A B C A A B B Failure Detection
+ Membership
Heartbeating
Evaluating Protocols Completeness Speed Accuracy Overhead
Evaluating Protocols Completeness Speed Accuracy Overhead heartbeating Yes Limit *
Interval High Nodes2 !
Key Insight Failure Detection State Updates. Solve separately from
Failure Detection ping! ack! A B C D {B,C,D}
Failure Detection ping! ack! A B C D {B,C,D}
Indirect Ping ping(C)! ack! B C D ping(C)! ping ack
fail {B,C,D}
Indirect Ping ping(C)! B C D ping(C)! fail ...C is
dead! fail
Key Insight Failure Detection State Updates. Solve separately from
Key Insight State Updates Failure Detection Piggyback onto messages.
State Updates ping! (B is dead) ack! (D just joined)
A B C D {B,C} {D}
Infection Style A B C D {B,C} {A,B,D} {A,B} weakly
consistent
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, eventually 1
* Interval High-ish O(N)
Improvements Time Bounded Completeness Increased Accuracy
Completeness A B {B, C, D, ..., N} N
Completeness A B {B, C, D, ..., N} N 1.
Shuffle List 2. Iterate
Completeness Fixed Time
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time
1 * Interval High-ish O(N)
Accuracy B C D ...C is MAYBE dead! A
Accuracy B C D I heard C might be dead.
A
Accuracy B C D I'm not dead! A
Accuracy
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time
1 * Interval High O(N)
Evaluating Protocols Completeness Speed Accuracy Overhead SWIM Yes, fixed time
1 * Interval High O(N)
None
Limitations Update Latency Problem Solution Separate Gossip Timer
Limitations Cannot Handle Network Partitions Problem Solution Track and Retry
Recently Dead Nodes
Limitations No Concept of Graceful Leave (vs Failure) Problem Solution
Broadcast and Tracking of "Intents"
Limitations New Nodes Take Too Long Materialize Initial State Problem
Solution Anti-entropy TCP State Syncs
Limitations No built-in facility for user data Problem Solution Implement
user payloads (ordering via lamport clocks)
Limitations No peer metadata, only IP Addresses Problem Solution Inject
versioned peer metadata into state messages
Limitations No encryption Problem Solution Implement AES-GCM with key rotation
56 nodes 2K nodes Performance
Implementations Memberlist https://github.com/hashicorp/memberlist Serf lib https://github.com/hashicorp/serf Serf http://www.serfdom.io Consul http://www.consul.io
Implementations events, queries, and scripts serf CLI serf lib memberlist
Implementations service discovery, K/V, health checks consul serf lib memberlist
Implementations service discovery, K/V, health checks
Thanks Cloud icons by Julien Deveaux from the Noun Project