Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
RAFT: Implementing Distributed Consensus with Erlang
Search
Tim McGilchrist
May 08, 2014
Programming
4
640
RAFT: Implementing Distributed Consensus with Erlang
Talk from Yow LambdaJam 2014 in Brisbane on RAFT Algorithm and implementing it in Erlang.
Tim McGilchrist
May 08, 2014
Tweet
Share
More Decks by Tim McGilchrist
See All by Tim McGilchrist
Dependently Typed State Machines
lambda_foo
0
130
Code reuse through polymorphic variants
lambda_foo
1
120
Either Error Success
lambda_foo
0
100
Idris States: Dependent types, not just for vectors?
lambda_foo
0
160
Other Decks in Programming
See All in Programming
How to use Macrobenchmark
veronikapj
0
160
わかりやすい正解を捨てて、コトに向き合う - スクラムフェス金沢2024 スポンサーセッション
yusukekokubo
0
170
Modern Angular: Renovation for Your Applications
manfredsteyer
PRO
0
140
Jetpack for KMP
fornewid
1
290
社内 LT 会を発足し、アウトプット文化を醸成させるために考えたこと・やったこと / Starting internal LT meetings and fostering an output culture
mackey0225
3
120
DynamoDB コスト最適化っぽいことの基本 with Terraform
kuro_kurorrr
2
250
Polarsの成長: v0.14からv1.0までの変遷と今後の展望
zerebom
1
350
最近追加した型の紹介とその振り返り
aki19035vc
0
180
リハビリmruby
kishima
1
160
20240706_CDKConf
takuyay0ne
0
1.2k
データカタログ運用物語 〜令和6年夏の理想と現実〜
kuro_kurorrr
0
110
Harnessing Large Language Models for Training-free Video Anomaly Detection
tereka114
1
1.3k
Featured
See All Featured
Clear Off the Table
cherdarchuk
89
320k
Building a Scalable Design System with Sketch
lauravandoore
458
32k
Building Flexible Design Systems
yeseniaperezcruz
323
37k
Mobile First: as difficult as doing things right
swwweet
219
8.8k
Learning to Love Humans: Emotional Interface Design
aarron
269
39k
How to train your dragon (web standard)
notwaldorf
79
5.5k
Speed Design
sergeychernyshev
9
270
Reflections from 52 weeks, 52 projects
jeffersonlam
346
19k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
245
1.2M
Building Better People: How to give real-time feedback that sticks.
wjessup
357
18k
Designing Experiences People Love
moore
136
23k
How to name files
jennybc
67
96k
Transcript
Tim McGilchrist @lambda_foo lambdafoo.com Raft Implementing Distributed Consensus with Erlang
Outline ❖ Goals! ❖ The consensus problem! ❖ Outline RAFT
algorithm! ❖ Implementing in Erlang
Goals What I want you to get out of this
talk?! • Understand core ideas in RAFT! • Erlang / OTP as a tool for building systems! • Build your own implementation
Consensus In a distributed system, agreement among multiple processes on
a single data value, despite failures. ! ! Once they reach a decision on a value, that decision is final.
Potential Use Case ❖ Configuration Management! ❖ Distributed Transactions! ❖
Distributed Lock Manager! ❖ DNS and Resource Discovery
RAFT ❖ Design for understandability! ❖ Strong leader! ❖ Practical
to implement Goals Raft is a consensus algorithm that is designed to be easy to understand.
Messages ❖ RAFT only needs 2 messages.! ❖ RequestVote includes
term! ❖ AppendEntries includes term and log entries! ❖ Term acts as a logical clock
States 3 states a node can be in. Follower Candidate!
Leader
Leader Leader • Only a single leader within a cluster!
• Receives commands from client! • Commits commands to the log
Follower Follower • Appends commands to log! • Votes for
candidates! • Otherwise passive
Candidate Candidate! • Initiates Election! • Coordinates Votes
Leader Election Follower Candidate! Leader starts up timeout new election
gets majority of votes step down step down timeout restart election
Log Replication add 1 F2 F1 Leader AppendEntries add 1,
index 0 add 1, index 0
Log Replication add 1 add 1 add 1 F2 F1
Leader Ok OK
Log Replication add 1 add 1 add 1 F2 F1
Leader Executes command
Log Replication add 1 add 4 add 1 add 1
F2 F1 Leader AppendEntries add 4, index 1 add 4, index 1 Executes command Executes command
RAFT Summary ❖ 2 types of messages, RequestVote and AppendEntries!
❖ 3 states, Leader, Follower and Candidate! ❖ Save Entries to persistent log
Erlang ❖ Functional language! ❖ Fundamentally a concurrent language! ❖
Actor model as basic abstraction! ❖ No shared state between actors! ❖ OTP behaviours like supervisors and gen_fsm! ❖ Location independent message sending
Implementation Overview ❖ github.com/tmcgilchrist/sloop ! ❖ github.com/andrewjstone/ rafter! ❖ Each
node has 2 supervised behaviours! ❖ gen_fsm implementing the consensus protocol! ❖ gen_server wraps the log store! ❖ passes erlang terms as messages
sloop_fsm ❖ state machine implements leader election and log replication!
❖ each state is a function with multiple clauses! !
Supervisors sloop_sup sloop_fsm sloop_store sloop_state sender
Implementations ! ❖ raftconsensus.github.io! ❖ github.com/tmcgilchrist/sloop ! ❖ github.com/andrewjstone/rafter !
❖ github.com/goraft/raft
Summary ❖ Defined Distributed Consensus! ❖ Looked at core ideas
of RAFT! ❖ Erlang suits distributed systems! ❖ Map Erlang to RAFT
Thanks!