Upgrade to Pro — share decks privately, control downloads, hide ads and more …

RAFT: Implementing Distributed Consensus with Erlang

RAFT: Implementing Distributed Consensus with Erlang

Talk from Yow LambdaJam 2014 in Brisbane on RAFT Algorithm and implementing it in Erlang.

Tim McGilchrist

May 08, 2014
Tweet

More Decks by Tim McGilchrist

Other Decks in Programming

Transcript

  1. Goals What I want you to get out of this

    talk?! • Understand core ideas in RAFT! • Erlang / OTP as a tool for building systems! • Build your own implementation
  2. Consensus In a distributed system, agreement among multiple processes on

    a single data value, despite failures. ! ! Once they reach a decision on a value, that decision is final.
  3. Potential Use Case ❖ Configuration Management! ❖ Distributed Transactions! ❖

    Distributed Lock Manager! ❖ DNS and Resource Discovery
  4. RAFT ❖ Design for understandability! ❖ Strong leader! ❖ Practical

    to implement Goals Raft is a consensus algorithm that is designed to be easy to understand.
  5. Messages ❖ RAFT only needs 2 messages.! ❖ RequestVote includes

    term! ❖ AppendEntries includes term and log entries! ❖ Term acts as a logical clock
  6. Leader Leader • Only a single leader within a cluster!

    • Receives commands from client! • Commits commands to the log
  7. Leader Election Follower Candidate! Leader starts up timeout new election

    gets majority of votes step down step down timeout restart election
  8. Log Replication add 1 add 1 add 1 F2 F1

    Leader Executes command
  9. Log Replication add 1 add 4 add 1 add 1

    F2 F1 Leader AppendEntries add 4, index 1 add 4, index 1 Executes command Executes command
  10. RAFT Summary ❖ 2 types of messages, RequestVote and AppendEntries!

    ❖ 3 states, Leader, Follower and Candidate! ❖ Save Entries to persistent log
  11. Erlang ❖ Functional language! ❖ Fundamentally a concurrent language! ❖

    Actor model as basic abstraction! ❖ No shared state between actors! ❖ OTP behaviours like supervisors and gen_fsm! ❖ Location independent message sending
  12. Implementation Overview ❖ github.com/tmcgilchrist/sloop ! ❖ github.com/andrewjstone/ rafter! ❖ Each

    node has 2 supervised behaviours! ❖ gen_fsm implementing the consensus protocol! ❖ gen_server wraps the log store! ❖ passes erlang terms as messages
  13. sloop_fsm ❖ state machine implements leader election and log replication!

    ❖ each state is a function with multiple clauses! !
  14. Summary ❖ Defined Distributed Consensus! ❖ Looked at core ideas

    of RAFT! ❖ Erlang suits distributed systems! ❖ Map Erlang to RAFT