Consensus: An Introduction to Raft

Consensus: An Introduction to Raft

B6a8f005f39d23ffc930508ac9da68b9?s=128

Patrick Van Stee

July 23, 2013
Tweet

Transcript

  1. Consensus An Introduction to Raft

  2. con·sen·sus /kənˈsensəs/ Agreeing upon state across distributed processes even in

    the presence of failures.
  3. Problem • Distributed System • Consistency • Partition tolerance

  4. Solution • Quorum • Replicated State Machines

  5. Consensus Data —

  6. We are sacrificing Availability

  7. Why not Paxos? • Difficult to understand • Not practical

    enough to implement
  8. Raft A Practical Paxos

  9. Components • Consensus Module • State Machine • Log

  10. Consensus Module • Roles: Leader, Follower, and Candidate • Time

    is divided into Terms • Commands: RequestVote and AppendEntries
  11. Leader Accept commands from clients, commit entries, and send heartbeats

    Follower Replicate state from leaders and vote for candidates Candidate Start and handle leader elections
  12. Follower Candidate Leader Times out, Starts election Times out, Restarts

    election Wins election Discovers new leader, Steps down Discovers current leader or new leader, Steps down
  13. Term Higher numbers are used to determine leaders and check

    log entries. The term is incremented each time an election is started. Any command with an old term is ignored.
  14. Example Happy Log Entry

  15. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] Role: Follower Term: 1 Commit Index: 0 Log: []
  16. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] Leader receives command ˒
  17. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Leader sends log entries to followers
  18. A B C Role: Leader Term: 1 Commit Index: 1

    Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Majority of followers respond with success
  19. A B C Role: Leader Term: 1 Commit Index: 1

    Log: [˒] Role: Follower Term: 1 Commit Index: 1 Log: [˒] Role: Follower Term: 1 Commit Index: 1 Log: [˒] Leader sends commit index to followers and responds to client
  20. Example Sad Log Entry

  21. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] Role: Follower Term: 1 Commit Index: 0 Log: []
  22. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] Leader receives command ˒
  23. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [] Leader sends log entries to followers
  24. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [] Majority of followers do not respond
  25. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Role: Follower Term: 1 Commit Index: 0 Log: [˒] Leader continues to retry log entry
  26. Example Leader Failure

  27. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] D Role: Follower Term: 1 Commit Index: 0 Log: []
  28. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] Followers do not receive heartbeat D Role: Follower Term: 1 Commit Index: 0 Log: []
  29. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [] Role: Candidate Term: 2 Commit Index: 0 Log: [] Role: Follower Term: 1 Commit Index: 0 Log: [] First follower to timeout becomes candidate D Role: Follower Term: 1 Commit Index: 0 Log: []
  30. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [] Role: Candidate Term: 2 Commit Index: 0 Log: [] Role: Follower Term: 2 Commit Index: 0 Log: [] Candidate starts election and requests votes D Role: Follower Term: 2 Commit Index: 0 Log: []
  31. A B C Role: Leader Term: 1 Commit Index: 0

    Log: [] Role: Leader Term: 2 Commit Index: 0 Log: [] Role: Follower Term: 2 Commit Index: 0 Log: [] Followers respond with votes D Role: Follower Term: 2 Commit Index: 0 Log: []
  32. Extras • Log safety and compaction • Cluster changes

  33. Real-life Application • Distributed lock server • Configuration management •

    Background job storage
  34. Smart People • Raft Paper by Diego Ongaro and John

    Ousterhout • Raft Implementation • ThinkDistributed