Consensus: An Introduction to Raft

Transcript

1. Consensus
   An Introduction to Raft
   

   View Slide

2. con·sen·sus
   /kənˈsensəs/
   Agreeing upon state across
   distributed processes even
   in the presence of failures.
   

   View Slide

3. Problem
   • Distributed System
   • Consistency
   • Partition tolerance
   

   View Slide

4. Solution
   • Quorum
   • Replicated State
   Machines
   

   View Slide

5. Consensus Data
   —
   

   View Slide

6. We are sacrificing
   Availability
   

   View Slide

7. Why not Paxos?
   • Difficult to understand
   • Not practical enough
   to implement
   

   View Slide

8. Raft
   A Practical Paxos
   

   View Slide

9. Components
   • Consensus Module
   • State Machine
   • Log
   

   View Slide

10. Consensus Module
    • Roles: Leader, Follower,
    and Candidate
    • Time is divided into Terms
    • Commands: RequestVote
    and AppendEntries
    

    View Slide

11. Leader
    Accept commands from clients, commit
    entries, and send heartbeats
    Follower
    Replicate state from leaders and vote for
    candidates
    Candidate
    Start and handle leader elections
    

    View Slide

12. Follower Candidate Leader
    Times out,
    Starts election
    Times out,
    Restarts election
    Wins election
    Discovers new leader,
    Steps down
    Discovers current leader
    or new leader,
    Steps down
    

    View Slide

13. Term
    Higher numbers are used to determine
    leaders and check log entries. The term
    is incremented each time an election is
    started. Any command with an old term
    is ignored.
    

    View Slide

14. Example
    Happy Log Entry
    

    View Slide

15. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    

    View Slide

16. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader receives command ˒
    

    View Slide

17. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Leader sends log entries to followers
    

    View Slide

18. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Majority of followers respond with success
    

    View Slide

19. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 1
    Log: [˒]
    Leader sends commit index to followers
    and responds to client
    

    View Slide

20. Example
    Sad Log Entry
    

    View Slide

21. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    

    View Slide

22. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader receives command ˒
    

    View Slide

23. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader sends log entries to followers
    

    View Slide

24. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Majority of followers do not respond
    

    View Slide

25. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Leader continues to retry log entry
    

    View Slide

26. Example
    Leader Failure
    

    View Slide

27. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    

    View Slide

28. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Followers do not receive heartbeat
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    

    View Slide

29. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Candidate
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    First follower to timeout becomes candidate
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    

    View Slide

30. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Candidate
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    Candidate starts election and requests votes
    D Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    

    View Slide

31. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Leader
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    Followers respond with votes
    D Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    

    View Slide

32. Extras
    • Log safety and
    compaction
    • Cluster changes
    

    View Slide

33. Real-life Application
    • Distributed lock server
    • Configuration
    management
    • Background job storage
    

    View Slide

34. Smart People
    • Raft Paper by Diego Ongaro
    and John Ousterhout
    • Raft Implementation
    • ThinkDistributed
    

    View Slide