Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Consensus: An Introduction to Raft

Consensus: An Introduction to Raft

Patrick Van Stee

July 23, 2013
Tweet

More Decks by Patrick Van Stee

Other Decks in Programming

Transcript

  1. Consensus
    An Introduction to Raft

    View Slide

  2. con·sen·sus
    /kənˈsensəs/
    Agreeing upon state across
    distributed processes even
    in the presence of failures.

    View Slide

  3. Problem
    • Distributed System
    • Consistency
    • Partition tolerance

    View Slide

  4. Solution
    • Quorum
    • Replicated State
    Machines

    View Slide

  5. Consensus Data

    View Slide

  6. We are sacrificing
    Availability

    View Slide

  7. Why not Paxos?
    • Difficult to understand
    • Not practical enough
    to implement

    View Slide

  8. Raft
    A Practical Paxos

    View Slide

  9. Components
    • Consensus Module
    • State Machine
    • Log

    View Slide

  10. Consensus Module
    • Roles: Leader, Follower,
    and Candidate
    • Time is divided into Terms
    • Commands: RequestVote
    and AppendEntries

    View Slide

  11. Leader
    Accept commands from clients, commit
    entries, and send heartbeats
    Follower
    Replicate state from leaders and vote for
    candidates
    Candidate
    Start and handle leader elections

    View Slide

  12. Follower Candidate Leader
    Times out,
    Starts election
    Times out,
    Restarts election
    Wins election
    Discovers new leader,
    Steps down
    Discovers current leader
    or new leader,
    Steps down

    View Slide

  13. Term
    Higher numbers are used to determine
    leaders and check log entries. The term
    is incremented each time an election is
    started. Any command with an old term
    is ignored.

    View Slide

  14. Example
    Happy Log Entry

    View Slide

  15. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View Slide

  16. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader receives command ˒

    View Slide

  17. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Leader sends log entries to followers

    View Slide

  18. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Majority of followers respond with success

    View Slide

  19. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 1
    Log: [˒]
    Leader sends commit index to followers
    and responds to client

    View Slide

  20. Example
    Sad Log Entry

    View Slide

  21. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View Slide

  22. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader receives command ˒

    View Slide

  23. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader sends log entries to followers

    View Slide

  24. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Majority of followers do not respond

    View Slide

  25. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Leader continues to retry log entry

    View Slide

  26. Example
    Leader Failure

    View Slide

  27. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View Slide

  28. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Followers do not receive heartbeat
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View Slide

  29. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Candidate
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    First follower to timeout becomes candidate
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View Slide

  30. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Candidate
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    Candidate starts election and requests votes
    D Role: Follower
    Term: 2
    Commit Index: 0
    Log: []

    View Slide

  31. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Leader
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    Followers respond with votes
    D Role: Follower
    Term: 2
    Commit Index: 0
    Log: []

    View Slide

  32. Extras
    • Log safety and
    compaction
    • Cluster changes

    View Slide

  33. Real-life Application
    • Distributed lock server
    • Configuration
    management
    • Background job storage

    View Slide

  34. Smart People
    • Raft Paper by Diego Ongaro
    and John Ousterhout
    • Raft Implementation
    • ThinkDistributed

    View Slide