Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Consensus: An Introduction to Raft

Consensus: An Introduction to Raft

Patrick Van Stee

July 23, 2013
Tweet

More Decks by Patrick Van Stee

Other Decks in Programming

Transcript

  1. Consensus
    An Introduction to Raft

    View full-size slide

  2. con·sen·sus
    /kənˈsensəs/
    Agreeing upon state across
    distributed processes even
    in the presence of failures.

    View full-size slide

  3. Problem
    • Distributed System
    • Consistency
    • Partition tolerance

    View full-size slide

  4. Solution
    • Quorum
    • Replicated State
    Machines

    View full-size slide

  5. Consensus Data

    View full-size slide

  6. We are sacrificing
    Availability

    View full-size slide

  7. Why not Paxos?
    • Difficult to understand
    • Not practical enough
    to implement

    View full-size slide

  8. Raft
    A Practical Paxos

    View full-size slide

  9. Components
    • Consensus Module
    • State Machine
    • Log

    View full-size slide

  10. Consensus Module
    • Roles: Leader, Follower,
    and Candidate
    • Time is divided into Terms
    • Commands: RequestVote
    and AppendEntries

    View full-size slide

  11. Leader
    Accept commands from clients, commit
    entries, and send heartbeats
    Follower
    Replicate state from leaders and vote for
    candidates
    Candidate
    Start and handle leader elections

    View full-size slide

  12. Follower Candidate Leader
    Times out,
    Starts election
    Times out,
    Restarts election
    Wins election
    Discovers new leader,
    Steps down
    Discovers current leader
    or new leader,
    Steps down

    View full-size slide

  13. Term
    Higher numbers are used to determine
    leaders and check log entries. The term
    is incremented each time an election is
    started. Any command with an old term
    is ignored.

    View full-size slide

  14. Example
    Happy Log Entry

    View full-size slide

  15. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View full-size slide

  16. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader receives command ˒

    View full-size slide

  17. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Leader sends log entries to followers

    View full-size slide

  18. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Majority of followers respond with success

    View full-size slide

  19. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 1
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 1
    Log: [˒]
    Leader sends commit index to followers
    and responds to client

    View full-size slide

  20. Example
    Sad Log Entry

    View full-size slide

  21. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View full-size slide

  22. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader receives command ˒

    View full-size slide

  23. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Leader sends log entries to followers

    View full-size slide

  24. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Majority of followers do not respond

    View full-size slide

  25. A
    B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: [˒]
    Leader continues to retry log entry

    View full-size slide

  26. Example
    Leader Failure

    View full-size slide

  27. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View full-size slide

  28. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    Followers do not receive heartbeat
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View full-size slide

  29. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Candidate
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 1
    Commit Index: 0
    Log: []
    First follower to timeout becomes candidate
    D Role: Follower
    Term: 1
    Commit Index: 0
    Log: []

    View full-size slide

  30. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Candidate
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    Candidate starts election and requests votes
    D Role: Follower
    Term: 2
    Commit Index: 0
    Log: []

    View full-size slide

  31. A B
    C
    Role: Leader
    Term: 1
    Commit Index: 0
    Log: []
    Role: Leader
    Term: 2
    Commit Index: 0
    Log: []
    Role: Follower
    Term: 2
    Commit Index: 0
    Log: []
    Followers respond with votes
    D Role: Follower
    Term: 2
    Commit Index: 0
    Log: []

    View full-size slide

  32. Extras
    • Log safety and
    compaction
    • Cluster changes

    View full-size slide

  33. Real-life Application
    • Distributed lock server
    • Configuration
    management
    • Background job storage

    View full-size slide

  34. Smart People
    • Raft Paper by Diego Ongaro
    and John Ousterhout
    • Raft Implementation
    • ThinkDistributed

    View full-size slide