Raft: Consensus for Rubyists

Transcript

1. Raft Consensus for Rubyists
2. @vanstee Big Nerd Ranch
3. Coffee Snob Scenario
4. Matt Amy Kim Dan
5. Matt Amy Kim Dan
6. Matt Amy Kim Dan Want to meet for coffee?
7. Matt Amy Kim Dan Totally! I’m in. zzzzz…
8. Matt Amy Kim Dan Ok, let’s meet at Octane.
9. Matt Amy Kim Dan Sounds good. I’ll be there. …
10. Matt Amy Kim Dan Who wants coffee? *wakes up
11. Matt Amy Kim Dan Who wants coffee?
12. Matt Amy Kim Dan Oh, we’re going to Octane.
13. Delicious Coffee Consensus
14. What did we just do?
15. • Leader election • State replication • Partition tolerance
16. Now write an algorithm.
17. Humans > Computers > Congress
18. Why Ruby
19. Multiple db servers? Multiple app servers? Multiple clients?
20. Multiple db servers? Distributed Multiple app servers? Multiple clients?
21. Multiple db servers? Distributed Multiple app servers? Distributed Multiple clients?
22. Multiple db servers? Distributed Multiple app servers? Distributed Multiple clients? Distributed
23. If you build webapps, you build distributed systems.
24. Raft
25. @ongardie Diego Ongaro John Ousterhout and
26. con·sen·sus /kənˈsensəs/ ! Agreeing upon state across distributed processes even in the presence of failures.
27. When should I use it?
28. • Distributed locks • Configuration • Background jobs
29. Why not Paxos?
30. Laying the Groundwork
31. {…} Log State Machine Consensus Module
32. Leader Follower Follower Follower
33. Leader Follower Follower Follower
34. Leader Follower Follower Follower
35. Follower
36. Leader Election Log Replication Safety
37. Follower Candidate Leader
38. Follower Candidate Leader
39. Follower Candidate Leader Times out, Starts election
40. Follower Candidate Leader Times out, Starts election Wins election
41. Follower Candidate Leader Times out, Starts election Times out, Restarts election Wins election
42. Follower Candidate Leader Times out, Starts election Times out, Restarts election Wins election Discovers current or new leader, Steps down
43. Follower Candidate Leader Times out, Starts election Times out, Restarts election Wins election Discovers new leader, Steps down Discovers current or new leader, Steps down
44. RequestVote Used by candidates to ask for votes during an election. Log information included for comparison. ! AppendEntries Used by leaders to tell followers which entries to replicate and commit. Also used as a heartbeat to remain leader. Follower Candidate Vote for me please. Follower Here are some entries for your log. Oh, and btw I’m still the leader. Leader
45. Term Higher numbers are used to determine leaders and check log entries. The term is incremented each time an election is started. Any command with an old term is ignored. Term 1 Term 2 Term 3 Election Normal Operation Split vote
46. Follower Follower Follower Follower
47. Candidate Follower Follower Follower Votes Times out, starts election
48. Candidate Follower Follower Follower Votes Votes for self
49. Candidate Follower Follower Follower Votes Sends out RequestVotes
50. Candidate Follower Follower Votes Responds with success
51. Leader Follower Votes Yay! I won! Follower
52. Leader Election Log Replication Safety
53. Log Entries Explained
54. SET X = 1 SET Y = 2 SET Z = 3 SET X = 4 SET X = 5 1 1 1 2 2 SET X = 1 SET Y = 2 SET Z = 3 SET X = 4 1 1 1 2 Leader Follower 1 2 3 4 5 6
55. { :entries => [{ 4 => 'SET X = 4' }], :term => 1, :prev_log_term => 1, :prev_log_index => 3, :leader_commit => 3, :leader_id => '192.168.1.101/7238' } APPEND ENTRIES EXAMPLE
56. SET X = 1 SET Y = 2 SET Z = 3 1 1 1 Before 1 1 1 1 After SET X = 1 SET Y = 2 SET Z = 3 SET X = 4 1 2 3 4 5 6 1 2 3 4 5 6
57. Log entries are always committed in the same order and are never uncommitted.
58. Happy Log Entry
59. Leader Follower Follower
60. Leader Follower Follower
61. Leader Follower Follower
62. Leader Follower Follower
63. Leader Follower Follower
64. Sad Log Entry
65. Leader Follower Follower
66. Leader Follower Follower
67. Leader Follower Follower
68. Leader Follower Follower
69. Leader Follower Leader
70. Leader Follower Leader
71. Leader Follower Leader
72. Leader Follower Leader
73. Leader Follower Leader
74. Follower Follower Follower Leader
75. Facepalm Log Entry
76. Leader Follower Follower
77. Leader Follower Follower
78. Leader Follower Follower
79. Leader Follower Follower
80. Leader Follower Follower
81. Leader Follower Follower
82. Leader Follower Leader
83. How do we guard against losing log entries?
84. Leader Election Log Replication Safety
85. Only cast votes for nodes with logs that contain at least as many entries as your own. 1
86. New leaders must commit a log entry form their new term before committing old entries. 2
87. Bonus Round
88. • Cluster changes • Log compaction • Client specifics
89. Why Ruby
91. Ruby is great for expressing complex problems.
92. Then why doesn’t the academic community love it?
93. • Community • Understandability • Learning Material
94. • Read Papers • Go To Conferences • Talk To People
95. Raft Paper Raft Implementations Raft Website ThinkDistributed Ricon Google Scholar
96. Question Time Make sure I repeat your questions.