Consensus is for Everyone: An Introduction

The 2019 version of my introduction to consensus algorithms, geared more towards generalist programmers.

1. Tess Rinearson, @_tessr consensus is for everyone an introduction November 2019
2. https://www.bloomberg.com/news/articles/2017-12-21/crypto-craze-sees-long-island-iced-tea-rename-as-long-blockchain
3. Consensus is for Everyone: An Introduction to Distributed Consensus 3 I should rewrite the financial system.
4. 4 blockchains distsys
6. 6 x x x x x GOAL: Consistency across all nodes
8. 7 x? x? x? x?
10. 8 ok ✔ ok ✔ ok ✔ ok ✔
11. 8 ok ✔ ok ✔ ok ✔ ok ✔ x
12. 8 ok ✔ ok ✔ ok ✔ ok ✔ x x x x x
13. Consensus is for Everyone: An Introduction to Distributed Consensus 9 this works, but it’s kind of terrible
14. 10 why does this node get to suggest a value? what if this one wants to suggest a value? or this one?
15. 11 what if there’s a network partition?
16. 12 what if we have A LOT of nodes???
17. 12 what if we have A LOT of nodes???
18. 13 what if one of the nodes goes offline?
19. 13 what if one of the nodes goes offline?
20. Consensus is for Everyone: An Introduction to Distributed Consensus also a lot of other problems that computer scientists get excited about (clock skew!!!) 14
21. Consensus is for Everyone: An Introduction to Distributed Consensus 15 what is a consensus algorithm?
22. Consensus is for Everyone: An Introduction to Distributed Consensus every node must decide on a value the value must be the same for every node that value must be produced by some process 16
23. Consensus is for Everyone: An Introduction to Distributed Consensus every node must decide on a value the value must be the same for every node that value must be produced by some process 16 termination
24. Consensus is for Everyone: An Introduction to Distributed Consensus every node must decide on a value the value must be the same for every node that value must be produced by some process 16 termination agreement
25. Consensus is for Everyone: An Introduction to Distributed Consensus every node must decide on a value the value must be the same for every node that value must be produced by some process 16 termination agreement validity
26. Consensus is for Everyone: An Introduction to Distributed Consensus 17 Paxos
27. Consensus is for Everyone: An Introduction to Distributed Consensus 18 no one actually understands Paxos
28. Consensus is for Everyone: An Introduction to Distributed Consensus “We found few people who were comfortable with Paxos, even among seasoned researchers. We… were not able to understand the complete protocol until after reading several simplified explanations and designing our own alternative protocol, a process that took almost a year.” (from the Raft paper) 19
29. Consensus is for Everyone: An Introduction to Distributed Consensus 20 • a lot of behavior is unspecified • the variants that make it more complete can be very complex the trouble with Paxos
30. Consensus is for Everyone: An Introduction to Distributed Consensus 21 you can understand (basic) Paxos
31. Consensus is for Everyone: An Introduction to Distributed Consensus 22 Naive Solution • proposer node sends new value to every other node in the system Paxos • proposer node sends out ballots that other nodes can vote on • each round of consensus requires two rounds of voting
32. Consensus is for Everyone: An Introduction to Distributed Consensus 23 value: ? prepared: n accepted: n
33. 24 Step 1: Prepare the ballot number nʹ value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n
34. 24 prepare(nʹ) prepare(nʹ) prepare(nʹ) prepare(nʹ) Step 1: Prepare the ballot number nʹ prepare(nʹ) value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n
35. 25 Step 2: Acknowledge ballot value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
36. 25 ack(nʹ, n) ack(nʹ, n) ack(nʹ, n) ack(nʹ, n) Step 2: Acknowledge ballot ack(nʹ, n) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
37. 26 I have received acks from 5/5 nodes Step 3a: Count acks value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
38. 27 Step 3b: Tell nodes to accept the value value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
39. 27 accept(nʹ, x) accept(nʹ, x) accept(nʹ, x) accept(nʹ, x) Step 3b: Tell nodes to accept the value accept(nʹ, x) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
40. 28 Step 4: Nodes adopt the value value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ
41. Consensus is for Everyone: An Introduction to Distributed Consensus what if a node goes offline? 29
42. 30 Step 1: Prepare the ballot number nʹ value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n
43. 30 prepare(nʹ) prepare(nʹ) prepare(nʹ) prepare(nʹ) Step 1: Prepare the ballot number nʹ prepare(nʹ) value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n
44. 30 prepare(nʹ) prepare(nʹ) prepare(nʹ) prepare(nʹ) Step 1: Prepare the ballot number nʹ prepare(nʹ) value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n
45. 31 Step 2: Acknowledge ballot value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
46. 31 ack(nʹ, n) ack(nʹ, n) ack(nʹ, n) Step 2: Acknowledge ballot ack(nʹ, n) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
47. 32 I have received acks from 4/5 nodes Step 3a: Count acks value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n
48. Consensus is for Everyone: An Introduction to Distributed Consensus we received acks from a majority of nodes, so we can proceed 33
49. Consensus is for Everyone: An Introduction to Distributed Consensus what if two nodes propose values at the same time? 34
50. 35 Step 1: Prepare the ballot number nʹ value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n
51. 35 prepare(nʹ) prepare(nʹ) prepare(nʹ) prepare(nʹ) Step 1: Prepare the ballot number nʹ prepare(nʹ) value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n
52. Consensus is for Everyone: An Introduction to Distributed Consensus 36 A I’m Proposer A, so my ballot are numbered with <n, a> B I’m Proposer B, so my ballot are numbered with <n, b>
53. 37 Step 1: Prepare the ballot number nʹ value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n B A
54. 37 prepare(<nʹ, a>) Step 1: Prepare the ballot number nʹ prepare(<nʹ, b>) value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n B A
55. 38 Step 2: Acknowledge ballot value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n B A
56. 38 ack(<nʹ, b>,n) ack(<nʹ,b>, n) ack(<nʹ,b>, n) Step 2: Acknowledge ballot ack(<nʹ,b>, n) value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n B A ack(<nʹ,b>, n)
57. 39 I have received acks from 5/5 nodes Step 3a: Count acks value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n I have received acks from 0/5 nodes, so I must stop value: prepared: nʹ,b accepted: n
58. 40 Step 3b: Tell nodes to accept the value value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n
59. 40 accept(<nʹ,b> x) accept(<nʹ,b>, x) accept(<nʹ,b>, x) accept(<nʹ,b>, x) Step 3b: Tell nodes to accept the value accept(<nʹ,b>, x) value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n value: prepared: nʹ,b accepted: n
60. 41 Step 4: Nodes adopt the value value: x prepared: nʹ,b accepted: nʹ,b value: x prepared: nʹ,b accepted: nʹ,b value: x prepared: nʹ,b accepted: nʹ,b value: x prepared: nʹ,b accepted: nʹ,b value: x prepared: nʹ,b accepted: nʹ,b
61. Consensus is for Everyone: An Introduction to Distributed Consensus we maintained consistency across all the nodes 42
62. Consensus is for Everyone: An Introduction to Distributed Consensus 43 Raft
63. 44 I’m the leader! I get to pick the next value.
64. 45 I want the next value to be x, so I’ll tell the leader.
65. 45 I want the next value to be x, so I’ll tell the leader. “please adopt x”
66. Consensus is for Everyone: An Introduction to Distributed Consensus who’s the leader? 46
68. 47 heartbeat
69. 48 dead
70. 49 dead
71. 50 dead
72. 50 dead “vote pls?” “vote pls?” “vote pls?”
73. 51 dead
74. 51 dead vote vote vote
75. 52 the replicated log it’s replicated here and here and here and here
76. 53 A Replicated Log value: x term: 1 index: 1 value: y term: 1 index: 2 value: z term: 2 index: 3 value: a term: 3 index: 4 represents the period between each election the position of this entry within the log this is an entry
77. Consensus is for Everyone: An Introduction to Distributed Consensus 54 • If two entries in different logs have the same index and term, then they store the same command. • If two entries in different logs have the same index and term, then the logs are identical in all preceding entries. the Log-Matching Property
78. 55 term: 1 index: 2 term: 1 index: 1 term: 1 index: 3 term: 1 index: 2 term: 1 index: 1
79. 55 term: 1 index: 2 term: 1 index: 1 term: 1 index: 3 term: 1 index: 2 term: 1 index: 1 I’d like to append to the log. My most recent entry has term 1 and index 3.
80. 55 term: 1 index: 2 term: 1 index: 1 term: 1 index: 3 term: 1 index: 2 term: 1 index: 1 I’d like to append to the log. My most recent entry has term 1 and index 3. Uh oh, I’m missing that entry. Can you replay your log from after index 2?
81. Consensus Paxos Raft
82. Consensus is for Everyone: An Introduction to Distributed Consensus everyone was following the rules: these are non-Byzantine failures 57
83. Consensus is for Everyone: An Introduction to Distributed Consensus 58 the Byzantine Generals
84. Consensus is for Everyone: An Introduction to Distributed Consensus 59
85. Consensus is for Everyone: An Introduction to Distributed Consensus 59 “attack” “attack” also, don’t attack
86. Consensus is for Everyone: An Introduction to Distributed Consensus 60 how can a system tolerate Byzantine faults?
87. Consensus is for Everyone: An Introduction to Distributed Consensus 61 PBFT
88. Consensus is for Everyone: An Introduction to Distributed Consensus 62 request pre-prepare prepare commit reply
89. Consensus is for Everyone: An Introduction to Distributed Consensus 63
90. Consensus Paxos Raft Byzantine-safe PBFT
91. Consensus is for Everyone: An Introduction to Distributed Consensus PBFT keeps working even if 1/3 of the nodes are malicious 65
92. Consensus is for Everyone: An Introduction to Distributed Consensus 66 problem: Sybil attacks
93. Consensus is for Everyone: An Introduction to Distributed Consensus classic PBFT requires you to vet your member set (no good for public networks!) 67
94. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe
95. Consensus is for Everyone: An Introduction to Distributed Consensus 69 how does Bitcoin handle consensus?
96. 70 If I choose a nonce x, will hash(x, transactions) yield a small enough hash?
97. 71 validator node miner
98. 72 nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: a19 hash: 8e43 nonce: 89 hash: 9876 nonce: abc hash: 89a0
99. nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: a19 hash: 8e43 nonce: 89 hash: 9876 nonce: abc hash: 89a0 * in Bitcoin, nodes gossip; not every node is connected to every other node
100. nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: a19 hash: 8e43 nonce: 89 hash: 9876 nonce: abc hash: 89a0 block, hash block, hash block, hash block, hash * in Bitcoin, nodes gossip; not every node is connected to every other node
101. 74 nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: 89 hash: 9876 nonce: abc hash: 89a0 block, hash block, hash block, hash block, hash nonce: a19 hash: 8e43
102. 74 nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: 89 hash: 9876 nonce: abc hash: 89a0 block, hash block, hash block, hash block, hash LGTM ✔ LGTM ✔ nonce: a19 hash: 8e43 LGTM ✔ LGTM ✔
103. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe Proof of Work
104. Consensus is for Everyone: An Introduction to Distributed Consensus every node must decide on a value the value must be the same for every node that value must be produced by some process 76 termination agreement validity Reminder
105. nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: bc8 hash: 0011 nonce: 89 hash: 9876 nonce: abc hash: 89a0
106. nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: bc8 hash: 0011 nonce: 89 hash: 9876 nonce: abc hash: 89a0
107. 78 nonce: 9a4 hash: 002a nonce: bc8 hash: 0011 nonce: f6e hash: 0002 nonce: 092 hash: 0013 A Blockchain, with a fork
108. 79 nonce: 9a4 hash: 002a nonce: f6e hash: 0002 nonce: 092 hash: 0013 nonce: bc8 hash: 0011 A Blockchain, with a fork
109. 79 nonce: 9a4 hash: 002a nonce: f6e hash: 0002 nonce: 092 hash: 0013 nonce: bc8 hash: 0011 nonce: 9bb hash: 0020 A Blockchain, with a fork
110. 79 nonce: 9a4 hash: 002a nonce: f6e hash: 0002 nonce: 092 hash: 0013 nonce: 9bb hash: 0020 A Blockchain, with a fork
111. Consensus is for Everyone: An Introduction to Distributed Consensus 80 • accidental forks • tricks like ASICs (application-specific integrated circuits, special hardware) can make it less fair • boils the ocean the trouble with Proof of Work
113. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe Doesn’t boil the ocean Proof of Work
114. 83 I’m a quorum
115. 84 Oops
116. Consensus is for Everyone: An Introduction to Distributed Consensus 85 federated consensus
117. 86 I trust my two friends
118. 87 I still trust my two friends
119. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe Doesn’t boil the ocean Proof of Work Fed. Con.
120. Consensus is for Everyone: An Introduction to Distributed Consensus 89 • How do nodes choose their personal quorums? • What should a node do if one of its trusted peers misbehaves? Out-of-band questions
121. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe Trust no one Doesn’t boil the ocean Proof of Work Fed. Con.
122. Consensus is for Everyone: An Introduction to Distributed Consensus blockchains track and transfer value 91
123. Consensus is for Everyone: An Introduction to Distributed Consensus 92 Proof of Stake
124. Consensus is for Everyone: An Introduction to Distributed Consensus 93 • probabilities are tied to amount staked • either amount held or a “security deposit” • nodes exhibiting Byzantine behavior will be “slashed” Proof of Stake
125. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe Trust no one Doesn’t boil the ocean Proof of Work Proof of Stake Fed. Con.
126. Consensus is for Everyone: An Introduction to Distributed Consensus how can we know if a node misbehaved? 95
127. Consensus is for Everyone: An Introduction to Distributed Consensus what if we let the other nodes vote on it? 96 …sounds like PBFT
128. Consensus is for Everyone: An Introduction to Distributed Consensus what if we let the other nodes vote on it? 96 …sounds like PBFT
129. Consensus is for Everyone: An Introduction to Distributed Consensus 97 Tendermint
130. 98 $$$$ $$ $ $$$ $
131. 98 $$$$ $$ $ $$$ $
132. Consensus is for Everyone: An Introduction to Distributed Consensus 99 new height propose prevote nil prevote block invalid block valid block wait for prevotes from +2/3 precommit block precommit nil +2/3 2/3 wait for precommits from +2/3 commit new round 2/3 +2/3
133. Consensus is for Everyone: An Introduction to Distributed Consensus tendermint.com/docs 100
134. Consensus is for Everyone: An Introduction to Distributed Consensus tendermint.com/docs 100
135. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe Trust no one Doesn’t boil the ocean Proof of Work Proof of Stake Fed. Con.
136. Consensus is for Everyone: An Introduction to Distributed Consensus in proof of stake, resources = control 102
137. Consensus Paxos Raft Byzantine-safe PBFT Sybil-safe Trust no one Doesn’t boil the ocean Non-kleptocratic Proof of Work Proof of Stake Fed. Con.
138. 104 Consistency Availability Partition Tolerance
139. 105 Efficiency Fairness Minimized Trust Federated Consensus Proof of Work Proof of Stake
140. Consensus is for Everyone: An Introduction to Distributed Consensus • Get in touch: @_tessr or tess@tendermint.com • Please don’t forget to rate this session • Thank you to Peter Bourgon, Jeremy Rubin and Peter Alvaro for their help with this talk thanks! 106
141. Consensus is for Everyone: An Introduction to Distributed Consensus • Paxos: http://www.cs.yale.edu/homes/aspnes/pinewiki/Paxos.html • Leslie Lamport on Paxos: https://lamport.azurewebsites.net/pubs/pubs.html#lamport-paxos • A Brief Tour of FLP Impossibility (Henry Robinson): https://www.the-paper-trail.org/post/2008-08-13-a-brief-tour- of-flp-impossibility/ • In Search of an Understandable Consensus Algorithm (Raft): https://raft.github.io/raft.pdf • Raft: Consensus made simple(r) (Brian Storti): https://www.brianstorti.com/raft/ • The Morning Paper: Practical Byzantine Fault Tolerance: https://blog.acolyer.org/2015/05/18/practical-byzantine- fault-tolerance • On the Practicality of ‘Practical’ Byzantine Fault Tolerance: https://arxiv.org/abs/1110.4854v1 • Mastering Bitcoin (Andreas Antonopolous): https://github.com/bitcoinbook/bitcoinbook/blob/develop/ ch10.asciidoc#forks • Ethereum Proof of Stake FAQ: https://github.com/ethereum/wiki/wiki/Proof-of-Stake-FAQs • Consensus Compare: Casper vs. Tendermint (Chjango Unchained): https://blog.cosmos.network/consensus- compare-casper-vs-tendermint-6df154ad56ae • Tendermint Consensus Overview: https://tendermint.com/docs/introduction/introduction.html#consensus- overview • The Stellar Consensus Protocol: A Federated Model for Internet-level Consensus: https://www.stellar.org/papers/ stellar-consensus-protocol.pdf • Monty Python picture from https://welovebudapest.com/en/event/gyalog-galopp/ references &
 further reading 107