All Together Now: An Introduction to Distributed Consensus

Tess Rinearson, @_tessr © Interstellar Inc. 2018 all together now:
an introduction to distributed consensus

© Interstellar Inc. 2018 3 x x x x x
GOAL: Consistency across all nodes

© Interstellar Inc. 2018 5 ok ✔ ok ✔ ok
✔ ok ✔

✔ ok ✔ x

✔ ok ✔ x x x x x

An Introduction to Distributed Consensus © Interstellar Inc. 2018 6
this works, but it also sucks!

© Interstellar Inc. 2018 7 why does this node get
to suggest a value? what if this one wants to suggest a value? or this one?

© Interstellar Inc. 2018 8 what if there’s a network
partition?

© Interstellar Inc. 2018 9 what if we have A
LOT of nodes???

© Interstellar Inc. 2018 10 what if one of the
nodes goes offline?

© Interstellar Inc. 2018 An Introduction to Distributed Consensus also
a lot of other problems that computer scientists get excited about (clock skew!!!) 11

solution: Paxos

no one actually understands Paxos

© Interstellar Inc. 2018 An Introduction to Distributed Consensus “We
found few people who were comfortable with Paxos, even among seasoned researchers. We… were not able to understand the complete protocol until after reading several simplified explanations and designing our own alternative protocol, a process that took almost a year.” (from the Raft paper) 14

© Interstellar Inc. 2018 An Introduction to Distributed Consensus 15
• a lot of behavior is unspecified • the variants that make it more complete can be very complex the trouble with Paxos

you can understand (basic) Paxos

Naive Solution • proposer node sends new value to every other node in the system Paxos • proposer node sends out ballots that other nodes can vote on • each round of consensus requires two rounds of voting

value: ? prepared: n accepted: n

© Interstellar Inc. 2018 19 Step 1: Prepare the ballot
number nʹ value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n

© Interstellar Inc. 2018 19 prepare(nʹ) prepare(nʹ) prepare(nʹ) prepare(nʹ) Step
1: Prepare the ballot number nʹ prepare(nʹ) value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n

© Interstellar Inc. 2018 20 Step 2: Acknowledge ballot value:
prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 20 ack(nʹ, n) ack(nʹ, n) ack(nʹ,
n) ack(nʹ, n) Step 2: Acknowledge ballot ack(nʹ, n) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 21 I have received acks from
5/5 nodes Step 3a: Count acks value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 22 Step 3b: Tell nodes to
accept the value value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 22 accept(nʹ, x) accept(nʹ, x) accept(nʹ,
x) accept(nʹ, x) Step 3b: Tell nodes to accept the value accept(nʹ, x) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 23 Step 4: Nodes adopt the
value value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ

© Interstellar Inc. 2018 An Introduction to Distributed Consensus what
if a node goes offline? 24

1: Prepare the ballot number nʹ prepare(nʹ) value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n value: prepared: n accepted: n

prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

n) Step 2: Acknowledge ballot ack(nʹ, n) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

4/5 nodes Step 3a: Count acks value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 An Introduction to Distributed Consensus we
received acks from a majority of nodes, so we can proceed 28

if two nodes propose values at the same time? 29

prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

n) Step 2: Acknowledge ballot ack(nʹ, n) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

4/5 nodes Step 3a: Count acks value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n I have received acks from 1/5 nodes, so I must stop

© Interstellar Inc. 2018 33 Step 3b: Tell nodes to
accept the value value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 33 accept(nʹ, x) accept(nʹ, x) accept(nʹ,
x) accept(nʹ, x) Step 3b: Tell nodes to accept the value accept(nʹ, x) value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n value: prepared: nʹ accepted: n

© Interstellar Inc. 2018 34 Step 4: Nodes adopt the
value value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ value: x prepared: nʹ accepted: nʹ

© Interstellar Inc. 2018 An Introduction to Distributed Consensus we
maintained consistency across all the nodes 35

© Interstellar Inc. 2018 An Introduction to Distributed Consensus everyone
was following the rules: these are non-Byzantine failures 37

the Byzantine Generals

“attack” “attack” also, don’t attack

prepare(n) prepare(n+100000)

prepare(n) prepare(n+100000) help I’m stuck

how can a system tolerate Byzantine faults?

solution: PBFT

request pre-prepare prepare commit reply

© Interstellar Inc. 2018 An Introduction to Distributed Consensus PBFT
keeps working even if 1/3 of the nodes are malicious 46

problem: Sybil attacks

© Interstellar Inc. 2018 An Introduction to Distributed Consensus classic
PBFT requires you to vet your member set (no good for public networks!) 48

what about proof of work?

© Interstellar Inc. 2018 51 nonce: 123 hash: fa6c nonce:
9a4 hash: 002a nonce: a19 hash: 8e43 nonce: 89 hash: 9876 nonce: abc hash: 89a0

nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: a19
hash: 8e43 nonce: 89 hash: 9876 nonce: abc hash: 89a0 * in Bitcoin, nodes gossip; not every node is connected to every other node

nonce: 123 hash: fa6c nonce: 9a4 hash: 002a nonce: a19
hash: 8e43 nonce: 89 hash: 9876 nonce: abc hash: 89a0 block, hash block, hash block, hash block, hash * in Bitcoin, nodes gossip; not every node is connected to every other node

• multiple nodes can propose blocks simultaneously • longest* chain is the valid one • nodes are incentivized to choose the most-work chain, so they’ll eventually converge Proof of Work

Proof of Work

• tricks like ASICs (application-specific integrated circuits, special hardware) can make it less fair • boils the ocean the trouble with Proof of Work

Doesn’t boil the ocean Proof of Work

Doesn’t boil the ocean Proof of Work Proof of Stake

• probabilities are tied to amount staked • either amount held or a “security deposit” Proof of Stake

if we let the other nodes vote on it? 61 …sounds like PBFT

Tendermint

• PBFT is vulnerable to Sybil attacks • gate membership through Proof of Stake • slash collateral for bad behavior Tendermint

Doesn’t boil the ocean Non-kleptocratic Proof of Work Proof of Stake

federated consensus

Trust no one Doesn’t boil the ocean Non-kleptocratic Proof of Work Proof of Stake

the Stellar Consensus Protocol (SCP)

ps://thenextweb.com/insider/2015/02/16/ways-people-described-computers-1990s-hilarious/

✔ ok ✔ I trust A I trust A I trust B I trust A and B

✔ ok ✔ ok ✔ I trust A I trust A I trust B I trust A and B

Trust no one Doesn’t boil the ocean Non-kleptocratic Proof of Work Proof of Stake SCP

© Interstellar Inc. 2018 An Introduction to Distributed Consensus •
Get in touch: @_tessr or [email protected] • Thank you to Peter Bourgon and Jeremy Rubin for their help with this talk thanks! 77

© Interstellar Inc. 2018 An Introduction to Distributed Consensus •
Paxos: http://www.cs.yale.edu/homes/aspnes/pinewiki/Paxos.html • Leslie Lamport on Paxos: https://lamport.azurewebsites.net/pubs/pubs.html#lamport-paxos • In Search of an Understandable Consensus Algorithm (Raft): https://raft.github.io/raft.pdf • The Morning Paper: Practical Byzantine Fault Tolerance: https://blog.acolyer.org/2015/05/18/practical- byzantine-fault-tolerance • On the Practicality of ‘Practical’ Byzantine Fault Tolerance: https://arxiv.org/abs/1110.4854v1 • Ethereum Proof of Stake FAQ: https://github.com/ethereum/wiki/wiki/Proof-of-Stake-FAQs • Consensus Compare: Casper vs. Tendermint: https://blog.cosmos.network/consensus-compare-casper- vs-tendermint-6df154ad56ae • Tendermint Consensus Overview: https://tendermint.com/docs/introduction/ introduction.html#consensus-overview • The Stellar Consensus Protocol: A Federated Model for Internet-level Consensus: https:// www.stellar.org/papers/stellar-consensus-protocol.pdf • Monty Python picture from https://welovebudapest.com/en/event/gyalog-galopp/ • Mr. Jones picture from https://www.amazon.com/Indiana-Jones-Complete-Adventure-Collection/dp/ B001E75QH0 references &  further reading 78

All Together Now: An Introduction to Distribute...

All Together Now: An Introduction to Distributed Consensus

Transcript

More Decks by Tess Rinearson

Other Decks in Technology

Featured