All Together Now: An Introduction to Distributed Consensus

A friendly, visual introduction to various consensus algorithms. Given at Crypto Springs, a generalist blockchain conference.

Transcript

1. Tess Rinearson, @_tessr
   © Interstellar Inc. 2018
   all together now:
   an introduction to
   distributed consensus
   

   View Slide

2. © Interstellar Inc. 2018 2
   

   View Slide

3. © Interstellar Inc. 2018 3
   x
   x
   x
   x
   x
   GOAL: Consistency across
   all nodes
   

   View Slide

4. © Interstellar Inc. 2018 4
   

   View Slide

5. © Interstellar Inc. 2018 4
   x?
   x?
   x?
   x?
   

   View Slide

6. © Interstellar Inc. 2018 5
   

   View Slide

7. © Interstellar Inc. 2018 5
   ok ✔
   ok ✔
   ok ✔
   ok ✔
   

   View Slide

8. © Interstellar Inc. 2018 5
   ok ✔
   ok ✔
   ok ✔
   ok ✔
   x
   

   View Slide

9. © Interstellar Inc. 2018 5
   ok ✔
   ok ✔
   ok ✔
   ok ✔
   x
   x
   x
   x
   x
   

   View Slide

10. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 6
    this works, but it
    also sucks!
    

    View Slide

11. © Interstellar Inc. 2018 7
    why does this
    node get to
    suggest a value?
    what if this one
    wants to suggest
    a value?
    or this one?
    

    View Slide

12. © Interstellar Inc. 2018 8
    what if there’s a network partition?
    

    View Slide

13. © Interstellar Inc. 2018 9
    what if we have A LOT of nodes???
    

    View Slide

14. © Interstellar Inc. 2018 9
    what if we have A LOT of nodes???
    

    View Slide

15. © Interstellar Inc. 2018 10
    what if one of the nodes goes offline?
    

    View Slide

16. © Interstellar Inc. 2018 10
    what if one of the nodes goes offline?
    

    View Slide

17. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    also a lot of other problems that
    computer scientists get excited about
    (clock skew!!!)
    11
    

    View Slide

18. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 12
    solution: Paxos
    

    View Slide

19. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 13
    no one actually
    understands Paxos
    

    View Slide

20. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    “We found few people who were comfortable with Paxos,
    even among seasoned researchers. We… were not able
    to understand the complete protocol until after reading
    several simplified explanations and designing our own
    alternative protocol, a process that took almost a year.”
    (from the Raft paper)
    14
    

    View Slide

21. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    15
    •
    a lot of behavior is
    unspecified
    •
    the variants that make it
    more complete can be very
    complex
    the trouble with
    Paxos
    

    View Slide

22. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 16
    you can understand
    (basic) Paxos
    

    View Slide

23. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    17
    Naive Solution
    •
    proposer node sends new value
    to every other node in the
    system
    Paxos
    •
    proposer node sends out
    ballots that other nodes can
    vote on
    •
    each round of consensus
    requires two rounds of voting
    

    View Slide

24. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    18
    value: ?
    prepared: n
    accepted: n
    

    View Slide

25. © Interstellar Inc. 2018 19
    Step 1: Prepare the ballot
    number nʹ
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    

    View Slide

26. © Interstellar Inc. 2018 19
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    Step 1: Prepare the ballot
    number nʹ
    prepare(nʹ)
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    

    View Slide

27. © Interstellar Inc. 2018 20
    Step 2: Acknowledge ballot
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

28. © Interstellar Inc. 2018 20
    ack(nʹ, n)
    ack(nʹ, n)
    ack(nʹ, n)
    ack(nʹ, n)
    Step 2: Acknowledge ballot
    ack(nʹ, n)
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

29. © Interstellar Inc. 2018 21
    I have received acks
    from 5/5 nodes
    Step 3a: Count acks
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

30. © Interstellar Inc. 2018 22
    Step 3b: Tell nodes to
    accept the value
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

31. © Interstellar Inc. 2018 22
    accept(nʹ, x)
    accept(nʹ, x)
    accept(nʹ, x)
    accept(nʹ, x)
    Step 3b: Tell nodes to
    accept the value
    accept(nʹ, x)
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

32. © Interstellar Inc. 2018 23
    Step 4: Nodes adopt the
    value
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    

    View Slide

33. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    what if a node goes offline?
    24
    

    View Slide

34. © Interstellar Inc. 2018 25
    Step 1: Prepare the ballot
    number nʹ
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    

    View Slide

35. © Interstellar Inc. 2018 25
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    Step 1: Prepare the ballot
    number nʹ
    prepare(nʹ)
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    

    View Slide

36. © Interstellar Inc. 2018 25
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    Step 1: Prepare the ballot
    number nʹ
    prepare(nʹ)
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    

    View Slide

37. © Interstellar Inc. 2018 26
    Step 2: Acknowledge ballot
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

38. © Interstellar Inc. 2018 26
    ack(nʹ, n)
    ack(nʹ, n)
    ack(nʹ, n)
    Step 2: Acknowledge ballot
    ack(nʹ, n)
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

39. © Interstellar Inc. 2018 27
    I have received acks
    from 4/5 nodes
    Step 3a: Count acks
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

40. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    we received acks from a majority of
    nodes, so we can proceed
    28
    

    View Slide

41. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    what if two nodes propose values at
    the same time?
    29
    

    View Slide

42. © Interstellar Inc. 2018 30
    Step 1: Prepare the ballot
    number nʹ
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    

    View Slide

43. © Interstellar Inc. 2018 30
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    prepare(nʹ)
    Step 1: Prepare the ballot
    number nʹ
    prepare(nʹ)
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    value:
    prepared: n
    accepted: n
    

    View Slide

44. © Interstellar Inc. 2018 31
    Step 2: Acknowledge ballot
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

45. © Interstellar Inc. 2018 31
    ack(nʹ, n)
    ack(nʹ, n)
    ack(nʹ, n)
    Step 2: Acknowledge ballot
    ack(nʹ, n)
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

46. © Interstellar Inc. 2018 32
    I have received acks
    from 4/5 nodes
    Step 3a: Count acks
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    I have received acks from
    1/5 nodes, so I must stop
    

    View Slide

47. © Interstellar Inc. 2018 33
    Step 3b: Tell nodes to
    accept the value
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

48. © Interstellar Inc. 2018 33
    accept(nʹ, x)
    accept(nʹ, x)
    accept(nʹ, x)
    accept(nʹ, x)
    Step 3b: Tell nodes to
    accept the value
    accept(nʹ, x)
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    value:
    prepared: nʹ
    accepted: n
    

    View Slide

49. © Interstellar Inc. 2018 34
    Step 4: Nodes adopt the
    value
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    value: x
    prepared: nʹ
    accepted: nʹ
    

    View Slide

50. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    we maintained consistency across all
    the nodes
    35
    

    View Slide

51. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    

    View Slide

52. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    everyone was following the rules:
    these are non-Byzantine failures
    37
    

    View Slide

53. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 38
    the Byzantine Generals
    

    View Slide

54. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 38
    the Byzantine Generals
    

    View Slide

55. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    39
    

    View Slide

56. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    39
    “attack”
    “attack”
    also, don’t attack
    

    View Slide

57. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    40
    prepare(n)
    prepare(n+100000)
    

    View Slide

58. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    40
    prepare(n)
    prepare(n+100000)
    help I’m stuck
    

    View Slide

59. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 41
    how can a system
    tolerate Byzantine
    faults?
    

    View Slide

60. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 42
    solution: PBFT
    

    View Slide

61. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    43
    request
    pre-prepare
    prepare
    commit
    reply
    

    View Slide

62. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 44
    

    View Slide

63. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    

    View Slide

64. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    PBFT keeps working even if 1/3 of
    the nodes are malicious
    46
    

    View Slide

65. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 47
    problem: Sybil attacks
    

    View Slide

66. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    classic PBFT requires you to vet
    your member set
    (no good for public networks!)
    48
    

    View Slide

67. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    Sybil-safe
    

    View Slide

68. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 50
    what about proof of
    work?
    

    View Slide

69. © Interstellar Inc. 2018 51
    nonce: 123
    hash: fa6c
    nonce: 9a4
    hash: 002a
    nonce: a19
    hash: 8e43
    nonce: 89
    hash: 9876
    nonce: abc
    hash: 89a0
    

    View Slide

70. nonce: 123
    hash: fa6c
    nonce: 9a4
    hash: 002a
    nonce: a19
    hash: 8e43
    nonce: 89
    hash: 9876
    nonce: abc
    hash: 89a0
    * in Bitcoin, nodes gossip; not every node is connected to every other node
    

    View Slide

71. nonce: 123
    hash: fa6c
    nonce: 9a4
    hash: 002a
    nonce: a19
    hash: 8e43
    nonce: 89
    hash: 9876
    nonce: abc
    hash: 89a0
    block, hash
    block, hash
    block, hash
    block, hash
    * in Bitcoin, nodes gossip; not every node is connected to every other node
    

    View Slide

72. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    53
    •
    multiple nodes can propose
    blocks simultaneously
    •
    longest* chain is the valid one
    • nodes are incentivized to
    choose the most-work chain,
    so they’ll eventually converge
    Proof of Work
    

    View Slide

73. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    Sybil-safe
    Proof of Work
    

    View Slide

74. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    55
    •
    tricks like ASICs
    (application-specific
    integrated circuits, special
    hardware) can make it less
    fair
    •
    boils the ocean
    the trouble with
    Proof of Work
    

    View Slide

75. View Slide

76. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    Sybil-safe
    Doesn’t boil the ocean
    Proof of Work
    

    View Slide

77. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    Sybil-safe
    Doesn’t boil the ocean
    Proof of Work
    Proof of Stake
    

    View Slide

78. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    59
    •
    probabilities are tied to
    amount staked
    • either amount held or a
    “security deposit”
    Proof of Stake
    

    View Slide

79. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    how can we know if a node
    misbehaved?
    60
    

    View Slide

80. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    what if we let the other
    nodes vote on it?
    61
    …sounds like PBFT
    

    View Slide

81. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    what if we let the other
    nodes vote on it?
    61
    …sounds like PBFT
    

    View Slide

82. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 62
    Tendermint
    

    View Slide

83. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    63
    • PBFT is vulnerable to Sybil
    attacks
    • gate membership through
    Proof of Stake
    •
    slash collateral for bad
    behavior
    Tendermint
    

    View Slide

84. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    in proof of stake,
    resources = control
    64
    

    View Slide

85. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    Sybil-safe
    Doesn’t boil the ocean
    Non-kleptocratic
    Proof of Work
    Proof of Stake
    

    View Slide

86. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 66
    federated consensus
    

    View Slide

87. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    Sybil-safe
    Trust no one
    Doesn’t boil the ocean
    Non-kleptocratic
    Proof of Work
    Proof of Stake
    

    View Slide

88. An Introduction to Distributed Consensus
    © Interstellar Inc. 2018 68
    the Stellar
    Consensus Protocol
    (SCP)
    

    View Slide

89. © Interstellar Inc. 2018 69
    A
    I trust A to issue tokens,
    stay solvent, redeem
    tokens, and validate
    their transactions
    

    View Slide

90. ps://thenextweb.com/insider/2015/02/16/ways-people-described-computers-1990s-hilarious/
    

    View Slide

91. © Interstellar Inc. 2018 71
    A B
    I trust A
    I trust A
    I trust B
    I trust A and B
    

    View Slide

92. © Interstellar Inc. 2018 72
    A B
    I trust A
    I trust A
    I trust B
    I trust A and B
    

    View Slide

93. © Interstellar Inc. 2018 72
    A B
    ok ✔
    ok ✔
    ok ✔
    I trust A
    I trust A
    I trust B
    I trust A and B
    

    View Slide

94. © Interstellar Inc. 2018 73
    A B
    ok ✔
    ok ✔
    ok ✔
    I trust A
    I trust A
    I trust B
    I trust A and B
    

    View Slide

95. © Interstellar Inc. 2018 73
    A B
    ok ✔
    ok ✔
    ok ✔ ok ✔
    I trust A
    I trust A
    I trust B
    I trust A and B
    

    View Slide

96. © Interstellar Inc. 2018
    Consensus
    Paxos Raft
    Byzantine-safe
    PBFT
    Sybil-safe
    Trust no one
    Doesn’t boil the ocean
    Non-kleptocratic
    Proof of Work
    Proof of Stake
    SCP
    

    View Slide

97. © Interstellar Inc. 2018 75
    Consistency
    Availability
    Partition
    Tolerance
    

    View Slide

98. © Interstellar Inc. 2018 76
    Efficiency
    Fairness
    Minimized
    Trust
    Federated Consensus (SCP)
    Proof of Work
    Proof of Stake
    

    View Slide

99. © Interstellar Inc. 2018
    An Introduction to Distributed Consensus
    •
    Get in touch: @_tessr or
    tess@interstellar.com
    •
    Thank you to Peter Bourgon and
    Jeremy Rubin for their help with
    this talk
    thanks!
    77
    

    View Slide

100. © Interstellar Inc. 2018
     An Introduction to Distributed Consensus
     •
     Paxos: http://www.cs.yale.edu/homes/aspnes/pinewiki/Paxos.html
     •
     Leslie Lamport on Paxos: https://lamport.azurewebsites.net/pubs/pubs.html#lamport-paxos
     •
     In Search of an Understandable Consensus Algorithm (Raft): https://raft.github.io/raft.pdf
     •
     The Morning Paper: Practical Byzantine Fault Tolerance: https://blog.acolyer.org/2015/05/18/practical-
     byzantine-fault-tolerance
     •
     On the Practicality of ‘Practical’ Byzantine Fault Tolerance: https://arxiv.org/abs/1110.4854v1
     •
     Ethereum Proof of Stake FAQ: https://github.com/ethereum/wiki/wiki/Proof-of-Stake-FAQs
     •
     Consensus Compare: Casper vs. Tendermint: https://blog.cosmos.network/consensus-compare-casper-
     vs-tendermint-6df154ad56ae
     •
     Tendermint Consensus Overview: https://tendermint.com/docs/introduction/
     introduction.html#consensus-overview
     •
     The Stellar Consensus Protocol: A Federated Model for Internet-level Consensus: https://
     www.stellar.org/papers/stellar-consensus-protocol.pdf
     •
     Monty Python picture from https://welovebudapest.com/en/event/gyalog-galopp/
     •
     Mr. Jones picture from https://www.amazon.com/Indiana-Jones-Complete-Adventure-Collection/dp/
     B001E75QH0
     references &

     further reading
     78
     

     View Slide