consensus_made_simple.pdf

2PC, 3PC and Paxos

Transcript

1. Consensus Made Simple 悟时@taobao
2. 提纲 • 2PC • 3PC • Paxos
3. 一致性协议 • 应用场景 – 数据库事务提交 – 时钟同步 – leader选举 – … • 形式定义 – 一致性(agreement) – 有效性(validity) – 可终止性(termination)
4. 2PC • 前提 – synchronized network – fail-stop • 过程 P1 C P2 P3 commit
5. 2PC • 前提 – synchronized network – fail-stop • 过程 P1 C P2 P3
6. 2PC • 前提 – synchronized network – fail-stop • 过程 P1 C P2 P3 commit
7. 2PC P1 P2 P3 C
8. 2PC P1 P2 P3 C
9. 2PC P1 P2 P3 C C’ 如果C挂了，让C’接管它的工作， 可以继续正确运行
10. 2PC P1 P2 P3 C C’ 如果C挂了，让C’接管它的工作， 可以继续正确运行 P2 P3 C P1
11. 2PC P1 P2 P3 C C’ 如果C挂了，让C’接管它的工作， 可以继续正确运行 P2 P3 C P1
12. 2PC P1 P2 P3 C C’ 如果C挂了，让C’接管它的工作， 可以继续正确运行 P2 P3 C P1
13. 2PC P1 P2 P3 C 如果C在第一阶段之后挂了，P3也挂了，C’来接 管，该如何继续？ 如果commit, 则因为P3可能是反对的，违反协议. 如果abort, 但是P3可能已经committed了，状态 又不一致了. C’ 如果C挂了，让C’接管它的工作， 可以继续正确运行 P2 P3 C C’ P1
14. 3PC • 3PC – 将2PC的第二阶段分成两个阶段 • 预提交 • 提交
15. 3PC P1 P2 P3 C C’ 如果其它人都收到了preCommit, 进继续进入提交阶段，最终提交。P3肯 定也同意了。 如果有人没收到preCommit, 则可以简单的abort. P3肯定也没有提交。 系统的表现是一致的。
16. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’
17. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’
18. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’
19. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C 暂停心跳
20. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’ 超时，接管 暂停心跳
21. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’ 超时，接管 询问状态 暂停心跳
22. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’ 超时，接管 询问状态 命令commit 暂停心跳
23. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’ 超时，接管 询问状态 命令commit 暂停心跳 恢复
24. 3PC • 问题 – 脑裂 – 无法处理fail-recovery P1 P2 P3 C C’ 超时，接管 询问状态 命令commit 暂停心跳 恢复 命令abort
25. paxos
26. paxos • 分布式文件系统[chubby|zookeeper] – 单机 • 一致性 • 缺少可用性 – 多机备份 • 可用性 • 如何保持一致性 – paxos
27. paxos • 系统角色 – proposer – acceptor – learner • 问题定义 – 决议只有在被 proposers 提出后才能被批准 – 提议被大多数acceptor接受后成为决议 – 在一次 执行实例中只批准一个 决议 – learners 只能获得被批准的决议
28. paxos • 思想 – 不再面面俱到 – 照顾多数人的利益，牺牲少数人的利益
29. paxos • 思想 – 不再面面俱到 – 照顾多数人的利益，牺牲少数人的利益 如果一个提案被大多数人接受，则该提案可以作为决议。
30. paxos 没有失败，没有丢消息，只有一个proposer提了一个提案，应该形成决议
31. paxos P1. 一个acceptor必须接受它第一次收到的提案. 没有失败，没有丢消息，只有一个proposer提了一个提案，应该形成决议
32. paxos P1. 一个acceptor必须接受它第一次收到的提案. 没有失败，没有丢消息，只有一个proposer提了一个提案，应该形成决议 各提各的，都不占大多数怎么办？
33. paxos acceptor可以接受 多个提案，只要这些提案的value相同就行了啊
34. paxos acceptor可以接受 多个提案，只要这些提案的value相同就行了啊 P2. 一旦一个具有value v的提案被批准，那么后续批准的提案也必须具有value v
35. paxos acceptor可以接受 多个提案，只要这些提案的value相同就行了啊 P2. 一旦一个具有value v的提案被批准，那么后续批准的提案也必须具有value v P2a. 一旦一个具有value v的提案被批准，那么所有acceptor后续接受的 提案也必须具有value v
36. paxos acceptor可以接受 多个提案，只要这些提案的value相同就行了啊 P2. 一旦一个具有value v的提案被批准，那么后续批准的提案也必须具有value v P2a. 一旦一个具有value v的提案被批准，那么所有acceptor后续接受的 提案也必须具有value v 万一有俩人都不知道已经有决议了，一个提了一个提案v’，另一个接受了咋办？
37. paxos 提了就不能拒绝，那就别提不同的提案了。
38. paxos P2b. 一旦一个具有value v的提案被批准，那么所有proposer后续提出的 提案也必须具有value v 提了就不能拒绝，那就别提不同的提案了。
39. paxos P2b. 一旦一个具有value v的提案被批准，那么所有proposer后续提出的 提案也必须具有value v 提了就不能拒绝，那就别提不同的提案了。 P2c. 如果一个编号为n的提案具有value v, 那么存在一个多数派， 要么他们当中所有人都没有接受编号小于n的任何提案， 要么他们已经接受的所有编号小于n的提案中编号最大的提案具有value v
40. paxos P2b. 一旦一个具有value v的提案被批准，那么所有proposer后续提出的 提案也必须具有value v 提了就不能拒绝，那就别提不同的提案了。 P2c. 如果一个编号为n的提案具有value v, 那么存在一个多数派， 要么他们当中所有人都没有接受编号小于n的任何提案， 要么他们已经接受的所有编号小于n的提案中编号最大的提案具有value v 证明一下P2c  P2b？
41. paxos • proposer – prepare请求n • 希望acceptor回应 – 不再接受比n小的提案的承诺 – 若已经接受过提案了，告知比n小的最大序号的提案value – accept请求 • 得到大多数acceptor回应后才提出自己的提案 • 提案内容(n, v) – value的选择
42. paxos • acceptor – 可以忽略任何请求，但不能乱响应 • 对prepare请求可以一直响应 • 对accept请求的响应不能违背承诺 P1a. acceptor可以接受序号为n的提案，iff 他没有响应过序号大于n的prepare请求
43. paxos • 优化措施 1.如果已经响应了n的prepare请求, 则不需要再响应m<n的prepare请求  acceptor需要记住自己已经响应的序号最大的prepare请求和序号最大 的已经接受的请求[先写日志再实际响应，写了日志没响应没关系滴]. proposer 只要保证序号增长就行了,不需要记录历史信息 2.如果一个acceptor已经响应了一个更高序号的prepare请求，它可以忽略 低序号的prepare或accept请求，但是如果直接告诉发低序号请求的 proposer放弃或许更好.
44. paxos • 完整算法 第一阶段： a. 一个proposer选择一个序号n, 向一个多数acceptor集合发出prepare请 求 b. 如果一个acceptor接收到一个比自己所有已经响应的序号都大的 prepare请求n, 则回应该proposer, 承诺不会再接受比n小的提案，并且 告知自己已经接受过的序号最大的提案value v(假如有的话). 第二阶段： a. 如果一个proposer收到了大多数acceptor的响应, 那么它可以向这些 acceptor发出一个提案(n, v). 其中v是所有响应中序号最大的提案value 或者自己的value(如果没有响应). b. 当一个acceptor收到一个序号为n的提案时，如果它没有响应过序号比 n大的prepare请求，那么可以接受该提案。
45. paxos sample1(1) Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2
46. paxos sample1(2) Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1
47. paxos sample1(3) Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null>
48. paxos sample1(4) Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1
49. paxos sample1(5) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted>
50. paxos sample1(6) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2
51. paxos sample1(7) Acceptor1 latest_preped: #2p2 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, <#1p1,v1>> <ok, null>
52. paxos sample1(8) Acceptor1 latest_preped: #2p2 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, <#1p1,v1>> <ok, null> accept, #1p1, v1
53. paxos sample1(9) Acceptor1 latest_preped: #2p2 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, <#1p1,v1>> <ok, null> accept, #1p1, v1 <accepted>
54. paxos sample1(10) Acceptor1 latest_preped: #2p2 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, <#1p1,v1>> <ok, null> accept, #1p1, v1 <accepted> ok
55. paxos sample1(11) Acceptor1 latest_preped: #2p2 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, <#1p1,v1>> <ok, null> accept, #1p1, v1 <accepted> ok accept, #2p2, v1 accept, #2p2, v1
56. paxos sample1(12) Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v1 Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, <#1p1,v1>> <ok, null> accept, #1p1, v1 <accepted> ok accept, #2p2, v1 <accepted> accept, #2p2, v1 <accepted>
57. paxos sample1(13) Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v1 Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v1 Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, <#1p1,v1>> <ok, null> accept, #1p1, v1 <accepted> ok accept, #2p2, v1 <accepted> accept, #2p2, v1 <accepted> ok
58. paxos sample2(1) Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2
59. paxos sample2(2) Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1
60. paxos sample2(3) Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null>
61. paxos sample2(4) Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1
62. paxos sample2(5) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted>
63. paxos sample2(6) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #1p1 latest_acpted: null Acceptor1 latest_preped: null latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2
64. paxos sample2(7) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: null Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null>
65. paxos sample2(8) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: null Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #1p1, v1
66. paxos sample2(9) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: null Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #1p1, v1 <rejected, #2p2>
67. paxos sample2(10) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: null Acceptor1 latest_preped: #2p2 latest_acpted: null Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #1p1, v1 <rejected, #2p2> accept, #2p2, v2 accept, #2p2, v2
68. paxos sample2(11) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #1p1, v1 <rejected, #2p2> accept, #2p2, v2 accept, #2p2, v2 accepted accepted
69. paxos sample2(12) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #1p1 prepare, #1p1 <ok, null> <ok, null> accept, #1p1, v1 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #1p1, v1 <rejected, #2p2> accept, #2p2, v2 accept, #2p2, v2 accepted accepted ok
70. paxos sample2(13) Acceptor1 latest_preped: #1p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #3p1 prepare, #3p1 prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #2p2, v2 accept, #2p2, v2 accepted accepted ok
71. paxos sample2(14) Acceptor1 latest_preped: #3p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #3p1 latest_acpted: #2p2,v2 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #3p1 prepare, #3p1 <ok, <#1p1,v1>> <ok,<#2p2,v2> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #2p2, v2 accept, #2p2, v2 accepted accepted ok
72. paxos sample2(15) Acceptor1 latest_preped: #3p1 latest_acpted: #1p1,v1 Acceptor1 latest_preped: #3p1 latest_acpted: #2p2,v2 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #3p1 prepare, #3p1 <ok, <#1p1,v1>> <ok,<#2p2,v2> accept, #3p1, v2 prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #3p1, v2 accept, #2p2, v2 accept, #2p2, v2 accepted accepted ok
73. paxos sample2(16) Acceptor1 latest_preped: #3p1 latest_acpted: #3p1,v2 Acceptor1 latest_preped: #3p1 latest_acpted: #3p1,v2 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #3p1 prepare, #3p1 <ok, <#1p1,v1>> <ok,<#2p2,v2> accept, #3p1, v2 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #3p1, v2 <accepted> accept, #2p2, v2 accept, #2p2, v2 accepted accepted ok
74. paxos sample2(17) Acceptor1 latest_preped: #3p1 latest_acpted: #3p1,v2 Acceptor1 latest_preped: #3p1 latest_acpted: #3p1,v2 Acceptor1 latest_preped: #2p2 latest_acpted: #2p2,v2 Proposer1 Proposer2 prepare, #3p1 prepare, #3p1 <ok, <#1p1,v1>> <ok,<#2p2,v2> accept, #3p1, v2 <accepted> prepare, #2p2 prepare, #2p2 <ok, null> <ok, null> accept, #3p1, v2 <accepted> accept, #2p2, v2 accept, #2p2, v2 accepted accepted ok ok
75. paxos • learner – 正常情况 • 每个acceptor接受到都告诉所有的learner – 消息量大 • 选一个learner, 只告诉它,它知道决议后再告诉其它人 – 消息量小，但是这个learner成了单点 • 选一个learners集合 – 在性能和可靠性之间权衡 – 异常 • 如果有acceptor挂了，可能导致无法推断出决议 – learner同机的proposer 发起针对该提案新一轮的决议
76. paxos • 活性(progress|liveness) – 必须选择唯一的learner才能保证活性 – 可能不动，一定不会动错
77. paxos • 状态机 – 分布式状态机 Quorum client client client Quorum Quorum client client op1 op2 op3 ….
78. 参考文献 • http://www.wikipedia.org/ – 2pc & 3pc & paxos & Byzantine_failure • << paxos made simple >> • http://the-paper-trail.org/blog/tag/consensus/