Consensus and Consistency

Slide 1

Slide 1 text

Consensus and Consistency: Why Should I Care? Neha Narula @neha Berlin Buzzwords 2014

Slide 2

Slide 2 text

2003-‐2010 2008-‐2014

Slide 3

Slide 3 text

This Talk •  Why do we need it? •  Types of consistency •  Consensus •  CAP theorem •  What to do with this?

Slide 4

Slide 4 text

How Messed Up Can Things Get?

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

“The hacker discovered that mul3ple simultaneous withdrawals are processed essen3ally at the same 3me and that the system's so9ware doesn't check quickly enough for a nega3ve balance” hNp://arstechnica.com/security/2014/03/yet-‐another-‐exchange-‐hacked-‐poloniex-‐loses-‐ around-‐50000-‐in-‐bitcoin/

Slide 7

Slide 7 text

Consistency guarantees help us reason about our code and avoid subtle bugs

Slide 8

Slide 8 text

Consistency •  Most misused word in distributed systems •  C as in ACID •  C as in CAP •  C as in sequenWal, causal, eventual, strict consistency

Slide 9

Slide 9 text

Cache Coherence P1 P2 P3 RAM

Slide 10

Slide 10 text

SequenWal Consistency The result of any execuWon is as if the reads and writes were executed in some order Order doesn’t have to match Wme! Order does have to match what each process sees

Slide 11

Slide 11 text

P1: W(x)a P2: W(x)b P3: R(x)b R(x)a P4: R(x)b R(x)a P1: W(x)a P2: W(x)b P3: R(x)b R(x)a P4: R(x)b R(x)a Wme

Slide 12

Slide 12 text

External Consistency Everything that sequenWal consistency has Except results actually match 3me. An external observer

Slide 13

Slide 13 text

P1: W(x)a P2: W(x)b P3: R(x)b R(x)a P4: R(x)b R(x)a The value of x is b! Then I read x=a? P3: P4: Not Externally Consistent Wme

Slide 14

Slide 14 text

Distributed System •  CommunicaWon, not shared memory •  ReplicaWon without cache coherence •  Time becomes a fuzzy concept

Slide 15

Slide 15 text

Eventual Consistency If no new updates are made to the object, eventually all accesses will return the last updated value.

Slide 16

Slide 16 text

Eventual Consistency If no new updates are made to the object, eventually all accesses will return the last updated value the same value. (What is last, really?) (And when do we stop wriWng?)

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

Paxos

Slide 19

Slide 19 text

Client Propose blue

Slide 20

Slide 20 text

Client Prepare (n, blue)

Slide 21

Slide 21 text

Client Prepare (n, blue)

Slide 22

Slide 22 text

Client Prepare OK

Slide 23

Slide 23 text

Client Prepare OK

Slide 24

Slide 24 text

Client Accept (n, blue)

Slide 25

Slide 25 text

Client Accept (n, blue)

Slide 26

Slide 26 text

Client Accept OK

Slide 27

Slide 27 text

Client Decided blue

Slide 28

Slide 28 text

Paxos Client Client

Slide 29

Slide 29 text

Paxos Client Client

Slide 30

Slide 30 text

ConﬂicWng Prepares Client Client

Slide 31

Slide 31 text

Highest Proposal # Wins Client Client

Slide 32

Slide 32 text

proposer(v): while not decided: choose n, unique and higher than any n seen so far send prepare(n) to all servers including self if prepare_ok(n_a, v_a) from majority: v' = v_a with highest n_a; choose own v otherwise send accept(n, v') to all if accept_ok(n) from majority: send decided(v') to all acceptor's state: n_p (highest prepare seen) n_a, v_a (highest accept seen) acceptor_prepare_handler(n): if n > n_p n_p = n reply prepare_ok(n_a, v_a) else reply prepare_reject acceptor_accept_handler(n, v): if n >= n_p n_p = n n_a = n v_a = v reply accept_ok(n) else reply accept_reject

Slide 33

Slide 33 text

Global Log PrimiWve •  Each operaWon (read or write) as an entry in the log •  Everyone agrees on the log •  Everyone applies operaWons in log order •  Externally consistent ZAB (Zookeeper), Viewstamped ReplicaWon

Slide 34

Slide 34 text

Paxos lets us guarantee correctness with a funcWoning majority Paxos does not guarantee liveness

Slide 35

Slide 35 text

CAP Theorem •  Brewer’s PODC talk: “Consistency, Availability, ParWWon-‐tolerance: choose two” in 2000 –  ParWWon-‐tolerance is a failure model –  Choice: can you process reads and writes during a parWWon or not? •  FLP result – “Impossibility of Distributed Consensus with One Faulty Process” in 1985 –  Asynchronous model; cannot tell the diﬀerence between message delay and failure

Slide 36

Slide 36 text

What does this mean? It’s impossible to decide anything on the internet?

Slide 37

Slide 37 text

NP-‐hard

Slide 38

Slide 38 text

What does CAP mean? It’s impossible to 100% of the Wme decide everything on the internet if we can’t rely on synchronous messaging We can 100% of the Wme decide everything if parWWons heal (we know the upper bound on message delays) We can sWll play Candy Crush

Slide 39

Slide 39 text

CAP Consistency vs. Performance Paxos is many rounds of messages. How do we reduce # messages while: •  Producing a correct ordering of reads and writes and •  Handling failures and making progress?

Slide 40

Slide 40 text

Real World Systems Google’s distributed database Spanner: “We believe it is beAer to have applica3on programmers deal with performance problems due to overuse of transac3ons as boAlenecks arise, rather than always coding around the lack of transac3ons.”

Slide 41

Slide 41 text

Summary •  Consistency makes our lives a lot easier and programming with guarantees is HARD. •  We should be focusing on how to improve the performance of consistent systems instead of worrying about impossibility results.

Slide 42

Slide 42 text

Further Reading •  Fischer, Lynch, Paterson: Impossiblity of Consensus with One Faulty Process. Journal of the ACM, 1985 •  Henry Robinson: hNp://the-‐paper-‐trail.org/blog/a-‐brief-‐tour-‐ of-‐ﬂp-‐impossibility/ •  Eric Brewer: hNp://www.infoq.com/arWcles/cap-‐twelve-‐ years-‐later-‐how-‐the-‐rules-‐have-‐changed

Slide 43

Slide 43 text

Thanks! The Stata Center via emax: hNp://hip.cat/emax/ narula@mit.edu hNp://nehanaru.la @neha