$30 off During Our Annual Pro Sale. View Details »

Consensus and Consistency

Neha
May 26, 2014

Consensus and Consistency

A delightful whirlwind on consistency definitions, consensus, the silliness of the CAP theorem, and what FLP really means. An appeal to developers to stop worrying about dropping C or A for CAP and instead work on maintaining consistency with performance.

Neha

May 26, 2014
Tweet

More Decks by Neha

Other Decks in Programming

Transcript

  1. Consensus  and  Consistency:  
    Why  Should  I  Care?  
    Neha  Narula  
    @neha  
    Berlin  Buzzwords  2014  

    View Slide

  2. 2003-­‐2010  
    2008-­‐2014  

    View Slide

  3. This  Talk  
    •  Why  do  we  need  it?  
    •  Types  of  consistency  
    •  Consensus  
    •  CAP  theorem  
    •  What  to  do  with  this?  

    View Slide

  4. How  Messed  Up  Can  Things  Get?  

    View Slide

  5. View Slide

  6. “The  hacker  discovered  that  mul3ple  
    simultaneous  withdrawals  are  
    processed  essen3ally  at  the  same  
    3me  and  that  the  system's  so9ware  
    doesn't  check  quickly  enough  for  a  
    nega3ve  balance”  
    hNp://arstechnica.com/security/2014/03/yet-­‐another-­‐exchange-­‐hacked-­‐poloniex-­‐loses-­‐
    around-­‐50000-­‐in-­‐bitcoin/  

    View Slide

  7. Consistency  guarantees  help  us  
    reason  about  our  code  and  avoid  
    subtle  bugs  

    View Slide

  8. Consistency  
    •  Most  misused  word  in  distributed  systems  
    •  C  as  in  ACID  
    •  C  as  in  CAP  
    •  C  as  in  sequenWal,  causal,  eventual,  strict  
    consistency  

    View Slide

  9. Cache  Coherence  
    P1   P2   P3  
    RAM  

    View Slide

  10. SequenWal  Consistency  
    The  result  of  any  execuWon  is  as  if  the  reads  and  
    writes  were  executed  in  some  order  
     
    Order  doesn’t  have  to  match  Wme!  
     
    Order  does  have  to  match  what  each  process  
    sees  

    View Slide

  11. P1:  W(x)a  
    P2:                          W(x)b  
    P3:                                                    R(x)b                                  R(x)a  
    P4:                                                                          R(x)b            R(x)a  
    P1:                                                                          W(x)a  
    P2:    W(x)b  
    P3:                                        R(x)b                                              R(x)a  
    P4:                                                        R(x)b                              R(x)a  
    Wme  

    View Slide

  12. External  Consistency  
    Everything  that  sequenWal  consistency  has  
     
    Except  results  actually  match  3me.  
     
    An  external  observer  

    View Slide

  13. P1:  W(x)a  
    P2:                          W(x)b  
    P3:                                                    R(x)b                                  R(x)a  
    P4:                                                                          R(x)b            R(x)a  
    The  value  of  x  
    is  b!  
    Then  I  read  
    x=a?  
     
     
    P3:                                                      
    P4:                                                          
    Not  Externally  Consistent  
    Wme  

    View Slide

  14. Distributed  System  
    •  CommunicaWon,  not  shared  memory  
    •  ReplicaWon  without  cache  coherence  
    •  Time  becomes  a  fuzzy  concept  

    View Slide

  15. Eventual  Consistency  
    If  no  new  updates  are  made  to  the  object,  
    eventually  all  accesses  will  return  the  last  
    updated  value.  

    View Slide

  16. Eventual  Consistency  
    If  no  new  updates  are  made  to  the  object,  
    eventually  all  accesses  will  return  the  last  
    updated  value    the  same  value.  
     
    (What  is  last,  really?)  
     
    (And  when  do  we  stop  wriWng?)  

    View Slide

  17. View Slide

  18. Paxos  

    View Slide

  19. Client  
    Propose  blue  

    View Slide

  20. Client  
    Prepare  (n,  blue)  

    View Slide

  21. Client  
    Prepare  (n,  blue)  

    View Slide

  22. Client  
    Prepare  OK  

    View Slide

  23. Client  
    Prepare  OK  

    View Slide

  24. Client  
    Accept  (n,  blue)  

    View Slide

  25. Client  
    Accept  (n,  blue)  

    View Slide

  26. Client  
    Accept  OK  

    View Slide

  27. Client  
    Decided  blue  

    View Slide

  28. Paxos  
    Client  
    Client  

    View Slide

  29. Paxos  
    Client  
    Client  

    View Slide

  30. ConflicWng  Prepares  
    Client  
    Client  

    View Slide

  31. Highest  Proposal  #  Wins  
    Client  
    Client  

    View Slide

  32. proposer(v):  
       while  not  decided:  
           choose  n,  unique  and  higher  than  any  n  seen  so  far  
           send  prepare(n)  to  all  servers  including  self  
           if  prepare_ok(n_a,  v_a)  from  majority:  
               v'  =  v_a  with  highest  n_a;  choose  own  v  otherwise  
               send  accept(n,  v')  to  all  
               if  accept_ok(n)  from  majority:  
                   send  decided(v')  to  all  
     
    acceptor's  state:  
       n_p  (highest  prepare  seen)  
       n_a,  v_a  (highest  accept  seen)  
     
    acceptor_prepare_handler(n):  
       if  n  >  n_p  
           n_p  =  n  
           reply  prepare_ok(n_a,  v_a)  
       else  
           reply  prepare_reject  
     
    acceptor_accept_handler(n,  v):  
       if  n  >=  n_p  
           n_p  =  n  
           n_a  =  n  
           v_a  =  v  
           reply  accept_ok(n)  
       else  
           reply  accept_reject  

    View Slide

  33. Global  Log  PrimiWve  
    •  Each  operaWon  (read  or  write)  as  an  entry  in  
    the  log  
    •  Everyone  agrees  on  the  log  
    •  Everyone  applies  operaWons  in  log  order  
    •  Externally  consistent  
    ZAB  (Zookeeper),  Viewstamped  ReplicaWon  

    View Slide

  34. Paxos  lets  us  guarantee  correctness  with  a  
    funcWoning  majority  
     
    Paxos  does  not  guarantee  liveness  

    View Slide

  35. CAP  Theorem  
    •  Brewer’s  PODC  talk:  “Consistency,  Availability,  
    ParWWon-­‐tolerance:  choose  two”  in  2000  
    –  ParWWon-­‐tolerance  is  a  failure  model  
    –  Choice:  can  you  process  reads  and  writes  during  a  
    parWWon  or  not?  
     
    •  FLP  result  –  “Impossibility  of  Distributed  
    Consensus  with  One  Faulty  Process”  in  1985  
    –  Asynchronous  model;  cannot  tell  the  difference  
    between  message  delay  and  failure  

    View Slide

  36. What  does  this  mean?  
     
     
    It’s  impossible  to  decide  anything  on  the  
    internet?  

    View Slide

  37. NP-­‐hard  

    View Slide

  38. What  does  CAP  mean?  
    It’s  impossible  to  100%  of  the  Wme  decide  
    everything  on  the  internet  if  we  can’t  rely  on  
    synchronous  messaging  
     
    We  can  100%  of  the  Wme  decide  everything  if  
    parWWons  heal  (we  know  the  upper  bound  on  
    message  delays)  
     
    We  can  sWll  play  Candy  Crush  

    View Slide

  39. CAP  
    Consistency  vs.  Performance    
    Paxos  is  many  rounds  of  messages.  
     
    How  do  we  reduce  #  messages  while:  
    •  Producing  a  correct  ordering  of  reads  and  
    writes  and  
    •  Handling  failures  and  making  progress?  

    View Slide

  40. Real  World  Systems  
    Google’s  distributed  database  Spanner:  
    “We  believe  it  is  beAer  to  have  applica3on  
    programmers  deal  with  performance  problems  
    due  to  overuse  of  transac3ons  as  boAlenecks  
    arise,  rather  than  always  coding  around  the  lack  
    of  transac3ons.”  

    View Slide

  41. Summary  
    •  Consistency  makes  our  lives  a  lot  easier  and  
    programming  with  guarantees  is  HARD.  
    •  We  should  be  focusing  on  how  to  improve  the  
    performance  of  consistent  systems  instead  of  
    worrying  about  impossibility  results.  

    View Slide

  42. Further  Reading  
    •  Fischer,  Lynch,  Paterson:  Impossiblity  of  
    Consensus  with  One  Faulty  Process.  Journal  of  
    the  ACM,  1985  
    •  Henry  Robinson:  
    hNp://the-­‐paper-­‐trail.org/blog/a-­‐brief-­‐tour-­‐
    of-­‐flp-­‐impossibility/  
    •  Eric  Brewer:  
    hNp://www.infoq.com/arWcles/cap-­‐twelve-­‐
    years-­‐later-­‐how-­‐the-­‐rules-­‐have-­‐changed  

    View Slide

  43. Thanks!  
     
    The  Stata  Center  via  emax:  hNp://hip.cat/emax/  
    [email protected]  
    hNp://nehanaru.la  
    @neha    

    View Slide