Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Russell Brown - Oh, you wanted to increment a counter? Good luck with that!

Russell Brown - Oh, you wanted to increment a counter? Good luck with that!

Presented at Hey! #22 on 29th March, 2016.

Hey! Presents

March 29, 2016
Tweet

More Decks by Hey! Presents

Other Decks in Programming

Transcript

  1. basho "Oh, you wanted to *increment a counter*?! Good luck

    with that!” [1] Russell Brown russelldb@basho.com [1] https://twitter.com/lindsey/status/575006945213485056
  2. basho https://twitter.com/lindsey/status/575006945213485056

  3. basho Jokes ARE better explained

  4. basho https://twitter.com/dizzyd/status/35566447145127936

  5. basho Number one feature request 2011 - Counters

  6. basho HOW HARD CAN IT BE?

  7. None
  8. basho

  9. basho

  10. basho Leslie Lamport ``A distributed system is one in which

    the failure of a computer you didn't even know existed can render your own computer unusable''
  11. basho https://en.wikipedia.org/wiki/Distributed_computing A distributed system is a software system in

    which components located on networked computers communicate and coordinate their actions by passing messages. The components interact with each other in order to achieve a common goal.
  12. basho Distributed Counters - WHY? • Many many updates -

    scale/perf • Fault tolerance - machines crash • Latency - local data centres • Containers/“microservices”
  13. basho CAP CAP CAP CAP CAP CAP CAP CAP CAP

    CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP CAP
  14. basho CAP • Consistency • Availability • Partition Tolerance

  15. Replica A Replica B Replica C Client X Client Y

    add 1 add 1 NO!!!! :( CP
  16. Replica A Replica B Replica C Client X Client Y

    Add 1 C’ Add 1 AP
  17. basho Why Are You Counting? • Money / Votes •

    Real world resources (seats on a place, rental cars) • Unique IDs • Events (likes, hits, plays)
  18. basho “Strong Consistency” • ZooKeeper/HyperDex-Replicant/Riak-Ensemble • ZAB/Chain Replication/Paxos/Raft/VSR • Compare

    and swap • Starvation • Single data centre
  19. basho Not Really Counting - IDs • Ticket Servers -

    flickr ticket server • Flake - boundary • SnowFlake - Twitter
  20. basho Eventually Consistent • Riak/Cassandra etc • CRDTs Counters •

    Drift/Partial failure • Idempotence • Bounded-Counters - reservations
  21. basho Probabilistic • HyperLogLog etc • Bounded error estimates •

    Trade off space/speed for accuracy • Redis (soon to-be riak) and others
  22. basho https://twitter.com/lindsey/status/575006945213485056

  23. basho Take Home • Understand your requirements • Know What

    you are counting and WHY • Understand the trade offs • Make them consciously • Chose the least consistency/accuracy your use case allows