Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Jepsen: Mon cluster est-il cohérent ?

Jepsen: Mon cluster est-il cohérent ?

A short introduction to jepsen, a distributed databases testing tool by @aphyr

Arnaud Bailly

June 10, 2014
Tweet

More Decks by Arnaud Bailly

Other Decks in Programming

Transcript

  1. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Jepsen: Testing Cluster Consistency
    Arnaud Bailly @abailly
    http://blog.foldlabs.com/
    2014-06

    View Slide

  2. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Jepsen?
    ▶ A tool for testing that distributed databases respect some
    consistency model in face of network partitions

    View Slide

  3. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Jepsen?
    ▶ A tool for testing that distributed databases respect some
    consistency model in face of network partitions
    ▶ A set of primitives to define and run such tests

    View Slide

  4. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Jepsen?
    ▶ A tool for testing that distributed databases respect some
    consistency model in face of network partitions
    ▶ A set of primitives to define and run such tests
    ▶ A Domain Specific Language for test and network partitions
    generators (aka. nemesis)

    View Slide

  5. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Jepsen?
    ▶ A tool for testing that distributed databases respect some
    consistency model in face of network partitions
    ▶ A set of primitives to define and run such tests
    ▶ A Domain Specific Language for test and network partitions
    generators (aka. nemesis)
    ▶ Model checkers, includingLinearizability Checker (aka.
    Knossos) for asserting correctness of test runs

    View Slide

  6. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Jepsen?
    ▶ A tool for testing that distributed databases respect some
    consistency model in face of network partitions
    ▶ A set of primitives to define and run such tests
    ▶ A Domain Specific Language for test and network partitions
    generators (aka. nemesis)
    ▶ Model checkers, includingLinearizability Checker (aka.
    Knossos) for asserting correctness of test runs
    ▶ With specific tests for RabbitMQ, etcd, ElasticSearch,
    DAtomic, Riak

    View Slide

  7. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Jepsen?
    ▶ A tool for testing that distributed databases respect some
    consistency model in face of network partitions
    ▶ A set of primitives to define and run such tests
    ▶ A Domain Specific Language for test and network partitions
    generators (aka. nemesis)
    ▶ Model checkers, includingLinearizability Checker (aka.
    Knossos) for asserting correctness of test runs
    ▶ With specific tests for RabbitMQ, etcd, ElasticSearch,
    DAtomic, Riak
    ▶ Written by @aphyr (aka. Kyle Kingsbury) of Call Me Maybe
    fame

    View Slide

  8. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Tools
    Figure : The NoSQL Database Zoo

    View Slide

  9. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees

    View Slide

  10. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes

    View Slide

  11. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration

    View Slide

  12. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery

    View Slide

  13. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery
    ▶ Distributed Consensus

    View Slide

  14. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery
    ▶ Distributed Consensus
    ▶ Sessions Synchronization

    View Slide

  15. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery
    ▶ Distributed Consensus
    ▶ Sessions Synchronization
    ▶ …

    View Slide

  16. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Mapping the CAP Landscape
    Figure : CAP as A Metric Space

    View Slide

  17. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Mapping the CAP Landscape
    Figure : Consistency and Partition Tolerance

    View Slide

  18. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Partitioning?

    View Slide

  19. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Partitioning?

    View Slide

  20. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Partitioning?

    View Slide

  21. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency

    View Slide

  22. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency

    View Slide

  23. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency

    View Slide

  24. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency

    View Slide

  25. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency
    ▶ Sequential Consistency

    View Slide

  26. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency
    ▶ Sequential Consistency
    ▶ Linearizability

    View Slide

  27. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency
    ▶ Sequential Consistency
    ▶ Linearizability
    ▶ …

    View Slide

  28. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Sequential Consistency

    View Slide

  29. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Linearizability

    View Slide

  30. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Demo

    View Slide

  31. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Questions?

    View Slide

  32. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper

    View Slide

  33. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper
    ▶ Knossos the linearizability checker

    View Slide

  34. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper
    ▶ Knossos the linearizability checker
    ▶ Kyle Kingsbury’s blog and the famous Call Me Maybe posts
    that introduced Jepsen

    View Slide

  35. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper
    ▶ Knossos the linearizability checker
    ▶ Kyle Kingsbury’s blog and the famous Call Me Maybe posts
    that introduced Jepsen
    ▶ ETH Zurich Distributed Systems Course

    View Slide