Jepsen: Mon cluster est-il cohérent ?

A short introduction to jepsen, a distributed databases testing tool by @aphyr

Transcript

1. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   Jepsen: Testing Cluster Consistency
   Arnaud Bailly @abailly
   http://blog.foldlabs.com/
   2014-06
   

   View Slide

2. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   What is Jepsen?
   ▶ A tool for testing that distributed databases respect some
   consistency model in face of network partitions
   

   View Slide

3. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   What is Jepsen?
   ▶ A tool for testing that distributed databases respect some
   consistency model in face of network partitions
   ▶ A set of primitives to define and run such tests
   

   View Slide

4. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   What is Jepsen?
   ▶ A tool for testing that distributed databases respect some
   consistency model in face of network partitions
   ▶ A set of primitives to define and run such tests
   ▶ A Domain Specific Language for test and network partitions
   generators (aka. nemesis)
   

   View Slide

5. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   What is Jepsen?
   ▶ A tool for testing that distributed databases respect some
   consistency model in face of network partitions
   ▶ A set of primitives to define and run such tests
   ▶ A Domain Specific Language for test and network partitions
   generators (aka. nemesis)
   ▶ Model checkers, includingLinearizability Checker (aka.
   Knossos) for asserting correctness of test runs
   

   View Slide

6. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   What is Jepsen?
   ▶ A tool for testing that distributed databases respect some
   consistency model in face of network partitions
   ▶ A set of primitives to define and run such tests
   ▶ A Domain Specific Language for test and network partitions
   generators (aka. nemesis)
   ▶ Model checkers, includingLinearizability Checker (aka.
   Knossos) for asserting correctness of test runs
   ▶ With specific tests for RabbitMQ, etcd, ElasticSearch,
   DAtomic, Riak
   

   View Slide

7. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   What is Jepsen?
   ▶ A tool for testing that distributed databases respect some
   consistency model in face of network partitions
   ▶ A set of primitives to define and run such tests
   ▶ A Domain Specific Language for test and network partitions
   generators (aka. nemesis)
   ▶ Model checkers, includingLinearizability Checker (aka.
   Knossos) for asserting correctness of test runs
   ▶ With specific tests for RabbitMQ, etcd, ElasticSearch,
   DAtomic, Riak
   ▶ Written by @aphyr (aka. Kyle Kingsbury) of Call Me Maybe
   fame
   

   View Slide

8. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   Why I Should Care - Tools
   Figure : The NoSQL Database Zoo
   

   View Slide

9. .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   .
   .
   ..
   .
   Why I Should Care - Usage
   ▶ Message Queues w/ different delivery guarantees
   

   View Slide

10. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    

    View Slide

11. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    

    View Slide

12. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery
    

    View Slide

13. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery
    ▶ Distributed Consensus
    

    View Slide

14. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery
    ▶ Distributed Consensus
    ▶ Sessions Synchronization
    

    View Slide

15. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Why I Should Care - Usage
    ▶ Message Queues w/ different delivery guarantees
    ▶ Locks & Mutexes
    ▶ Configuration
    ▶ Service Discovery
    ▶ Distributed Consensus
    ▶ Sessions Synchronization
    ▶ …
    

    View Slide

16. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Mapping the CAP Landscape
    Figure : CAP as A Metric Space
    

    View Slide

17. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Mapping the CAP Landscape
    Figure : Consistency and Partition Tolerance
    

    View Slide

18. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Partitioning?
    

    View Slide

19. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Partitioning?
    

    View Slide

20. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Partitioning?
    

    View Slide

21. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    

    View Slide

22. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    

    View Slide

23. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    

    View Slide

24. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency
    

    View Slide

25. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency
    ▶ Sequential Consistency
    

    View Slide

26. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency
    ▶ Sequential Consistency
    ▶ Linearizability
    

    View Slide

27. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    What is Consistency?
    ▶ Quiescent Consistency
    ▶ Eventual Consistency
    ▶ Causal Consistency
    ▶ Read-Your-Writes Consistency
    ▶ Sequential Consistency
    ▶ Linearizability
    ▶ …
    

    View Slide

28. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Sequential Consistency
    

    View Slide

29. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Linearizability
    

    View Slide

30. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Demo
    

    View Slide

31. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Questions?
    

    View Slide

32. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper
    

    View Slide

33. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper
    ▶ Knossos the linearizability checker
    

    View Slide

34. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper
    ▶ Knossos the linearizability checker
    ▶ Kyle Kingsbury’s blog and the famous Call Me Maybe posts
    that introduced Jepsen
    

    View Slide

35. .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    .
    .
    ..
    .
    Links
    ▶ jepsen proper
    ▶ Knossos the linearizability checker
    ▶ Kyle Kingsbury’s blog and the famous Call Me Maybe posts
    that introduced Jepsen
    ▶ ETH Zurich Distributed Systems Course
    

    View Slide