Amazon's Dynamo

Transcript

1. Dynamo: Amazon's Highly Available Key-value Store Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and Werner Vogels Amazon.com Muhammet Orazov
2. Outline • Introduction • System Architecture • Evaluation • Conclusion
3. Introduction • Key-value store • Highly available • Sacrifices consistency • Consists of different techniques
4. System Architecture • Partitioning • Replication • Data Versioning • Get and Put Execution • Temporary Failures • Permanent Failures • Membership and Failure Detection
5. Partitioning • Scale incrementally • Consistent hashing (ring) • Departure/arrival only affects neighbours
6. Partitioning • Scale incrementally • Consistent hashing (ring) • Departure/arrival only affects neighbours • Drawback?
7. Partitioning • Scale incrementally • Consistent hashing (ring) • Departure/arrival only affects neighbours • Drawback? ◦ Non-uniform load ◦ Heterogeneity of nodes
8. Partitioning • Scale incrementally • Consistent hashing (ring) • Departure/arrival only affects neighbours • Drawback? ◦ Non-uniform load ◦ Heterogeneity of nodes • Solution: virtual nodes
9. Replication • Replicate on N nodes • Preference-list (next N successor nodes)
10. Replication • Replicate on N nodes • Preference-list (next N successor nodes) • Drawback?
11. Replication • Replicate on N nodes • Preference-list (next N successor nodes) • Drawback? ◦ next N nodes may map to one physical node
12. Data Versioning • Eventual consistency • Conflict resolution • Vector clocks
13. Data Versioning • Eventual consistency • Conflict resolution • Vector clocks
14. Data Versioning • Eventual consistency • Conflict resolution • Vector clocks • Drawback?
15. Data Versioning • Eventual consistency • Conflict resolution • Vector clocks • Drawback? ◦ vector clock grows
16. Data Versioning • Eventual consistency • Conflict resolution • Vector clocks • Drawback? ◦ vector clock grows • Solution ◦ only size of N ◦ truncation schema
17. Get and Put Execution • Maintaining consistency • Quorum-like system ◦ R + W > N ◦ min # nodes participating in read and write
18. Get and Put Execution • Maintaining consistency • Quorum-like system ◦ R + W > N ◦ min # nodes participating in read and write • Drawback?
19. Get and Put Execution • Maintaining consistency • Quorum-like system ◦ R + W > N ◦ min # nodes participating in read and write • Drawback? ◦ latency is dominated by slowest of R (or W) replicas
20. Get and Put Execution • Maintaining consistency • Quorum-like system ◦ R + W > N ◦ min # nodes participating in read and write • Drawback? ◦ latency is dominated by slowest of R (or W) replicas • Solution ◦ R (or W) configured to be less than N
21. Temporary Failures • Sloppy quorum ◦ first N healthy nodes • Hinted handoff ◦ checked periodically • Highest level availability ◦ set W to 1
22. Permanent Failures • Replica synchronization (anti-entropy) • Merkle trees ◦ fast inconsistency detection ◦ minimize amount of data transfer
23. Permanent Failures • Replica synchronization (anti-entropy) • Merkle trees ◦ fast inconsistency detection ◦ minimize amount of data transfer • Drawback?
24. Permanent Failures • Replica synchronization (anti-entropy) • Merkle trees ◦ fast inconsistency detection ◦ minimize amount of data transfer • Drawback? ◦ key ranges change when nodes join and leave ◦ merkle trees needs to be recalculated
25. Permanent Failures • Replica synchronization (anti-entropy) • Merkle trees ◦ fast inconsistency detection ◦ minimize amount of data transfer • Drawback? ◦ key ranges change when nodes join and leave ◦ merkle trees needs to be recalculated • Solution: partition refinement (paper)
26. Membership and Failure Detection • Ring membership ◦ gossip-based protocol • External discovery ◦ logical partitions ◦ seed nodes • Failure detection ◦ gossip-based protocol
27. Evaluation • Client latencies
28. Evaluation • Client latencies
29. Evaluation • Buffered writes
30. Questions?
31. Dynamo: Amazon's Highly Available Key-value Store Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and Werner Vogels Amazon.com Muhammet Orazov