Databases, Part 1: The Theory of Scale

This talk will compare and contrast the different approaches to modern database design. Starting with history, we will look at how databases have evolved, and what use cases have driven their development. Ending with theory, we will look at the tradeoffs between different database designs.

Transcript

1. The Origin of Databases How SQL and NoSQL Evolved
2. Databases Part 1: The Theory of Scale
3. /whois • Myles Megyesi • Software Craftsman at 8th Light
4. The CAP Theorem It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: • Consistency • Availability • Partition tolerance
5. Define the CAP theorem in terms of relational databases
6. Consistency All nodes see the same data at the same time Scenario: Two customers on Amazon are trying to buy a book at the same time, but there is only one book left. How do we guarantee one success and one failure? • DB Transaction: ◦ decrement the quantity of the book ◦ assign the book to the customer
7. Availability Simply put: the server responds • Lots of requests • Lots of data Solutions: • Vertical scaling ◦ Mecha server (68 GB cache) • Horizontal scaling (increase number of servers) ◦ read slaves ◦ shards
8. Partition tolerance the system continues to operate despite arbitrary message loss or failure of part of the system
9. Partition tolerance: read slaves Client Client Client Master (writes) Slave (read) Slave (read) ?
11. Partition tolerance: sharding Client Client Client Other tables Some tables
12. Relational databases (C camp) • Consistent (via atomic operations) • Available (via vertical scaling or some horizontal scale) • Not partition tolerant
13. Dealing with CAP: allow partitions Guarantee Consistency/Availability • Master/Slave ◦ Wait until master comes back online • Sharding ◦ Wait until shard(s) comes back online • On every write, wait until data is consistent before serving more requests ◦ Extreme latency cost
14. Dealing with CAP: allow inconsistency Guarantee Availability/Partition Tolerance • At some point, all writes will be propagated throughout the system • Keep accepting writes when a partition is down, figure out the conflicts when the system is whole again
15. NoSQL (A camp)
16. Eventual Consistency the storage system guarantees that if no new updates are made to the object, eventually all accesses will return the last updated value Can write to any node, not just master
17. Availability • All nodes are equal ◦ No single point of failure • Horizontal Scaling ◦ Add as many nodes as needed ◦ In different regions
18. Node Node Node Node Node Node Client Client Client
19. Partition Tolerance Node Node Node Node Node Node Node Node Portland Virginia Node Node Node Node London
20. Partition Tolerance • Read/Write to any node ◦ Figure out the conflicts later
21. Consistency Revisited Scenario: Two customers on Amazon are trying to buy a book at the same time, but there is only one book left. How do we guarantee one success and one failure?
22. Conclusion • C camp ◦ Consistency over partition tolerance • A Camp ◦ Availability over consistency