由Spanner來看Google資料庫的前世今生

2012年秋，網際網路資料庫 @ 國立中正大學資工所

Transcript

1. 由 Spanner來看 Google資料庫 的 前世今⽣生 Szu-Kai Hsu (brucehsu)
2. Spanner is a scalable multi-version globally-distributed synchronously-replicated database
3. BigTable
4. Handling
5. Handling really
6. Handling really BIG DATA
7. key-value
8. key-value { “CCU”: “123”, “NCTU”: “113”, “NTU”: “112” }; key
9. key-value { “CCU”: “123”, “NCTU”: “113”, “NTU”: “112” }; value
10. distributed
11. Lack of transaction, think of our first project.
12. CAP
13. C A P
14. Consistency A P
15. Consistency Availability P
16. Consistency Availability Partition tolerance
17. Consistency Availability Partition tolerance Consistency
18. Megastore
19. NoSQL datastores are highly scalable, but their limited API and loose consistency models complicate application development. “ “
20. In Megastore, data model is declared in a strong-typed schema strong-typed schema CREATE TABLE User { required int64 user_id; required string name; } PRIMARY KEY(user_id), ENTITY GROUP ROOT;
21. Based on BigTable BigTable
22. PRIMARY user_id PRIMARY user_id, nyan_id
23. Local and Global Indexes are introduced: Local Index Find corresponding data in entity group Global Index Find corresponding data in external groups Local Index Global Index
24. (user_id, born,nyan_id) For local index CREATE LOCAL INDEX NyanByBorn ON Nyan(user_id, born); CREATE LOCAL INDEX NyanByBorn ON Nyan(user_id, born);
25. Consistency achieved via Paxos algorithm Paxos 2 Replicas 1 Witness At least
26. Replica consists of Replication server and Coordinator Replication server Coordinator write oversee
27. Witness’ Replication server only writes logs logs
28. Average Latency: 100-400ms Poor write throughput 100-400ms
29. Spanner ,finally.
30. We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions. “ “
31. Data model is almost identical to Megastore almost identical Basic unit defined as Directory Directory
32. Data model is almost identical to Megastore almost identical Basic unit defined as Directory Directory Same prefix key, therefore adjacent
33. Data model is almost identical to Megastore almost identical Basic unit defined as Directory Directory Same prefix key, therefore adjacent Fine-grained mapping
34. Data model is almost identical to Megastore almost identical Basic unit defined as Directory Directory Same prefix key, therefore adjacent Fine-grained mapping Interleaved rows gain performance
35. Two-phase commit for distributed transactions Two-phase commit 1Vote Coordinator Participants
36. Two-phase commit for distributed transactions Two-phase commit 2Commit Coordinator Participants
37. Locking remains a big issue Locking Especially when someone went down, causing deadlock, literally.
38. Paxos is here to rescue, again Paxos will make sure ALL logs are copied to every replicas. ALL logs
39. Real Innovation lies in time TrueTime API utilizes atomic clock & GPS to determine the order of each transactions atomic clock GPS
40. NewSQL is the new NoSQL and Spanner is the best example so far.