Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
由Spanner來看Google資料庫的前世今生
Search
Szu-Kai Hsu (brucehsu)
November 07, 2012
Technology
4
270
由Spanner來看Google資料庫的前世今生
2012年秋,網際網路資料庫 @ 國立中正大學資工所
Szu-Kai Hsu (brucehsu)
November 07, 2012
Tweet
Share
More Decks by Szu-Kai Hsu (brucehsu)
See All by Szu-Kai Hsu (brucehsu)
Running Life Lean
brucehsu
0
160
Core Unleashed Part II: Introduction to GobiesVM (and STM) @ RubyKaigi 2014
brucehsu
0
1.9k
[RubyConf.tw 2014] Cores unleashed - Exploiting Parallelism in Ruby with STM
brucehsu
0
2.1k
用 Go 打造程式語言執行環境:實例剖析 [OSDC.tw 2014]
brucehsu
3
2.3k
pickbox @ OSDC.tw 2013 Lightning Talk
brucehsu
0
53
Building Web 2.0 APIs
brucehsu
1
140
Rapid Web Development by Example
brucehsu
3
3.1k
TechWed@CCU #0
brucehsu
2
450
Chromium OS
brucehsu
2
190
Other Decks in Technology
See All in Technology
次は君だ。~Japan AWS Jr. Champions 受賞までの奇跡~
fukuchiiinu
0
120
多数のWebサービスをECS/Fargate構成で効率よく構築・運用するなら copilot-cli
interu
2
170
Cosmos DB で持続可能な RAG を実現しよう!~ AOAI Dev Day ふりかえりを添えて / Sustainable RAG with Cosmos DB with recap AOAI Dev Day
miyake
0
180
RAG: from dumb implementation to serious results
glaforge
0
670
Amazon Managed Grafana で AWS IoT TwinMaker によるデジタルツインアプリケーションを動かしてみた
wakatsuki
0
120
不要なリソースを自動で定期的に整理する方法 ~Sandboxアカウントのコストを削減しよう!~
amixedcolor
4
240
パートナー企業のテクニカルサポートエンジニアとして気になる、より良い AWS サポートの利活用について
kazzpapa3
1
290
Demystifying Vite Internals
nozomuikuta
3
760
塩野義製薬様のAWS統合管理戦略:Organizations設計と運用の具体例
tkikuchi
0
320
APIs for AI: Have we failed?
zdne
0
130
管理画面とユーザー機能の調和を取り戻す!~クエリパフォーマンス改善の成功物語~ / Restore harmony between administrative and user functions!
minisera
1
320
寒冷地稲作の歴史にみるコミュニティ
miu_crescent
2
140
Featured
See All Featured
Happy Clients
brianwarren
97
6.7k
Creating an realtime collaboration tool: Agile Flush - .NET Oxford
marcduiker
25
1.8k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
5
130
A better future with KSS
kneath
237
17k
The Invisible Side of Design
smashingmag
297
50k
The Art of Programming - Codeland 2020
erikaheidi
51
13k
Automating Front-end Workflow
addyosmani
1365
200k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
13
1.8k
Optimizing for Happiness
mojombo
376
69k
Writing Fast Ruby
sferik
626
60k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
3
340
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
167
49k
Transcript
由 Spanner來看 Google資料庫 的 前世今⽣生 Szu-Kai Hsu (brucehsu)
Spanner is a scalable multi-version globally-distributed synchronously-replicated database
BigTable
Handling
Handling really
Handling really BIG DATA
key-value
key-value { “CCU”: “123”, “NCTU”: “113”, “NTU”: “112” }; key
key-value { “CCU”: “123”, “NCTU”: “113”, “NTU”: “112” }; value
distributed
Lack of transaction, think of our first project.
CAP
C A P
Consistency A P
Consistency Availability P
Consistency Availability Partition tolerance
Consistency Availability Partition tolerance Consistency
Megastore
NoSQL datastores are highly scalable, but their limited API and
loose consistency models complicate application development. “ “
In Megastore, data model is declared in a strong-typed schema
strong-typed schema CREATE TABLE User { required int64 user_id; required string name; } PRIMARY KEY(user_id), ENTITY GROUP ROOT;
Based on BigTable BigTable
PRIMARY user_id PRIMARY user_id, nyan_id
Local and Global Indexes are introduced: Local Index Find corresponding
data in entity group Global Index Find corresponding data in external groups Local Index Global Index
(user_id, born,nyan_id) For local index CREATE LOCAL INDEX NyanByBorn ON
Nyan(user_id, born); CREATE LOCAL INDEX NyanByBorn ON Nyan(user_id, born);
Consistency achieved via Paxos algorithm Paxos 2 Replicas 1 Witness
At least
Replica consists of Replication server and Coordinator Replication server Coordinator
write oversee
Witness’ Replication server only writes logs logs
Average Latency: 100-400ms Poor write throughput 100-400ms
Spanner ,finally.
We believe it is better to have application programmers deal
with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions. “ “
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory Same prefix key, therefore adjacent
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory Same prefix key, therefore adjacent Fine-grained mapping
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory Same prefix key, therefore adjacent Fine-grained mapping Interleaved rows gain performance
Two-phase commit for distributed transactions Two-phase commit 1Vote Coordinator Participants
Two-phase commit for distributed transactions Two-phase commit 2Commit Coordinator Participants
Locking remains a big issue Locking Especially when someone went
down, causing deadlock, literally.
Paxos is here to rescue, again Paxos will make sure
ALL logs are copied to every replicas. ALL logs
Real Innovation lies in time TrueTime API utilizes atomic clock
& GPS to determine the order of each transactions atomic clock GPS
NewSQL is the new NoSQL and Spanner is the best
example so far.