Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
由Spanner來看Google資料庫的前世今生
Search
Sponsored
·
SiteGround - Reliable hosting with speed, security, and support you can count on.
→
Szu-Kai Hsu (brucehsu)
November 07, 2012
Technology
400
4
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
由Spanner來看Google資料庫的前世今生
2012年秋,網際網路資料庫 @ 國立中正大學資工所
Szu-Kai Hsu (brucehsu)
November 07, 2012
Transcript
40 slides
由 Spanner來看 Google資料庫 的 前世今⽣生 Szu-Kai Hsu (brucehsu)
Spanner is a scalable multi-version globally-distributed synchronously-replicated database
BigTable
Handling
Handling really
Handling really BIG DATA
key-value
key-value { “CCU”: “123”, “NCTU”: “113”, “NTU”: “112” }; key
key-value { “CCU”: “123”, “NCTU”: “113”, “NTU”: “112” }; value
distributed
Lack of transaction, think of our first project.
CAP
C A P
Consistency A P
Consistency Availability P
Consistency Availability Partition tolerance
Consistency Availability Partition tolerance Consistency
Megastore
NoSQL datastores are highly scalable, but their limited API and
loose consistency models complicate application development. “ “
In Megastore, data model is declared in a strong-typed schema
strong-typed schema CREATE TABLE User { required int64 user_id; required string name; } PRIMARY KEY(user_id), ENTITY GROUP ROOT;
Based on BigTable BigTable
PRIMARY user_id PRIMARY user_id, nyan_id
Local and Global Indexes are introduced: Local Index Find corresponding
data in entity group Global Index Find corresponding data in external groups Local Index Global Index
(user_id, born,nyan_id) For local index CREATE LOCAL INDEX NyanByBorn ON
Nyan(user_id, born); CREATE LOCAL INDEX NyanByBorn ON Nyan(user_id, born);
Consistency achieved via Paxos algorithm Paxos 2 Replicas 1 Witness
At least
Replica consists of Replication server and Coordinator Replication server Coordinator
write oversee
Witness’ Replication server only writes logs logs
Average Latency: 100-400ms Poor write throughput 100-400ms
Spanner ,finally.
We believe it is better to have application programmers deal
with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions. “ “
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory Same prefix key, therefore adjacent
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory Same prefix key, therefore adjacent Fine-grained mapping
Data model is almost identical to Megastore almost identical Basic
unit defined as Directory Directory Same prefix key, therefore adjacent Fine-grained mapping Interleaved rows gain performance
Two-phase commit for distributed transactions Two-phase commit 1Vote Coordinator Participants
Two-phase commit for distributed transactions Two-phase commit 2Commit Coordinator Participants
Locking remains a big issue Locking Especially when someone went
down, causing deadlock, literally.
Paxos is here to rescue, again Paxos will make sure
ALL logs are copied to every replicas. ALL logs
Real Innovation lies in time TrueTime API utilizes atomic clock
& GPS to determine the order of each transactions atomic clock GPS
NewSQL is the new NoSQL and Spanner is the best
example so far.
More Decks by Szu-Kai Hsu (brucehsu)
See All by Szu-Kai Hsu (brucehsu)
Running Life Lean
brucehsu
0
200
Core Unleashed Part II: Introduction to GobiesVM (and STM) @ RubyKaigi 2014
brucehsu
0
2.2k
[RubyConf.tw 2014] Cores unleashed - Exploiting Parallelism in Ruby with STM
brucehsu
0
2.3k
用 Go 打造程式語言執行環境:實例剖析 [OSDC.tw 2014]
brucehsu
3
2.5k
pickbox @ OSDC.tw 2013 Lightning Talk
brucehsu
0
150
Building Web 2.0 APIs
brucehsu
1
180
Rapid Web Development by Example
brucehsu
3
3.2k
TechWed@CCU #0
brucehsu
2
630
Chromium OS
brucehsu
2
250
Other Decks in Technology
See All in Technology
小さなエージェントから始めてみよう!実践コンテキストエンジニアリング
iwamot
PRO
0
130
Terraform 0 to 100:100個以上のリポジトリを常に最新にし続けるための自動化
sue445
2
140
E2E Test というハーネス — AI時代の自動テスト民主化
daishukojima
10
5.8k
RAGの改善を速く回すための評価基盤
yosukeyoshida
6
3.3k
異夢同船〜異なる夢でも同じ船でなんとかする〜
fortegp05
1
530
HTTP通信に使おうとすると実はめんどくさいfile_get_contents
maguroalternative
0
220
変更差分を読むために環境を整備しよう
pipepasta
1
470
エンタープライズIAMについて - ID・権限をどう統制するか
oidfj
1
220
マネジメントされる側の仕事は「文句」を言うこと
riyaamemiya
7
5.8k
Biohub - Cell Tracking During Development 5th place solution
rist
0
180
gem install とrequire のむこうがわ
ydah
0
230
さようならSSH鍵、こんにちはゼロトラスト ― AIフレンドリーで快適な自宅鯖SSH
atalie2m
14
7.2k
Featured
See All Featured
Site-Speed That Sticks
csswizardry
13
1.6k
What does AI have to do with Human Rights?
axbom
PRO
1
2.5k
Self-Hosted WebAssembly Runtime for Runtime-Neutral Checkpoint/Restore in Edge–Cloud Continuum
chikuwait
0
960
Data-driven link building: lessons from a $708K investment (BrightonSEO talk)
szymonslowik
1
1.4k
The Hidden Cost of Media on the Web [PixelPalooza 2025]
tammyeverts
2
580
[SF Ruby Conf 2025] Rails X
palkan
3
1.5k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
203
77k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
250
1.3M
<Decoding/> the Language of Devs - We Love SEO 2024
nikkihalliwell
1
380
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
28
3.8k
4 Signs Your Business is Dying
shpigford
187
23k