Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale the Google Way
Search
Terrence Ryan
July 30, 2015
Technology
1
83
Scale the Google Way
A discussion of issues with Google scaling technologies
Terrence Ryan
July 30, 2015
Tweet
Share
More Decks by Terrence Ryan
See All by Terrence Ryan
Vms, Serverless, or Containers
tpryan
0
540
Go for PHP Developers
tpryan
2
940
Navigating Google Cloud Platform
tpryan
0
350
Which Engine?
tpryan
0
310
Introduction to Containers and Kubernetes
tpryan
2
330
Cloud Next 2017 Roundup
tpryan
2
120
LAMP in Containers
tpryan
1
230
GCP and IoT
tpryan
0
290
30,000,000 Requests in a Hour in the Cloud
tpryan
2
320
Other Decks in Technology
See All in Technology
LLMアプリケーションの評価の実践と課題 ~PharmaXにおける今後の展望~
pharma_x_tech
2
160
ここがすごいよ! AWS Systems Manager!
saichan11
0
1.8k
初中級者用如何使用backlog -VALE TUDOEDITION-
in0u
0
140
セキュリティ研修 Day1【MIXI 24新卒技術研修】
mixi_engineers
PRO
0
160
AIアシスタントの活用で品質の向上と開発ワークフローのスピードアップ
nagix
1
200
開発と事業を繋ぐ!SREのオブザーバビリティ戦略 ~ Developers Summit 2024 Summer ~
leveragestech
0
630
dxd2024-生成AIに振り回された3か月間の成功と失敗/dxd2024-link-and-motivation
lmi
2
260
地理情報とAPIのトレンド
nagix
0
160
シフトレフトで挑む セキュリティの生産性向上
sekido
PRO
0
270
運用改善、不都合な真実 / 20240722-ssmjp-kaizen
opelab
17
8k
テストケースの自動生成に生成AIの導入を試みた話と生成AIによる今後の期待
shift_evolve
0
180
DevIO2024_レガシー運用からの脱却 -クラウド活用の実践事例とベストプラクティス-
jun2882
0
210
Featured
See All Featured
GraphQLの誤解/rethinking-graphql
sonatard
59
9.6k
Building Flexible Design Systems
yeseniaperezcruz
323
37k
Git: the NoSQL Database
bkeepers
PRO
423
64k
A Philosophy of Restraint
colly
200
16k
From Idea to $5000 a Month in 5 Months
shpigford
377
46k
Making Projects Easy
brettharned
111
5.7k
Writing Fast Ruby
sferik
623
60k
Thoughts on Productivity
jonyablonski
64
4.1k
Web development in the modern age
philhawksworth
203
10k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
34
1.9k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
90
47k
The Illustrated Children's Guide to Kubernetes
chrisshort
39
47k
Transcript
@tpryan Scale the Google Way Terry Ryan Developer Advocate
@tpryan Who are you?
None
None
None
None
None
None
None
@tpryan First Cluster History
@tpryan First Cluster •Every machine had a role History
@tpryan First Cluster •Every machine had a role •Every machine
had a name History
@tpryan First Rack History
@tpryan First Rack •Machines belong to groups History
@tpryan First Rack •Machines belong to groups •Slack capacity goes
to waste History
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000
@tpryan Today’s Numbers History
@tpryan Today’s Numbers •Over 3 Billion searches a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History
@tpryan Handling this growth took fundamental changes
Networking
70 POP’s 33 Countries B4 WAN
Jupiter Superblock
Jupiter Superblock • 1 Petabit/sec
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
• 10 Libraries of Congress a second
Storage
Colossus Datacenter wide storage
Processing
Productionized Containers
Productionized Containers • Developed as only practical way to manage
Google-scale compute
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.
@tpryan What does this mean for you?
@tpryan
@tpryan Big Table
@tpryan Big Table
@tpryan Big Table MapReduce
@tpryan Big Table MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan
@tpryan Google Cloud Platform
@tpryan Google Cloud Platform Compute
@tpryan Google Cloud Platform Compute Storage
@tpryan Google Cloud Platform Compute Connectivity Storage
@tpryan Google Cloud Platform Compute Connectivity Big Data Storage
@tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big
Data Storage Management
@tpryan Conclusions
None
None
Your App Here
@tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work
is licensed under a Creative Commons Attribution 2.0 Generic License.