Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scale the Google Way
Search
Terrence Ryan
July 30, 2015
Technology
1
81
Scale the Google Way
A discussion of issues with Google scaling technologies
Terrence Ryan
July 30, 2015
Tweet
Share
More Decks by Terrence Ryan
See All by Terrence Ryan
Vms, Serverless, or Containers
tpryan
0
510
Go for PHP Developers
tpryan
2
880
Navigating Google Cloud Platform
tpryan
0
350
Which Engine?
tpryan
0
300
Introduction to Containers and Kubernetes
tpryan
2
300
Cloud Next 2017 Roundup
tpryan
2
110
LAMP in Containers
tpryan
1
220
GCP and IoT
tpryan
0
260
30,000,000 Requests in a Hour in the Cloud
tpryan
2
290
Other Decks in Technology
See All in Technology
DevOpsメトリクスとアウトカムの接続にトライ!開発プロセスを通して計測できるメトリクスの活用方法
ham0215
2
230
Cloud Native Java with Spring Boot (CNCF Aarhus, April 2024)
thomasvitale
1
160
データベース02: データベースの概念
trycycle
0
150
KubeCon EU 2024 Recap “Kubernetes Policy Time Machine: Where to Next?”
ryysud
0
210
日本におけるデータエンジニアリングのこれまでとこれから
foursue
16
4.2k
よく聞くけど使ったことないソフトウェアNo.1 KafkaとSnowflake
foursue
4
350
FrontDoorとWebAppsを組み合わせた際のリダイレクト処理の注意点
kenichirokimura
1
500
プラットフォームってつくることより計測することが重要なんじゃないかという話 / Platform Engineering Meetup #8
taishin
1
340
サーバー間 GraphQL と webmock-graphql の話 / server-to-server graphql and webmock-graphql
qsona
2
180
Python と Snowflake はズッ友だょ!~ Snowflake の Python 関連機能をふりかえる ~
__allllllllez__
1
110
現代CSSフレームワークの内部実装とその仕組み
poteboy
8
3.6k
複雑な構成要素を持つUIとの向き合い方 〜新・支出グラフでの実例〜 / B43 TECH TALK
nakamuuu
0
140
Featured
See All Featured
Building Adaptive Systems
keathley
31
1.9k
The World Runs on Bad Software
bkeepers
PRO
61
6.7k
Done Done
chrislema
178
15k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
21
1.6k
Building Better People: How to give real-time feedback that sticks.
wjessup
355
18k
The Language of Interfaces
destraynor
151
23k
Infographics Made Easy
chrislema
238
18k
Clear Off the Table
cherdarchuk
84
310k
Automating Front-end Workflow
addyosmani
1356
200k
Music & Morning Musume
bryan
41
5.6k
Fireside Chat
paigeccino
21
2.6k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
19
1.7k
Transcript
@tpryan Scale the Google Way Terry Ryan Developer Advocate
@tpryan Who are you?
None
None
None
None
None
None
None
@tpryan First Cluster History
@tpryan First Cluster •Every machine had a role History
@tpryan First Cluster •Every machine had a role •Every machine
had a name History
@tpryan First Rack History
@tpryan First Rack •Machines belong to groups History
@tpryan First Rack •Machines belong to groups •Slack capacity goes
to waste History
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000
November 1998 December 1998 January 1999 February 1999 March 1999
April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000
@tpryan Today’s Numbers History
@tpryan Today’s Numbers •Over 3 Billion searches a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day History
@tpryan Today’s Numbers •Over 3 Billion searches a day •On
a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History
@tpryan Handling this growth took fundamental changes
Networking
70 POP’s 33 Countries B4 WAN
Jupiter Superblock
Jupiter Superblock • 1 Petabit/sec
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s
• 10 Libraries of Congress a second
Storage
Colossus Datacenter wide storage
Processing
Productionized Containers
Productionized Containers • Developed as only practical way to manage
Google-scale compute
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container
Productionized Containers • Developed as only practical way to manage
Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.
@tpryan What does this mean for you?
@tpryan
@tpryan Big Table
@tpryan Big Table
@tpryan Big Table MapReduce
@tpryan Big Table MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan Big Table Borg Dremel MapReduce
@tpryan
@tpryan Google Cloud Platform
@tpryan Google Cloud Platform Compute
@tpryan Google Cloud Platform Compute Storage
@tpryan Google Cloud Platform Compute Connectivity Storage
@tpryan Google Cloud Platform Compute Connectivity Big Data Storage
@tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big
Data Storage Management
@tpryan Conclusions
None
None
Your App Here
@tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work
is licensed under a Creative Commons Attribution 2.0 Generic License.