Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scale the Google Way

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Terrence Ryan Terrence Ryan
July 30, 2015
Technology
1
110

Scale the Google Way

A discussion of issues with Google scaling technologies

Avatar for Terrence Ryan

Terrence Ryan

July 30, 2015
Tweet

More Decks by Terrence Ryan

See All by Terrence Ryan
Vms, Serverless, or Containers
Avatar for Terrence Ryan tpryan
0
640
Go for PHP Developers
Avatar for Terrence Ryan tpryan
2
1.1k
Navigating Google Cloud Platform
Avatar for Terrence Ryan tpryan
0
450
Which Engine?
Avatar for Terrence Ryan tpryan
0
400
Introduction to Containers and Kubernetes
Avatar for Terrence Ryan tpryan
2
390
Cloud Next 2017 Roundup
Avatar for Terrence Ryan tpryan
2
180
LAMP in Containers
Avatar for Terrence Ryan tpryan
1
310
GCP and IoT
Avatar for Terrence Ryan tpryan
0
500
30,000,000 Requests in a Hour in the Cloud
Avatar for Terrence Ryan tpryan
2
400

Other Decks in Technology

See All in Technology
僕、S3 　シンプルって名前だけど全然シンプルじゃありません　よろしくお願いします
Avatar for Yuuki Yamashita yama3133
1
230
生成AIで速度と品質を両立する、QAエンジニア・開発者連携のAI協調型テストプロセス
Avatar for butatamasoba / Shota Kusaba shota_kusaba
0
210
It’s “Time” to use Temporal
Avatar for Saji sajikix
3
230
コンテキスト・ハーネスエンジニアリングの現在
Avatar for Hirosato Gamo hirosatogamo
PRO
5
550
猫でもわかるKiro CLI(AI 駆動開発への道編)
Avatar for Hiroo Katoh kentapapa
0
270
CyberAgentの生成AI戦略 〜変わるものと変わらないもの〜
Avatar for katayan katayan
0
280
Zeal of the Convert: Taming Shai-Hulud with AI
Avatar for Rami McCarthy ramimac
0
150
AlloyDB 奮闘記
Avatar for hatappi hatappi
0
160
Go 1.26 Genericsにおける再帰的型制約 / Recursive Type Constraints in Go 1.26 Generics
Avatar for turbofish ryokotmng
0
140
プラットフォームエンジニアリングはAI時代の開発者をどう救うのか
Avatar for Kazuto Kusama jacopen
7
3.9k
実践 Datadog MCP Server
Avatar for 株式会社ヌーラボ nulabinc
PRO
2
240
複数クラスタ運用と検索の高度化：ビズリーチにおけるElastic活用事例 / ElasticON Tokyo2026
Avatar for Visional Engineering ＆ Design visional_engineering_and_design
0
170

Featured

See All Featured
How to Grow Your eCommerce with AI & Automation
Avatar for Katarina Dahlin katarinadahlin
PRO
1
150
The Hidden Cost of Media on the Web [PixelPalooza 2025]
Avatar for Tammy Everts tammyeverts
2
250
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
Avatar for Dr. Kim W Petersen kimpetersen
PRO
0
270
New Earth Scene 8
Avatar for Sydney Stone popppiees
1
1.7k
Pawsitive SEO: Lessons from My Dog (and Many Mistakes) on Thriving as a Consultant in the Age of AI
Avatar for David Carrasco davidcarrasco
0
89
Measuring Dark Social's Impact On Conversion and Attribution
Avatar for Stephen Akadiri stephenakadiri
1
160
What the history of the web can teach us about the future of AI
Avatar for Ines Montani inesmontani
PRO
1
480
B2B Lead Gen: Tactics, Traps & Triumph
Avatar for Sophie Logan marketingsoph
0
80
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
51
52k
Conquering PDFs: document understanding beyond plain text
Avatar for Ines Montani inesmontani
PRO
4
2.5k
Speed Design
Avatar for Sergey Chernyshev sergeychernyshev
33
1.6k
RailsConf 2023
Avatar for Aaron Patterson tenderlove
30
1.4k

Transcript

  1. @tpryan Scale the Google Way Terry Ryan Developer Advocate

  2. @tpryan Who are you?

  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None

  10. @tpryan First Cluster History

  11. @tpryan First Cluster •Every machine had a role History

  12. @tpryan First Cluster •Every machine had a role •Every machine

    had a name History

  13. @tpryan First Rack History

  14. @tpryan First Rack •Machines belong to groups History

  15. @tpryan First Rack •Machines belong to groups •Slack capacity goes

    to waste History

  16. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999

  17. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000

  18. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000

  19. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000

  20. @tpryan Today’s Numbers History

  21. @tpryan Today’s Numbers •Over 3 Billion searches a day History

  22. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. History

  23. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day History

  24. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History

  25. @tpryan Handling this growth took fundamental changes

  26. Networking

  27. 70 POP’s 33 Countries B4 WAN

  28. Jupiter Superblock

  29. Jupiter Superblock • 1 Petabit/sec

  30. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

  31. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

    • 10 Libraries of Congress a second

  32. Storage

  33. Colossus Datacenter wide storage

  34. Processing

  35. Productionized Containers

  36. Productionized Containers • Developed as only practical way to manage

    Google-scale compute

  37. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container

  38. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.

  39. @tpryan What does this mean for you?

  40. @tpryan

  41. @tpryan Big Table

  42. @tpryan Big Table

  43. @tpryan Big Table MapReduce

  44. @tpryan Big Table MapReduce

  45. @tpryan Big Table Dremel MapReduce

  46. @tpryan Big Table Dremel MapReduce

  47. @tpryan Big Table Borg Dremel MapReduce

  48. @tpryan Big Table Borg Dremel MapReduce

  49. @tpryan

  50. @tpryan Google Cloud Platform

  51. @tpryan Google Cloud Platform Compute

  52. @tpryan Google Cloud Platform Compute Storage

  53. @tpryan Google Cloud Platform Compute Connectivity Storage

  54. @tpryan Google Cloud Platform Compute Connectivity Big Data Storage

  55. @tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big

    Data Storage Management

  56. @tpryan Conclusions

  57. None
  58. None

  59. Your App Here

  60. @tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work

    is licensed under a Creative Commons Attribution 2.0 Generic License.