Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scale the Google Way

Avatar for Terrence Ryan Terrence Ryan
July 30, 2015
Technology
1
93

Scale the Google Way

A discussion of issues with Google scaling technologies
Avatar for Terrence Ryan

Terrence Ryan

July 30, 2015
Tweet

More Decks by Terrence Ryan

See All by Terrence Ryan
Vms, Serverless, or Containers
Avatar for Terrence Ryan tpryan
0
610
Go for PHP Developers
Avatar for Terrence Ryan tpryan
2
1k
Navigating Google Cloud Platform
Avatar for Terrence Ryan tpryan
0
400
Which Engine?
Avatar for Terrence Ryan tpryan
0
370
Introduction to Containers and Kubernetes
Avatar for Terrence Ryan tpryan
2
360
Cloud Next 2017 Roundup
Avatar for Terrence Ryan tpryan
2
150
LAMP in Containers
Avatar for Terrence Ryan tpryan
1
270
GCP and IoT
Avatar for Terrence Ryan tpryan
0
390
30,000,000 Requests in a Hour in the Cloud
Avatar for Terrence Ryan tpryan
2
350

Other Decks in Technology

See All in Technology
「良さそう」と「とても良い」の間には 「良さそうだがホンマか」がたくさんある / 2025.07.01 LLM品質Night
Avatar for Shumpei Miyawaki smiyawaki0820
1
500
CursorによるPMO業務の代替 / Automating PMO Tasks with Cursor
Avatar for Motoyoshi Kosei motoyoshi_kakaku
2
930
Yamla: Rustでつくるリアルタイム性を追求した機械学習基盤 / Yamla: A Rust-Based Machine Learning Platform Pursuing Real-Time Capabilities
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
4
230
AWS認定を取る中で感じたこと
Avatar for Siromi siromi
1
170
PO初心者が考えた ”POらしさ”
Avatar for (Ha)rady nb_rady
0
190
「クラウドコスト絶対削減」を支える技術—FinOpsを超えた徹底的なクラウドコスト削減の実践論
Avatar for 株式会社DELTA delta_tech
4
140
改めてAWS WAFを振り返る～業務で使うためのポイント～
Avatar for モブエンジニア masakiokuda
2
240
整頓のジレンマとの戦い〜Tidy First?で振り返る事業とキャリアの歩み〜/Fighting the tidiness dilemma〜Business and Career Milestones Reflected on in Tidy First?〜
Avatar for 株式会社ビットキー / Bitkey Inc. bitkey
2
14k
asken AI勉強会(Android)
Avatar for 佐藤忠 tadashi_sato
0
180
React開発にStorybookとCopilotを導入して、爆速でUIを編集・確認する方法
Avatar for yut yu_kod
1
230
【５分でわかる】セーフィー エンジニア向け会社紹介
Avatar for Safie safie_recruit
0
27k
fukabori.fm 出張版: 売上高617億円と高稼働率を陰で支えた社内ツール開発のあれこれ話 / 20250704 Yoshimasa Iwase & Tomoo Morikawa
Avatar for SHIFT EVOLVE shift_evolve
PRO
2
6.8k

Featured

See All Featured
The Cost Of JavaScript in 2023
Avatar for Addy Osmani addyosmani
51
8.5k
Facilitating Awesome Meetings
Avatar for Lara Hogan lara
54
6.4k
How to Think Like a Performance Engineer
Avatar for Harry Roberts csswizardry
24
1.7k
The Language of Interfaces
Avatar for Des Traynor destraynor
158
25k
Unsuck your backbone
Avatar for Amy Palamountain ammeep
671
58k
Testing 201, or: Great Expectations
Avatar for Joseph Mastey jmmastey
42
7.6k
Balancing Empowerment & Direction
Avatar for Lara Hogan lara
1
420
Raft: Consensus for Rubyists
Avatar for Patrick Van Stee vanstee
140
7k
Designing for Performance
Avatar for Lara Hogan lara
610
69k
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
357
30k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
Avatar for Kevin Liebholz kevinliebholz
34
5.9k
The Art of Programming - Codeland 2020
Avatar for Erika Heidi erikaheidi
54
13k

Transcript

  1. @tpryan Scale the Google Way Terry Ryan Developer Advocate

  2. @tpryan Who are you?

  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None

  10. @tpryan First Cluster History

  11. @tpryan First Cluster •Every machine had a role History

  12. @tpryan First Cluster •Every machine had a role •Every machine

    had a name History

  13. @tpryan First Rack History

  14. @tpryan First Rack •Machines belong to groups History

  15. @tpryan First Rack •Machines belong to groups •Slack capacity goes

    to waste History

  16. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999

  17. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000

  18. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000

  19. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000

  20. @tpryan Today’s Numbers History

  21. @tpryan Today’s Numbers •Over 3 Billion searches a day History

  22. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. History

  23. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day History

  24. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History

  25. @tpryan Handling this growth took fundamental changes

  26. Networking

  27. 70 POP’s 33 Countries B4 WAN

  28. Jupiter Superblock

  29. Jupiter Superblock • 1 Petabit/sec

  30. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

  31. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

    • 10 Libraries of Congress a second

  32. Storage

  33. Colossus Datacenter wide storage

  34. Processing

  35. Productionized Containers

  36. Productionized Containers • Developed as only practical way to manage

    Google-scale compute

  37. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container

  38. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.

  39. @tpryan What does this mean for you?

  40. @tpryan

  41. @tpryan Big Table

  42. @tpryan Big Table

  43. @tpryan Big Table MapReduce

  44. @tpryan Big Table MapReduce

  45. @tpryan Big Table Dremel MapReduce

  46. @tpryan Big Table Dremel MapReduce

  47. @tpryan Big Table Borg Dremel MapReduce

  48. @tpryan Big Table Borg Dremel MapReduce

  49. @tpryan

  50. @tpryan Google Cloud Platform

  51. @tpryan Google Cloud Platform Compute

  52. @tpryan Google Cloud Platform Compute Storage

  53. @tpryan Google Cloud Platform Compute Connectivity Storage

  54. @tpryan Google Cloud Platform Compute Connectivity Big Data Storage

  55. @tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big

    Data Storage Management

  56. @tpryan Conclusions

  57. None
  58. None

  59. Your App Here

  60. @tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work

    is licensed under a Creative Commons Attribution 2.0 Generic License.