Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scale the Google Way

Avatar for Terrence Ryan Terrence Ryan
July 30, 2015
Technology
120
1

Scale the Google Way

A discussion of issues with Google scaling technologies

Avatar for Terrence Ryan

Terrence Ryan

July 30, 2015

More Decks by Terrence Ryan

See All by Terrence Ryan
Vms, Serverless, or Containers
Avatar for Terrence Ryan tpryan
0
660
Go for PHP Developers
Avatar for Terrence Ryan tpryan
2
1.1k
Navigating Google Cloud Platform
Avatar for Terrence Ryan tpryan
0
470
Which Engine?
Avatar for Terrence Ryan tpryan
0
420
Introduction to Containers and Kubernetes
Avatar for Terrence Ryan tpryan
2
410
Cloud Next 2017 Roundup
Avatar for Terrence Ryan tpryan
2
200
LAMP in Containers
Avatar for Terrence Ryan tpryan
1
320
GCP and IoT
Avatar for Terrence Ryan tpryan
0
510
30,000,000 Requests in a Hour in the Cloud
Avatar for Terrence Ryan tpryan
2
410

Other Decks in Technology

See All in Technology
BigQuery を検索ソースとした AI Agent の作り方って 〇〇 通りあんねん
Avatar for SatohJohn satohjohn
0
150
plamo-3-translateの開発
Avatar for Preferred Networks pfn
PRO
0
240
13年運用タイトルのサーバーサイドが辿り着いた現在地 ― モンスターストライクにおける技術・組織・AI活用から得た知見
Avatar for MIXI ENGINEERS mixi_engineers
PRO
1
380
PLaMoを毎日の開発で使い育てていく
Avatar for Preferred Networks pfn
PRO
0
170
エンタープライズデータへ安全につなぐ Production-ready なエージェント設計 ― AI × MCP リファレンスアーキテクチャ ― #AIDevDay
Avatar for CData Software Japan cdataj
1
450
AI研修(Day1)【MIXI 26新卒技術研修】
Avatar for MIXI ENGINEERS mixi_engineers
PRO
2
3.2k
テックカンファレンス三大ステークホルダーの文化人類学 ─ 違いを認め合う関係性作り
Avatar for bash0C7 bash0c7
5
1.5k
AIがコードを書く時代、人間は何を保証するのか———馬場さんと考える、開発者に求められる新しい責任と価値 - TECH PLAY
Avatar for Toshiaki Baba netmarkjp
0
1k
AI工学特論: MLOps・継続的評価
Avatar for Asei Sugiyama asei
11
3.1k
データと地図で読む 大井町の「かわるもの、かわらないもの」
Avatar for じゅんじゅん yoshiyama_hana
0
900
SnowflakeCoCoでデータエンジニアリング！
Avatar for numa foursue
0
110
論語・武士道・産業革命から見る　かわるもの、かわらないもの
Avatar for nakamichi ichimichi
8
2.1k

Featured

See All Featured
The agentic SEO stack - context over prompts
Avatar for schlessera schlessera
0
860
How to Align SEO within the Product Triangle To Get 
Buy-In & Support - #RIMC
Avatar for Aleyda Solis aleyda
2
1.7k
Skip the Path - Find Your Career Trail
Avatar for Mark Kilby mkilby
1
170
Digital Ethics as a Driver of Design Innovation
Avatar for Per Axbom axbom
PRO
1
350
Making Projects Easy
Avatar for Brett Harned brettharned
120
6.7k
Designing for Timeless Needs
Avatar for Cassini Nazir cassininazir
1
430
Documentation Writing (for coders)
Avatar for Carmen Chung carmenintech
77
5.4k
Thoughts on Productivity
Avatar for Jon Yablonski jonyablonski
76
5.3k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
56
3.4k
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
408
67k
Designing Powerful Visuals for Engaging Learning
Avatar for Mike Taylor tmiket
1
470
Design of three-dimensional binary manipulators for pick-and-place task avoiding obstacles (IECON2024)
Avatar for konakalab konakalab
0
500

Transcript

  1. @tpryan Scale the Google Way Terry Ryan Developer Advocate

  2. @tpryan Who are you?

  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None

  10. @tpryan First Cluster History

  11. @tpryan First Cluster •Every machine had a role History

  12. @tpryan First Cluster •Every machine had a role •Every machine

    had a name History

  13. @tpryan First Rack History

  14. @tpryan First Rack •Machines belong to groups History

  15. @tpryan First Rack •Machines belong to groups •Slack capacity goes

    to waste History

  16. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999

  17. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000

  18. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000

  19. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000

  20. @tpryan Today’s Numbers History

  21. @tpryan Today’s Numbers •Over 3 Billion searches a day History

  22. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. History

  23. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day History

  24. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History

  25. @tpryan Handling this growth took fundamental changes

  26. Networking

  27. 70 POP’s 33 Countries B4 WAN

  28. Jupiter Superblock

  29. Jupiter Superblock • 1 Petabit/sec

  30. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

  31. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

    • 10 Libraries of Congress a second

  32. Storage

  33. Colossus Datacenter wide storage

  34. Processing

  35. Productionized Containers

  36. Productionized Containers • Developed as only practical way to manage

    Google-scale compute

  37. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container

  38. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.

  39. @tpryan What does this mean for you?

  40. @tpryan

  41. @tpryan Big Table

  42. @tpryan Big Table

  43. @tpryan Big Table MapReduce

  44. @tpryan Big Table MapReduce

  45. @tpryan Big Table Dremel MapReduce

  46. @tpryan Big Table Dremel MapReduce

  47. @tpryan Big Table Borg Dremel MapReduce

  48. @tpryan Big Table Borg Dremel MapReduce

  49. @tpryan

  50. @tpryan Google Cloud Platform

  51. @tpryan Google Cloud Platform Compute

  52. @tpryan Google Cloud Platform Compute Storage

  53. @tpryan Google Cloud Platform Compute Connectivity Storage

  54. @tpryan Google Cloud Platform Compute Connectivity Big Data Storage

  55. @tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big

    Data Storage Management

  56. @tpryan Conclusions

  57. None
  58. None

  59. Your App Here

  60. @tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work

    is licensed under a Creative Commons Attribution 2.0 Generic License.