Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Scale the Google Way

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on. →
Avatar for Terrence Ryan Terrence Ryan
July 30, 2015
Technology
110
1

Scale the Google Way

A discussion of issues with Google scaling technologies

Avatar for Terrence Ryan

Terrence Ryan

July 30, 2015

More Decks by Terrence Ryan

See All by Terrence Ryan
Vms, Serverless, or Containers
Avatar for Terrence Ryan tpryan
0
650
Go for PHP Developers
Avatar for Terrence Ryan tpryan
2
1.1k
Navigating Google Cloud Platform
Avatar for Terrence Ryan tpryan
0
460
Which Engine?
Avatar for Terrence Ryan tpryan
0
410
Introduction to Containers and Kubernetes
Avatar for Terrence Ryan tpryan
2
400
Cloud Next 2017 Roundup
Avatar for Terrence Ryan tpryan
2
200
LAMP in Containers
Avatar for Terrence Ryan tpryan
1
320
GCP and IoT
Avatar for Terrence Ryan tpryan
0
510
30,000,000 Requests in a Hour in the Cloud
Avatar for Terrence Ryan tpryan
2
400

Other Decks in Technology

See All in Technology
権限管理設計を完全に理解した
Avatar for r-sugi rsugi
1
180
【禁断】Obsidianの第二の脳に「知の巨人」と呼ばれた師匠の脳をロードしてみた
Avatar for 長津孝輔 nagatsu
0
5.4k
Fラン学生が考える、AI時代のデザインに執着した突破口
Avatar for husensan husengs7
1
240
Orchestration Development Workshopを半期実施して
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
310
Kaggle未経験社員をメダリストに育てる「AIドラゴン桜」
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
290
Personal knowledge bases using LLM
Avatar for LINEヤフーTech (LY Corporation Tech) lycorptech_jp
PRO
0
310
ソフトウェアサプライチェーン攻撃対策として今からサクッとできること
Avatar for GMO Flatt Security flatt_security
2
110
AWSアップデートから考える継続的な運用改善
Avatar for Toru_Kubota toru_kubota
2
360
続 運用改善、不都合な真実 〜 物理制約のない運用改善はほとんど無価値 / 20260518-ssmjp-kaizen-no-value-without-physical-constraints
Avatar for opelab opelab
3
330
Oracle AI Database@Azure：サービス概要のご紹介
Avatar for oracle4engineer oracle4engineer
PRO
6
1.7k
ルール・ロール・ツールを創る / Creating Rules, Roles and Tools
Avatar for Kenji Saito ks91
PRO
0
150
AI時代に改めて考える、ドメイン駆動設計 - モデリングが「AIへの共通言語」になる
Avatar for Koichiro Matsuoka littlehands
6
1.4k

Featured

See All Featured
A Modern Web Designer's Workflow
Avatar for Chris Coyier chriscoyier
698
190k
KATA
Avatar for Megan Lloyd mclloyd
PRO
35
15k
Chasing Engaging Ingredients in Design
Avatar for Sebastian Deterding codingconduct
0
190
Dealing with People You Can't Stand - Big Design 2015
Avatar for Cassini Nazir cassininazir
367
27k
AI Search:
Where Are We & What Can We Do About It?
Avatar for Aleyda Solis aleyda
0
7.5k
The Director’s Chair: Orchestrating AI for Truly Effective Learning
Avatar for Mike Taylor tmiket
1
170
The State of eCommerce SEO: How to Win in Today's Products SERPs - #SEOweek
Avatar for Aleyda Solis aleyda
2
10k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
Avatar for Addy Osmani addyosmani
10
1.2k
CSS Pre-Processors: Stylus, Less & Sass
Avatar for Bermon Painter bermonpainter
360
30k
Build your cross-platform service in a week with App Engine
Avatar for Jose L jlugia
234
18k
Mobile First: as difficult as doing things right
Avatar for Swwweet swwweet
225
10k
sira's awesome portfolio website redesign presentation
Avatar for ElsiraPls elsirapls
0
250

Transcript

  1. @tpryan Scale the Google Way Terry Ryan Developer Advocate

  2. @tpryan Who are you?

  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None

  10. @tpryan First Cluster History

  11. @tpryan First Cluster •Every machine had a role History

  12. @tpryan First Cluster •Every machine had a role •Every machine

    had a name History

  13. @tpryan First Rack History

  14. @tpryan First Rack •Machines belong to groups History

  15. @tpryan First Rack •Machines belong to groups •Slack capacity goes

    to waste History

  16. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999

  17. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000

  18. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000

  19. November 1998 December 1998 January 1999 February 1999 March 1999

    April 1999 December 1998 50,000 January 1999 150,000 April 1999 500,000

  20. @tpryan Today’s Numbers History

  21. @tpryan Today’s Numbers •Over 3 Billion searches a day History

  22. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. History

  23. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day History

  24. @tpryan Today’s Numbers •Over 3 Billion searches a day •On

    a universe of over 60 Trillion urls. •We index over 20 billion a day •Average time to deliver under 250ms History

  25. @tpryan Handling this growth took fundamental changes

  26. Networking

  27. 70 POP’s 33 Countries B4 WAN

  28. Jupiter Superblock

  29. Jupiter Superblock • 1 Petabit/sec

  30. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

  31. Jupiter Superblock • 1 Petabit/sec • 100,000 Servers at 10GB/s

    • 10 Libraries of Congress a second

  32. Storage

  33. Colossus Datacenter wide storage

  34. Processing

  35. Productionized Containers

  36. Productionized Containers • Developed as only practical way to manage

    Google-scale compute

  37. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container

  38. Productionized Containers • Developed as only practical way to manage

    Google-scale compute • Everything at Google runs in a container • We launch over 2 Billion containers per week.

  39. @tpryan What does this mean for you?

  40. @tpryan

  41. @tpryan Big Table

  42. @tpryan Big Table

  43. @tpryan Big Table MapReduce

  44. @tpryan Big Table MapReduce

  45. @tpryan Big Table Dremel MapReduce

  46. @tpryan Big Table Dremel MapReduce

  47. @tpryan Big Table Borg Dremel MapReduce

  48. @tpryan Big Table Borg Dremel MapReduce

  49. @tpryan

  50. @tpryan Google Cloud Platform

  51. @tpryan Google Cloud Platform Compute

  52. @tpryan Google Cloud Platform Compute Storage

  53. @tpryan Google Cloud Platform Compute Connectivity Storage

  54. @tpryan Google Cloud Platform Compute Connectivity Big Data Storage

  55. @tpryan Google Cloud Platform Compute Connectivity Developer Tools Mobile Big

    Data Storage Management

  56. @tpryan Conclusions

  57. None
  58. None

  59. Your App Here

  60. @tpryan Thank You! terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-scale This work

    is licensed under a Creative Commons Attribution 2.0 Generic License.