Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Google Developers Summit Tokyo - The Googliness of Google Cloud Platform

Google Developers Summit Tokyo - The Googliness of Google Cloud Platform



April 09, 2015

More Decks by GoogleCloudPlatformJapan

Other Decks in Programming


  1. Kazunori Sato Developer Advocate, Cloud Platform The Googliness of Google

    Cloud Platform
  2. +Kazunori Sato @kazunori_279 Developer Advocate, Cloud Platform - Developer Community

    Support - gcp ja night, Docker Meetup, GCPUG - Product Launch Support
  3. None
  4. None
  5. Cloud Technology Innovations 2012 2013 MapReduce Spanner/F1 2003 2006 2007

    2010 2011 GFS Omega Colossus Cloud Storage Dremel BigQuery Bigtable Cloud Datastore Paxos impl. 2004
  6. Management Mobile Developer Tools Compute Networkin g Big Data Storage

  7. Google Compute Engine Google App Engine Google BigQuery Data Analytics

    with Google’s massively parallel query engine IaaS with Google’s network and computing infra PaaS with Google’s container technology
  8. Google BigQuery

  9. 72 hours / min 100 petabytes 500M active users How

    Big is Big Data in Google?
  10. In Google, we use Google BigQuery for processing Big Data

    SELECT top(appId, 20) AS app, count(*) AS count FROM installlog.2013 ORDER BY count DESC Scans 100B rows in 10 sec, without index.
  11. Google BigQuery: Massively Parallel Query Service

  12. Cost of BigQuery Storing 1TB = $26 / month Query

    on 10GB = $0.05 / query
  13. select top(title), count(*) from publicdata:samples.wikipedia To scan 1 TB in

    1 sec, you need 5,000 disks in parallel Massively Parallel Processing
  14. Mixer 0 Mixer 1 Mixer 1 Shard Shard Shard Shard

    ColumnIO on Colossus SELECT state, year COUNT(*) GROUP BY state WHERE year >= 1980 and year < 1990 ORDER BY count_babies DESC LIMIT 10 COUNT(*) GROUP BY state Tree structure for fast aggregation
  15. Streaming API + Fluentd: real-time import at 1M+ rows/s

  16. Google App Engine

  17. Google App Engine: PaaS from Google

  18. Google confidential | Do not distribute Google’s Container Technology Every

    Google service runs in containers Launches 2B containers every week
  19. Google confidential | Do not distribute Scaling Out by Google’s

    Container Technology Instantiation in 40 ms
  20. GAE Architecture And Optmization The Traditional Web Development Hardware Failures

    Traffic Spike Growing Big Data Complex Design Complex Dev. Complex Admin Cost
  21. GAE Architecture And Optmization Google App Engine Encapsulates Them All

    Hardware Failures Traffic Spike Growing Big Data Simpler Design Simpler Dev No Admin No Initial Funding
  22. Source: Google Internal Data 4.75 Million active applications 作用中應用程式    active

  23. 700 million “App Engine enabled us to focus on developing

    the application. We wouldn’t have gotten here without the ease of development that App Engine gave us.” Bobby Murphy, CTO Snapchat sends photos and videos each day Google App Engine scaled seamlessly during growth to millions of users Small team is able to innovate quickly and expand globally
  24. Google Compute Engine

  25. The State of the Art Network Technology

  26. Andromeda: The latest Software Defined Network (SDN) - Intra/Inter zone

    throughput = 9G bps - Inter regional private network by default
  27.  Google-scale Load Balancing region 2 region 3 region 1 VMs

    VMs VMs ::::::::::::::::::::::::::::::::::::::::::::::: HTTP LB DNS :) One global IP: • Load Balancing infra shared with Google.com • Easily handles 1M req/s • TCP anycast with single global IP • Fast fail-over in seconds
  28. Live Migration source: Google

  29. Run and manage an optimized cluster Kubernetes Open Source Workload

    API & Tools Google Container Engine (GKE) Kubernetes • Growing Community • Linux containers support • Open source orchestration create and manage Container Engine • クラスタをすぐに起動可能 • クラスタ環境の最適化 • Kubernetesを実行管理する GKE Master GKE Nodes Compute Engine VMs Google Container Engine
  30. Getting Started

  31. http://gcpug.jp/

  32. Thanks !