Google Developers Summit Tokyo - The Googliness of Google Cloud Platform

Transcript

1. Kazunori Sato Developer Advocate, Cloud Platform The Googliness of Google

   Cloud Platform

2. +Kazunori Sato @kazunori_279 Developer Advocate, Cloud Platform - Developer Community

   Support - gcp ja night, Docker Meetup, GCPUG - Product Launch Support
3. None
4. None

5. Cloud Technology Innovations 2012 2013 MapReduce Spanner/F1 2003 2006 2007

   2010 2011 GFS Omega Colossus Cloud Storage Dremel BigQuery Bigtable Cloud Datastore Paxos impl. 2004

6. Management Mobile Developer Tools Compute Networkin g Big Data Storage

7. Google Compute Engine Google App Engine Google BigQuery Data Analytics

   with Google’s massively parallel query engine IaaS with Google’s network and computing infra PaaS with Google’s container technology

8. Google BigQuery

9. 72 hours / min 100 petabytes 500M active users How

   Big is Big Data in Google?

10. In Google, we use Google BigQuery for processing Big Data

    SELECT top(appId, 20) AS app, count(*) AS count FROM installlog.2013 ORDER BY count DESC Scans 100B rows in 10 sec, without index.

11. Google BigQuery: Massively Parallel Query Service

12. Cost of BigQuery Storing 1TB = $26 / month Query

    on 10GB = $0.05 / query

13. select top(title), count(*) from publicdata:samples.wikipedia To scan 1 TB in

    1 sec, you need 5,000 disks in parallel Massively Parallel Processing

14. Mixer 0 Mixer 1 Mixer 1 Shard Shard Shard Shard

    ColumnIO on Colossus SELECT state, year COUNT(*) GROUP BY state WHERE year >= 1980 and year < 1990 ORDER BY count_babies DESC LIMIT 10 COUNT(*) GROUP BY state Tree structure for fast aggregation

15. Streaming API + Fluentd: real-time import at 1M+ rows/s

16. Google App Engine

17. Google App Engine: PaaS from Google

18. Google confidential | Do not distribute Google’s Container Technology Every

    Google service runs in containers Launches 2B containers every week

19. Google confidential | Do not distribute Scaling Out by Google’s

    Container Technology Instantiation in 40 ms

20. GAE Architecture And Optmization The Traditional Web Development Hardware Failures

    Traffic Spike Growing Big Data Complex Design Complex Dev. Complex Admin Cost

21. GAE Architecture And Optmization Google App Engine Encapsulates Them All

    Hardware Failures Traffic Spike Growing Big Data Simpler Design Simpler Dev No Admin No Initial Funding

22. Source: Google Internal Data 4.75 Million active applications 作用中應用程式 　　　active

    applications

23. 700 million “App Engine enabled us to focus on developing

    the application. We wouldn’t have gotten here without the ease of development that App Engine gave us.” Bobby Murphy, CTO Snapchat sends photos and videos each day Google App Engine scaled seamlessly during growth to millions of users Small team is able to innovate quickly and expand globally

24. Google Compute Engine

25. The State of the Art Network Technology

26. Andromeda: The latest Software Defined Network (SDN) - Intra/Inter zone

    throughput = 9G bps - Inter regional private network by default

27. 　Google-scale Load Balancing region 2 region 3 region 1 VMs

    VMs VMs ::::::::::::::::::::::::::::::::::::::::::::::: HTTP LB DNS :) One global IP: 11.22.33.44 • Load Balancing infra shared with Google.com • Easily handles 1M req/s • TCP anycast with single global IP • Fast fail-over in seconds

28. Live Migration source: Google

29. Run and manage an optimized cluster Kubernetes Open Source Workload

    API & Tools Google Container Engine (GKE) Kubernetes • Growing Community • Linux containers support • Open source orchestration create and manage Container Engine • クラスタをすぐに起動可能 • クラスタ環境の最適化 • Kubernetesを実行管理する GKE Master GKE Nodes Compute Engine VMs Google Container Engine

30. Getting Started

31. http://gcpug.jp/

32. Thanks !