Google Developers Summit Tokyo - The Googliness of Google Cloud Platform

Transcript

1. Kazunori Sato Developer Advocate, Cloud Platform The Googliness of Google Cloud Platform
2. +Kazunori Sato @kazunori_279 Developer Advocate, Cloud Platform - Developer Community Support - gcp ja night, Docker Meetup, GCPUG - Product Launch Support
5. Cloud Technology Innovations 2012 2013 MapReduce Spanner/F1 2003 2006 2007 2010 2011 GFS Omega Colossus Cloud Storage Dremel BigQuery Bigtable Cloud Datastore Paxos impl. 2004
6. Management Mobile Developer Tools Compute Networkin g Big Data Storage
7. Google Compute Engine Google App Engine Google BigQuery Data Analytics with Google’s massively parallel query engine IaaS with Google’s network and computing infra PaaS with Google’s container technology
8. Google BigQuery
9. 72 hours / min 100 petabytes 500M active users How Big is Big Data in Google?
10. In Google, we use Google BigQuery for processing Big Data SELECT top(appId, 20) AS app, count(*) AS count FROM installlog.2013 ORDER BY count DESC Scans 100B rows in 10 sec, without index.
11. Google BigQuery: Massively Parallel Query Service
12. Cost of BigQuery Storing 1TB = $26 / month Query on 10GB = $0.05 / query
13. select top(title), count(*) from publicdata:samples.wikipedia To scan 1 TB in 1 sec, you need 5,000 disks in parallel Massively Parallel Processing
14. Mixer 0 Mixer 1 Mixer 1 Shard Shard Shard Shard ColumnIO on Colossus SELECT state, year COUNT(*) GROUP BY state WHERE year >= 1980 and year < 1990 ORDER BY count_babies DESC LIMIT 10 COUNT(*) GROUP BY state Tree structure for fast aggregation
15. Streaming API + Fluentd: real-time import at 1M+ rows/s
16. Google App Engine
17. Google App Engine: PaaS from Google
18. Google confidential | Do not distribute Google’s Container Technology Every Google service runs in containers Launches 2B containers every week
19. Google confidential | Do not distribute Scaling Out by Google’s Container Technology Instantiation in 40 ms
20. GAE Architecture And Optmization The Traditional Web Development Hardware Failures Traffic Spike Growing Big Data Complex Design Complex Dev. Complex Admin Cost
21. GAE Architecture And Optmization Google App Engine Encapsulates Them All Hardware Failures Traffic Spike Growing Big Data Simpler Design Simpler Dev No Admin No Initial Funding
22. Source: Google Internal Data 4.75 Million active applications 作用中應用程式 　　　active applications
23. 700 million “App Engine enabled us to focus on developing the application. We wouldn’t have gotten here without the ease of development that App Engine gave us.” Bobby Murphy, CTO Snapchat sends photos and videos each day Google App Engine scaled seamlessly during growth to millions of users Small team is able to innovate quickly and expand globally
24. Google Compute Engine
25. The State of the Art Network Technology
26. Andromeda: The latest Software Defined Network (SDN) - Intra/Inter zone throughput = 9G bps - Inter regional private network by default
27. 　Google-scale Load Balancing region 2 region 3 region 1 VMs VMs VMs ::::::::::::::::::::::::::::::::::::::::::::::: HTTP LB DNS :) One global IP: 11.22.33.44 • Load Balancing infra shared with Google.com • Easily handles 1M req/s • TCP anycast with single global IP • Fast fail-over in seconds
28. Live Migration source: Google
29. Run and manage an optimized cluster Kubernetes Open Source Workload API & Tools Google Container Engine (GKE) Kubernetes • Growing Community • Linux containers support • Open source orchestration create and manage Container Engine • クラスタをすぐに起動可能 • クラスタ環境の最適化 • Kubernetesを実行管理する GKE Master GKE Nodes Compute Engine VMs Google Container Engine
30. Getting Started
31. http://gcpug.jp/
32. Thanks !