Google Developers Summit Tokyo - The Googliness of Google Cloud Platform

Transcript

1. Kazunori Sato
   Developer Advocate, Cloud Platform
   The Googliness of
   Google Cloud Platform
   

   View Slide

2. +Kazunori Sato
   @kazunori_279
   Developer Advocate, Cloud Platform
   - Developer Community Support
   - gcp ja night, Docker Meetup, GCPUG
   - Product Launch Support
   

   View Slide

3. View Slide

4. View Slide

5. Cloud Technology Innovations
   2012 2013
   MapReduce
   Spanner/F1
   2003 2006 2007 2010 2011
   GFS
   Omega
   Colossus
   Cloud Storage
   Dremel
   BigQuery
   Bigtable
   Cloud Datastore
   Paxos impl.
   2004
   

   View Slide

6. Management Mobile
   Developer Tools
   Compute
   Networkin
   g
   Big Data
   Storage
   

   View Slide

7. Google Compute Engine
   Google App Engine
   Google BigQuery
   Data Analytics with Google’s
   massively parallel query engine
   IaaS with Google’s
   network and computing infra
   PaaS with Google’s
   container technology
   

   View Slide

8. Google BigQuery
   

   View Slide

9. 72 hours / min
   100 petabytes
   500M active users
   How Big is Big Data in Google?
   

   View Slide

10. In Google, we use
    Google BigQuery
    for processing
    Big Data
    SELECT
    top(appId, 20) AS app,
    count(*) AS count
    FROM installlog.2013
    ORDER BY
    count DESC
    Scans 100B rows
    in 10 sec, without index.
    

    View Slide

11. Google BigQuery: Massively Parallel Query Service
    

    View Slide

12. Cost of BigQuery
    Storing 1TB = $26 / month
    Query on 10GB = $0.05 / query
    

    View Slide

13. select top(title), count(*)
    from publicdata:samples.wikipedia
    To scan 1 TB in 1 sec,
    you need 5,000 disks in parallel
    Massively Parallel Processing
    

    View Slide

14. Mixer 0
    Mixer 1 Mixer 1
    Shard Shard Shard Shard
    ColumnIO on Colossus SELECT state, year
    COUNT(*)
    GROUP BY state
    WHERE year >= 1980 and year < 1990
    ORDER BY count_babies DESC
    LIMIT 10
    COUNT(*)
    GROUP BY state
    Tree structure for fast aggregation
    

    View Slide

15. Streaming API + Fluentd: real-time import at 1M+ rows/s
    

    View Slide

16. Google App Engine
    

    View Slide

17. Google App Engine: PaaS from Google
    

    View Slide

18. Google confidential | Do not distribute
    Google’s Container Technology
    Every Google service runs in containers
    Launches 2B containers every week
    

    View Slide

19. Google confidential | Do not distribute
    Scaling Out by Google’s Container Technology
    Instantiation
    in 40 ms
    

    View Slide

20. GAE Architecture And Optmization
    The Traditional Web Development
    Hardware Failures
    Traffic Spike
    Growing Big Data
    Complex Design
    Complex Dev.
    Complex Admin
    Cost
    

    View Slide

21. GAE Architecture And Optmization
    Google App Engine Encapsulates Them All
    Hardware Failures
    Traffic Spike
    Growing Big Data
    Simpler Design
    Simpler Dev
    No Admin
    No Initial Funding
    

    View Slide

22. Source: Google Internal Data
    4.75 Million
    active applications
    作用中應用程式 　　　active applications
    

    View Slide

23. 700 million
    “App Engine enabled us to focus on developing the
    application. We wouldn’t have gotten here without the
    ease of development that App Engine gave us.”
    Bobby Murphy, CTO
    Snapchat sends
    photos and videos each day Google App Engine
    scaled seamlessly
    during growth to
    millions of users
    Small team is able
    to innovate quickly
    and expand
    globally
    

    View Slide

24. Google Compute Engine
    

    View Slide

25. The State of the Art Network Technology
    

    View Slide

26. Andromeda: The latest Software Defined Network (SDN)
    - Intra/Inter zone throughput = 9G bps
    - Inter regional private network by default
    

    View Slide

27. 　Google-scale Load Balancing
    region 2 region 3
    region 1
    VMs VMs VMs
    :::::::::::::::::::::::::::::::::::::::::::::::
    HTTP
    LB
    DNS :)
    One global IP:
    11.22.33.44
    ● Load Balancing infra
    shared with Google.com
    ● Easily handles 1M req/s
    ● TCP anycast with single
    global IP
    ● Fast fail-over in seconds
    

    View Slide

28. Live Migration
    source: Google
    

    View Slide

29. Run and manage an optimized cluster
    Kubernetes
    Open Source
    Workload
    API & Tools
    Google Container Engine (GKE)
    Kubernetes
    ● Growing Community
    ● Linux containers support
    ● Open source orchestration
    create and manage
    Container Engine
    ● クラスタをすぐに起動可能
    ● クラスタ環境の最適化
    ● Kubernetesを実行管理する
    GKE
    Master GKE Nodes
    Compute
    Engine
    VMs
    Google Container Engine
    

    View Slide

30. Getting Started
    

    View Slide

31. http://gcpug.jp/
    

    View Slide

32. Thanks !
    

    View Slide