Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Google Developers Summit Tokyo - The Googliness of Google Cloud Platform

Google Developers Summit Tokyo - The Googliness of Google Cloud Platform

GoogleCloudPlatformJapan

April 09, 2015
Tweet

More Decks by GoogleCloudPlatformJapan

Other Decks in Programming

Transcript

  1. Kazunori Sato
    Developer Advocate, Cloud Platform
    The Googliness of
    Google Cloud Platform

    View Slide

  2. +Kazunori Sato
    @kazunori_279
    Developer Advocate, Cloud Platform
    - Developer Community Support
    - gcp ja night, Docker Meetup, GCPUG
    - Product Launch Support

    View Slide

  3. View Slide

  4. View Slide

  5. Cloud Technology Innovations
    2012 2013
    MapReduce
    Spanner/F1
    2003 2006 2007 2010 2011
    GFS
    Omega
    Colossus
    Cloud Storage
    Dremel
    BigQuery
    Bigtable
    Cloud Datastore
    Paxos impl.
    2004

    View Slide

  6. Management Mobile
    Developer Tools
    Compute
    Networkin
    g
    Big Data
    Storage

    View Slide

  7. Google Compute Engine
    Google App Engine
    Google BigQuery
    Data Analytics with Google’s
    massively parallel query engine
    IaaS with Google’s
    network and computing infra
    PaaS with Google’s
    container technology

    View Slide

  8. Google BigQuery

    View Slide

  9. 72 hours / min
    100 petabytes
    500M active users
    How Big is Big Data in Google?

    View Slide

  10. In Google, we use
    Google BigQuery
    for processing
    Big Data
    SELECT
    top(appId, 20) AS app,
    count(*) AS count
    FROM installlog.2013
    ORDER BY
    count DESC
    Scans 100B rows
    in 10 sec, without index.

    View Slide

  11. Google BigQuery: Massively Parallel Query Service

    View Slide

  12. Cost of BigQuery
    Storing 1TB = $26 / month
    Query on 10GB = $0.05 / query

    View Slide

  13. select top(title), count(*)
    from publicdata:samples.wikipedia
    To scan 1 TB in 1 sec,
    you need 5,000 disks in parallel
    Massively Parallel Processing

    View Slide

  14. Mixer 0
    Mixer 1 Mixer 1
    Shard Shard Shard Shard
    ColumnIO on Colossus SELECT state, year
    COUNT(*)
    GROUP BY state
    WHERE year >= 1980 and year < 1990
    ORDER BY count_babies DESC
    LIMIT 10
    COUNT(*)
    GROUP BY state
    Tree structure for fast aggregation

    View Slide

  15. Streaming API + Fluentd: real-time import at 1M+ rows/s

    View Slide

  16. Google App Engine

    View Slide

  17. Google App Engine: PaaS from Google

    View Slide

  18. Google confidential | Do not distribute
    Google’s Container Technology
    Every Google service runs in containers
    Launches 2B containers every week

    View Slide

  19. Google confidential | Do not distribute
    Scaling Out by Google’s Container Technology
    Instantiation
    in 40 ms

    View Slide

  20. GAE Architecture And Optmization
    The Traditional Web Development
    Hardware Failures
    Traffic Spike
    Growing Big Data
    Complex Design
    Complex Dev.
    Complex Admin
    Cost

    View Slide

  21. GAE Architecture And Optmization
    Google App Engine Encapsulates Them All
    Hardware Failures
    Traffic Spike
    Growing Big Data
    Simpler Design
    Simpler Dev
    No Admin
    No Initial Funding

    View Slide

  22. Source: Google Internal Data
    4.75 Million
    active applications
    作用中應用程式    active applications

    View Slide

  23. 700 million
    “App Engine enabled us to focus on developing the
    application. We wouldn’t have gotten here without the
    ease of development that App Engine gave us.”
    Bobby Murphy, CTO
    Snapchat sends
    photos and videos each day Google App Engine
    scaled seamlessly
    during growth to
    millions of users
    Small team is able
    to innovate quickly
    and expand
    globally

    View Slide

  24. Google Compute Engine

    View Slide

  25. The State of the Art Network Technology

    View Slide

  26. Andromeda: The latest Software Defined Network (SDN)
    - Intra/Inter zone throughput = 9G bps
    - Inter regional private network by default

    View Slide

  27.  Google-scale Load Balancing
    region 2 region 3
    region 1
    VMs VMs VMs
    :::::::::::::::::::::::::::::::::::::::::::::::
    HTTP
    LB
    DNS :)
    One global IP:
    11.22.33.44
    ● Load Balancing infra
    shared with Google.com
    ● Easily handles 1M req/s
    ● TCP anycast with single
    global IP
    ● Fast fail-over in seconds

    View Slide

  28. Live Migration
    source: Google

    View Slide

  29. Run and manage an optimized cluster
    Kubernetes
    Open Source
    Workload
    API & Tools
    Google Container Engine (GKE)
    Kubernetes
    ● Growing Community
    ● Linux containers support
    ● Open source orchestration
    create and manage
    Container Engine
    ● クラスタをすぐに起動可能
    ● クラスタ環境の最適化
    ● Kubernetesを実行管理する
    GKE
    Master GKE Nodes
    Compute
    Engine
    VMs
    Google Container Engine

    View Slide

  30. Getting Started

    View Slide

  31. http://gcpug.jp/

    View Slide

  32. Thanks !

    View Slide