Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Compute at Google - An Insider's View

Compute at Google - An Insider's View

from Google Cloud Platform Live 2014
YouTube video: https://www.youtube.com/watch?v=zZdoL5d7KC8


Kazunori Sato

April 24, 2014


  1. Compute at Google An Insider’s View Navneet Joneja Product Manager,

    App Engine & Compute Engine
  2. How Google builds applications How that’s reflected in the Cloud

    Platform 1 2 3 What that means for you as a public cloud user Agenda
  3. The Infrastructure of Yesteryear...

  4. Images by Connie Zhou • 100,000+ Devices • MTBF of

    10 years = 1 failure per hour Failure is the norm
  5. People = Scarce Resource Image by Joe Beda

  6. • Plan for failure instead of reacting • Software +

    Best Practices + Platforms • Investments at all layers The (Google) Solution
  7. Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute Engine

    App Engine App Services BigQuery Cloud Endpoints Google Cloud Platform
  8. Building Applications at Google Joe Beda Senior Staff Software Engineer

  9. The Google Cloud Platform is built on the internal Google

    Platform We are continually exposing more of that internal platform The toolbox at Google is the toolbox you get with the Cloud Platform. Image: "Toolbox" florianric CC-BY-2.0 http://www.flickr.com/photos/florianric/7263382550/ The Platform
  10. Networking Storage Flexible Building Blocks Compute Services

  11. Image: "hópehely /snowflake/" bkaree CC-BY-2.0 http://www.flickr.com/photos/bkaree/6850865695/ No Snowflakes! Compute

  12. Declarative Management Reproducible Deployment • State what you want •

    System makes it happen • Idempotency • Reproduce any compute environment • Staging/dev/test environments mirror production • Automated build, test and deploy Avoiding Snowflakes
  13. From the cluster to across the globe Managed QoS, SDN

    pioneers High Capacity Load Balancing Network
  14. VM Block Storage Blob Storage and Serving Structured Storage and

    Analytics Storage
  15. Common data formats and API Mechanisms Monitoring and alerting Configuration

    Automated build, testing and lifecycle management Services and Glue
  16. Uniquely Google: Datastore

  17. Uniquely Google: BigQuery

  18. Cloud Platform Updates What’s new in Compute Engine & App

  19. Compute Engine

  20. Transparent Maintenance

  21. Simple, Predictable and Cost-Effective Pricing

  22. Example pricing: n1-standard-1 Virtual Machines • More use = Lower

    Price • Automatic • No upfront payments • No lock-in • No complex planning -53%
  23. Sustained Use Discounts April 1 April 15 April 30 VM

    1 VM 2 VM 3
  24. Sustained Use Discounts April 1 April 15 April 30

  25. Sustained Use Discounts April 1 April 15 April 30 100%

    use, 30% discount 50% use 10% discount VM1 VM2
  26. Persistent Disk • Simple, Low Pricing • Predictable Performance •

    Streamlined Operations • Security and Data Consistency $0.04/GB I/O included; scales with size burst capability
  27. App Engine Drastically simplified pricing. Highlights: instance hours dedicated memcache

    datastore writes Many services, including SNI SSL and PageSpeed are now offered to all applications at no extra cost. 37.5% 50% 33%
  28. New Networking Capabilities Advanced Routing L3 Load Balancing Protocol Forwarding

  29. cloud.google.com/cloud-dns • Google’s DNS infrastructure • Command Line and API

    access • Low latency worldwide serving Announcing: Google Cloud DNS
  30. Expanded OS support Red Hat SuSE FreeBSD <your distro>

  31. Windows http://goo.gl/7KNJhC In Preview ®

  32. App Engine

  33. Modules Componentize w/ independent • Scaling • Performance characteristics •

    Programming language
  34. None
  35. • Fully managed memcache solution • 1 GB rated at

    10K ops/s (small values) • $0.06/GB/hr Dedicated Memcache 10+ terabytes
  36. Git Push to Deploy

  37. The Next-Generation Cloud Blurring the lines between IaaS and PaaS

  38. PaaS IaaS Your code Your code Flexibility Agility Today: Cloud

    as a binary choice
  39. Auto-Managed Open / Flexible Productive Next Gen Cloud

  40. Ending the False Dichotomy Chris Ramsdale Product Manager

  41. • Flexibility of Compute Engine • Access to native resources,

    machine types • Auto-management of App Engine • High Availability • Image updates, security patches • Access to all Managed Services • Co-location with other services • Developer productivity Virtualized hardware Abstracted computing power IaaS Infrastructure-as-a-Service PaaS Platform-as-a-Service Managed VMs Managed VMs
  42. VM VM Replica Pools In Preview Template image metadata bash

    cmds Replica Pool Service VM create/destroy Resource View Load Balancer
  43. In Preview Deployment Managed VMs Disks Network Load balancer Autoscaler

    Template name: template-a modules: virtualMachineModule: type: REPLICA_POOL replicaPoolModule: numReplicas: 4 replicaPoolParams: v1beta1: machineType: n1-standard-1 zone: us-central2-a disksToCreate: ... networkInterfaces: ... Deployment Manager
  44. Flexibility Agility Google Compute Engine Compute as a spectrum Your

    Code Manage your infrastructure
  45. Manage your infrastructure Flexibility Agility Google Compute Engine Your Code

    Compute as a spectrum Your Code Replica Pools Provisioning and health checking
  46. Manage your infrastructure Flexibility Agility Google Compute Engine Your Code

    Compute as a spectrum Replica Pools Provisioning and health checking Managed VMs OS management, deployments, logging and monitoring Your Code Your Code
  47. Manage your infrastructure Flexibility Agility Google Compute Engine Your Code

    Compute as a spectrum Replica Pools Provisioning and health checking Managed VMs OS management, deployments, logging and monitoring Your Code Your Code App Engine Managed Runtimes Manage your serving stack Your Code
  48. Managed VMs Demo Brian Dorsey Developer Advocate

  49. Managed VMs Demo Sudoku Example 5 1 7 3 9

    3 9 8 8 2 3 6 5 7 3 4 8 7 7 5 1 3 9 7 2 8 8 5 1 8 4 9 4 6 7 9 3 8 4 1
  50. Peter Norvig’s Sudoku solver

  51. OpenCV

  52. ?

  53. Backend Workers App Engine VM VM VM VM VM Task

    Queue Sudoku architecture
  54. application: compute-demo version: 1 runtime: python27 api_version: 1 threadsafe: yes

    handlers: - url: /favicon\.ico static_files: favicon.ico upload: favicon\.ico - url: .* script: main.APP libraries: - name: webapp2 version: "2.5.2" - name: jinja2 version: latest app.yaml application: compute-demo version: 1 module: solver runtime: python27 vm: true api_version: 1 threadsafe: yes handlers: - url: .* script: main_solver.APP libraries: - name: webapp2 version: "2.5.2" manual_scaling: instances: 42 vm_settings: machine_type: n1-standard-1 apt_get_install: python-opencv backend.yaml
  55. $ gcloud app update . Deployment command line

  56. Live Demo!

  57. Sudoku architecture Backend Workers App Engine VM VM VM VM

    VM Task Queue
  58. from google.appengine.api import modules modules.set_num_instances(42) Application specific scaling Python

  59. We’ve learned a lot -- and we’re happy to share

    Iaas or PaaS is a false choice We’re only getting started! 1 2 3 Summary http://developers.google.com/cloud/managed-vms
  60. cloud.google.com Images by Connie Zhou resources: http://goo.gl/ckOvK1

  61. Backup Auto-Managed Open / Flexible Productive

  62. http://cloud.google.com/r/managed-vms • Great technologies for changing cloud development We’re just

    getting started
  63. The Cloud, Today + Flexibility + Control + Open /

    Familiar - “plumbing” - BYO Management + Productivity + Auto-managed - Opinionated / Cliffs - Lock-in IaaS PaaS
  64. create/ destroy VM VM Replica Pools Replica Pool actuator VM

    Resource View Template image metadata bash cmds Auto-Scaler controller +1/-1 CPU stats Load Balancer
  65. cloud.google.com resources: http://goo.gl/ckOvK1