Compute at Google - An Insider's View

from Google Cloud Platform Live 2014
YouTube video: https://www.youtube.com/watch?v=zZdoL5d7KC8

Transcript

1. Compute at Google An Insider’s View Navneet Joneja Product Manager, App Engine & Compute Engine
2. How Google builds applications How that’s reflected in the Cloud Platform 1 2 3 What that means for you as a public cloud user Agenda
3. The Infrastructure of Yesteryear...
4. Images by Connie Zhou • 100,000+ Devices • MTBF of 10 years = 1 failure per hour Failure is the norm
5. People = Scarce Resource Image by Joe Beda
6. • Plan for failure instead of reacting • Software + Best Practices + Platforms • Investments at all layers The (Google) Solution
7. Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute Engine App Engine App Services BigQuery Cloud Endpoints Google Cloud Platform
8. Building Applications at Google Joe Beda Senior Staff Software Engineer
9. The Google Cloud Platform is built on the internal Google Platform We are continually exposing more of that internal platform The toolbox at Google is the toolbox you get with the Cloud Platform. Image: "Toolbox" florianric CC-BY-2.0 http://www.flickr.com/photos/florianric/7263382550/ The Platform
10. Networking Storage Flexible Building Blocks Compute Services
11. Image: "hópehely /snowflake/" bkaree CC-BY-2.0 http://www.flickr.com/photos/bkaree/6850865695/ No Snowflakes! Compute
12. Declarative Management Reproducible Deployment • State what you want • System makes it happen • Idempotency • Reproduce any compute environment • Staging/dev/test environments mirror production • Automated build, test and deploy Avoiding Snowflakes
13. From the cluster to across the globe Managed QoS, SDN pioneers High Capacity Load Balancing Network
14. VM Block Storage Blob Storage and Serving Structured Storage and Analytics Storage
15. Common data formats and API Mechanisms Monitoring and alerting Configuration Automated build, testing and lifecycle management Services and Glue
16. Uniquely Google: Datastore
17. Uniquely Google: BigQuery
18. Cloud Platform Updates What’s new in Compute Engine & App Engine
19. Compute Engine
20. Transparent Maintenance
21. Simple, Predictable and Cost-Effective Pricing
22. Example pricing: n1-standard-1 Virtual Machines • More use = Lower Price • Automatic • No upfront payments • No lock-in • No complex planning -53%
23. Sustained Use Discounts April 1 April 15 April 30 VM 1 VM 2 VM 3
24. Sustained Use Discounts April 1 April 15 April 30
25. Sustained Use Discounts April 1 April 15 April 30 100% use, 30% discount 50% use 10% discount VM1 VM2
26. Persistent Disk • Simple, Low Pricing • Predictable Performance • Streamlined Operations • Security and Data Consistency $0.04/GB I/O included; scales with size burst capability
27. App Engine Drastically simplified pricing. Highlights: instance hours dedicated memcache datastore writes Many services, including SNI SSL and PageSpeed are now offered to all applications at no extra cost. 37.5% 50% 33%
28. New Networking Capabilities Advanced Routing L3 Load Balancing Protocol Forwarding
29. cloud.google.com/cloud-dns • Google’s DNS infrastructure • Command Line and API access • Low latency worldwide serving Announcing: Google Cloud DNS
30. Expanded OS support Red Hat SuSE FreeBSD <your distro>
31. Windows http://goo.gl/7KNJhC In Preview ®
32. App Engine
33. Modules Componentize w/ independent • Scaling • Performance characteristics • Programming language
35. • Fully managed memcache solution • 1 GB rated at 10K ops/s (small values) • $0.06/GB/hr Dedicated Memcache 10+ terabytes
36. Git Push to Deploy
37. The Next-Generation Cloud Blurring the lines between IaaS and PaaS
38. PaaS IaaS Your code Your code Flexibility Agility Today: Cloud as a binary choice
39. Auto-Managed Open / Flexible Productive Next Gen Cloud
40. Ending the False Dichotomy Chris Ramsdale Product Manager
41. • Flexibility of Compute Engine • Access to native resources, machine types • Auto-management of App Engine • High Availability • Image updates, security patches • Access to all Managed Services • Co-location with other services • Developer productivity Virtualized hardware Abstracted computing power IaaS Infrastructure-as-a-Service PaaS Platform-as-a-Service Managed VMs Managed VMs
42. VM VM Replica Pools In Preview Template image metadata bash cmds Replica Pool Service VM create/destroy Resource View Load Balancer
43. In Preview Deployment Managed VMs Disks Network Load balancer Autoscaler Template name: template-a modules: virtualMachineModule: type: REPLICA_POOL replicaPoolModule: numReplicas: 4 replicaPoolParams: v1beta1: machineType: n1-standard-1 zone: us-central2-a disksToCreate: ... networkInterfaces: ... Deployment Manager
44. Flexibility Agility Google Compute Engine Compute as a spectrum Your Code Manage your infrastructure
45. Manage your infrastructure Flexibility Agility Google Compute Engine Your Code Compute as a spectrum Your Code Replica Pools Provisioning and health checking
46. Manage your infrastructure Flexibility Agility Google Compute Engine Your Code Compute as a spectrum Replica Pools Provisioning and health checking Managed VMs OS management, deployments, logging and monitoring Your Code Your Code
47. Manage your infrastructure Flexibility Agility Google Compute Engine Your Code Compute as a spectrum Replica Pools Provisioning and health checking Managed VMs OS management, deployments, logging and monitoring Your Code Your Code App Engine Managed Runtimes Manage your serving stack Your Code
48. Managed VMs Demo Brian Dorsey Developer Advocate
49. Managed VMs Demo Sudoku Example 5 1 7 3 9 3 9 8 8 2 3 6 5 7 3 4 8 7 7 5 1 3 9 7 2 8 8 5 1 8 4 9 4 6 7 9 3 8 4 1
50. Peter Norvig’s Sudoku solver
51. OpenCV
52. ?
53. Backend Workers App Engine VM VM VM VM VM Task Queue Sudoku architecture
54. application: compute-demo version: 1 runtime: python27 api_version: 1 threadsafe: yes handlers: - url: /favicon\.ico static_files: favicon.ico upload: favicon\.ico - url: .* script: main.APP libraries: - name: webapp2 version: "2.5.2" - name: jinja2 version: latest app.yaml application: compute-demo version: 1 module: solver runtime: python27 vm: true api_version: 1 threadsafe: yes handlers: - url: .* script: main_solver.APP libraries: - name: webapp2 version: "2.5.2" manual_scaling: instances: 42 vm_settings: machine_type: n1-standard-1 apt_get_install: python-opencv backend.yaml
55. $ gcloud app update . Deployment command line
56. Live Demo!
57. Sudoku architecture Backend Workers App Engine VM VM VM VM VM Task Queue
58. from google.appengine.api import modules modules.set_num_instances(42) Application specific scaling Python
59. We’ve learned a lot -- and we’re happy to share Iaas or PaaS is a false choice We’re only getting started! 1 2 3 Summary http://developers.google.com/cloud/managed-vms
60. cloud.google.com Images by Connie Zhou resources: http://goo.gl/ckOvK1
61. Backup Auto-Managed Open / Flexible Productive
62. http://cloud.google.com/r/managed-vms • Great technologies for changing cloud development We’re just getting started
63. The Cloud, Today + Flexibility + Control + Open / Familiar - “plumbing” - BYO Management + Productivity + Auto-managed - Opinionated / Cliffs - Lock-in IaaS PaaS
64. create/ destroy VM VM Replica Pools Replica Pool actuator VM Resource View Template image metadata bash cmds Auto-Scaler controller +1/-1 CPU stats Load Balancer
65. cloud.google.com resources: http://goo.gl/ckOvK1