Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to Design, Build and Run a Cloud App (Cloud...

How to Design, Build and Run a Cloud App (Cloud Developer Roadshow 2014)

This deck is an end to end walkthrough of designing and building an application on Google Cloud Platform. It's based on the real world example of the app that we built for the Google I/O 2014 keynote and sessions, an app called Walkshare. This deck was delivered during the Google Cloud Platform Developer Roadshow events in 2014.

GoogleCloudPlatform

August 20, 2014
Tweet

More Decks by GoogleCloudPlatform

Other Decks in Technology

Transcript

  1. How to Design, Build and Run a Cloud app Google

    Cloud Platform Developer Roadshow - 2014
  2. - Persistent Storage Accessible from Mobile - Autoscaling, Modular, Frontend

    and API - Ease of Development and maintenance - Robust commenting - DB for Temp/Summary data
  3. - Persistent Storage Accessible from Mobile - Autoscaling, Modular, Frontend

    and API - Ease of Development and maintenance - Robust commenting - DB for Temp/Summary data
  4. NoSQL Store Autoscale and management Automatically scales as your needs

    do and is fully managed. sharding and replication is taken care of for you Schemaless access, no need to think about underlying data structure. Robust query engine and abstraction libraries Schemaless access Cloud Datastore ACID Transactions with Strong consistency on reads and ancestor queries
  5. • Same High Replication Datastore Used By App Engine Apps

    Today • Accessible Anywhere via RESTful API • Equally Fast Queries For Any Sized Dataset • Data Replicated Across Multiple Data Centers • Use From Any Application or Language • Serving 4.5+ Trillion Requests Per Month Cloud Datastore
  6. MySQL in the Cloud Easy Migration; No Lock-in Standard connections

    and tools such as mysqldump, MySQL Wire Protocol, and JDBC, easy to migrate and avoids lock-in. Data replicated in many geographic locations as standard, failover handled automatically. Fully managed backups Security, Availability, Durability Cloud SQL Cloud SQL offers managed MySQL instances, up to 16GB RAM, 100GB storage.
  7. • Flexible Charging ◦ Pay per use ◦ Package options

    • EU, US and Asia Data Centers ◦ Colocate Cloud SQL and App Engine Apps in EU or US Cloud SQL
  8. Object Store Encryption and Access Control Store any amount of

    data, up to 5TB/Object. With full versioning, change notifications and resumable uploads and downloads Data encrypted at rest. Fine grained access control 99.95% SLA through highly available, geo-redundant data-replication system. Strong read-after-write consistency for each object Reliable and Consistent Cloud Storage
  9. Persistent Storage Accessible from Mobile - Autoscaling, Modular, Frontend and

    API - Ease of Development and maintenance - Robust commenting - DB for Temp/Summary data ✓
  10. Persistent Storage Accessible from Mobile - Autoscaling, Modular, Frontend and

    API - Ease of Development and maintenance - Robust commenting - DB for Temp/Summary data ✓
  11. App Engine • Very easy to use Datastore from web

    • We created an API and HTML5 client app
  12. Simple to Scale - Autoscale Easy to develop - Free

    to start - Build and test locally - Focus on App Code Trivial to manage - Fully managed - No patches/updates - 24x7 operation by Google SREs Google App Engine
  13. Local Dev Environment... $ gcloud app run . INFO: Module

    instance default is running at http: //localhost:8080/ com.google.appengine.tools.development.AbstractModule startup INFO: The admin console is running at http://localhost: 8080/_ah/admin com.google.appengine.tools.development.DevAppServerImpl doStart INFO: Dev App Server is now running
  14. Factor large Apps into logical components Modules: - Can access

    and share stateful Services - Communicate Securely - Have Performance settings and Versions - Have URI, e.g. my-module. myapp.appspot.com - Can be backed by Compute Engine VMs (Managed VMs) App Engine Modules Module Module Module Version Version Version Version VM Instance Instance Version Version VM Instance App
  15. Backend running on Google App Engine iOS Cloud Endpoints •

    Create APIs for Mobile and Gaming Backends • Decorate client interface application code or generate Endpoint classes from Models • Discoverable, Restful APIs implemented on top of Google’s API Infrastructure • Tools for generating mobile optimized Client Libraries for Android, iOS and Web • out-of-the-box DoS protection, OAuth 2.0 support and client key management Google Cloud Endpoints
  16. Java - App Engine @Api(name=”comment” version=”v1”) public class CommentsEndpoint @ApiMethod(httpMethod

    = "GET", path = "{userid}/{walkid}") public List<Comment> getComments( @Named("walkid") long walkId, @Named(“userid”) long userId) throws UnauthorizedException, NotFoundException { List<Comment> results = comments.query(userid, walkid); return results; } } // REST call = /comment/v1/{userid}/{walkid} public class CommentsTask extends ServiceTask<Void> { protected void executeEndpointCall() { service.commentsEndpoint().getComments(userId, walkId).execute(); } Android App
  17. App Engine shared memcache Application (walkshare) prod Instance 1 Instance

    N DataAnalysis Backend 1 Backend 2 Queues Application (snipchut) one Instance 1 Instance N DataAnalysis Backend 1 Backend 2 Queues Application (someapp) prod Backend 1 Backend 2 test Backend 1 Backend 2 Queues Datastore Memcache
  18. memcache App Engine shared memcache Application (walkshare) prod Instance 1

    Instance N DataAnalysis Backend 1 Backend 2 Queues Application (snipchut) one Instance 1 Instance N DataAnalysis Backend 1 Backend 2 Queues Application (someapp) prod Backend 1 Backend 2 test Backend 1 Backend 2 Queues Datastore Dedicated Memcache Dedicated Memcache Per GB reservation No service-level evictions
  19. Persistent Storage Accessible from Mobile Autoscaling, Modular, Frontend and API

    Ease of Development and maintenance - Robust commenting - DB for Temp/Summary data ✓ ✓ ✓
  20. Persistent Storage Accessible from Mobile Autoscaling, Modular, Frontend and API

    Ease of Development and maintenance - Robust commenting - DB for Temp/Summary data ✓ ✓ ✓
  21. Persistent Storage Accessible from Mobile Autoscaling, Modular, Frontend and API

    Ease of Development and maintenance Robust commenting - DB for Temp/Summary data ✓ ✓ ✓ ✓
  22. Persistent Storage Accessible from Mobile Autoscaling, Modular, Frontend and API

    Ease of Development and maintenance Robust commenting - DB for Temp/Summary data ✓ ✓ ✓ ✓
  23. Redis on Compute Engine • Which users are getting the

    most comments on their walks? Redis +
  24. Compute Resources - US, Europe and Asia zones - Fast

    SDN-Based virtual networking Consistently Fast - Fast VM Provisioning - Consistent Performance - Cloud Native Load Balancing Cost Effective - Sub-Hour Billing - No IOPS charges for Block Storage - VMs not required for Load balancing Compute Engine
  25. Cores CPU to Memory ratio More memory 1 2 4

    8 More CPU (Shared) 16 High CPU Standard High Memory Standard High Memory High CPU Compute Engine VMs: • Debian, CentOS, SUSE, RHEL • shared core or 1 - 16 cores • Up to 60GB of RAM Instances to suit diverse workloads
  26. • Per GB pricing; no IO charges • Consistent Performance

    • Encryption • Live Migration • Storage redundancy • Checksums • Snapshots • 1GB - 1TB Persistent Disk SSD PD Standard PD
  27. Local SSD Persistent Disk • Per GB pricing; no IO

    charges • Consistent Performance • Encryption • Live Migration • Sub-ms latency • 1-4 x 375 GB partitions
  28. Local SSD Persistent Disk 200 30 80 Local SSD IOPS/GB

    (log scale) 0 Standard PD 15 6 SSD PD 2 SSD PD Standard PD
  29. • TCP and UDP (plus VPN protocols) • IP addresses

    • public: static and ephemeral • internal: ephemeral with automatic DNS • Network groups & firewall rules • rules applied to instances via tags • Gateways, routing and VPNs • Load Balancing • Client Affinity Networking Internet
  30. Persistent Storage Accessible from Mobile Autoscaling, Modular, Frontend and API

    Ease of Development and maintenance Robust commenting DB for Temp/Summary data ✓ ✓ ✓ ✓ ✓
  31. Replica Pool name: io-demo modules: replica_pool: type: REPLICA_POOL replicaPoolModule: numReplicas:

    1 replicaPoolParams: v1beta1: machineType: n1-standard-2 zone: us-central1-a baseInstanceName: io-demo disksToCreate: - boot: true autodelete: true mode: READ_WRITE … load_balancer: type: LOAD_BALANCING … firewall: type: … autoscaler: type: AUTOSCALING autoscalingModule: targetModule: "replica_pool" targetUtilization: 0.5 maxNumReplicas: 10 minNumReplicas: 2 coolDownPeriodSec: 45 Resource View create/ destroy Load Balancer Agent Agent VM Monitoring Agent Compute Engine Replica Pool
  32. Load Balancing: 1m requests/sec Load Balancing at Scale • Single

    load-balanced IP Address • Serving 1 million qps sustained • Total cost of test: $10
  33. Focus on the product • Minimize risk • Scale down

    • Deployment • Ease of management
  34. App Engine Modules Module Module Module Version Version Version Version

    VM Instance Instance Version Version VM Instance App
  35. Monitoring the product • data from monitoring the tech •

    great source of usage data, etc. • custom? • we set up Redis on GCE for leaderboards • you can do anything your app needs • leaderboard stats over time? ad hoc, exploratory queries? • BigQuery!
  36. Compute Engine Core OS Kubernetes App Engine managed VMs NEW

    NEW Blurring the line between Iaas and Paas
  37. Dockerfile app.yaml All you need Your app Cloud SDK gcloud

    app run or gcloud app deploy Docker registry VM* container dockerd *A VM running Docker…
  38. … … machineType: n1-standard-1 zone: us-central1-a baseInstanceName: io-demo disksToCreate: -

    boot: true autodelete: true mode: READ_WRITE … load_balancer: type: LOAD_BALANCING … firewall: type: … autoscaler: type: AUTOSCALING autoscalingModule: targetModule: "replica_pool" targetUtilization: 0.5 maxNumReplicas: 10 minNumReplicas: 2 coolDownPeriodSec: 45 Replica Pool Resource View create/ destroy Load Balancer Agent Agent VM Monitoring Agent Actuator Compute Engine Autoscaler
  39. Your Project Your App running on Google App Engine Google

    Cloud Datastore Cloud Save for Android Google Cloud Save for Android