Introduction to Google Cloud Platform

For the past 15 years, Google has been building the most powerful cloud infrastructure on the planet. Since not too far ago this platform has been opened to the public, and now it's possible to publish your own applications sharing infrastructure with Google Search, GMail o Youtube.

Presented at Barcelona DevFest 2015, this talk introduces some products and features of Google Cloud Platform that can be combined to address different business needs in creative ways.

Transcript

1. Intro to Google Cloud Platform Nacho Coloma — CTO at Extrema Sistemas Google Developer Expert @nachocoloma http://gplus.to/icoloma
2. Doing everything by yourself is tempting
3. Cloud Platform solutions IaaS PaaS SaaS Infrastructure-as-a-Service Platform-as-a-Service Software-as-a-Service Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Packaged Software Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking You Manage Vendor Managed
4. For the past 15 years, Google has been building the most powerful cloud infrastructure on the planet. Images by Connie Zhou
5. Google innovations in the last twelve years Spanner Dremel MapReduce Big Table Colossus 2012 2013 2002 2004 2006 2008 2010 GFS Compute Engine
6. Google Cloud Platform Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute Engine (IaaS) App Engine (PaaS) Services BigQuery Cloud Endpoints
7. Google Cloud Platform Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute Engine (IaaS) App Engine (PaaS) Services BigQuery Cloud Endpoints
8. Simple to Scale - Autoscale Easy to develop - Free to start - Build and test locally - Focus on App Code Trivial to manage - Fully managed - No patches/updates - 24x7 operation by Google SREs About Google App Engine
9. My system is not always down But when it is, I have a team of Googlers fixing it
10. App Engine If you don’t have a DevOps team to guarantee these: • Infinite scaling • High availability. • Transparent security upgrades. and instead just want to focus on delivering new features, that’s what App Engine is for.
11. Cloud Endpoints runs on App Engine • Generates REST API automatically • JS, Android and iOS client libraries • Server/Client communication is hidden • Runs on same powerful infrastructure, scales infinitely
12. (Almost) Complete Endpoint example @Api(name = “MyAPI”, version = “v1”, namespace = @ApiNamespace(ownerDomain=”foo.com”,ownerName=”mycompany”), clientIds = { Constants.WEB_CLIENT_ID, Constants.ANDROID_CLIENT_ID, Constants.IOS_CLIENT_ID} ) public class MyAPI { @ApiMethod(path="helloworld", name="helloworld") public String hello() { return “Hello World!”; } }
13. Loading your API gapi.client.load(‘oauth2’, ‘v2’, callback); gapi.client.load('myAPI', 'v1', callback, ‘//’ + location.host + ‘/ah/api’);
14. Client side - JS - Calling myApi To call the different APIs we need to use the following pattern: gapi.client.{{API}}.{{methodName}}([params]).execute(callback) Examples: gapi.client.myAPI.helloworld().execute(function(resp) { … }); gapi.client.oauth2.userinfo.get().execute(function(resp) { … }); gapi.client.anotherAPI.entries.delete({ id: 2 }).execute(function(resp) { … });
15. Java Client Example Once we have an instance of our API client, we just have to use it like with JS: myAPIService.{{methodName}}.execute() Examples: String message = myAPIService.helloworld().execute(); // helloworld method name Entry entry = myAPIService.entries().get().execute(); // entries.get method name myAPIService.entries().create(entry).execute(); //entries.create method name
16. Reasons to use endpoints Pros: • Generate client library for Android, iOS, JS • Transparent OAuth2 • SSL is required • Hosted on App Engine: scalability, managed, performance • Use as any other Google API Cons: • only *.appspot.com domains • only JSON responses (you can use App Engine for other formats) • only Java and Python • Standard App Engine limits apply (like the 60s request timeout)
17. Google Identity Toolkit Multiple authentication options Currently Google, Facebook, Yahoo, Microsoft, Paypal, and AOL > go to demo site Based on standards from the OpenID Foundation You can test the standard at AccountChooser.com/ Can be used right now A seamless integration with other products in Google Cloud Platform is in the works.
18. Google Compute Engine
19. Google Cloud Platform Stack IaaS Infrastructure-as-a-Service PaaS Platform-as-a-Service SaaS Software-as-a-Service
20. build and deploy vm images • Curated runtimes • Rich services • Auto-everything • … just add code • Managed collections • Declarative + Dynamic Platform Cluster VM More Agility More flexibility build and deploy clusters build and deploy apps • Basic atom • Run anything Compute as a Continuum
21. GCE: Speed and Performance Latency (ms) Other 90 68 45 23 0 GCE 300 225 150 75 0 Bandwidth (Mbit/s) GCE Other 300 225 150 75 0 Min (s) Max (s) Avg (s) GCE Low Latency High Bandwidth between regions Fast Instance Creation Other Source: "By the Numbers..." Performance Observations
22. GCE: Speed and Performance “We were able to reproduce the result of sorting 1.5 terabytes of data in less than 1 minute consistently across multiple runs and multiple cluster configurations, thanks to the consistent and uniform performance of the Google Compute Environment.” -Yuliya Feldman, MapR MapR broke the MinuteSort record using Google Cloud Platform The previous record had been set using customized software and hardware
23. GCE: Speed and Performance MapR/GCE World Record[1] Previous Record Data Sorted 1.5 TB 1.4 TB Number of Servers (virtual instances) 2,103 2,200 Cost per Server 58¢ per instance hour $4,545 Total Cost $20.33 $10,000,000 Time to Build Cluster Minutes Months [1] Source: Hadoop Minutesort Record Minute Sort Test
24. Sub-hour Billing 1 minute granularity 10 minute minimum
25. Enhanced Reliability us-central1-a scheduled maintenance event Live Migration • No downtime during scheduled datacenter maintenance events Automatic Restart • Instances automatically restarted if subjected to system events such as hardware failure
26. Operating System Supported images • Windows 2008 server • Several flavors of Linux: CentOS, Debian, Red Hat, SUSE • Community supported: CoreOS, FreeBSD, Ubuntu Bring your own Image • any common Linux distro • must have Python 2.6 or higher & sshd • must contain some Google packages (startup script support, google-daemon, etc) • should have other settings configured (e.g. DHCP, SSH, firewall) Full details at: https://developers.google.com/compute/docs/images#buildingimage
27. Docker is a lightweight Linux container to deploy sealed, contained applications
28. Why would you do that? • Reliable deployments: no stress while deploying. • Repeatable artifacts: release the exact same container that you use for development. • Loosely coupled: compose applications from microservices.
29. Containers everywhere · Everything at Google runs in a container · Google starts over 2 billion containers per week · With these numbers, we need better abstractions
30. · Manage a cluster of containers, coupling them as needed for your architecture. · Open source · Contributions from IBM, Red Hat, Docker, Mesosphere, SaltStack, CoreOS, etc. · It’s portable · Google Cloud Platform · Other cloud providers · Your own hardware · Your development environment. Kubernetes
31. Interacting with the user Your own HTML / JSON API Your app on App Engine
32. Interacting with the user Cloud endpoints Your own HTML / JSON API Your app on App Engine (Java, Python, SSL)
33. Interacting with the user Cloud endpoints Your own HTML / JSON API Push notifications Your app on App Engine (Java, Python, SSL)
34. Interacting with the user Cloud endpoints Your own HTML / JSON API Push notifications Your app on App Engine Web Sockets (GCE or custom VMs) (Java, Python, SSL)
35. Storage Options
36. Storage options on App Engine Google Cloud Datastore Managed noSQL storage Unlimited scale Limited query capabilities Entities < 1MB Google Cloud Storage Store big files in the cloud Reliable storage Encrypted at rest Resumable uploads / downloads using HTTP
37. More storage options on App Engine Google Cloud SQL Managed plain ol’ MySQL Max database size is 500GB BigQuery Blazing fast analytics and reporting Scales indefinitely (though you may want to break data in chunks for cost) Can be used via API, command line or web interface Cloud Storage Cloud Datastore
38. Even more storage options on App Engine Google Compute Engine · Deploy your own storage solution using persistent disks: PostgreSQL, Redis, MongoDB, etc. · Some of these are available using a preconfigured stack that can be deployed with a single click. · Choose the type of storage: Standard Persistent Disks, SSD Persistent Disks, Local SSD Disks (upcoming). · Choose size: bigger is faster. Cloud Storage Cloud Datastore Cloud SQL BigQuery
39. Yet Even more storage options on App Engine Google Drive API · Store your data in rows using Google Spreadsheets · Store files in Google Drive Cloud Storage Cloud Datastore Cloud SQL BigQuery Compute Engine For Android · Cloud Save: Save and load a small amount of data for each user (4 x 256KB) · Saved Games: Like Cloud Save for games (since Jul 2014). Includes a default UI, and counts against the Drive quota of the user.
40. New and shiny! Firebase · Store your data in the cloud · Get multiple clients updated in real time · JavaScript, Android, iOS · No server side needed Cloud Storage Cloud Datastore Cloud SQL BigQuery Compute Engine Drive Android
41. Storage options on App Engine Cloud Storage Cloud Datastore Cloud SQL BigQuery Compute Engine Drive Android Firebase
42. Storage options on App Engine Compute Engine Cloud Storage Cloud Datastore Cloud SQL BigQuery Compute Engine Drive Android Firebase
43. Also works for hybrid applications Cloud Storage Cloud Datastore Cloud SQL BigQuery Compute Engine Drive Android + Firebase
44. Performance and cache
45. Caching options on App Engine Memcache Shared or dedicated Maximum size of an entity is approx. 1MB Up to 20GB (dedicated) Roll your own cache service (e.g. Redis) Cached entities can be up to 512MB in size More features: sorted sets, queries, pub/sub, etc. Flexible configuration: persist to disk, max. memory, eviction policy, etc.
46. Caching options on App Engine (2) Edge cache A CDN distributed around the world No configuration needed Will try to honor Cache-Control headers Available for App Engine and Google Cloud Storage PageSpeed Create sprites, inline JS, concatenate CSS, minify, resize/recompress images… Available as an App Engine Service and a Nginx / Apache module.
47. PERFORMANCE MATTERS
48. Global network and Edge Cache https://peering.google.com/about/delivery_ecosystem.html
49. HTTP 2.0 (based on SPDY) Enabled out-of-the-box on App Engine You don’t have to do anything Supported in other environments Included with the latest Nginx Adding mod_spdy with Apache · Up to 50% reduction in Page Load Time by reducing network latency · Requires SSL (in practice) and is backwards-compatible · Supported in all browsers (even Explorer) · Learn if you are already supporting it: http://spdycheck.org/ · Check the speed difference: https://www.httpvshttps.com/
50. Big Data
51. Google BigQuery The Big Data product in Google Cloud Platform
52. Google BigQuery Now, let’s talk about BigQuery... SHUT THE FUCK UP! The Big Data product in Google Cloud Platform
53. Google BigQuery Now, let’s talk about BigQuery... SHUT THE FUCK UP! The Big Data product in Google Cloud Platform (Best introduced by Javier Ramírez)
54. Wrap-up: Google Cloud Platform Storage Cloud Storage Cloud SQL Cloud Datastore Compute Compute Engine (IaaS) App Engine (PaaS) Services BigQuery Cloud Endpoints
55. • The Official Google Cloud Platform Course is available in Europe. • Official exam and title emitted by Google. • 5 days of the best training: · App Engine, Compute Engine, Cloud Storage, Cloud SQL and BigQuery • Upcoming: Barcelona Feb 9 2015 · Other cities also available! • Register here. CP300 IS HERE!
56. Q? A! Nacho Coloma — CTO at Extrema Sistemas Google Developer Expert @nachocoloma http://gplus.to/icoloma