Serverless on Google Cloud - Deep Dive

Presentation from Google New York (NYC) advocate Bret McGowen in 2018:

Serverless on Google Cloud covers a lot: compute, Cloud Functions, Cloud Run, App Engine, containers, Kubernetes, Firebase and much more. We'll also cover storage, containers vs apps vs functions, ML and AI, and much more.

Coming soon, see the video for this talk on Bret McGowen's YouTube channel at https://www.youtube.com/c/BretMcG or his playlist at https://www.youtube.com/playlist?list=PLlCd2ljeqltbJQQ79eyxbresnaKkP0TgS

You can find source code for some of the above demos at https://github.com/bretmcg

Transcript

1. @BretMcG Bret McGowen Deep dive into serverless on Google Cloud Bret McGowen Google New York City Developer Advocate @BretMcG bretmcg.com
2. @BretMcG Bret McGowen Serverless swear words Contai***s Kubern****s Infrastr****re Offl*ne Cluster of virt*** mach***s .
3. @BretMcG Bret McGowen Hosted FaaS/compute Easiest dev experience Code focused Limited runtime options Serverless anywhere Knative / GKE serverless addon On prem or multi-cloud Code or containers Custom hardware (GPU, TPU, IoT, etc) Offline You manage infra Serverless operational spectrum Managed by your cloud Container orchestration (k8s) Not serverless for programming or ops Container focused You manage infra Managed by your team Serverless containers Custom languages & runtimes Infra still cloud-managed
4. @BretMcG Bret McGowen Serverless containers Custom languages & runtimes Infra still cloud-managed Hosted FaaS/compute Easiest dev experience Code focused Limited runtime options Serverless anywhere Knative / GKE serverless addon On prem or multi-cloud Code or containers Custom hardware (GPU, TPU, IoT, etc) Offline You manage infra Serverless operational spectrum Managed by your cloud Container orchestration (k8s) Not serverless for programming or ops Container focused You manage infra Managed by your team Serverless containers Custom languages & runtimes Infra still cloud-managed
5. @BretMcG Bret McGowen First, a word about containers.... Containers != Kubernetes (Blasphemy, I know)
6. @BretMcG Bret McGowen Middleware Compute Databases Cloud Functions App Engine Functions as a Service, event-driven Platform as a Service (standard environment) Firestore Cloud Datastore NoSQL document store & sync BigQuery NoSQL Data warehouse & analytics Machine Learning Cloud ML Engine Serverless Tensorflow training & prediction AutoML Training & prediction from examples Cloud Dataflow Stream & batch data processing Cloud Pub/Sub Global real-time messaging Frontend Data Studio Firebase
7. @BretMcG Bret McGowen Serverless on Cloud Platform Compute Storage Big Data AI/ML App Engine Cloud Functions Cloud Datastore Cloud Firestore Cloud Storage BigQuery Cloud Dataflow Cloud Pub/Sub Cloud ML Engine Cloud AutoML
8. @BretMcG Bret McGowen Serverless Compute on Cloud Platform App Engine Standard Highly scalable, serverless web applications. Deploy and scale Applications that react to Requests Cloud Functions Event-driven serverless compute platform. Deploy and scale Functions that react to Events
9. @BretMcG Bret McGowen Serverless for apps and compute Mobile apps Functions as a Service App backends (e.g. APIs, web apps) App Engine Standard Cloud Functions Firebase Fully managed, auto-scales Versioning, traffic splitting Java, Python, Go, PHP, Node.js Event-driven functions Microservices, cloud "glue" Node.js, Python (beta), Go (alpha) Real-time syncing databases Authentication, analytics, notifications, hosting, storage, ML Kit, much more...
10. @BretMcG Bret McGowen @BretMcG Bret McGowen App Engine Standard
11. @BretMcG Bret McGowen App Engine (Standard) A Quick Recap Launched at I/O in May of 2008 Python Runtime and memcache api Java Runtime in April of 2009 Moves out of preview in November of 2011 Go Runtime in March of 2012 PHP Runtime in October 2013 Java 8 in June of 2017 Node.js 8 in June of 2018 with 2nd Generation Runtime
12. @BretMcG Bret McGowen New 2nd Generation Runtimes • Open-source, idiomatic experience • Use any extension, binary, or framework • Supported Runtimes ◦ Node.js 8 ◦ Python 3.7 ◦ PHP 7.2 ◦ Go 1.11
13. @BretMcG Bret McGowen New! Go 1.11 support 2nd Gen runtime Modules support Idiomatic Any library Regular project structure Web frameworks (e.g gin) vendor/ dependencies Today - beta in App Engine Standard GCF coming soon
14. @BretMcG Bret McGowen Cloud Tasks App Engine Standard 2nd gen runtimes (Node.js, Python 3.7, PHP App Engine Flex Distributed task queues Scalable and fully managed Rate and retry controls App Engine targets Future scheduling
15. @BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions
16. @BretMcG Bret McGowen Cloud Functions: triggered by events Database changes Firestore database events (create/update/delete/write) Cloud Storage files App events Firebase Authentication Crashlytics Google Analytics for Firebase Infrastructure changes Stackdriver logging More coming soon! Scheduled Cloud Scheduler Explicit invocations HTTP Webhooks for 3rd party Messages Pub/Sub messages
17. @BretMcG Bret McGowen Cloud Functions A Quick Recap Beta Launch @ Next 2017 Node.js Runtime HTTP/S in the box Stackdriver Integration Pay only when code runs
18. @BretMcG Bret McGowen Cloud Function event triggers Cloud storage (incl. Firebase) Cloud Pub/Sub HTTPS Firebase realtime database & the new Firestore database Firebase authentication Google analytics for Firebase Firebase hosting Firebase Crashlytics
19. @BretMcG Bret McGowen Access 20+ Google services from GCF Cloud Storage Cloud Pub/Sub HTTPS Firebase Cloud Scheduler Cloud Functions as cloud glue
20. @BretMcG Bret McGowen Cloud Console UI Deploy from $ gcloud functions deploy helloWorld \ --trigger-http Command line API
21. @BretMcG Bret McGowen General Availability Cloud Functions is GA! • Open to all developers • Stable and ready for production use. • Service Level Agreement ◦ https://cloud.google.com/functions/sla
22. @BretMcG Bret McGowen Global Footprint $ gcloud functions deploy --region europe-west1 Iowa Belgium Tokyo South Carolina
23. @BretMcG Bret McGowen Single Project, Multiple Regions Tokyo Belgium Iowa My Project
24. @BretMcG Bret McGowen @BretMcG Bret McGowen New runtimes on Cloud Functions
25. @BretMcG Bret McGowen New Cloud Functions languages!
26. @BretMcG Bret McGowen Language Support Node 8 (Beta) • Write Cloud Functions using Node 8.11 • Support for async/await • New function signature! $ gcloud functions deploy --runtime nodejs8
27. @BretMcG Bret McGowen Language Support exports.helloPubsubCallback = (event, callback) => { const pubsubMessageData = event.data.data; doSomething(pubsubMessageData, callback); }; exports.helloPubsubPromise = (event) => { const pubsubMessageData = event.data.data; return doSomethingWithAPromise(pubsubMessageData) }; Current (Node 6) Behavior
28. @BretMcG Bret McGowen Language Support New (Node 8+) Behavior // Note the new (optional) context object exports.helloPubsub = async (data, context) => { return await doSomethingAsynchronous(data) }; // HTTP Functions are unchanged, but also support async/await exports.helloHTTP = async (req, res) => { return await doSomethingAsynchronous(req) };
29. @BretMcG Bret McGowen Language Support Python 3.7!!! (Beta) Write Cloud Functions using Python 3.7 $ gcloud functions deploy --runtime python37 def hello_pubsub(data, context): return do_something(data) def hello_http(request): return do_something(request)
30. @BretMcG Bret McGowen Language Support Python HTTP Functions • GET, PUT, POST, DELETE and OPTIONS • Based on the Flask microframework • Requests are flask.Request • Responses should be compatible with flask.make_response
31. @BretMcG Bret McGowen Language Support Python Background Functions • As with Node, called with data (dict) and context (google.cloud.functions.Context) • To signal successful completion, just return from your function • To signal that your function has failed to complete, just raise an exception
32. @BretMcG Bret McGowen Language Support Common Features • Logs (stdout, stderr) automatically sent to Stackdriver Logging • Uncaught exceptions automatically handled by Stackdriver Error Reporting (for alerts etc) • Automatic build and dependency resolution (package.json, requirements.txt) in the cloud
33. @BretMcG Bret McGowen Language Support The Context Object Python Node Description Type event_id eventId A unique ID for the event. For example: "70172329041928" String timestamp timestamp The date/time this event was created. For example: "2018-04-09T07:56:12.975Z" String (ISO 8601) event_type eventType The type of the event. For example: "google.pubsub.topic.publish" String resource resource The resource that emitted the event. This dictionary has attributes `service`, `name` and `type`. Dictionary / Object
34. @BretMcG Bret McGowen Language Support Sample Context Object (Node.js) { eventId:"122286916458880", timestamp:"2018-06-19T23:12:19.340Z", eventType:"google.pubsub.topic.publish", resource:{ service:"pubsub.googleapis.com", name:"projects/my-project/topics/foo", type:"type.googleapis.com/google.pubsub.v1.PubsubMessage" } }
35. @BretMcG Bret McGowen Cloud Functions for Firebase New CLI (v4.0.0) & New SDK (v2.0.0) • Cloud Functions for Firebase is GA • Full support for Node 8 ◦ ECMAScript 2017 support ◦ Async/Await • New runtime configuration options ◦ Region, memory, timeout • Firebase Events (Analytics, Firestore, Realtime DB, Authentication), now available directly in Cloud Functions
36. @BretMcG Bret McGowen Ubuntu Base Image Ubuntu 18.04 LTS Base Image Unforked linux distribution and broader set of system libraries • Headless Chrome libs! • imagemagick • ffmpeg • libcairo2
37. @BretMcG Bret McGowen Moar System Libraries fontconfig ffmpeg flvmeta geoip-database git imagemagick jq libatlas3-base libblas3 libbz2-1.0 libcurl4-openssl-dev libdb5.3 libenchant1c2a libexpat1 libffi6 libfftw3-double3 libflac8 libfontconfig1 libfontenc1 libfreetype6 libgcrypt20 libgd3 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgmp10 libgmpxx4ldbl libgdbm5 libgoogle-perftools4 libgraphite2-3 libgs9 libgs9-common libicu60 libhashkit2 libjbig0 libjbig2dec0 libjpeg8 libjpeg-turbo8 liblapack3 libldap-2.4-2 liblzma5 libmagickcore-6.q16-3 libmagickcore-6.q16-3-extra libmagickwand-6.q16-3 libmemcached11 libmemcachedutil2 libmpc3 libmpdec2 libmysqlclient20 libncursesw5 libnetpbm10 libpng16-16 libprotoc10 libpq5 librabbitmq4 librdkafka1 libreadline7 librsvg2-2 librsvg2-common libsasl2-2 libsasl2-modules libsasl2-modules-db libsqlite3-0 libssl1.0.0 libtiff5 libtiffxx5 libtidy5 libuuid1 libvpx5 libwebp6 libxml2 libxslt1.1 libyaml-0-2 libzip4 locales lsb-release mime-support netpbm python3-chardet tzdata uuid-runtime
38. @BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Environment Variables
39. @BretMcG Bret McGowen Environment Variables Environment Variables Store configuration outside your source code $ gcloud functions deploy --set-env-vars FOO=bar
40. @BretMcG Bret McGowen Environment Variables $ … deploy --set-env-vars FOO=bar env FOO="bar" process.env.FOO Environment Variables are: • Set at deploy-time • Bound to a single function • Surfaced as literal environment variables
41. @BretMcG Bret McGowen Environment Variables { "environmentVariables": { string: string, ... } } Cloud Console API
42. @BretMcG Bret McGowen @BretMcG Bret McGowen Demo: headless Chrome
43. @BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Cloud SQL Direct Connect
44. @BretMcG Bret McGowen Cloud SQL Direct Connect Cloud SQL Direct Connect Direct connection to Cloud SQL instances const pool = mysql.createPool({ connectionLimit : 1, socketPath: '/cloudsql/foo', user: dbUser, password: dbPass, database: dbName });
45. @BretMcG Bret McGowen Cloud SQL Direct Connect Internal External
46. @BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Scaling Controls
47. @BretMcG Bret McGowen Example: Connection Pools
48. @BretMcG Bret McGowen Scaling Controls Scaling Controls Limit scaling on a per-function basis $ gcloud functions deploy --max-instances 100
49. @BretMcG Bret McGowen Scaling Controls Scaling Controls • Controls and limits are per-function (not per project) • Default limit of up to 1,000 (varies by region) • Can be increased upon request for HTTP Functions • Can be reduced on a per-function basis via API/CLI/UI • Requests exceeding the limit are queued with a timeout (60s)
50. @BretMcG Bret McGowen Example: Connection Pools
51. @BretMcG Bret McGowen @BretMcG Bret McGowen Serverless Scheduling
52. @BretMcG Bret McGowen Cloud Scheduler Pub/Sub 1. Establish a Cloud Scheduler schedule (down to 1 min intervals) 2. Cloud Scheduler invokes Cloud Functions over HTTPS or via Cloud Pub/Sub 3. Invokes App Engine on a relative url handler HTTPS Now in Beta!
53. @BretMcG Bret McGowen Cloud Scheduler Cloud Scheduler & Cloud Functions Execute Cloud Functions on a repeating schedule $ gcloud scheduler jobs create-http-job my-http-job \ --message-body 'Hello World!' \ --schedule 'Every 1 mins' \ --url 'https://foo.cloudfunctions.net/bar' \ --http-method POST
54. @BretMcG Bret McGowen Cloud Scheduler Cloud Scheduler & App Engine Execute App Engine on a repeating schedule $ gcloud scheduler jobs create-app-engine-job my-appengine-job \ --message-body 'Hello World!' \ --schedule 'Every 1 mins' \ --relative-url '/my-handler/' \ --service default
55. @BretMcG Bret McGowen Demo setup
56. @BretMcG Bret McGowen HTTP
57. @BretMcG Bret McGowen HTTP
58. @BretMcG Bret McGowen $ gcloud functions deploy ... --retry HTTP
59. @BretMcG Bret McGowen Dead le er queue writeToDB Failed messages HTTP
60. @BretMcG Bret McGowen Dead le er queue writeToDB Failed messages HTTP
61. @BretMcG Bret McGowen Dead le er queue writeToDB Failed messages HTTP
62. @BretMcG Bret McGowen HTTP Dead le er queue writeToDB retryFailed Failed messages
63. @BretMcG Bret McGowen Demo: Cloud Scheduler
64. @BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Access & IAM
65. @BretMcG Bret McGowen Network A VPC • Create a network with a name • Add your Compute Engine instances to that network • Add your Cloud Functions or App Engine resources to that network • Egress to Compute Engine IP addresses Network A
66. @BretMcG Bret McGowen VPC VPC & VPN Access Compute Engine VMs from your Cloud Function $ gcloud functions deploy --connected-vpc my-network
67. @BretMcG Bret McGowen Security Controls Security Controls Control access to function invocation using IAM $ gcloud functions add-iam-policy-binding helloWorld \ --member='user:alice@foo.com' \ --role='roles/cloudfunctions.invoker' $ gcloud functions add-iam-policy-binding helloWorld \ --member='allUsers' \ --role='roles/cloudfunctions.invoker'
68. @BretMcG Bret McGowen Security Controls Serving HTTP Function HTTP Function IAM Check HTTP
69. @BretMcG Bret McGowen @BretMcG Bret McGowen Shhh...
70. @BretMcG Bret McGowen Serverless Containers Serverless Containers Provide arbitrary container images and run them "serverlessly" • Takes a pre-built Docker image • Use arbitrary base images • Use arbitrary system libraries • Use arbitrary language runtime • Same serverless execution environment ◦ No servers ◦ Pay only while code runs
71. @BretMcG Bret McGowen @BretMcG Bret McGowen Demo
72. @BretMcG Bret McGowen @BretMcG Bret McGowen Serverless challenges
73. @BretMcG Bret McGowen Challenges in serverless today Dependencies Constrained runtimes, frameworks and packages Multi-Cloud Unable to run your workloads on-prem, in the cloud or on a third party service provider 1 2
74. @BretMcG Bret McGowen FROM python RUN apt-get update && apt-get install -y blender ENV APP_HOME /app COPY . $APP_HOME WORKDIR $APP_HOME RUN pip install Flask EXPOSE 8080 CMD ["python", "app.py"] Containers Image It’s a method of packaging an application executable and its dependencies (runtime, libraries, configuration) Runtime and running as a set of resource-isolated processes.
75. @BretMcG Bret McGowen Coming soon: GCF serverless containers Serverless containers Fully managed BYO workloads Pay for use Alpha invitations coming later this year Sign up: g.co/serverlesscontainers
76. @BretMcG Bret McGowen Demo
77. @BretMcG Bret McGowen Challenges in serverless today Dependencies Constrained runtimes, frameworks and packages Multi-Cloud Unable to run your workloads on-prem, in the cloud or on a third party service provider 1 2
78. @BretMcG Bret McGowen Knative
79. @BretMcG Bret McGowen Kubernetes is the de facto platform for running containers.
80. @BretMcG Bret McGowen Kubernetes keeps your applications running while you're asleep. Container died? Restart it. Server or container unhealthy? Reschedule to another node. Container overloaded? Add more replicas automatically.
81. @BretMcG Bret McGowen Kubernetes is not easy 1. It was never meant to be used by developers directly. 2. Creating and operating Kubernetes clusters in production is pretty much a full time job.
82. @BretMcG Bret McGowen Google Kubernetes Engine (GKE) The zero ops cluster experience: • update your cluster to new versions of Kubernetes • scale the cluster up/down automatically • detect and replace broken nodes of the cluster
83. @BretMcG Bret McGowen Kubernetes isn't actually for developers It's not the right abstraction for end-developer experience. (This did not stop developers from using Kubernetes directly!) But it's a great platform for building a PaaS on top of.
84. @BretMcG Bret McGowen Why use Kubernetes for serverless? Reduce lock-in Performance Multi-cloud Customizability Use custom hardware (GPU, TPU, IoT, etc) Offline/reduced connectivity Existing on-prem infrastructure Existing applications
85. @BretMcG Bret McGowen What should it take to deploy a function or app? Spin up a VM instance Provision server capacity Specify DB requirements Write code Patch server Scale capacity depending on workload size Spin up a VM instance Provision server capacity Specify DB requirements Write code Patch server Scale capacity depending on workload size Write code
86. @BretMcG Bret McGowen Meet Knative Building blocks for creating serverless experiences on top of Kubernetes. github.com/knative
87. @BretMcG Bret McGowen Knative partners
88. @BretMcG Bret McGowen Knative stack Kubernetes + Istio Platform
89. @BretMcG Bret McGowen Knative stack Knative Build Serving Events Kubernetes + Istio Platform Primitives
90. @BretMcG Bret McGowen Knative Build Serving Events Kubernetes + Istio Platform Products Serverless Containers on GCF GKE Serverless Add-on SAP Kyma Pivotal Function Service IBM Cloud Functions Red Hat Cloud Functions Primitives riff OpenFaaS Jazz
91. @BretMcG Bret McGowen What Knative is • An open source project • Set of building blocks to construct your own FaaS/PaaS ◦ abstracts common tasks through custom Kubernetes API objects • An abstraction on top of Kubernetes. ◦ It's still Kubernetes: Runs containers at the end of the day.
92. @BretMcG Bret McGowen What Knative is not • It's not a Google product. • It's not a FaaS.
93. @BretMcG Bret McGowen What can you do with Knative? [Developers] Use it directly to deploy stuff (not easy, but works fine) [Operators] Put a level of abstraction between your devs and Kubernetes. [Platform Architects] Use it to build your own serverless platform. e.g. DIY Heroku or GCF/Lambda.
94. @BretMcG Bret McGowen Infrastructure Primitives Knative Kubernetes Off-the-shelf FaaS (riff, OpenFaaS, Apache Whisk…) Developer Experience Your in-house FaaS platform Developers
95. @BretMcG Bret McGowen Serverless Compute on Cloud Platform Release Schedule New features rolling out in the coming weeks Check the Cloud Platform Blog for updates and early access
96. @BretMcG Bret McGowen Recap: new in Cloud Functions Generally Available (with SLA!) Python 3.7, Node 8 Regions: Tokyo, Belgium, 2 in US Environment Variables Cloud Scheduler Ubuntu 18.04 with many packages (ffmpeg, imagemagick, headless Chrome) Security Controls: VPC, IAM Scaling Controls Cloud SQL Direct Connect New! New!
97. @BretMcG Bret McGowen App Engine: 2nd gen runtimes New! Pay for what you use, scale to zero Open-source, idiomatic experience Use any module, extension, or framework New supported runtimes: ◦ Node.js 8 ◦ Python 3.7 ◦ PHP 7.2 ◦ Go 1.11!
98. @BretMcG Bret McGowen Knative on Kubernetes New!
99. @BretMcG Bret McGowen cloud.google.com/serverless Knative github.com/knative slack.knative.dev GKE serverless add-on (sign-up) g.co/serverlessaddon Containers on Cloud Functions (sign-up) g.co/serverlesscontainers Bret McGowen @BretMcG bretmcg.com StackOverflow Thank you!
100. @BretMcG Bret McGowen Resources cloud.google.com/serverless Knative github.com/knative slack.knative.dev GKE serverless add-on (sign-up) g.co/serverlessaddon Containers on Cloud Functions (sign-up) g.co/serverlesscontainers Thank you! Bret McGowen @BretMcG
101. @BretMcG Bret McGowen Thank you. @BretMcG