Serverless on Google Cloud - Deep Dive

by Bret McGowen - NYC Google Developer Advocate

Slide 1

Slide 1 text

@BretMcG Bret McGowen Deep dive into serverless on Google Cloud Bret McGowen Google New York City Developer Advocate @BretMcG bretmcg.com

Slide 2

Slide 2 text

@BretMcG Bret McGowen Serverless swear words Contai***s Kubern****s Infrastr****re Offl*ne Cluster of virt*** mach***s .

Slide 3

Slide 3 text

@BretMcG Bret McGowen Hosted FaaS/compute Easiest dev experience Code focused Limited runtime options Serverless anywhere Knative / GKE serverless addon On prem or multi-cloud Code or containers Custom hardware (GPU, TPU, IoT, etc) Offline You manage infra Serverless operational spectrum Managed by your cloud Container orchestration (k8s) Not serverless for programming or ops Container focused You manage infra Managed by your team Serverless containers Custom languages & runtimes Infra still cloud-managed

Slide 4

Slide 4 text

@BretMcG Bret McGowen Serverless containers Custom languages & runtimes Infra still cloud-managed Hosted FaaS/compute Easiest dev experience Code focused Limited runtime options Serverless anywhere Knative / GKE serverless addon On prem or multi-cloud Code or containers Custom hardware (GPU, TPU, IoT, etc) Offline You manage infra Serverless operational spectrum Managed by your cloud Container orchestration (k8s) Not serverless for programming or ops Container focused You manage infra Managed by your team Serverless containers Custom languages & runtimes Infra still cloud-managed

Slide 5

Slide 5 text

@BretMcG Bret McGowen First, a word about containers.... Containers != Kubernetes (Blasphemy, I know)

Slide 6

Slide 6 text

@BretMcG Bret McGowen Middleware Compute Databases Cloud Functions App Engine Functions as a Service, event-driven Platform as a Service (standard environment) Firestore Cloud Datastore NoSQL document store & sync BigQuery NoSQL Data warehouse & analytics Machine Learning Cloud ML Engine Serverless Tensorflow training & prediction AutoML Training & prediction from examples Cloud Dataflow Stream & batch data processing Cloud Pub/Sub Global real-time messaging Frontend Data Studio Firebase

Slide 7

Slide 7 text

@BretMcG Bret McGowen Serverless on Cloud Platform Compute Storage Big Data AI/ML App Engine Cloud Functions Cloud Datastore Cloud Firestore Cloud Storage BigQuery Cloud Dataflow Cloud Pub/Sub Cloud ML Engine Cloud AutoML

Slide 8

Slide 8 text

@BretMcG Bret McGowen Serverless Compute on Cloud Platform App Engine Standard Highly scalable, serverless web applications. Deploy and scale Applications that react to Requests Cloud Functions Event-driven serverless compute platform. Deploy and scale Functions that react to Events

Slide 9

Slide 9 text

@BretMcG Bret McGowen Serverless for apps and compute Mobile apps Functions as a Service App backends (e.g. APIs, web apps) App Engine Standard Cloud Functions Firebase Fully managed, auto-scales Versioning, traffic splitting Java, Python, Go, PHP, Node.js Event-driven functions Microservices, cloud "glue" Node.js, Python (beta), Go (alpha) Real-time syncing databases Authentication, analytics, notifications, hosting, storage, ML Kit, much more...

Slide 10

Slide 10 text

@BretMcG Bret McGowen @BretMcG Bret McGowen App Engine Standard

Slide 11

Slide 11 text

@BretMcG Bret McGowen App Engine (Standard) A Quick Recap Launched at I/O in May of 2008 Python Runtime and memcache api Java Runtime in April of 2009 Moves out of preview in November of 2011 Go Runtime in March of 2012 PHP Runtime in October 2013 Java 8 in June of 2017 Node.js 8 in June of 2018 with 2nd Generation Runtime

Slide 12

Slide 12 text

@BretMcG Bret McGowen New 2nd Generation Runtimes ● Open-source, idiomatic experience ● Use any extension, binary, or framework ● Supported Runtimes ○ Node.js 8 ○ Python 3.7 ○ PHP 7.2 ○ Go 1.11

Slide 13

Slide 13 text

@BretMcG Bret McGowen New! Go 1.11 support 2nd Gen runtime Modules support Idiomatic Any library Regular project structure Web frameworks (e.g gin) vendor/ dependencies Today - beta in App Engine Standard GCF coming soon

Slide 14

Slide 14 text

@BretMcG Bret McGowen Cloud Tasks App Engine Standard 2nd gen runtimes (Node.js, Python 3.7, PHP App Engine Flex Distributed task queues Scalable and fully managed Rate and retry controls App Engine targets Future scheduling

Slide 15

Slide 15 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions

Slide 16

Slide 16 text

@BretMcG Bret McGowen Cloud Functions: triggered by events Database changes Firestore database events (create/update/delete/write) Cloud Storage files App events Firebase Authentication Crashlytics Google Analytics for Firebase Infrastructure changes Stackdriver logging More coming soon! Scheduled Cloud Scheduler Explicit invocations HTTP Webhooks for 3rd party Messages Pub/Sub messages

Slide 17

Slide 17 text

@BretMcG Bret McGowen Cloud Functions A Quick Recap Beta Launch @ Next 2017 Node.js Runtime HTTP/S in the box Stackdriver Integration Pay only when code runs

Slide 18

Slide 18 text

@BretMcG Bret McGowen Cloud Function event triggers Cloud storage (incl. Firebase) Cloud Pub/Sub HTTPS Firebase realtime database & the new Firestore database Firebase authentication Google analytics for Firebase Firebase hosting Firebase Crashlytics

Slide 19

Slide 19 text

@BretMcG Bret McGowen Access 20+ Google services from GCF Cloud Storage Cloud Pub/Sub HTTPS Firebase Cloud Scheduler Cloud Functions as cloud glue

Slide 20

Slide 20 text

@BretMcG Bret McGowen Cloud Console UI Deploy from $ gcloud functions deploy helloWorld \ --trigger-http Command line API

Slide 21

Slide 21 text

@BretMcG Bret McGowen General Availability Cloud Functions is GA! ● Open to all developers ● Stable and ready for production use. ● Service Level Agreement ○ https://cloud.google.com/functions/sla

Slide 22

Slide 22 text

@BretMcG Bret McGowen Global Footprint $ gcloud functions deploy --region europe-west1 Iowa Belgium Tokyo South Carolina

Slide 23

Slide 23 text

@BretMcG Bret McGowen Single Project, Multiple Regions Tokyo Belgium Iowa My Project

Slide 24

Slide 24 text

@BretMcG Bret McGowen @BretMcG Bret McGowen New runtimes on Cloud Functions

Slide 25

Slide 25 text

@BretMcG Bret McGowen New Cloud Functions languages!

Slide 26

Slide 26 text

@BretMcG Bret McGowen Language Support Node 8 (Beta) ● Write Cloud Functions using Node 8.11 ● Support for async/await ● New function signature! $ gcloud functions deploy --runtime nodejs8

Slide 27

Slide 27 text

@BretMcG Bret McGowen Language Support exports.helloPubsubCallback = (event, callback) => { const pubsubMessageData = event.data.data; doSomething(pubsubMessageData, callback); }; exports.helloPubsubPromise = (event) => { const pubsubMessageData = event.data.data; return doSomethingWithAPromise(pubsubMessageData) }; Current (Node 6) Behavior

Slide 28

Slide 28 text

@BretMcG Bret McGowen Language Support New (Node 8+) Behavior // Note the new (optional) context object exports.helloPubsub = async (data, context) => { return await doSomethingAsynchronous(data) }; // HTTP Functions are unchanged, but also support async/await exports.helloHTTP = async (req, res) => { return await doSomethingAsynchronous(req) };

Slide 29

Slide 29 text

@BretMcG Bret McGowen Language Support Python 3.7!!! (Beta) Write Cloud Functions using Python 3.7 $ gcloud functions deploy --runtime python37 def hello_pubsub(data, context): return do_something(data) def hello_http(request): return do_something(request)

Slide 30

Slide 30 text

@BretMcG Bret McGowen Language Support Python HTTP Functions ● GET, PUT, POST, DELETE and OPTIONS ● Based on the Flask microframework ● Requests are flask.Request ● Responses should be compatible with flask.make_response

Slide 31

Slide 31 text

@BretMcG Bret McGowen Language Support Python Background Functions ● As with Node, called with data (dict) and context (google.cloud.functions.Context) ● To signal successful completion, just return from your function ● To signal that your function has failed to complete, just raise an exception

Slide 32

Slide 32 text

@BretMcG Bret McGowen Language Support Common Features ● Logs (stdout, stderr) automatically sent to Stackdriver Logging ● Uncaught exceptions automatically handled by Stackdriver Error Reporting (for alerts etc) ● Automatic build and dependency resolution (package.json, requirements.txt) in the cloud

Slide 33

Slide 33 text

@BretMcG Bret McGowen Language Support The Context Object Python Node Description Type event_id eventId A unique ID for the event. For example: "70172329041928" String timestamp timestamp The date/time this event was created. For example: "2018-04-09T07:56:12.975Z" String (ISO 8601) event_type eventType The type of the event. For example: "google.pubsub.topic.publish" String resource resource The resource that emitted the event. This dictionary has attributes `service`, `name` and `type`. Dictionary / Object

Slide 34

Slide 34 text

@BretMcG Bret McGowen Language Support Sample Context Object (Node.js) { eventId:"122286916458880", timestamp:"2018-06-19T23:12:19.340Z", eventType:"google.pubsub.topic.publish", resource:{ service:"pubsub.googleapis.com", name:"projects/my-project/topics/foo", type:"type.googleapis.com/google.pubsub.v1.PubsubMessage" } }

Slide 35

Slide 35 text

@BretMcG Bret McGowen Cloud Functions for Firebase New CLI (v4.0.0) & New SDK (v2.0.0) ● Cloud Functions for Firebase is GA ● Full support for Node 8 ○ ECMAScript 2017 support ○ Async/Await ● New runtime configuration options ○ Region, memory, timeout ● Firebase Events (Analytics, Firestore, Realtime DB, Authentication), now available directly in Cloud Functions

Slide 36

Slide 36 text

@BretMcG Bret McGowen Ubuntu Base Image Ubuntu 18.04 LTS Base Image Unforked linux distribution and broader set of system libraries ● Headless Chrome libs! ● imagemagick ● ffmpeg ● libcairo2

Slide 37

Slide 37 text

@BretMcG Bret McGowen Moar System Libraries fontconfig ffmpeg flvmeta geoip-database git imagemagick jq libatlas3-base libblas3 libbz2-1.0 libcurl4-openssl-dev libdb5.3 libenchant1c2a libexpat1 libffi6 libfftw3-double3 libflac8 libfontconfig1 libfontenc1 libfreetype6 libgcrypt20 libgd3 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-common libgmp10 libgmpxx4ldbl libgdbm5 libgoogle-perftools4 libgraphite2-3 libgs9 libgs9-common libicu60 libhashkit2 libjbig0 libjbig2dec0 libjpeg8 libjpeg-turbo8 liblapack3 libldap-2.4-2 liblzma5 libmagickcore-6.q16-3 libmagickcore-6.q16-3-extra libmagickwand-6.q16-3 libmemcached11 libmemcachedutil2 libmpc3 libmpdec2 libmysqlclient20 libncursesw5 libnetpbm10 libpng16-16 libprotoc10 libpq5 librabbitmq4 librdkafka1 libreadline7 librsvg2-2 librsvg2-common libsasl2-2 libsasl2-modules libsasl2-modules-db libsqlite3-0 libssl1.0.0 libtiff5 libtiffxx5 libtidy5 libuuid1 libvpx5 libwebp6 libxml2 libxslt1.1 libyaml-0-2 libzip4 locales lsb-release mime-support netpbm python3-chardet tzdata uuid-runtime

Slide 38

Slide 38 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Environment Variables

Slide 39

Slide 39 text

@BretMcG Bret McGowen Environment Variables Environment Variables Store configuration outside your source code $ gcloud functions deploy --set-env-vars FOO=bar

Slide 40

Slide 40 text

@BretMcG Bret McGowen Environment Variables $ … deploy --set-env-vars FOO=bar env FOO="bar" process.env.FOO Environment Variables are: ● Set at deploy-time ● Bound to a single function ● Surfaced as literal environment variables

Slide 41

Slide 41 text

@BretMcG Bret McGowen Environment Variables { "environmentVariables": { string: string, ... } } Cloud Console API

Slide 42

Slide 42 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Demo: headless Chrome

Slide 43

Slide 43 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Cloud SQL Direct Connect

Slide 44

Slide 44 text

@BretMcG Bret McGowen Cloud SQL Direct Connect Cloud SQL Direct Connect Direct connection to Cloud SQL instances const pool = mysql.createPool({ connectionLimit : 1, socketPath: '/cloudsql/foo', user: dbUser, password: dbPass, database: dbName });

Slide 45

Slide 45 text

@BretMcG Bret McGowen Cloud SQL Direct Connect Internal External

Slide 46

Slide 46 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Scaling Controls

Slide 47

Slide 47 text

@BretMcG Bret McGowen Example: Connection Pools

Slide 48

Slide 48 text

@BretMcG Bret McGowen Scaling Controls Scaling Controls Limit scaling on a per-function basis $ gcloud functions deploy --max-instances 100

Slide 49

Slide 49 text

@BretMcG Bret McGowen Scaling Controls Scaling Controls ● Controls and limits are per-function (not per project) ● Default limit of up to 1,000 (varies by region) ● Can be increased upon request for HTTP Functions ● Can be reduced on a per-function basis via API/CLI/UI ● Requests exceeding the limit are queued with a timeout (60s)

Slide 50

Slide 50 text

@BretMcG Bret McGowen Example: Connection Pools

Slide 51

Slide 51 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Serverless Scheduling

Slide 52

Slide 52 text

@BretMcG Bret McGowen Cloud Scheduler Pub/Sub 1. Establish a Cloud Scheduler schedule (down to 1 min intervals) 2. Cloud Scheduler invokes Cloud Functions over HTTPS or via Cloud Pub/Sub 3. Invokes App Engine on a relative url handler HTTPS Now in Beta!

Slide 53

Slide 53 text

@BretMcG Bret McGowen Cloud Scheduler Cloud Scheduler & Cloud Functions Execute Cloud Functions on a repeating schedule $ gcloud scheduler jobs create-http-job my-http-job \ --message-body 'Hello World!' \ --schedule 'Every 1 mins' \ --url 'https://foo.cloudfunctions.net/bar' \ --http-method POST

Slide 54

Slide 54 text

@BretMcG Bret McGowen Cloud Scheduler Cloud Scheduler & App Engine Execute App Engine on a repeating schedule $ gcloud scheduler jobs create-app-engine-job my-appengine-job \ --message-body 'Hello World!' \ --schedule 'Every 1 mins' \ --relative-url '/my-handler/' \ --service default

Slide 55

Slide 55 text

@BretMcG Bret McGowen Demo setup

Slide 56

Slide 56 text

@BretMcG Bret McGowen HTTP

Slide 57

Slide 57 text

@BretMcG Bret McGowen HTTP

Slide 58

Slide 58 text

@BretMcG Bret McGowen $ gcloud functions deploy ... --retry HTTP

Slide 59

Slide 59 text

@BretMcG Bret McGowen Dead le er queue writeToDB Failed messages HTTP

Slide 60

Slide 60 text

@BretMcG Bret McGowen Dead le er queue writeToDB Failed messages HTTP

Slide 61

Slide 61 text

@BretMcG Bret McGowen Dead le er queue writeToDB Failed messages HTTP

Slide 62

Slide 62 text

@BretMcG Bret McGowen HTTP Dead le er queue writeToDB retryFailed Failed messages

Slide 63

Slide 63 text

@BretMcG Bret McGowen Demo: Cloud Scheduler

Slide 64

Slide 64 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Cloud Functions Access & IAM

Slide 65

Slide 65 text

@BretMcG Bret McGowen Network A VPC ● Create a network with a name ● Add your Compute Engine instances to that network ● Add your Cloud Functions or App Engine resources to that network ● Egress to Compute Engine IP addresses Network A

Slide 66

Slide 66 text

@BretMcG Bret McGowen VPC VPC & VPN Access Compute Engine VMs from your Cloud Function $ gcloud functions deploy --connected-vpc my-network

Slide 67

Slide 67 text

@BretMcG Bret McGowen Security Controls Security Controls Control access to function invocation using IAM $ gcloud functions add-iam-policy-binding helloWorld \ --member='user:[email protected]' \ --role='roles/cloudfunctions.invoker' $ gcloud functions add-iam-policy-binding helloWorld \ --member='allUsers' \ --role='roles/cloudfunctions.invoker'

Slide 68

Slide 68 text

@BretMcG Bret McGowen Security Controls Serving HTTP Function HTTP Function IAM Check HTTP

Slide 69

Slide 69 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Shhh...

Slide 70

Slide 70 text

@BretMcG Bret McGowen Serverless Containers Serverless Containers Provide arbitrary container images and run them "serverlessly" ● Takes a pre-built Docker image ● Use arbitrary base images ● Use arbitrary system libraries ● Use arbitrary language runtime ● Same serverless execution environment ○ No servers ○ Pay only while code runs

Slide 71

Slide 71 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Demo

Slide 72

Slide 72 text

@BretMcG Bret McGowen @BretMcG Bret McGowen Serverless challenges

Slide 73

Slide 73 text

@BretMcG Bret McGowen Challenges in serverless today Dependencies Constrained runtimes, frameworks and packages Multi-Cloud Unable to run your workloads on-prem, in the cloud or on a third party service provider 1 2

Slide 74

Slide 74 text

@BretMcG Bret McGowen FROM python RUN apt-get update && apt-get install -y blender ENV APP_HOME /app COPY . $APP_HOME WORKDIR $APP_HOME RUN pip install Flask EXPOSE 8080 CMD ["python", "app.py"] Containers Image It’s a method of packaging an application executable and its dependencies (runtime, libraries, configuration) Runtime and running as a set of resource-isolated processes.

Slide 75

Slide 75 text

@BretMcG Bret McGowen Coming soon: GCF serverless containers Serverless containers Fully managed BYO workloads Pay for use Alpha invitations coming later this year Sign up: g.co/serverlesscontainers

Slide 76

Slide 76 text

@BretMcG Bret McGowen Demo

Slide 77

Slide 77 text

Slide 78

Slide 78 text

@BretMcG Bret McGowen Knative

Slide 79

Slide 79 text

@BretMcG Bret McGowen Kubernetes is the de facto platform for running containers.

Slide 80

Slide 80 text

@BretMcG Bret McGowen Kubernetes keeps your applications running while you're asleep. Container died? Restart it. Server or container unhealthy? Reschedule to another node. Container overloaded? Add more replicas automatically.

Slide 81

Slide 81 text

@BretMcG Bret McGowen Kubernetes is not easy 1. It was never meant to be used by developers directly. 2. Creating and operating Kubernetes clusters in production is pretty much a full time job.

Slide 82

Slide 82 text

@BretMcG Bret McGowen Google Kubernetes Engine (GKE) The zero ops cluster experience: ● update your cluster to new versions of Kubernetes ● scale the cluster up/down automatically ● detect and replace broken nodes of the cluster

Slide 83

Slide 83 text

@BretMcG Bret McGowen Kubernetes isn't actually for developers It's not the right abstraction for end-developer experience. (This did not stop developers from using Kubernetes directly!) But it's a great platform for building a PaaS on top of.

Slide 84

Slide 84 text

@BretMcG Bret McGowen Why use Kubernetes for serverless? Reduce lock-in Performance Multi-cloud Customizability Use custom hardware (GPU, TPU, IoT, etc) Offline/reduced connectivity Existing on-prem infrastructure Existing applications

Slide 85

Slide 85 text

@BretMcG Bret McGowen What should it take to deploy a function or app? Spin up a VM instance Provision server capacity Specify DB requirements Write code Patch server Scale capacity depending on workload size Spin up a VM instance Provision server capacity Specify DB requirements Write code Patch server Scale capacity depending on workload size Write code

Slide 86

Slide 86 text

@BretMcG Bret McGowen Meet Knative Building blocks for creating serverless experiences on top of Kubernetes. github.com/knative

Slide 87

Slide 87 text

@BretMcG Bret McGowen Knative partners

Slide 88

Slide 88 text

@BretMcG Bret McGowen Knative stack Kubernetes + Istio Platform

Slide 89

Slide 89 text

@BretMcG Bret McGowen Knative stack Knative Build Serving Events Kubernetes + Istio Platform Primitives

Slide 90

Slide 90 text

@BretMcG Bret McGowen Knative Build Serving Events Kubernetes + Istio Platform Products Serverless Containers on GCF GKE Serverless Add-on SAP Kyma Pivotal Function Service IBM Cloud Functions Red Hat Cloud Functions Primitives riﬀ OpenFaaS Jazz

Slide 91

Slide 91 text

@BretMcG Bret McGowen What Knative is ● An open source project ● Set of building blocks to construct your own FaaS/PaaS ○ abstracts common tasks through custom Kubernetes API objects ● An abstraction on top of Kubernetes. ○ It's still Kubernetes: Runs containers at the end of the day.

Slide 92

Slide 92 text

@BretMcG Bret McGowen What Knative is not ● It's not a Google product. ● It's not a FaaS.

Slide 93

Slide 93 text

@BretMcG Bret McGowen What can you do with Knative? [Developers] Use it directly to deploy stuff (not easy, but works fine) [Operators] Put a level of abstraction between your devs and Kubernetes. [Platform Architects] Use it to build your own serverless platform. e.g. DIY Heroku or GCF/Lambda.

Slide 94

Slide 94 text

@BretMcG Bret McGowen Infrastructure Primitives Knative Kubernetes Oﬀ-the-shelf FaaS (riﬀ, OpenFaaS, Apache Whisk…) Developer Experience Your in-house FaaS platform Developers

Slide 95

Slide 95 text

@BretMcG Bret McGowen Serverless Compute on Cloud Platform Release Schedule New features rolling out in the coming weeks Check the Cloud Platform Blog for updates and early access

Slide 96

Slide 96 text

@BretMcG Bret McGowen Recap: new in Cloud Functions Generally Available (with SLA!) Python 3.7, Node 8 Regions: Tokyo, Belgium, 2 in US Environment Variables Cloud Scheduler Ubuntu 18.04 with many packages (ffmpeg, imagemagick, headless Chrome) Security Controls: VPC, IAM Scaling Controls Cloud SQL Direct Connect New! New!

Slide 97

Slide 97 text

@BretMcG Bret McGowen App Engine: 2nd gen runtimes New! Pay for what you use, scale to zero Open-source, idiomatic experience Use any module, extension, or framework New supported runtimes: ○ Node.js 8 ○ Python 3.7 ○ PHP 7.2 ○ Go 1.11!

Slide 98

Slide 98 text

@BretMcG Bret McGowen Knative on Kubernetes New!

Slide 99

Slide 99 text

@BretMcG Bret McGowen cloud.google.com/serverless Knative github.com/knative slack.knative.dev GKE serverless add-on (sign-up) g.co/serverlessaddon Containers on Cloud Functions (sign-up) g.co/serverlesscontainers Bret McGowen @BretMcG bretmcg.com StackOverflow Thank you!

Slide 100

Slide 100 text

@BretMcG Bret McGowen Resources cloud.google.com/serverless Knative github.com/knative slack.knative.dev GKE serverless add-on (sign-up) g.co/serverlessaddon Containers on Cloud Functions (sign-up) g.co/serverlesscontainers Thank you! Bret McGowen @BretMcG

Slide 101

Slide 101 text

@BretMcG Bret McGowen Thank you. @BretMcG