Google Cloud for Serverless Compute

Slide 1

Slide 1 text

Google Cloud for Serverless Compute Sandeep Parikh, Cloud Native Advocate

Slide 2

Slide 2 text

Today we’re going to cover the what, why, and how of serverless compute on Google Cloud.

Slide 3

Slide 3 text

About me I’m a Cloud Native Advocate at Google. I focus on helping developers and operators use Kubernetes and Istio. Before Google, I worked at MongoDB, Apple, and a bunch of other places. You can find me on Twitter @crcsmnky Github @crcsmnky Hi!

Slide 4

Slide 4 text

How did we get here?

Slide 5

Slide 5 text

First, some history

Slide 6

Slide 6 text

Wind the clocks back to 2008 In the beginning, there was App Engine. You could write code and deploy your app within minutes.

Slide 7

Slide 7 text

Not As Good Only certain runtimes Support for specific services Sandboxing Lack of VPC connectivity App Engine Good From code to app in minutes Scale to zero Traffic splitting, versions, etc. No instances to manage

Slide 8

Slide 8 text

Over time, the product line grew into App Engine and App Engine Flex. Next came Flex

Slide 9

Slide 9 text

Not As Good Lengthy startup times No support for scale to zero Higher costs (building images, running instances) App Engine Flex Good Containers and custom runtimes Traffic splitting, versions, etc. VPC connectivity Easy access to GCP services

Slide 10

Slide 10 text

Cloud Functions for event-driven serverless compute. Along came Functions

Slide 11

Slide 11 text

Not As Good Few runtimes Basic authn/authz Few event-driven integrations Function-level granularity Cloud Functions Good Great developer experience Autoscaling, no instances Pay only when your code runs Event-driven integrations

Slide 12

Slide 12 text

But wait, there’s more! App Engine added support for 2nd generation runtimes, which combined the flexibility of Flex and stability of Standard.

Slide 13

Slide 13 text

App Engine 2nd Generation Runtimes Standard Standard 2nd generation Flexible Python 2.7 Java 8 PHP 5.5 Go 1.9 and Go 1.11 Python 3.7 PHP 7.2 Node.js Go 1.12 (beta) Ruby (alpha) Node.js, Ruby, Java, Python, Go, PHP, .NET & custom container images Sandboxed processes Runs on managed VMs GAE specific APIs + GCP APIs Use GCP APIs directly No binary modules/libraries any extension, binary, or framework open-source, idiomatic experience

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Where did that leave us?

Slide 16

Slide 16 text

{ your containers your code { containers existing systems web applications* web applications Kubernetes Engine Compute Engine App Engine Flex App Engine Cloud Functions event driven * app containers too

Slide 17

Slide 17 text

So what comes next?

Slide 18

Slide 18 text

Operational Model No Infra Management Fully Managed Security Pay only for usage Programming Model Service-based Event-driven Open Stateless Let’s start with the basics of serverless

Slide 19

Slide 19 text

Containers Flexibility Serverless Velocity

Slide 20

Slide 20 text

Introducing Cloud Run The next step in bringing serverless to containers

Slide 21

Slide 21 text

Cloud Run Container to production in seconds Natively Serverless One experience, where you want it

Slide 22

Slide 22 text

Cloud Run Just ‘deploy’ Any stateless container Any language, any library URL in seconds Focus on writing code Scale up fast Scale down to zero Pay for exact usage No servers to manage

Slide 23

Slide 23 text

Autoscaling based on requests Scales up fast Scales down to zero

Slide 24

Slide 24 text

Pay only for what you use 100ms CPU / Memory / Requests

Slide 25

Slide 25 text

Billable time Instance Billable Time Request 1 Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable

Slide 26

Slide 26 text

Cloud Run use cases Public ● Website ● API endpoint ● Mobile backend ● Webhook Private ● Microservices ● Asynchronous tasks

Slide 27

Slide 27 text

Cloud Run What does this mean? Container to production in seconds Natively Serverless One experience, where you want it

Slide 28

Slide 28 text

Introducing Cloud Run on GKE Same great Cloud Run, but on Kubernetes More flexibility and control, operator required. Integrates with k8s-based policy, control & mgmt Custom nodes, hardware accelerators, VPC Build on your existing investment in Kubernetes

Slide 29

Slide 29 text

Cloud Run Fully serverless, no cluster Pay for what you use Cloud Run on GKE Serverless developer experience Runs in your GKE cluster Serverless containers, where you want them

Slide 30

Slide 30 text

Cloud Run on GKE: Fixed Price Priced as part of Kubernetes Engine Uses the provisioned resources in your cluster Works with GKE Usage Metering Instances $

Slide 31

Slide 31 text

Runs in your GKE cluster Provisioned resources Kubernetes operations Custom machine types Hardware accelerators (GPUs) Fully managed, no cluster Pay-per-use Minimal operations Limited instance size Autoscaling Stackdriver UI & CLI Custom URLs Knative Cloud Run Cloud Run on GKE

Slide 32

Slide 32 text

One experience, where you want it

Slide 33

Slide 33 text

Serverless + Portability

Slide 34

Slide 34 text

Knative open source building blocks for serverless on Kubernetes

Slide 35

Slide 35 text

Activates & scales up/down based on requests Manages code and config revisions Service mesh integration for request path/service access control Custom domains, certificate management Orchestrates on/off cluster resources Bindings for event sources, triggers, and services Scales from few events to full streaming Builds on CloudEvents Reproducible builds Source to serving URL templates No need for Docker or cross-compilation Supports de-coupled CI/CD Support for policy and audit controls What you kneed to know about Knative Serving Eventing Build

Slide 36

Slide 36 text

Cloud Run & Knative Portable via common API and runtime environment. Cloud Run implements Knative Serving and Knative Runtime Contract.

Slide 37

Slide 37 text

Products Google Cloud Run Red Hat OpenShift SAP Kyma Google Cloud Run on GKE IBM Cloud Kubernetes Service TriggerMesh Build Serving Kubernetes Platform Primitives Events ... Knative ecosystem

Slide 38

Slide 38 text

Knative community 20% MoM Participation Growth >50 Companies contributing 3.7K Pull Requests 400+ Contributors 9 Working Groups v0.5 Released late April!

Slide 39

Slide 39 text

You probably want more details about Cloud Run

Slide 40

Slide 40 text

Authorization

Slide 41

Slide 41 text

GCP Invoker permissions Service IAM Requests Auth check: "allUsers" "user:mail@domain.com" "serviceAccount:..."

Slide 42

Slide 42 text

Public service Frontend IAM: role: "roles/run.invoker" member: "allUsers"

Slide 43

Slide 43 text

Leverage "Invoker" IAM role and service identity. Private service to service Frontend Backend IAM: role: "roles/run.invoker" member: "serviceAccount:frontend@..." header:"Authorization: Bearer ID_TOKEN"

Slide 44

Slide 44 text

Push Events with Pub/Sub Pub/Sub push to Cloud Run URL with authentication token. Leverage "Invoker" IAM role to authorize push. No need to validate URL. Cloud Run Service Cloud Pub/Sub IAM: role: "roles/run.invoker" member: "serviceAccount:pubsub@..." gcloud alpha pubsub subscriptions create my-sub --topic my-topic --push-endpoint=https://service.run.app --push-auth-service-account=pubsub@...

Slide 45

Slide 45 text

Async tasks Cloud Tasks HTTP targets (Beta soon) push to Cloud Run URL with authentication token Leverage "Invoker" IAM role. Service Cloud Tasks IAM: role: "roles/run.invoker" member: "serviceAccount:tasks@..." HTTP target

Slide 46

Slide 46 text

Scheduled services Cloud Scheduler with authentication token Leverage "Invoker" IAM role. Service Cloud Scheduler IAM: role: "roles/run.invoker" member: "serviceAccount:scheduler@..."

Slide 47

Slide 47 text

Concurrency

Slide 48

Slide 48 text

Concurrency in Cloud Run Each Service is autoscaled to many container instances. Concurrency = "maximum number of requests that can be sent at the same time to a given container instance" AWS Lambda or Google Cloud Functions: only one request at a time to each instance, "concurrency = 1". With Cloud Run: set concurrency value from 1 to 80 (default: 80) → optimized resource consumption → optimized costs concurrency = 1 concurrency = 80

Slide 49

Slide 49 text

concurrency = 1 concurrency = 80 400 clients, making 3 req/sec

Slide 50

Slide 50 text

Other details

Slide 51

Slide 51 text

Container runtime contract State Listen for HTTP requests on $PORT CPU outside of requests

Slide 52

Slide 52 text

Monitoring & Logging ✓ Monitoring Out of the box: ✓ Error Reporting ✓ Logging Stackdriver

Slide 53

Slide 53 text

gVisor Container sandbox runtime gvisor.dev Secure container isolation. Most applications run well. Contact GCP support if you encounter a limitation due to unsupported system call. Container gVisor Host System calls Limited system calls Secure isolation }

Slide 54

Slide 54 text

Current limits ● Max to 1 vCPU and 2GB RAM ● No access to GPUs ● No Cloud SQL Coming Soon ● No VPC access Coming Soon → No Cloud Memorystore ● No Global Load Balancer Cloud Run on GKE Solution