Serverless on Google Cloud - Deep Dive

@BretMcG
Bret McGowen
Deep dive into serverless on
Google Cloud
Bret McGowen
Google New York City
Developer Advocate
@BretMcG
bretmcg.com

View Slide

@BretMcG
Bret McGowen
Serverless swear words
Contai***s
Kubern****s
Infrastr****re
Offl*ne
Cluster of virt*** mach***s
.

View Slide

@BretMcG
Bret McGowen
Hosted FaaS/compute
Easiest dev experience
Code focused
Limited runtime options
Serverless anywhere
Knative / GKE serverless addon
On prem or multi-cloud
Code or containers
Custom hardware (GPU, TPU, IoT, etc)
Offline
You manage infra
Serverless operational spectrum
Managed by your cloud
Container orchestration (k8s)
Not serverless for programming or ops
Container focused
You manage infra
Managed by your team
Serverless containers
Custom languages & runtimes
Infra still cloud-managed

View Slide

@BretMcG
Bret McGowen
Hosted FaaS/compute
Easiest dev experience
Code focused
Limited runtime options
Serverless anywhere
Knative / GKE serverless addon
On prem or multi-cloud
Code or containers
Custom hardware (GPU, TPU, IoT, etc)
Offline
You manage infra
Serverless operational spectrum
Managed by your cloud
Container orchestration (k8s)
Not serverless for programming or ops
Container focused
You manage infra
Managed by your team

View Slide

@BretMcG
Bret McGowen
First, a word about
containers....
Containers != Kubernetes
(Blasphemy, I know)

View Slide

@BretMcG
Bret McGowen
Middleware
Compute Databases
Cloud Functions App Engine
Functions as a Service,
event-driven
Platform as a Service
(standard environment)
Firestore Cloud Datastore
NoSQL document
store & sync
BigQuery
NoSQL Data warehouse &
analytics
Machine Learning
Cloud ML Engine
Serverless Tensorflow
training & prediction
AutoML
Training & prediction
from examples
Cloud Dataflow
Stream & batch data
processing
Cloud Pub/Sub
Global real-time
messaging
Frontend
Data Studio
Firebase

View Slide

@BretMcG
Bret McGowen
Serverless on Cloud Platform
Compute Storage Big Data AI/ML
App Engine
Cloud Functions
Cloud Datastore
Cloud Firestore
Cloud Storage
BigQuery
Cloud Dataflow
Cloud Pub/Sub
Cloud ML Engine
Cloud AutoML

View Slide

@BretMcG
Bret McGowen
Serverless Compute on Cloud Platform
App Engine Standard
Highly scalable, serverless web
applications.
Deploy and scale Applications
that react to Requests
Cloud Functions
Event-driven serverless
compute platform.
Deploy and scale Functions
that react to Events

View Slide

@BretMcG
Bret McGowen
Serverless for apps and compute
Mobile apps
Functions as a Service
App backends (e.g. APIs,
web apps)
App Engine Standard Cloud Functions Firebase
Fully managed, auto-scales
Versioning, traffic splitting
Java, Python, Go, PHP, Node.js
Event-driven functions
Microservices, cloud "glue"
Node.js, Python (beta), Go
(alpha)
Real-time syncing databases
Authentication, analytics,
notifications, hosting, storage,
ML Kit, much more...

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
App Engine Standard

View Slide

@BretMcG
Bret McGowen
App Engine (Standard)
A Quick Recap
Launched at I/O in May of 2008
Python Runtime and memcache api
Java Runtime in April of 2009
Moves out of preview in November of 2011
Go Runtime in March of 2012
PHP Runtime in October 2013
Java 8 in June of 2017
Node.js 8 in June of 2018 with 2nd Generation Runtime

View Slide

@BretMcG
Bret McGowen
New 2nd Generation Runtimes
● Open-source, idiomatic experience
● Use any extension, binary, or
framework
● Supported Runtimes
○ Node.js 8
○ Python 3.7
○ PHP 7.2
○ Go 1.11

View Slide

@BretMcG
Bret McGowen
New! Go 1.11 support
2nd Gen runtime
Modules support
Idiomatic
Any library
Regular project structure
Web frameworks (e.g gin)
vendor/ dependencies
Today - beta in App
Engine Standard
GCF coming soon

View Slide

@BretMcG
Bret McGowen
Cloud Tasks
App Engine Standard 2nd
gen runtimes
(Node.js, Python 3.7, PHP
App Engine Flex
Distributed task queues
Scalable and fully managed
Rate and retry controls
App Engine targets
Future scheduling

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Cloud Functions

View Slide

@BretMcG
Bret McGowen
Cloud Functions: triggered by events
Database changes
Firestore database events
(create/update/delete/write)
Cloud Storage files
App events
Firebase Authentication
Crashlytics
Google Analytics for Firebase
Infrastructure changes
Stackdriver logging
More coming soon!
Scheduled
Cloud Scheduler
Explicit invocations
HTTP
Webhooks for 3rd party
Messages
Pub/Sub messages

View Slide

@BretMcG
Bret McGowen
Cloud Functions
A Quick Recap
Beta Launch @ Next 2017
Node.js Runtime
HTTP/S in the box
Stackdriver Integration
Pay only when code runs

View Slide

@BretMcG
Bret McGowen
Cloud Function event triggers
Cloud storage (incl. Firebase)
Cloud Pub/Sub
HTTPS
Firebase realtime database
& the new Firestore database
Firebase authentication
Google analytics for Firebase
Firebase hosting
Firebase Crashlytics

View Slide

@BretMcG
Bret McGowen
Access 20+ Google
services from GCF
Cloud Storage
Cloud Pub/Sub
HTTPS
Firebase
Cloud Scheduler
Cloud Functions as cloud glue

View Slide

@BretMcG
Bret McGowen
Cloud Console UI
Deploy from
$ gcloud functions deploy helloWorld \
--trigger-http
Command line
API

View Slide

@BretMcG
Bret McGowen
General Availability
Cloud Functions is GA!
● Open to all developers
● Stable and ready for production use.
● Service Level Agreement
○ https://cloud.google.com/functions/sla

View Slide

@BretMcG
Bret McGowen
Global Footprint
$ gcloud functions deploy --region europe-west1
Iowa
Belgium
Tokyo
South Carolina

View Slide

@BretMcG
Bret McGowen
Single Project, Multiple Regions
Tokyo Belgium Iowa
My Project

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
New runtimes on
Cloud Functions

View Slide

@BretMcG
Bret McGowen
New Cloud Functions languages!

View Slide

@BretMcG
Bret McGowen
Language Support
Node 8 (Beta)
● Write Cloud Functions using Node 8.11
● Support for async/await
● New function signature!
$ gcloud functions deploy --runtime nodejs8

View Slide

@BretMcG
Bret McGowen
Language Support
exports.helloPubsubCallback = (event, callback) => {
const pubsubMessageData = event.data.data;
doSomething(pubsubMessageData, callback);
};
exports.helloPubsubPromise = (event) => {
const pubsubMessageData = event.data.data;
return doSomethingWithAPromise(pubsubMessageData)
};
Current (Node 6) Behavior

View Slide

@BretMcG
Bret McGowen
Language Support
New (Node 8+) Behavior
// Note the new (optional) context object
exports.helloPubsub = async (data, context) => {
return await doSomethingAsynchronous(data)
};
// HTTP Functions are unchanged, but also support async/await
exports.helloHTTP = async (req, res) => {
return await doSomethingAsynchronous(req)
};

View Slide

@BretMcG
Bret McGowen
Language Support
Python 3.7!!! (Beta)
Write Cloud Functions using Python 3.7
$ gcloud functions deploy --runtime python37
def hello_pubsub(data, context):
return do_something(data)
def hello_http(request):
return do_something(request)

View Slide

@BretMcG
Bret McGowen
Language Support
Python HTTP Functions
● GET, PUT, POST, DELETE and OPTIONS
● Based on the Flask microframework
● Requests are flask.Request
● Responses should be compatible with
flask.make_response

View Slide

@BretMcG
Bret McGowen
Language Support
Python Background Functions
● As with Node, called with data (dict) and context
(google.cloud.functions.Context)
● To signal successful completion, just return from
your function
● To signal that your function has failed to complete,
just raise an exception

View Slide

@BretMcG
Bret McGowen
Language Support
Common Features
● Logs (stdout, stderr) automatically sent to Stackdriver Logging
● Uncaught exceptions automatically handled by Stackdriver
Error Reporting (for alerts etc)
● Automatic build and dependency resolution (package.json,
requirements.txt) in the cloud

View Slide

@BretMcG
Bret McGowen
Language Support
The Context Object
Python Node Description Type
event_id eventId A unique ID for the event.
For example: "70172329041928"
String
timestamp timestamp The date/time this event was created.
For example: "2018-04-09T07:56:12.975Z"
String (ISO 8601)
event_type eventType The type of the event.
For example: "google.pubsub.topic.publish"
String
resource resource The resource that emitted the event. This
dictionary has attributes `service`, `name`
and `type`.
Dictionary /
Object

View Slide

@BretMcG
Bret McGowen
Language Support
Sample Context Object (Node.js)
{
eventId:"122286916458880",
timestamp:"2018-06-19T23:12:19.340Z",
eventType:"google.pubsub.topic.publish",
resource:{
service:"pubsub.googleapis.com",
name:"projects/my-project/topics/foo",
type:"type.googleapis.com/google.pubsub.v1.PubsubMessage"
}
}

View Slide

@BretMcG
Bret McGowen
Cloud Functions for Firebase
New CLI (v4.0.0) & New SDK (v2.0.0)
● Cloud Functions for Firebase is GA
● Full support for Node 8
○ ECMAScript 2017 support
○ Async/Await
● New runtime configuration options
○ Region, memory, timeout
● Firebase Events (Analytics, Firestore, Realtime
DB, Authentication), now available directly in
Cloud Functions

View Slide

@BretMcG
Bret McGowen
Ubuntu Base Image
Ubuntu 18.04 LTS Base Image
Unforked linux distribution and broader set of system libraries
● Headless Chrome libs!
● imagemagick
● ffmpeg
● libcairo2

View Slide

@BretMcG
Bret McGowen
Moar System Libraries
fontconfig
ffmpeg
flvmeta
geoip-database
git
imagemagick
jq
libatlas3-base
libblas3
libbz2-1.0
libcurl4-openssl-dev
libdb5.3
libenchant1c2a
libexpat1
libffi6
libfftw3-double3
libflac8
libfontconfig1
libfontenc1
libfreetype6
libgcrypt20
libgd3
libgdk-pixbuf2.0-0
libgdk-pixbuf2.0-common
libgmp10
libgmpxx4ldbl
libgdbm5
libgoogle-perftools4
libgraphite2-3
libgs9
libgs9-common
libicu60
libhashkit2
libjbig0
libjbig2dec0
libjpeg8
libjpeg-turbo8
liblapack3
libldap-2.4-2
liblzma5
libmagickcore-6.q16-3
libmagickcore-6.q16-3-extra
libmagickwand-6.q16-3
libmemcached11
libmemcachedutil2
libmpc3
libmpdec2
libmysqlclient20
libncursesw5
libnetpbm10
libpng16-16
libprotoc10
libpq5
librabbitmq4
librdkafka1
libreadline7
librsvg2-2
librsvg2-common
libsasl2-2
libsasl2-modules
libsasl2-modules-db
libsqlite3-0
libssl1.0.0
libtiff5
libtiffxx5
libtidy5
libuuid1
libvpx5
libwebp6
libxml2
libxslt1.1
libyaml-0-2
libzip4
locales
lsb-release
mime-support
netpbm
python3-chardet
tzdata
uuid-runtime

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Cloud Functions
Environment Variables

View Slide

@BretMcG
Bret McGowen
Store configuration outside your source code
$ gcloud functions deploy --set-env-vars FOO=bar

View Slide

@BretMcG
Bret McGowen
$ … deploy --set-env-vars FOO=bar
env FOO="bar"
process.env.FOO
Environment Variables are:
● Set at deploy-time
● Bound to a single
function
● Surfaced as literal
environment variables

View Slide

@BretMcG
Bret McGowen
{
"environmentVariables": {
string: string,
...
}
}
Cloud Console
API

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Demo: headless Chrome

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Cloud Functions
Cloud SQL Direct Connect

View Slide

@BretMcG
Bret McGowen
Direct connection to Cloud SQL instances
const pool = mysql.createPool({
connectionLimit : 1,
socketPath: '/cloudsql/foo',
user: dbUser,
password: dbPass,
database: dbName
});

View Slide

@BretMcG
Bret McGowen
Internal
External

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Cloud Functions
Scaling Controls

View Slide

@BretMcG
Bret McGowen
Example: Connection Pools

View Slide

@BretMcG
Bret McGowen
Scaling Controls
Scaling Controls
Limit scaling on a per-function basis
$ gcloud functions deploy --max-instances 100

View Slide

@BretMcG
Bret McGowen
Scaling Controls
Scaling Controls
● Controls and limits are per-function (not per project)
● Default limit of up to 1,000 (varies by region)
● Can be increased upon request for HTTP Functions
● Can be reduced on a per-function basis via API/CLI/UI
● Requests exceeding the limit are queued with a timeout
(60s)

View Slide

@BretMcG
Bret McGowen
Example: Connection Pools

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Serverless Scheduling

View Slide

@BretMcG
Bret McGowen
Cloud Scheduler
Pub/Sub
1. Establish a Cloud Scheduler schedule
(down to 1 min intervals)
2. Cloud Scheduler invokes Cloud Functions
over HTTPS or via Cloud Pub/Sub
3. Invokes App Engine on a relative url
handler
HTTPS
Now
in Beta!

View Slide

@BretMcG
Bret McGowen
Cloud Scheduler
Cloud Scheduler & Cloud Functions
Execute Cloud Functions on a repeating schedule
$ gcloud scheduler jobs create-http-job my-http-job \
--message-body 'Hello World!' \
--schedule 'Every 1 mins' \
--url 'https://foo.cloudfunctions.net/bar' \
--http-method POST

View Slide

@BretMcG
Bret McGowen
Cloud Scheduler
Cloud Scheduler & App Engine
Execute App Engine on a repeating schedule
$ gcloud scheduler jobs create-app-engine-job my-appengine-job \
--message-body 'Hello World!' \
--schedule 'Every 1 mins' \
--relative-url '/my-handler/' \
--service default

View Slide

@BretMcG
Bret McGowen
Demo setup

View Slide

@BretMcG
Bret McGowen
HTTP

View Slide

@BretMcG
Bret McGowen
$ gcloud functions deploy ... --retry
HTTP

View Slide

@BretMcG
Bret McGowen
Dead le er queue
writeToDB
Failed messages
HTTP

View Slide

@BretMcG
Bret McGowen
HTTP
Dead le er queue
writeToDB
retryFailed
Failed messages

View Slide

@BretMcG
Bret McGowen
Demo: Cloud
Scheduler

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Cloud Functions
Access & IAM

View Slide

@BretMcG
Bret McGowen
Network A
VPC
● Create a network with
a name
● Add your Compute Engine
instances to that network
● Add your Cloud Functions
or App Engine resources to
that network
● Egress to Compute Engine
IP addresses
Network A

View Slide

@BretMcG
Bret McGowen
VPC
VPC & VPN
Access Compute Engine VMs from your Cloud Function
$ gcloud functions deploy --connected-vpc my-network

View Slide

@BretMcG
Bret McGowen
Security Controls
Security Controls
Control access to function invocation using IAM
$ gcloud functions add-iam-policy-binding helloWorld \
--member='user:[email protected]' \
--role='roles/cloudfunctions.invoker'
$ gcloud functions add-iam-policy-binding helloWorld \
--member='allUsers' \
--role='roles/cloudfunctions.invoker'

View Slide

@BretMcG
Bret McGowen
Security Controls
Serving
HTTP
Function
HTTP
Function
IAM
Check
HTTP

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Shhh...

View Slide

@BretMcG
Bret McGowen
Serverless Containers
Serverless Containers
Provide arbitrary container images and run them "serverlessly"
● Takes a pre-built Docker image
● Use arbitrary base images
● Use arbitrary system libraries
● Use arbitrary language runtime
● Same serverless execution environment
○ No servers
○ Pay only while code runs

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Demo

View Slide

@BretMcG
Bret McGowen
@BretMcG
Bret McGowen
Serverless challenges

View Slide

@BretMcG
Bret McGowen
Challenges in serverless today
Dependencies
Constrained runtimes,
frameworks and packages
Multi-Cloud
Unable to run your workloads
on-prem, in the cloud or on a
third party service provider
1 2

View Slide

@BretMcG
Bret McGowen
FROM python
RUN apt-get update && apt-get install -y blender
ENV APP_HOME /app
COPY . $APP_HOME
WORKDIR $APP_HOME
RUN pip install Flask
EXPOSE 8080
CMD ["python", "app.py"]
Containers
Image
It’s a method of packaging
an application executable and
its dependencies (runtime,
libraries, configuration)
Runtime
and running as a set
of resource-isolated
processes.

View Slide

@BretMcG
Bret McGowen
Coming soon: GCF serverless containers
Fully managed
BYO workloads
Pay for use
Alpha invitations coming later this year
Sign up:
g.co/serverlesscontainers

View Slide

@BretMcG
Bret McGowen
Demo

View Slide

@BretMcG
Bret McGowen
Challenges in serverless today
Dependencies
Constrained runtimes,
frameworks and packages
Multi-Cloud
Unable to run your workloads
on-prem, in the cloud or on a
third party service provider
1 2

View Slide

@BretMcG
Bret McGowen
Knative

View Slide

@BretMcG
Bret McGowen
Kubernetes is the de facto
platform for running containers.

View Slide

@BretMcG
Bret McGowen
Kubernetes keeps your applications running while you're
asleep.
Container died?
Restart it.
Server or container unhealthy?
Reschedule to another node.
Container overloaded?
Add more replicas automatically.

View Slide

@BretMcG
Bret McGowen
Kubernetes is not easy
1. It was never meant to be used by developers directly.
2. Creating and operating Kubernetes clusters in production
is pretty much a full time job.

View Slide

@BretMcG
Bret McGowen
Google Kubernetes Engine (GKE)
The zero ops cluster experience:
● update your cluster to new versions of Kubernetes
● scale the cluster up/down automatically
● detect and replace broken nodes of the cluster

View Slide

@BretMcG
Bret McGowen
Kubernetes isn't actually for developers
It's not the right abstraction for end-developer experience.
(This did not stop developers from using Kubernetes directly!)
But it's a great platform for building a PaaS on top of.

View Slide

@BretMcG
Bret McGowen
Why use Kubernetes for serverless?
Reduce lock-in
Performance
Multi-cloud
Customizability
Use custom hardware (GPU, TPU, IoT, etc)
Offline/reduced connectivity
Existing on-prem infrastructure
Existing applications

View Slide

@BretMcG
Bret McGowen
What should it take to
deploy a function or app?
Spin up a VM instance
Provision server capacity
Specify DB requirements
Write code
Patch server
Scale capacity depending on workload size
Spin up a VM instance
Provision server capacity
Specify DB requirements
Write code
Patch server
Scale capacity depending on workload size
Write code

View Slide

@BretMcG
Bret McGowen
Meet Knative
Building blocks for creating
serverless experiences
on top of Kubernetes.
github.com/knative

View Slide

@BretMcG
Bret McGowen
Knative partners

View Slide

@BretMcG
Bret McGowen
Knative stack
Kubernetes + Istio
Platform

View Slide

@BretMcG
Bret McGowen
Knative stack
Knative
Build Serving Events
Kubernetes + Istio
Platform
Primitives

View Slide

@BretMcG
Bret McGowen
Knative
Build Serving Events
Kubernetes + Istio
Platform
Products
Serverless Containers on GCF GKE Serverless Add-on SAP Kyma
Pivotal Function Service IBM Cloud Functions Red Hat Cloud Functions
Primitives
riﬀ OpenFaaS Jazz

View Slide

@BretMcG
Bret McGowen
What Knative is
● An open source project
● Set of building blocks to construct your own FaaS/PaaS
○ abstracts common tasks through custom Kubernetes API objects
● An abstraction on top of Kubernetes.
○ It's still Kubernetes: Runs containers at the end of the day.

View Slide

@BretMcG
Bret McGowen
What Knative is not
● It's not a Google product.
● It's not a FaaS.

View Slide

@BretMcG
Bret McGowen
What can you do with Knative?
[Developers] Use it directly to deploy stuff (not easy, but works fine)
[Operators] Put a level of abstraction between your devs and
Kubernetes.
[Platform Architects] Use it to build your own serverless platform.
e.g. DIY Heroku or GCF/Lambda.

View Slide

@BretMcG
Bret McGowen
Infrastructure
Primitives
Knative
Kubernetes
Oﬀ-the-shelf FaaS
(riﬀ, OpenFaaS, Apache Whisk…)
Developer
Experience
Your in-house
FaaS platform
Developers

View Slide

@BretMcG
Bret McGowen
Serverless Compute on Cloud Platform
Release Schedule
New features rolling out in
the coming weeks
Check the Cloud Platform
Blog for updates and early
access

View Slide

@BretMcG
Bret McGowen
Recap: new in Cloud Functions
Generally Available (with SLA!)
Python 3.7, Node 8
Regions: Tokyo, Belgium, 2 in US
Cloud Scheduler
Ubuntu 18.04 with many packages
(ffmpeg, imagemagick, headless Chrome)
Security Controls: VPC, IAM
Scaling Controls
New!
New!

View Slide

@BretMcG
Bret McGowen
App Engine: 2nd gen runtimes
New!
Pay for what you use, scale to zero
Open-source, idiomatic experience
Use any module, extension, or
framework
New supported runtimes:
○ Node.js 8
○ Python 3.7
○ PHP 7.2
○ Go 1.11!

View Slide

@BretMcG
Bret McGowen
Knative on Kubernetes
New!

View Slide

@BretMcG
Bret McGowen
cloud.google.com/serverless
Knative
github.com/knative
slack.knative.dev
GKE serverless add-on (sign-up)
g.co/serverlessaddon
Containers on Cloud Functions (sign-up)
Bret McGowen
@BretMcG
bretmcg.com
StackOverflow
Thank you!

View Slide

@BretMcG
Bret McGowen
Resources
cloud.google.com/serverless
Knative
github.com/knative
slack.knative.dev
GKE serverless add-on (sign-up)
g.co/serverlessaddon
Containers on Cloud Functions (sign-up)
Thank you!
Bret McGowen
@BretMcG

View Slide

@BretMcG
Bret McGowen
Thank you.
@BretMcG

View Slide

Serverless on Google Cloud - Deep Dive

Serverless on Google Cloud - Deep Dive

Bret McGowen - NYC Google Developer Advocate

More Decks by Bret McGowen - NYC Google Developer Advocate

Other Decks in Technology

Featured

Transcript