Going Serverless with Artifactory and Containers on Cloud Run

Slide 1

Slide 1 text

Going Serverless with Artifactory and Containers on Cloud Run Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, Hybrid Cloud Specialist @ Google @glaforge @archyufa

Slide 2

Slide 2 text

Proprietary + Confidential

Slide 3

Slide 3 text

Proprietary + Confidential The serverless models and characteristics Operational Model Programming Model No Infra Management Managed Security Pay only for usage Service-based Event-driven Stateless

Slide 4

Slide 4 text

Proprietary + Confidential FaaS App platform Container Virtual machine The serverless spectrum Dedicated server Physical hardware Virtual hardware Operation system Application runtime Application Function Virtual hardware Operation system Application runtime Application Function Operation system Application runtime Application Function Application Function Function Serverless Zone

Slide 5

Slide 5 text

Proprietary + Confidential Develop and deploy highly scalable containerized applications on a fully managed serverless platform ● Write code your way using your favorite languages (Go, Python, Java, Ruby, Node.js, and more) ● Abstract away all infrastructure management for a simple developer experience ● Built upon an open standard Knative, enabling the portability of your applications Cloud Run

Slide 6

Slide 6 text

Proprietary + Confidential Serverless containers with Knative and Cloud Run Cloud Run Fully managed, deploy your workloads and don’t see the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative.

Slide 7

Slide 7 text

Proprietary + Confidential Containers Any language Rich ecosystem Any library of base images Any binary .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1 Containers Flexibility Serverless Velocity

Slide 8

Slide 8 text

Proprietary + Confidential Container contract... and resources ● Listen on 0.0.0.0 on port $PORT (default 8080) ● HTTP server must start < 4 min (timeout → 504) ● Request time < 15 min (default → 5 min, up to 60 min) ● Stateless (in-memory file system) ● Computation only within request (No background activity) ● 1 vCPU per container instance (configurable to 4 vCPU) ● 256 MiB of memory up to a max of 8 GiB (configurable, soon 16 GiB) ● 80 concurrent requests per container (configurable 1-80, soon 250) ● 100 max containers by default (configurable 1-1000, support overridable) ● Sandboxed by gVisor

Slide 9

Slide 9 text

Proprietary + Confidential Pay per use CPU / Memory / Requests 100ms

Slide 10

Slide 10 text

Proprietary + Confidential Billable time Instance Billable Time Request 1 Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable

Slide 11

Slide 11 text

Proprietary + Confidential Concurrency model concurrency = 1 concurrency = 80

Slide 12

Slide 12 text

What else?

Slide 13

Slide 13 text

Cloud Run is available Planned Future GCP region

Slide 14

Slide 14 text

Gradual rollouts & Rollbacks Specify % traffic between revisions Blue / Green deployments Get URLs for specific revisions # Gradual rollout $ gcloud beta run deploy myservice \ --image gcr.io/project/image:f5bd774 \ --no-traffic \ --tag green $ gcloud beta run services update-traffic myservice \ --to-tags green=1 $ gcloud beta run services update-traffic myservice \ --to-tags green=10 $ gcloud beta run services update-traffic myservice \ --to-tags green=50 $ gcloud beta run services update-traffic myservice \ --to-tags green=100 # Rollback $ gcloud run services update-traffic myservice --to-revisions my-service-0002-joy=100 $ curl https://green---myservice-12345-us.a.run.app

Slide 15

Slide 15 text

GCP Project VPC Access Connect to Cloud Memorystore Redis and Memcached Connect to private IPs Shared VPC Cloud Run Serverless VPC Connector Cloud Memorystore VM Compute Engine Virtual Private Cloud Private IP Private IP

Slide 16

Slide 16 text

Cloud Workflows Orchestrate serverless tasks Fully managed. Use cases: ● Process events ● Chaining API calls ● Automate infra management ● Implement retry policies GCP Project Start Every day at 6pm Get list of all dev VMs End For each VM: Is it running ? Extract status Stop VM Email owner

Slide 17

Slide 17 text

Min instances Keep a number of instances warm Use to avoid cold starts Lower instance price when not in use Container Instances min = 2 0

Slide 18

Slide 18 text

Graceful instance termination Receive a SIGTERM signal before container instance is terminated. If handled, CPU is allocated for 10s max $ cat index.js const process = require('process'); process.on('SIGTERM', () => { console.log('Container is shutting down...'); // TODO: Close database connections // TODO: Send any buffered telemetry data }); ...

Slide 19

Slide 19 text

gRPC support $ cat main.go … func main() { port := os.Getenv("PORT") listener, err := net.Listen("tcp", ":"+port) grpcServer := grpc.NewServer() pb.RegisterPingServiceServer(grpcServer, &pingService{}) if err = grpcServer.Serve(listener); err != nil { log.Fatal(err) } }

Slide 20

Slide 20 text

Server-side streaming Server-Sent Events, WebSockets. Stream HTTP or gRPC responses Responses no more buffered and limited to 32MB $ cat main.go … http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "video/mp4") w.Header().Set("Transfer-Encoding", "chunked") f, _ := os.Open("videos/demo.mp4") io.Copy(w, f) }) port := os.Getenv("PORT") log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil)) (coming soon)

Slide 21

Slide 21 text

DEMO TIME!

Slide 22

Slide 22 text

Proprietary + Confidential Cloud Run Use cases More at https://cloud.run Serverless Use Cases Cloud Run Build a web or mobile app HTTP web app Bidirectional streaming WebSockets & gRPC Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Event driven reactive automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Stateful Workloads ? Healthcare Gaming Retail Banking

Slide 23

Slide 23 text

Proprietary + Confidential Attached clusters Anthos on Azure Anthos on AWS Google Anthos and Jfrog Partnership

Slide 24

Slide 24 text

Cloud Run with Jfrog Demo Stack JFrog Cloud Platform on GCP - Jfrog Pipelines - Jfrog Docker Artifactory - Jfrog Xray Pull request Trigger Docker Repository CI CD Update Image tag Cloud Run Fully Managed Cloud Run for Anthos Thanks @jenn_viau for helping to build this amazing demo! Code Quality Build Image Scan Image

Slide 25

Slide 25 text

Online Store application

Slide 26

Slide 26 text

Amazing Cloud Run Links ● Slides - https://speakerdeck.com/cncfcanada ● Awesome Cloud Run - github.com/steren/awesome-cloudrun ● Cloud Run FAQ - github.com/ahmetb/cloud-run-faq ● Vault on Cloud Run - gh/kelseyhightower/serverless-vault-with-cloud-run Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, Hybrid Cloud Specialist @ Google @glaforge @archyufa

Slide 27

Slide 27 text

THANK YOU! Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, GDG, GDE Hybrid Cloud Specialist @ Google @glaforge @archyufa

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

STEREN’S CLOUD RUN DECK

Slide 31

Slide 31 text

Cloud Run 2021-04

Slide 32

Slide 32 text

Intro

Slide 33

Slide 33 text

33 Proprietary + Confidential Application Development Performance Monitoring Serverless Physical Servers Network Hardware Physical Security Internet Connectivity Scaling VM Provisioning Ops & Security Management The Responsibility Pyramid Managed by customer Fully Managed by Google

Slide 34

Slide 34 text

34 Risk Simplicity Velocity Elasticity Key values of developing, deploying and scaling apps in a fully managed environment

Slide 35

Slide 35 text

35 Serverless Compute Deploy and scale applications fast and securely in a fully managed environment No Infra Management Speed to Market Auto-scaling

Slide 36

Slide 36 text

“We can’t be locked in.” “How can we use existing binaries?” “Why do I have to choose between containers and serverless?” “Can you support language ____ ?” But… customers ask us:

Slide 37

Slide 37 text

Containers • Any Language • Any Library • Any Binary • Ecosystem of base images .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1

Slide 38

Slide 38 text

Proprietary + Confidential Cloud Run Deploy in seconds Automatic HTTPS, Custom domains Any language, any library Portability No cluster management Run containers on a fully managed environment

Slide 39

Slide 39 text

Deploy in seconds Demo

Slide 40

Slide 40 text

Proprietary + Confidential Build a web app Web app Request-based services Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Use cases More at https://cloud.run

Slide 41

Slide 41 text

Get Started Setup Compute CICD Run Other Create Project Setup Compute Core Infra Load Balancing Service Mesh Specify Volumes NAT Shared VPC Workload YAML Auto-Scale YAML Cloud Build YAML Deploy Time Test Image Rep. Source Version Control Run 8x faster time to market with Cloud Run! Shared VPC Create Project Deploy Time & Test Source Version Control Image Rep Cloud Build YAML Run

Slide 42

Slide 42 text

Proprietary + Confidential Engineers 3 Days 2 Containerized app 1 5.5M Requests per day 700ms Latency €500 a month 40% cost reduction and 80% less effort

Slide 43

Slide 43 text

News and Roadmap

Slide 44

Slide 44 text

Cloud Run is available Planned Future GCP region

Slide 45

Slide 45 text

Delight developers ● Deploy from YAML / Export to YAML ● Easy Continuous Deployment set up from Git ● Develop and run locally ● Graceful instance termination ● Trigger from 60+ GCP sources (Eventarc) ● Cloud Workflows Expand addressable workloads ● Min instances ● Up to 16GB RAM ● Up to 4 CPU ● bidirectional gRPC streaming Preview ● websockets Preview ● HTTP/2 Preview ● Long running instances (1hr) What's new? Enterprise Class ● VPC connectors: Shared VPC support ● Ingress = internal ● Egress controls ● VPC-SC ● Secret Manager integration ● Customer Managed Encryption Keys ● Binary Authorization ● API Gateway support ● Identity Aware Proxy support ● Cloud Armor support ● Committed Use Discounts

Slide 46

Slide 46 text

Keep delighting developers Enterprise-class Expand addressable workloads ● Access and Transparency ● Liveness and Readiness checks ● Always on CPU ● Autoscaling improvements ● Deterministic URLs, Service Discovery and Service Mesh ✓ Current Current 🠻 🠻 🠻 🠻 ● Local development experience (with Cloud Code) ● Trigger with events (with Eventarc) ● Recommendations and insights (with Recommendation Hub) ● Better observability: new metrics, platform errors, tracing (with Cloud Ops) ● Infrastructure as code via Conﬁg Connector 1. Expand the Cloud Run Service: ● Instances: ﬁlesystem access, more CPU, more RAM… ● Spec: multiple container 2. New resource types: Jobs, non-HTTP services...

Slide 47

Slide 47 text

Docs: https://cloud.run PM hotline: [email protected] Q&A Help

Slide 48

Slide 48 text

Deep dives

Slide 49

Slide 49 text

Deep dive: Pricing

Slide 50

Slide 50 text

Cloud Run (fully managed): Pay-per-use CPU / Memory / Requests 100ms

Slide 51

Slide 51 text

Billable time Instance Billable Time Request 1 Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable

Slide 52

Slide 52 text

Deep dive: Advanced Features

Slide 53

Slide 53 text

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Cloud Workflows Orchestrate serverless tasks. Fully managed. Use cases: ● Process events ● Chaining API calls ● Automate infra management ● Implement retry policies GCP Project Start Every day at 6pm Get list of all dev VMs End For each VM: Is it running ? Extract status Stop VM Email owner

Slide 56

Slide 56 text

Min instances Keep a number of instances warm. Use to avoid cold starts. Lower instance price when not in use. Container Instances min = 2 0

Slide 57

Slide 57 text

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Server-side streaming Stream HTTP or gRPC responses. Responses no more buffered Responses no more limited to 32MB $ cat main.go … http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "video/mp4") w.Header().Set("Transfer-Encoding", "chunked") f, _ := os.Open("videos/demo.mp4") io.Copy(w, f) }) port := os.Getenv("PORT") log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil)) (coming soon)

Slide 60

Slide 60 text

Deep dive: GCLB

Slide 61

Slide 61 text

Multi-region load balancing via External HTTP(S) Load Balancing Expose a global endpoint that routes requests to the closest service. Be resilient to regional outages GCP Project Cloud Load Balancing Cloud Run europe-west1 Cloud Run us-central1 Cloud Run asia-northeast1 San Francisco Paris https://example.com https://example.com

Slide 62

Slide 62 text

GCP Project Cloud CDN via External HTTP(S) Load Balancing Reduce the load on your service and improve its performance by returning Cache-Control headers Cloud Run Cache-Control: public, max-age=30, s-maxage=300 1 2 Cloud CDN Cloud Load Balancing

Slide 63

Slide 63 text

GCP Project Identity Aware Proxy via External HTTP(S) Load Balancing Create internal web apps: ● automatic login screen ● grant access to users in your org (coming soon) Cloud Run Identity Aware Proxy Cloud Load Balancing ✓ x

Slide 64

Slide 64 text

Cloud Armor via External HTTP(S) Load Balancing DDoS defense Filter traffic by: ● IP address ● geography Firewall rules Cloud Run Cloud Armor ✓ x Cloud Load Balancing ip-ranges: "198.51.100.0/24" action: "allow" 198.51.100.1 192.0.2.0 GCP Project

Slide 65

Slide 65 text

Deep dive: IAM invoker

Slide 66

Slide 66 text

GCP Invoker permissions Service IAM Requests Auth check: "allUsers" "user:[email protected]" "serviceAccount:..."

Slide 67

Slide 67 text

No authentication required Public service Frontend IAM: role: "roles/run.invoker" member: "allUsers"

Slide 68

Slide 68 text

Leverage "Invoker" IAM role and service identity. Private service to service Frontend Backend IAM: role: "roles/run.invoker" member: "serviceAccount:frontend@..." header:"Authorization: Bearer ID_TOKEN"

Slide 69

Slide 69 text

Push Events with Pub/Sub Pub/Sub push to Cloud Run URL with authentication token. Leverage "Invoker" IAM role to authorize push. No need to validate URL. Cloud Run Service Cloud Pub/Sub IAM: role: "roles/run.invoker" member: "serviceAccount:pubsub@..." gcloud alpha pubsub subscriptions create my-sub --topic my-topic --push-endpoint=https://service.run.app --push-auth-service-account=pubsub@...

Slide 70

Slide 70 text

Async tasks Cloud Tasks HTTP targets (Beta soon) push to Cloud Run URL with authentication token Leverage "Invoker" IAM role. Service Cloud Tasks IAM: role: "roles/run.invoker" member: "serviceAccount:tasks@..." HTTP target

Slide 71

Slide 71 text

Scheduled services Cloud Scheduler with authentication token Leverage "Invoker" IAM role. Service Cloud Scheduler IAM: role: "roles/run.invoker" member: "serviceAccount:scheduler@..."

Slide 72

Slide 72 text

Deep dive: Concurrency

Slide 73

Slide 73 text

Concurrency Each Service is autoscaled to many container instances. Concurrency = "maximum number of requests that can be sent at the same time to a given container instance" AWS Lambda or Google Cloud Functions: only one request at a time to each instance, "concurrency = 1". With Cloud Run: set concurrency value from 1 to 250 (default: 80) → optimized resource consumption → optimized costs concurrency = 1 concurrency = 80

Slide 74

Slide 74 text

400 Clients Targeting 3 Req / S Concurrency = 1

Slide 75

Slide 75 text

400 Clients Targeting 3 Req / S Concurrency = 80

Slide 76

Slide 76 text

Deep dive: Choosing a serverless product

Slide 77

Slide 77 text

Proprietary + Confidential Google Cloud Serverless Compute Product Portfolio App Engine Cloud Run Cloud Functions Event-driven Functions-as-a-Service Run containers on a fully managed environment Run source-based web applications on a fully managed environment

Slide 78

Slide 78 text

Proprietary + Confidential Serverless Use Cases App Engine Cloud Run Cloud Functions Build a web app Web app Request-based services Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Best use cases for Serverless compute products More at https://cloud.run

Slide 79

Slide 79 text

Docs: https://cloud.run PM hotline: [email protected] Help