Going Serverless with Artifactory and Containers on Cloud Run — SwampUp 2021

On Google Cloud, modern workloads are running containers, in a serverless fashion. With Cloud Run, you can easily deploy your workloads, and have them scale up and down transparently to accommodate traffic spikes as well as low demand. In this session, we’ll introduce you to the serverless container world of Cloud Run, how to build them with Cloud Build, and we’ll see how we can take advantage of JFrog Artifactory for hosting our container artifacts. In a series of live demos, we will demonstrate different Cloud Run use cases. Legacy Monolith, Machine Learning API, Vault, Microservices or Event Based Architecture — Google Cloud Run and JFrog will get you covered!

Transcript

1. Going Serverless with Artifactory and Containers on Cloud Run Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, Hybrid Cloud Specialist @ Google @glaforge @archyufa
2. Proprietary + Confidential
3. Proprietary + Confidential The serverless models and characteristics Operational Model Programming Model No Infra Management Managed Security Pay only for usage Service-based Event-driven Stateless
4. Proprietary + Confidential FaaS App platform Container Virtual machine The serverless spectrum Dedicated server Physical hardware Virtual hardware Operation system Application runtime Application Function Virtual hardware Operation system Application runtime Application Function Operation system Application runtime Application Function Application Function Function Serverless Zone
5. Proprietary + Confidential Develop and deploy highly scalable containerized applications on a fully managed serverless platform • Write code your way using your favorite languages (Go, Python, Java, Ruby, Node.js, and more) • Abstract away all infrastructure management for a simple developer experience • Built upon an open standard Knative, enabling the portability of your applications Cloud Run
6. Proprietary + Confidential Serverless containers with Knative and Cloud Run Cloud Run Fully managed, deploy your workloads and don’t see the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative.
7. Proprietary + Confidential Containers Any language Rich ecosystem Any library of base images Any binary .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1 Containers Flexibility Serverless Velocity
8. Proprietary + Confidential Container contract... and resources • Listen on 0.0.0.0 on port $PORT (default 8080) • HTTP server must start < 4 min (timeout → 504) • Request time < 15 min (default → 5 min, up to 60 min) • Stateless (in-memory file system) • Computation only within request (No background activity) • 1 vCPU per container instance (configurable to 4 vCPU) • 256 MiB of memory up to a max of 8 GiB (configurable, soon 16 GiB) • 80 concurrent requests per container (configurable 1-80, soon 250) • 100 max containers by default (configurable 1-1000, support overridable) • Sandboxed by gVisor
9. Proprietary + Confidential Pay per use CPU / Memory / Requests 100ms
10. Proprietary + Confidential Billable time Instance Billable Time Request 1 Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable
11. Proprietary + Confidential Concurrency model concurrency = 1 concurrency = 80
12. What else?
13. Cloud Run is available Planned Future GCP region
14. Gradual rollouts & Rollbacks Specify % traffic between revisions Blue / Green deployments Get URLs for specific revisions # Gradual rollout $ gcloud beta run deploy myservice \ --image gcr.io/project/image:f5bd774 \ --no-traffic \ --tag green $ gcloud beta run services update-traffic myservice \ --to-tags green=1 $ gcloud beta run services update-traffic myservice \ --to-tags green=10 $ gcloud beta run services update-traffic myservice \ --to-tags green=50 $ gcloud beta run services update-traffic myservice \ --to-tags green=100 # Rollback $ gcloud run services update-traffic myservice --to-revisions my-service-0002-joy=100 $ curl https://green---myservice-12345-us.a.run.app
15. GCP Project VPC Access Connect to Cloud Memorystore Redis and Memcached Connect to private IPs Shared VPC Cloud Run Serverless VPC Connector Cloud Memorystore VM Compute Engine Virtual Private Cloud Private IP Private IP
16. Cloud Workflows Orchestrate serverless tasks Fully managed. Use cases: • Process events • Chaining API calls • Automate infra management • Implement retry policies GCP Project Start Every day at 6pm Get list of all dev VMs End For each VM: Is it running ? Extract status Stop VM Email owner
17. Min instances Keep a number of instances warm Use to avoid cold starts Lower instance price when not in use Container Instances min = 2 0
18. Graceful instance termination Receive a SIGTERM signal before container instance is terminated. If handled, CPU is allocated for 10s max $ cat index.js const process = require('process'); process.on('SIGTERM', () => { console.log('Container is shutting down...'); // TODO: Close database connections // TODO: Send any buffered telemetry data }); ...
19. gRPC support $ cat main.go … func main() { port := os.Getenv("PORT") listener, err := net.Listen("tcp", ":"+port) grpcServer := grpc.NewServer() pb.RegisterPingServiceServer(grpcServer, &pingService{}) if err = grpcServer.Serve(listener); err != nil { log.Fatal(err) } }
20. Server-side streaming Server-Sent Events, WebSockets. Stream HTTP or gRPC responses Responses no more buffered and limited to 32MB $ cat main.go … http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "video/mp4") w.Header().Set("Transfer-Encoding", "chunked") f, _ := os.Open("videos/demo.mp4") io.Copy(w, f) }) port := os.Getenv("PORT") log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil)) (coming soon)
21. DEMO TIME!
22. Proprietary + Confidential Cloud Run Use cases More at https://cloud.run Serverless Use Cases Cloud Run Build a web or mobile app HTTP web app Bidirectional streaming WebSockets & gRPC Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Event driven reactive automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Stateful Workloads ? Healthcare Gaming Retail Banking
23. Proprietary + Confidential Attached clusters Anthos on Azure Anthos on AWS Google Anthos and Jfrog Partnership
24. Cloud Run with Jfrog Demo Stack JFrog Cloud Platform on GCP - Jfrog Pipelines - Jfrog Docker Artifactory - Jfrog Xray Pull request Trigger Docker Repository CI CD Update Image tag Cloud Run Fully Managed Cloud Run for Anthos Thanks @jenn_viau for helping to build this amazing demo! Code Quality Build Image Scan Image
25. Online Store application
26. Amazing Cloud Run Links • Slides - https://speakerdeck.com/cncfcanada • Awesome Cloud Run - github.com/steren/awesome-cloudrun • Cloud Run FAQ - github.com/ahmetb/cloud-run-faq • Vault on Cloud Run - gh/kelseyhightower/serverless-vault-with-cloud-run Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, Hybrid Cloud Specialist @ Google @glaforge @archyufa
27. THANK YOU! Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, GDG, GDE Hybrid Cloud Specialist @ Google @glaforge @archyufa
30. STEREN’S CLOUD RUN DECK
31. Cloud Run 2021-04
32. Intro
33. 33 Proprietary + Confidential Application Development Performance Monitoring Serverless Physical Servers Network Hardware Physical Security Internet Connectivity Scaling VM Provisioning Ops & Security Management The Responsibility Pyramid Managed by customer Fully Managed by Google
34. 34 Risk Simplicity Velocity Elasticity Key values of developing, deploying and scaling apps in a fully managed environment
35. 35 Serverless Compute Deploy and scale applications fast and securely in a fully managed environment No Infra Management Speed to Market Auto-scaling
36. “We can’t be locked in.” “How can we use existing binaries?” “Why do I have to choose between containers and serverless?” “Can you support language ____ ?” But… customers ask us:
37. Containers • Any Language • Any Library • Any Binary • Ecosystem of base images .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1
38. Proprietary + Confidential Cloud Run Deploy in seconds Automatic HTTPS, Custom domains Any language, any library Portability No cluster management Run containers on a fully managed environment
39. Deploy in seconds Demo
40. Proprietary + Confidential Build a web app Web app Request-based services Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Use cases More at https://cloud.run
41. Get Started Setup Compute CICD Run Other Create Project Setup Compute Core Infra Load Balancing Service Mesh Specify Volumes NAT Shared VPC Workload YAML Auto-Scale YAML Cloud Build YAML Deploy Time Test Image Rep. Source Version Control Run 8x faster time to market with Cloud Run! Shared VPC Create Project Deploy Time & Test Source Version Control Image Rep Cloud Build YAML Run
42. Proprietary + Confidential Engineers 3 Days 2 Containerized app 1 5.5M Requests per day 700ms Latency €500 a month 40% cost reduction and 80% less effort
43. News and Roadmap
44. Cloud Run is available Planned Future GCP region
45. Delight developers • Deploy from YAML / Export to YAML • Easy Continuous Deployment set up from Git • Develop and run locally • Graceful instance termination • Trigger from 60+ GCP sources (Eventarc) • Cloud Workflows Expand addressable workloads • Min instances • Up to 16GB RAM • Up to 4 CPU • bidirectional gRPC streaming Preview • websockets Preview • HTTP/2 Preview • Long running instances (1hr) What's new? Enterprise Class • VPC connectors: Shared VPC support • Ingress = internal • Egress controls • VPC-SC • Secret Manager integration • Customer Managed Encryption Keys • Binary Authorization • API Gateway support • Identity Aware Proxy support • Cloud Armor support • Committed Use Discounts
46. Keep delighting developers Enterprise-class Expand addressable workloads • Access and Transparency • Liveness and Readiness checks • Always on CPU • Autoscaling improvements • Deterministic URLs, Service Discovery and Service Mesh ✓ Current Current 🠻 🠻 🠻 🠻 • Local development experience (with Cloud Code) • Trigger with events (with Eventarc) • Recommendations and insights (with Recommendation Hub) • Better observability: new metrics, platform errors, tracing (with Cloud Ops) • Infrastructure as code via Config Connector 1. Expand the Cloud Run Service: • Instances: filesystem access, more CPU, more RAM… • Spec: multiple container 2. New resource types: Jobs, non-HTTP services...
47. Docs: https://cloud.run PM hotline: steren@google.com Q&A Help
48. Deep dives
49. Deep dive: Pricing
50. Cloud Run (fully managed): Pay-per-use CPU / Memory / Requests 100ms
51. Billable time Instance Billable Time Request 1 Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable
52. Deep dive: Advanced Features
53. Gradual rollouts & Rollbacks Specify % traffic between revisions Blue / Green deployments Get URLs for specific revisions # Gradual rollout $ gcloud beta run deploy myservice \ --image gcr.io/project/image:f5bd774 \ --no-traffic \ --tag green $ gcloud beta run services update-traffic myservice \ --to-tags green=1 $ gcloud beta run services update-traffic myservice \ --to-tags green=10 $ gcloud beta run services update-traffic myservice \ --to-tags green=50 $ gcloud beta run services update-traffic myservice \ --to-tags green=100 # Rollback $ gcloud run services update-traffic myservice --to-revisions my-service-0002-joy=100 $ curl https://green---myservice-12345-us.a.run.app
54. GCP Project VPC Access Connect to Cloud Memorystore Redis and Memcached Connect to private IPs Shared VPC Cloud Run Serverless VPC Connector Cloud Memorystore VM Compute Engine Virtual Private Cloud Private IP Private IP
55. Cloud Workflows Orchestrate serverless tasks. Fully managed. Use cases: • Process events • Chaining API calls • Automate infra management • Implement retry policies GCP Project Start Every day at 6pm Get list of all dev VMs End For each VM: Is it running ? Extract status Stop VM Email owner
56. Min instances Keep a number of instances warm. Use to avoid cold starts. Lower instance price when not in use. Container Instances min = 2 0
57. Graceful instance termination Receive a SIGTERM signal before container instance is terminated. If handled, CPU is allocated for 10s max $ cat index.js const process = require('process'); process.on('SIGTERM', () => { console.log('Container is shutting down...'); // TODO: Close database connections // TODO: Send any buffered telemetry data }); ...
58. gRPC support $ cat main.go … func main() { port := os.Getenv("PORT") listener, err := net.Listen("tcp", ":"+port) grpcServer := grpc.NewServer() pb.RegisterPingServiceServer(grpcServer, &pingService{}) if err = grpcServer.Serve(listener); err != nil { log.Fatal(err) } }
59. Server-side streaming Stream HTTP or gRPC responses. Responses no more buffered Responses no more limited to 32MB $ cat main.go … http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "video/mp4") w.Header().Set("Transfer-Encoding", "chunked") f, _ := os.Open("videos/demo.mp4") io.Copy(w, f) }) port := os.Getenv("PORT") log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil)) (coming soon)
60. Deep dive: GCLB
61. Multi-region load balancing via External HTTP(S) Load Balancing Expose a global endpoint that routes requests to the closest service. Be resilient to regional outages GCP Project Cloud Load Balancing Cloud Run europe-west1 Cloud Run us-central1 Cloud Run asia-northeast1 San Francisco Paris https://example.com https://example.com
62. GCP Project Cloud CDN via External HTTP(S) Load Balancing Reduce the load on your service and improve its performance by returning Cache-Control headers Cloud Run Cache-Control: public, max-age=30, s-maxage=300 1 2 Cloud CDN Cloud Load Balancing
63. GCP Project Identity Aware Proxy via External HTTP(S) Load Balancing Create internal web apps: • automatic login screen • grant access to users in your org (coming soon) Cloud Run Identity Aware Proxy Cloud Load Balancing ✓ x
64. Cloud Armor via External HTTP(S) Load Balancing DDoS defense Filter traffic by: • IP address • geography Firewall rules Cloud Run Cloud Armor ✓ x Cloud Load Balancing ip-ranges: "198.51.100.0/24" action: "allow" 198.51.100.1 192.0.2.0 GCP Project
65. Deep dive: IAM invoker
66. GCP Invoker permissions Service IAM Requests Auth check: "allUsers" "user:mail@domain.com" "serviceAccount:..."
67. No authentication required Public service Frontend IAM: role: "roles/run.invoker" member: "allUsers"
68. Leverage "Invoker" IAM role and service identity. Private service to service Frontend Backend IAM: role: "roles/run.invoker" member: "serviceAccount:frontend@..." header:"Authorization: Bearer ID_TOKEN"
69. Push Events with Pub/Sub Pub/Sub push to Cloud Run URL with authentication token. Leverage "Invoker" IAM role to authorize push. No need to validate URL. Cloud Run Service Cloud Pub/Sub IAM: role: "roles/run.invoker" member: "serviceAccount:pubsub@..." gcloud alpha pubsub subscriptions create my-sub --topic my-topic --push-endpoint=https://service.run.app --push-auth-service-account=pubsub@...
70. Async tasks Cloud Tasks HTTP targets (Beta soon) push to Cloud Run URL with authentication token Leverage "Invoker" IAM role. Service Cloud Tasks IAM: role: "roles/run.invoker" member: "serviceAccount:tasks@..." HTTP target
71. Scheduled services Cloud Scheduler with authentication token Leverage "Invoker" IAM role. Service Cloud Scheduler IAM: role: "roles/run.invoker" member: "serviceAccount:scheduler@..."
72. Deep dive: Concurrency
73. Concurrency Each Service is autoscaled to many container instances. Concurrency = "maximum number of requests that can be sent at the same time to a given container instance" AWS Lambda or Google Cloud Functions: only one request at a time to each instance, "concurrency = 1". With Cloud Run: set concurrency value from 1 to 250 (default: 80) → optimized resource consumption → optimized costs concurrency = 1 concurrency = 80
74. 400 Clients Targeting 3 Req / S Concurrency = 1
75. 400 Clients Targeting 3 Req / S Concurrency = 80
76. Deep dive: Choosing a serverless product
77. Proprietary + Confidential Google Cloud Serverless Compute Product Portfolio App Engine Cloud Run Cloud Functions Event-driven Functions-as-a-Service Run containers on a fully managed environment Run source-based web applications on a fully managed environment
78. Proprietary + Confidential Serverless Use Cases App Engine Cloud Run Cloud Functions Build a web app Web app Request-based services Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Best use cases for Serverless compute products More at https://cloud.run
79. Docs: https://cloud.run PM hotline: steren@google.com Help