Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Going Serverless with Artifactory and Containers on Cloud Run — SwampUp 2021

Going Serverless with Artifactory and Containers on Cloud Run — SwampUp 2021

On Google Cloud, modern workloads are running containers, in a serverless fashion. With Cloud Run, you can easily deploy your workloads, and have them scale up and down transparently to accommodate traffic spikes as well as low demand. In this session, we’ll introduce you to the serverless container world of Cloud Run, how to build them with Cloud Build, and we’ll see how we can take advantage of JFrog Artifactory for hosting our container artifacts. In a series of live demos, we will demonstrate different Cloud Run use cases. Legacy Monolith, Machine Learning API, Vault, Microservices or Event Based Architecture — Google Cloud Run and JFrog will get you covered!

137d3908243acfc30e126615d59d4e6d?s=128

Guillaume Laforge

June 02, 2021
Tweet

Transcript

  1. Going Serverless with Artifactory and Containers on Cloud Run Guillaume

    Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, Hybrid Cloud Specialist @ Google @glaforge @archyufa
  2. Proprietary + Confidential

  3. Proprietary + Confidential The serverless models and characteristics Operational Model

    Programming Model No Infra Management Managed Security Pay only for usage Service-based Event-driven Stateless
  4. Proprietary + Confidential FaaS App platform Container Virtual machine The

    serverless spectrum Dedicated server Physical hardware Virtual hardware Operation system Application runtime Application Function Virtual hardware Operation system Application runtime Application Function Operation system Application runtime Application Function Application Function Function Serverless Zone
  5. Proprietary + Confidential Develop and deploy highly scalable containerized applications

    on a fully managed serverless platform • Write code your way using your favorite languages (Go, Python, Java, Ruby, Node.js, and more) • Abstract away all infrastructure management for a simple developer experience • Built upon an open standard Knative, enabling the portability of your applications Cloud Run
  6. Proprietary + Confidential Serverless containers with Knative and Cloud Run

    Cloud Run Fully managed, deploy your workloads and don’t see the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative.
  7. Proprietary + Confidential Containers Any language Rich ecosystem Any library

    of base images Any binary .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1 Containers Flexibility Serverless Velocity
  8. Proprietary + Confidential Container contract... and resources • Listen on

    0.0.0.0 on port $PORT (default 8080) • HTTP server must start < 4 min (timeout → 504) • Request time < 15 min (default → 5 min, up to 60 min) • Stateless (in-memory file system) • Computation only within request (No background activity) • 1 vCPU per container instance (configurable to 4 vCPU) • 256 MiB of memory up to a max of 8 GiB (configurable, soon 16 GiB) • 80 concurrent requests per container (configurable 1-80, soon 250) • 100 max containers by default (configurable 1-1000, support overridable) • Sandboxed by gVisor
  9. Proprietary + Confidential Pay per use CPU / Memory /

    Requests 100ms
  10. Proprietary + Confidential Billable time Instance Billable Time Request 1

    Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable
  11. Proprietary + Confidential Concurrency model concurrency = 1 concurrency =

    80
  12. What else?

  13. Cloud Run is available Planned Future GCP region

  14. Gradual rollouts & Rollbacks Specify % traffic between revisions Blue

    / Green deployments Get URLs for specific revisions # Gradual rollout $ gcloud beta run deploy myservice \ --image gcr.io/project/image:f5bd774 \ --no-traffic \ --tag green $ gcloud beta run services update-traffic myservice \ --to-tags green=1 $ gcloud beta run services update-traffic myservice \ --to-tags green=10 $ gcloud beta run services update-traffic myservice \ --to-tags green=50 $ gcloud beta run services update-traffic myservice \ --to-tags green=100 # Rollback $ gcloud run services update-traffic myservice --to-revisions my-service-0002-joy=100 $ curl https://green---myservice-12345-us.a.run.app
  15. GCP Project VPC Access Connect to Cloud Memorystore Redis and

    Memcached Connect to private IPs Shared VPC Cloud Run Serverless VPC Connector Cloud Memorystore VM Compute Engine Virtual Private Cloud Private IP Private IP
  16. Cloud Workflows Orchestrate serverless tasks Fully managed. Use cases: •

    Process events • Chaining API calls • Automate infra management • Implement retry policies GCP Project Start Every day at 6pm Get list of all dev VMs End For each VM: Is it running ? Extract status Stop VM Email owner
  17. Min instances Keep a number of instances warm Use to

    avoid cold starts Lower instance price when not in use Container Instances min = 2 0
  18. Graceful instance termination Receive a SIGTERM signal before container instance

    is terminated. If handled, CPU is allocated for 10s max $ cat index.js const process = require('process'); process.on('SIGTERM', () => { console.log('Container is shutting down...'); // TODO: Close database connections // TODO: Send any buffered telemetry data }); ...
  19. gRPC support $ cat main.go … func main() { port

    := os.Getenv("PORT") listener, err := net.Listen("tcp", ":"+port) grpcServer := grpc.NewServer() pb.RegisterPingServiceServer(grpcServer, &pingService{}) if err = grpcServer.Serve(listener); err != nil { log.Fatal(err) } }
  20. Server-side streaming Server-Sent Events, WebSockets. Stream HTTP or gRPC responses

    Responses no more buffered and limited to 32MB $ cat main.go … http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "video/mp4") w.Header().Set("Transfer-Encoding", "chunked") f, _ := os.Open("videos/demo.mp4") io.Copy(w, f) }) port := os.Getenv("PORT") log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil)) (coming soon)
  21. DEMO TIME!

  22. Proprietary + Confidential Cloud Run Use cases More at https://cloud.run

    Serverless Use Cases Cloud Run Build a web or mobile app HTTP web app Bidirectional streaming WebSockets & gRPC Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Event driven reactive automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Stateful Workloads ? Healthcare Gaming Retail Banking
  23. Proprietary + Confidential Attached clusters Anthos on Azure Anthos on

    AWS Google Anthos and Jfrog Partnership
  24. Cloud Run with Jfrog Demo Stack JFrog Cloud Platform on

    GCP - Jfrog Pipelines - Jfrog Docker Artifactory - Jfrog Xray Pull request Trigger Docker Repository CI CD Update Image tag Cloud Run Fully Managed Cloud Run for Anthos Thanks @jenn_viau for helping to build this amazing demo! Code Quality Build Image Scan Image
  25. Online Store application

  26. Amazing Cloud Run Links • Slides - https://speakerdeck.com/cncfcanada • Awesome

    Cloud Run - github.com/steren/awesome-cloudrun • Cloud Run FAQ - github.com/ahmetb/cloud-run-faq • Vault on Cloud Run - gh/kelseyhightower/serverless-vault-with-cloud-run Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov CNCF Ambassador, Hybrid Cloud Specialist @ Google @glaforge @archyufa
  27. THANK YOU! Guillaume Laforge Developer Advocate @ Google Ayrat Khayretdinov

    CNCF Ambassador, GDG, GDE Hybrid Cloud Specialist @ Google @glaforge @archyufa
  28. None
  29. None
  30. STEREN’S CLOUD RUN DECK

  31. Cloud Run 2021-04

  32. Intro

  33. 33 Proprietary + Confidential Application Development Performance Monitoring Serverless Physical

    Servers Network Hardware Physical Security Internet Connectivity Scaling VM Provisioning Ops & Security Management The Responsibility Pyramid Managed by customer Fully Managed by Google
  34. 34 Risk Simplicity Velocity Elasticity Key values of developing, deploying

    and scaling apps in a fully managed environment
  35. 35 Serverless Compute Deploy and scale applications fast and securely

    in a fully managed environment No Infra Management Speed to Market Auto-scaling
  36. “We can’t be locked in.” “How can we use existing

    binaries?” “Why do I have to choose between containers and serverless?” “Can you support language ____ ?” But… customers ask us:
  37. Containers • Any Language • Any Library • Any Binary

    • Ecosystem of base images .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1
  38. Proprietary + Confidential Cloud Run Deploy in seconds Automatic HTTPS,

    Custom domains Any language, any library Portability No cluster management Run containers on a fully managed environment
  39. Deploy in seconds Demo

  40. Proprietary + Confidential Build a web app Web app Request-based

    services Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Use cases More at https://cloud.run
  41. Get Started Setup Compute CICD Run Other Create Project Setup

    Compute Core Infra Load Balancing Service Mesh Specify Volumes NAT Shared VPC Workload YAML Auto-Scale YAML Cloud Build YAML Deploy Time Test Image Rep. Source Version Control Run 8x faster time to market with Cloud Run! Shared VPC Create Project Deploy Time & Test Source Version Control Image Rep Cloud Build YAML Run
  42. Proprietary + Confidential Engineers 3 Days 2 Containerized app 1

    5.5M Requests per day 700ms Latency €500 a month 40% cost reduction and 80% less effort
  43. News and Roadmap

  44. Cloud Run is available Planned Future GCP region

  45. Delight developers • Deploy from YAML / Export to YAML

    • Easy Continuous Deployment set up from Git • Develop and run locally • Graceful instance termination • Trigger from 60+ GCP sources (Eventarc) • Cloud Workflows Expand addressable workloads • Min instances • Up to 16GB RAM • Up to 4 CPU • bidirectional gRPC streaming Preview • websockets Preview • HTTP/2 Preview • Long running instances (1hr) What's new? Enterprise Class • VPC connectors: Shared VPC support • Ingress = internal • Egress controls • VPC-SC • Secret Manager integration • Customer Managed Encryption Keys • Binary Authorization • API Gateway support • Identity Aware Proxy support • Cloud Armor support • Committed Use Discounts
  46. Keep delighting developers Enterprise-class Expand addressable workloads • Access and

    Transparency • Liveness and Readiness checks • Always on CPU • Autoscaling improvements • Deterministic URLs, Service Discovery and Service Mesh ✓ Current Current 🠻 🠻 🠻 🠻 • Local development experience (with Cloud Code) • Trigger with events (with Eventarc) • Recommendations and insights (with Recommendation Hub) • Better observability: new metrics, platform errors, tracing (with Cloud Ops) • Infrastructure as code via Config Connector 1. Expand the Cloud Run Service: • Instances: filesystem access, more CPU, more RAM… • Spec: multiple container 2. New resource types: Jobs, non-HTTP services...
  47. Docs: https://cloud.run PM hotline: steren@google.com Q&A Help

  48. Deep dives

  49. Deep dive: Pricing

  50. Cloud Run (fully managed): Pay-per-use CPU / Memory / Requests

    100ms
  51. Billable time Instance Billable Time Request 1 Start Request 1

    End Request 2 Start Request 2 End Instance Time Billable Non-billable
  52. Deep dive: Advanced Features

  53. Gradual rollouts & Rollbacks Specify % traffic between revisions Blue

    / Green deployments Get URLs for specific revisions # Gradual rollout $ gcloud beta run deploy myservice \ --image gcr.io/project/image:f5bd774 \ --no-traffic \ --tag green $ gcloud beta run services update-traffic myservice \ --to-tags green=1 $ gcloud beta run services update-traffic myservice \ --to-tags green=10 $ gcloud beta run services update-traffic myservice \ --to-tags green=50 $ gcloud beta run services update-traffic myservice \ --to-tags green=100 # Rollback $ gcloud run services update-traffic myservice --to-revisions my-service-0002-joy=100 $ curl https://green---myservice-12345-us.a.run.app
  54. GCP Project VPC Access Connect to Cloud Memorystore Redis and

    Memcached Connect to private IPs Shared VPC Cloud Run Serverless VPC Connector Cloud Memorystore VM Compute Engine Virtual Private Cloud Private IP Private IP
  55. Cloud Workflows Orchestrate serverless tasks. Fully managed. Use cases: •

    Process events • Chaining API calls • Automate infra management • Implement retry policies GCP Project Start Every day at 6pm Get list of all dev VMs End For each VM: Is it running ? Extract status Stop VM Email owner
  56. Min instances Keep a number of instances warm. Use to

    avoid cold starts. Lower instance price when not in use. Container Instances min = 2 0
  57. Graceful instance termination Receive a SIGTERM signal before container instance

    is terminated. If handled, CPU is allocated for 10s max $ cat index.js const process = require('process'); process.on('SIGTERM', () => { console.log('Container is shutting down...'); // TODO: Close database connections // TODO: Send any buffered telemetry data }); ...
  58. gRPC support $ cat main.go … func main() { port

    := os.Getenv("PORT") listener, err := net.Listen("tcp", ":"+port) grpcServer := grpc.NewServer() pb.RegisterPingServiceServer(grpcServer, &pingService{}) if err = grpcServer.Serve(listener); err != nil { log.Fatal(err) } }
  59. Server-side streaming Stream HTTP or gRPC responses. Responses no more

    buffered Responses no more limited to 32MB $ cat main.go … http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "video/mp4") w.Header().Set("Transfer-Encoding", "chunked") f, _ := os.Open("videos/demo.mp4") io.Copy(w, f) }) port := os.Getenv("PORT") log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil)) (coming soon)
  60. Deep dive: GCLB

  61. Multi-region load balancing via External HTTP(S) Load Balancing Expose a

    global endpoint that routes requests to the closest service. Be resilient to regional outages GCP Project Cloud Load Balancing Cloud Run europe-west1 Cloud Run us-central1 Cloud Run asia-northeast1 San Francisco Paris https://example.com https://example.com
  62. GCP Project Cloud CDN via External HTTP(S) Load Balancing Reduce

    the load on your service and improve its performance by returning Cache-Control headers Cloud Run Cache-Control: public, max-age=30, s-maxage=300 1 2 Cloud CDN Cloud Load Balancing
  63. GCP Project Identity Aware Proxy via External HTTP(S) Load Balancing

    Create internal web apps: • automatic login screen • grant access to users in your org (coming soon) Cloud Run Identity Aware Proxy Cloud Load Balancing ✓ x
  64. Cloud Armor via External HTTP(S) Load Balancing DDoS defense Filter

    traffic by: • IP address • geography Firewall rules Cloud Run Cloud Armor ✓ x Cloud Load Balancing ip-ranges: "198.51.100.0/24" action: "allow" 198.51.100.1 192.0.2.0 GCP Project
  65. Deep dive: IAM invoker

  66. GCP Invoker permissions Service IAM Requests Auth check: "allUsers" "user:mail@domain.com"

    "serviceAccount:..."
  67. No authentication required Public service Frontend IAM: role: "roles/run.invoker" member:

    "allUsers"
  68. Leverage "Invoker" IAM role and service identity. Private service to

    service Frontend Backend IAM: role: "roles/run.invoker" member: "serviceAccount:frontend@..." header:"Authorization: Bearer ID_TOKEN"
  69. Push Events with Pub/Sub Pub/Sub push to Cloud Run URL

    with authentication token. Leverage "Invoker" IAM role to authorize push. No need to validate URL. Cloud Run Service Cloud Pub/Sub IAM: role: "roles/run.invoker" member: "serviceAccount:pubsub@..." gcloud alpha pubsub subscriptions create my-sub --topic my-topic --push-endpoint=https://service.run.app --push-auth-service-account=pubsub@...
  70. Async tasks Cloud Tasks HTTP targets (Beta soon) push to

    Cloud Run URL with authentication token Leverage "Invoker" IAM role. Service Cloud Tasks IAM: role: "roles/run.invoker" member: "serviceAccount:tasks@..." HTTP target
  71. Scheduled services Cloud Scheduler with authentication token Leverage "Invoker" IAM

    role. Service Cloud Scheduler IAM: role: "roles/run.invoker" member: "serviceAccount:scheduler@..."
  72. Deep dive: Concurrency

  73. Concurrency Each Service is autoscaled to many container instances. Concurrency

    = "maximum number of requests that can be sent at the same time to a given container instance" AWS Lambda or Google Cloud Functions: only one request at a time to each instance, "concurrency = 1". With Cloud Run: set concurrency value from 1 to 250 (default: 80) → optimized resource consumption → optimized costs concurrency = 1 concurrency = 80
  74. 400 Clients Targeting 3 Req / S Concurrency = 1

  75. 400 Clients Targeting 3 Req / S Concurrency = 80

  76. Deep dive: Choosing a serverless product

  77. Proprietary + Confidential Google Cloud Serverless Compute Product Portfolio App

    Engine Cloud Run Cloud Functions Event-driven Functions-as-a-Service Run containers on a fully managed environment Run source-based web applications on a fully managed environment
  78. Proprietary + Confidential Serverless Use Cases App Engine Cloud Run

    Cloud Functions Build a web app Web app Request-based services Developing APIs Web & mobile backends Internal APIs and services Data Processing Automation Workflow & Orchestration Event driven reactive automation Connecting Cloud Services Best use cases for Serverless compute products More at https://cloud.run
  79. Docs: https://cloud.run PM hotline: steren@google.com Help