Going Serverless with Artifactory and Containers on Cloud Run — SwampUp 2021

On Google Cloud, modern workloads are running containers, in a serverless fashion. With Cloud Run, you can easily deploy your workloads, and have them scale up and down transparently to accommodate traffic spikes as well as low demand. In this session, we’ll introduce you to the serverless container world of Cloud Run, how to build them with Cloud Build, and we’ll see how we can take advantage of JFrog Artifactory for hosting our container artifacts. In a series of live demos, we will demonstrate different Cloud Run use cases. Legacy Monolith, Machine Learning API, Vault, Microservices or Event Based Architecture — Google Cloud Run and JFrog will get you covered!

Transcript

1. Going Serverless
   with Artifactory
   and Containers
   on Cloud Run
   Guillaume Laforge
   Developer Advocate @ Google
   Ayrat Khayretdinov
   CNCF Ambassador,
   Hybrid Cloud Specialist @ Google
   @glaforge
   @archyufa
   

   View Slide

2. Proprietary + Confidential
   

   View Slide

3. Proprietary + Confidential
   The serverless models and characteristics
   Operational
   Model
   Programming
   Model
   No Infra Management Managed Security Pay only for usage
   Service-based Event-driven Stateless
   

   View Slide

4. Proprietary + Confidential
   FaaS
   App platform
   Container
   Virtual machine
   The serverless spectrum
   Dedicated server
   Physical
   hardware
   Virtual
   hardware
   Operation
   system
   Application
   runtime
   Application
   Function
   Virtual
   hardware
   Operation
   system
   Application
   runtime
   Application
   Function
   Operation
   system
   Application
   runtime
   Application
   Function
   Application
   Function Function
   Serverless Zone
   

   View Slide

5. Proprietary + Confidential
   Develop and deploy highly scalable containerized
   applications on a fully managed serverless platform
   ● Write code your way using your favorite languages
   (Go, Python, Java, Ruby, Node.js, and more)
   ● Abstract away all infrastructure management
   for a simple developer experience
   ● Built upon an open standard Knative,
   enabling the portability of your applications
   Cloud Run
   

   View Slide

6. Proprietary + Confidential
   Serverless containers with Knative and Cloud Run
   Cloud Run
   Fully managed, deploy your
   workloads and don’t see the
   cluster.
   Cloud Run on Anthos
   Deploy into Anthos, run
   serverless side-by-side with
   your existing workloads.
   Knative everywhere
   Use the same APIs and
   tooling anywhere you run
   Kubernetes with Knative.
   

   View Slide

7. Proprietary + Confidential
   Containers
   Any language Rich ecosystem
   Any library of base images
   Any binary
   .js .rb .go
   .py .sh …
   0 1 0
   1 0 0
   1 1 1
   Containers
   Flexibility
   Serverless
   Velocity
   

   View Slide

8. Proprietary + Confidential
   Container contract... and resources
   ● Listen on 0.0.0.0 on port $PORT
   (default 8080)
   ● HTTP server must start < 4 min
   (timeout → 504)
   ● Request time < 15 min
   (default → 5 min, up to 60 min)
   ● Stateless
   (in-memory file system)
   ● Computation only within request
   (No background activity)
   ● 1 vCPU per container instance
   (configurable to 4 vCPU)
   ● 256 MiB of memory up to a max of 8 GiB
   (configurable, soon 16 GiB)
   ● 80 concurrent requests per container
   (configurable 1-80, soon 250)
   ● 100 max containers by default
   (configurable 1-1000, support overridable)
   ● Sandboxed by gVisor
   

   View Slide

9. Proprietary + Confidential
   Pay per use
   CPU / Memory / Requests 100ms
   

   View Slide

10. Proprietary + Confidential
    Billable time
    Instance
    Billable Time
    Request 1 Start Request 1 End
    Request 2 Start Request 2 End
    Instance Time
    Billable
    Non-billable
    

    View Slide

11. Proprietary + Confidential
    Concurrency model
    concurrency = 1
    concurrency = 80
    

    View Slide

12. What
    else?
    

    View Slide

13. Cloud Run is available
    Planned
    Future GCP region
    

    View Slide

14. Gradual
    rollouts &
    Rollbacks
    Specify % traffic
    between revisions
    Blue / Green deployments
    Get URLs for specific
    revisions
    # Gradual rollout
    $ gcloud beta run deploy myservice \
    --image gcr.io/project/image:f5bd774 \
    --no-traffic \
    --tag green
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=1
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=10
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=50
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=100
    # Rollback
    $ gcloud run services update-traffic myservice
    --to-revisions my-service-0002-joy=100
    $ curl https://green---myservice-12345-us.a.run.app
    

    View Slide

15. GCP Project
    VPC
    Access
    Connect to Cloud
    Memorystore Redis and
    Memcached
    Connect to private IPs
    Shared VPC
    Cloud Run
    Serverless
    VPC
    Connector
    Cloud
    Memorystore
    VM
    Compute
    Engine
    Virtual Private Cloud
    Private IP
    Private IP
    

    View Slide

16. Cloud
    Workflows
    Orchestrate serverless tasks
    Fully managed.
    Use cases:
    ● Process events
    ● Chaining API calls
    ● Automate infra management
    ● Implement retry policies
    GCP Project
    Start
    Every day at
    6pm
    Get list
    of all
    dev VMs
    End
    For each VM:
    Is it
    running
    ?
    Extract
    status
    Stop VM
    Email
    owner
    

    View Slide

17. Min
    instances
    Keep a number of
    instances warm
    Use to avoid cold starts
    Lower instance price
    when not in use
    Container Instances
    min = 2
    0
    

    View Slide

18. Graceful
    instance
    termination
    Receive a SIGTERM signal
    before container instance
    is terminated.
    If handled, CPU is
    allocated for 10s max
    $ cat index.js
    const process = require('process');
    process.on('SIGTERM', () => {
    console.log('Container is shutting down...');
    // TODO: Close database connections
    // TODO: Send any buffered telemetry data
    });
    ...
    

    View Slide

19. gRPC
    support
    $ cat main.go
    …
    func main() {
    port := os.Getenv("PORT")
    listener, err := net.Listen("tcp", ":"+port)
    grpcServer := grpc.NewServer()
    pb.RegisterPingServiceServer(grpcServer, &pingService{})
    if err = grpcServer.Serve(listener); err != nil {
    log.Fatal(err)
    }
    }
    

    View Slide

20. Server-side
    streaming
    Server-Sent Events,
    WebSockets.
    Stream HTTP or gRPC
    responses
    Responses no more buffered
    and limited to 32MB
    $ cat main.go
    …
    http.HandleFunc("/", func(w http.ResponseWriter, r
    *http.Request) {
    w.Header().Set("Content-Type", "video/mp4")
    w.Header().Set("Transfer-Encoding", "chunked")
    f, _ := os.Open("videos/demo.mp4")
    io.Copy(w, f)
    })
    port := os.Getenv("PORT")
    log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
    (coming soon)
    

    View Slide

21. DEMO TIME!
    

    View Slide

22. Proprietary + Confidential
    Cloud Run Use cases
    More at https://cloud.run
    Serverless Use Cases Cloud Run
    Build a web or mobile app
    HTTP web app
    Bidirectional streaming
    WebSockets & gRPC
    Developing APIs
    Web & mobile backends
    Internal APIs and services
    Data Processing
    Automation
    Event driven reactive automation
    Workflow & Orchestration
    Event driven reactive automation
    Connecting Cloud Services
    Stateful Workloads ?
    Healthcare Gaming
    Retail Banking
    

    View Slide

23. Proprietary + Confidential
    Attached
    clusters
    Anthos on Azure
    Anthos on AWS
    Google Anthos and Jfrog Partnership
    

    View Slide

24. Cloud Run with Jfrog Demo Stack
    JFrog Cloud Platform on GCP
    - Jfrog Pipelines
    - Jfrog Docker Artifactory
    - Jfrog Xray
    Pull request
    Trigger
    Docker Repository
    CI CD
    Update Image tag
    Cloud Run Fully
    Managed
    Cloud Run for
    Anthos
    Thanks @jenn_viau for helping to build this amazing demo!
    Code Quality
    Build Image
    Scan Image
    

    View Slide

25. Online Store application
    

    View Slide

26. Amazing Cloud Run Links
    ● Slides - https://speakerdeck.com/cncfcanada
    ● Awesome Cloud Run - github.com/steren/awesome-cloudrun
    ● Cloud Run FAQ - github.com/ahmetb/cloud-run-faq
    ● Vault on Cloud Run - gh/kelseyhightower/serverless-vault-with-cloud-run
    Guillaume Laforge
    Developer Advocate @ Google
    Ayrat Khayretdinov
    CNCF Ambassador,
    Hybrid Cloud Specialist @ Google
    @glaforge
    @archyufa
    

    View Slide

27. THANK YOU!
    Guillaume Laforge
    Developer Advocate @ Google
    Ayrat Khayretdinov
    CNCF Ambassador, GDG, GDE
    Hybrid Cloud Specialist @ Google
    @glaforge
    @archyufa
    

    View Slide

28. View Slide

29. View Slide

30. STEREN’S
    CLOUD RUN
    DECK
    

    View Slide

31. Cloud Run
    2021-04
    

    View Slide

32. Intro
    

    View Slide

33. 33
    Proprietary + Confidential
    Application
    Development
    Performance
    Monitoring
    Serverless
    Physical
    Servers
    Network
    Hardware
    Physical
    Security
    Internet
    Connectivity
    Scaling
    VM
    Provisioning
    Ops & Security
    Management
    The Responsibility
    Pyramid
    Managed by
    customer
    Fully Managed
    by Google
    

    View Slide

34. 34
    Risk
    Simplicity
    Velocity
    Elasticity
    Key values of developing, deploying and
    scaling apps in a fully managed environment
    

    View Slide

35. 35
    Serverless
    Compute
    Deploy and scale applications fast and
    securely in a fully managed environment
    No Infra
    Management
    Speed to
    Market
    Auto-scaling
    

    View Slide

36. “We can’t be locked in.”
    “How can we use
    existing binaries?”
    “Why do I have to choose between
    containers and serverless?”
    “Can you support language ____ ?”
    But… customers ask us:
    

    View Slide

37. Containers
    • Any Language
    • Any Library
    • Any Binary
    • Ecosystem of base images
    .js .rb .go
    .py .sh …
    0 1 0
    1 0 0
    1 1 1
    

    View Slide

38. Proprietary + Confidential
    Cloud Run
    Deploy in seconds
    Automatic HTTPS, Custom domains
    Any language, any library
    Portability
    No cluster management
    Run containers on a fully managed environment
    

    View Slide

39. Deploy in
    seconds
    Demo
    

    View Slide

40. Proprietary + Confidential
    Build a web app
    Web app
    Request-based services
    Developing APIs
    Web & mobile backends
    Internal APIs and services
    Data Processing
    Automation
    Workflow & Orchestration
    Event driven reactive automation
    Connecting Cloud Services
    Use cases
    More at https://cloud.run
    

    View Slide

41. Get Started
    Setup Compute
    CICD
    Run
    Other
    Create
    Project
    Setup
    Compute
    Core
    Infra
    Load
    Balancing
    Service
    Mesh
    Specify
    Volumes NAT
    Shared
    VPC
    Workload
    YAML
    Auto-Scale
    YAML
    Cloud Build
    YAML
    Deploy
    Time Test
    Image
    Rep.
    Source Version
    Control
    Run
    8x faster time to market
    with Cloud Run!
    Shared
    VPC
    Create
    Project
    Deploy
    Time &
    Test
    Source
    Version
    Control
    Image
    Rep
    Cloud
    Build
    YAML
    Run
    

    View Slide

42. Proprietary + Confidential
    Engineers
    3
    Days
    2
    Containerized
    app
    1
    5.5M
    Requests per day
    700ms
    Latency
    €500
    a month
    40% cost reduction and 80% less
    effort
    

    View Slide

43. News and Roadmap
    

    View Slide

44. Cloud Run is available
    Planned
    Future GCP region
    

    View Slide

45. Delight developers
    ● Deploy from YAML / Export to YAML
    ● Easy Continuous Deployment set up from Git
    ● Develop and run locally
    ● Graceful instance termination
    ● Trigger from 60+ GCP sources (Eventarc)
    ● Cloud Workflows
    Expand addressable workloads
    ● Min instances
    ● Up to 16GB RAM
    ● Up to 4 CPU
    ● bidirectional gRPC streaming Preview
    ● websockets Preview
    ● HTTP/2 Preview
    ● Long running instances (1hr)
    What's new?
    Enterprise Class
    ● VPC connectors: Shared VPC support
    ● Ingress = internal
    ● Egress controls
    ● VPC-SC
    ● Secret Manager integration
    ● Customer Managed Encryption Keys
    ● Binary Authorization
    ● API Gateway support
    ● Identity Aware Proxy support
    ● Cloud Armor support
    ● Committed Use Discounts
    

    View Slide

46. Keep delighting developers
    Enterprise-class
    Expand addressable workloads
    ● Access and Transparency
    ● Liveness and Readiness checks
    ● Always on CPU
    ● Autoscaling improvements
    ● Deterministic URLs, Service Discovery and Service Mesh
    ✓
    Current
    Current
    🠻
    🠻
    🠻
    🠻
    ● Local development experience (with Cloud Code)
    ● Trigger with events (with Eventarc)
    ● Recommendations and insights (with Recommendation Hub)
    ● Better observability: new metrics, platform errors, tracing (with Cloud Ops)
    ● Infrastructure as code via Config Connector
    1. Expand the Cloud Run Service:
    ● Instances: filesystem access, more CPU, more RAM…
    ● Spec: multiple container
    2. New resource types: Jobs, non-HTTP services...
    

    View Slide

47. Docs: https://cloud.run
    PM hotline: steren@google.com
    Q&A Help
    

    View Slide

48. Deep dives
    

    View Slide

49. Deep dive:
    Pricing
    

    View Slide

50. Cloud Run (fully managed): Pay-per-use
    CPU / Memory / Requests 100ms
    

    View Slide

51. Billable time
    Instance
    Billable Time
    Request 1 Start Request 1 End
    Request 2 Start Request 2 End
    Instance Time
    Billable
    Non-billable
    

    View Slide

52. Deep dive:
    Advanced Features
    

    View Slide

53. Gradual
    rollouts &
    Rollbacks
    Specify % traffic between
    revisions
    Blue / Green deployments
    Get URLs for specific
    revisions
    # Gradual rollout
    $ gcloud beta run deploy myservice \
    --image gcr.io/project/image:f5bd774 \
    --no-traffic \
    --tag green
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=1
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=10
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=50
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=100
    # Rollback
    $ gcloud run services update-traffic myservice
    --to-revisions my-service-0002-joy=100
    $ curl https://green---myservice-12345-us.a.run.app
    

    View Slide

54. GCP Project
    VPC
    Access
    Connect to Cloud
    Memorystore Redis and
    Memcached
    Connect to private IPs
    Shared VPC
    Cloud Run
    Serverless
    VPC
    Connector
    Cloud
    Memorystore
    VM
    Compute
    Engine
    Virtual Private Cloud
    Private IP
    Private IP
    

    View Slide

55. Cloud
    Workflows
    Orchestrate serverless tasks.
    Fully managed.
    Use cases:
    ● Process events
    ● Chaining API calls
    ● Automate infra management
    ● Implement retry policies
    GCP Project
    Start
    Every day at
    6pm
    Get list
    of all
    dev VMs
    End
    For each VM:
    Is it
    running
    ?
    Extract
    status
    Stop VM
    Email
    owner
    

    View Slide

56. Min
    instances
    Keep a number of instances
    warm.
    Use to avoid cold starts.
    Lower instance price when
    not in use.
    Container Instances
    min = 2
    0
    

    View Slide

57. Graceful
    instance
    termination
    Receive a SIGTERM signal
    before
    container instance is
    terminated.
    If handled, CPU is allocated
    for 10s max
    $ cat index.js
    const process = require('process');
    process.on('SIGTERM', () => {
    console.log('Container is shutting down...');
    // TODO: Close database connections
    // TODO: Send any buffered telemetry data
    });
    ...
    

    View Slide

58. gRPC
    support
    $ cat main.go
    …
    func main() {
    port := os.Getenv("PORT")
    listener, err := net.Listen("tcp", ":"+port)
    grpcServer := grpc.NewServer()
    pb.RegisterPingServiceServer(grpcServer, &pingService{})
    if err = grpcServer.Serve(listener); err != nil {
    log.Fatal(err)
    }
    }
    

    View Slide

59. Server-side
    streaming
    Stream HTTP or gRPC
    responses.
    Responses no more buffered
    Responses no more limited
    to 32MB
    $ cat main.go
    …
    http.HandleFunc("/", func(w http.ResponseWriter, r
    *http.Request) {
    w.Header().Set("Content-Type", "video/mp4")
    w.Header().Set("Transfer-Encoding", "chunked")
    f, _ := os.Open("videos/demo.mp4")
    io.Copy(w, f)
    })
    port := os.Getenv("PORT")
    log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
    (coming soon)
    

    View Slide

60. Deep dive:
    GCLB
    

    View Slide

61. Multi-region
    load
    balancing
    via External HTTP(S) Load Balancing
    Expose a global endpoint that
    routes requests to the closest
    service.
    Be resilient to regional outages
    GCP Project
    Cloud
    Load
    Balancing
    Cloud Run
    europe-west1
    Cloud Run
    us-central1
    Cloud Run
    asia-northeast1
    San Francisco
    Paris
    https://example.com
    https://example.com
    

    View Slide

62. GCP Project
    Cloud CDN
    via External HTTP(S) Load Balancing
    Reduce the load on your service
    and improve its performance
    by returning Cache-Control headers
    Cloud Run
    Cache-Control: public, max-age=30, s-maxage=300
    1
    2
    Cloud
    CDN
    Cloud
    Load
    Balancing
    

    View Slide

63. GCP Project
    Identity
    Aware
    Proxy
    via External HTTP(S) Load
    Balancing
    Create internal web apps:
    ● automatic login screen
    ● grant access to users in
    your org
    (coming soon)
    Cloud
    Run
    Identity
    Aware
    Proxy
    Cloud
    Load
    Balancing
    ✓
    x
    

    View Slide

64. Cloud
    Armor
    via External HTTP(S) Load Balancing
    DDoS defense
    Filter traffic by:
    ● IP address
    ● geography
    Firewall rules
    Cloud Run
    Cloud
    Armor
    ✓
    x
    Cloud
    Load
    Balancing
    ip-ranges: "198.51.100.0/24"
    action: "allow"
    198.51.100.1
    192.0.2.0
    GCP Project
    

    View Slide

65. Deep dive:
    IAM invoker
    

    View Slide

66. GCP
    Invoker permissions
    Service
    IAM
    Requests
    Auth check:
    "allUsers"
    "user:mail@domain.com"
    "serviceAccount:..."
    

    View Slide

67. No authentication required
    Public service
    Frontend
    IAM:
    role: "roles/run.invoker"
    member: "allUsers"
    

    View Slide

68. Leverage "Invoker" IAM role and
    service identity.
    Private
    service to service
    Frontend
    Backend
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:frontend@..."
    header:"Authorization: Bearer ID_TOKEN"
    

    View Slide

69. Push Events with
    Pub/Sub
    Pub/Sub push to Cloud Run URL
    with authentication token.
    Leverage "Invoker" IAM role to
    authorize push.
    No need to validate URL.
    Cloud Run
    Service
    Cloud Pub/Sub
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:pubsub@..."
    gcloud alpha pubsub subscriptions create my-sub
    --topic my-topic
    --push-endpoint=https://service.run.app
    --push-auth-service-account=pubsub@...
    

    View Slide

70. Async tasks
    Cloud Tasks HTTP targets (Beta soon)
    push to Cloud Run URL
    with authentication token
    Leverage "Invoker" IAM role.
    Service
    Cloud Tasks
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:tasks@..."
    HTTP target
    

    View Slide

71. Scheduled services
    Cloud Scheduler
    with authentication token
    Leverage "Invoker" IAM role.
    Service
    Cloud Scheduler
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:scheduler@..."
    

    View Slide

72. Deep dive:
    Concurrency
    

    View Slide

73. Concurrency
    Each Service is autoscaled to many container instances.
    Concurrency = "maximum number of requests that can be sent at the
    same time to a given container instance"
    AWS Lambda or Google Cloud Functions:
    only one request at a time to each instance, "concurrency = 1".
    With Cloud Run: set concurrency value from 1 to 250 (default: 80)
    → optimized resource consumption
    → optimized costs
    concurrency = 1
    concurrency = 80
    

    View Slide

74. 400 Clients
    Targeting 3
    Req / S
    Concurrency = 1
    

    View Slide

75. 400 Clients
    Targeting 3
    Req / S
    Concurrency = 80
    

    View Slide

76. Deep dive:
    Choosing a serverless
    product
    

    View Slide

77. Proprietary + Confidential
    Google Cloud Serverless Compute
    Product Portfolio
    App Engine
    Cloud Run
    Cloud Functions Event-driven Functions-as-a-Service
    Run containers on a fully managed
    environment
    Run source-based web applications on a
    fully managed environment
    

    View Slide

78. Proprietary + Confidential
    Serverless Use Cases App Engine Cloud Run
    Cloud
    Functions
    Build a web app
    Web app
    Request-based services
    Developing APIs
    Web & mobile backends
    Internal APIs and services
    Data Processing
    Automation
    Workflow & Orchestration
    Event driven reactive automation
    Connecting Cloud Services
    Best use cases for Serverless compute products
    More at https://cloud.run
    

    View Slide

79. Docs: https://cloud.run
    PM hotline: steren@google.com
    Help
    

    View Slide