Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Going Serverless with Artifactory and Containers on Cloud Run — SwampUp 2021

Going Serverless with Artifactory and Containers on Cloud Run — SwampUp 2021

On Google Cloud, modern workloads are running containers, in a serverless fashion. With Cloud Run, you can easily deploy your workloads, and have them scale up and down transparently to accommodate traffic spikes as well as low demand. In this session, we’ll introduce you to the serverless container world of Cloud Run, how to build them with Cloud Build, and we’ll see how we can take advantage of JFrog Artifactory for hosting our container artifacts. In a series of live demos, we will demonstrate different Cloud Run use cases. Legacy Monolith, Machine Learning API, Vault, Microservices or Event Based Architecture — Google Cloud Run and JFrog will get you covered!

Guillaume Laforge

June 02, 2021
Tweet

More Decks by Guillaume Laforge

Other Decks in Technology

Transcript

  1. Going Serverless
    with Artifactory
    and Containers
    on Cloud Run
    Guillaume Laforge
    Developer Advocate @ Google
    Ayrat Khayretdinov
    CNCF Ambassador,
    Hybrid Cloud Specialist @ Google
    @glaforge
    @archyufa

    View full-size slide

  2. Proprietary + Confidential

    View full-size slide

  3. Proprietary + Confidential
    The serverless models and characteristics
    Operational
    Model
    Programming
    Model
    No Infra Management Managed Security Pay only for usage
    Service-based Event-driven Stateless

    View full-size slide

  4. Proprietary + Confidential
    FaaS
    App platform
    Container
    Virtual machine
    The serverless spectrum
    Dedicated server
    Physical
    hardware
    Virtual
    hardware
    Operation
    system
    Application
    runtime
    Application
    Function
    Virtual
    hardware
    Operation
    system
    Application
    runtime
    Application
    Function
    Operation
    system
    Application
    runtime
    Application
    Function
    Application
    Function Function
    Serverless Zone

    View full-size slide

  5. Proprietary + Confidential
    Develop and deploy highly scalable containerized
    applications on a fully managed serverless platform
    ● Write code your way using your favorite languages
    (Go, Python, Java, Ruby, Node.js, and more)
    ● Abstract away all infrastructure management
    for a simple developer experience
    ● Built upon an open standard Knative,
    enabling the portability of your applications
    Cloud Run

    View full-size slide

  6. Proprietary + Confidential
    Serverless containers with Knative and Cloud Run
    Cloud Run
    Fully managed, deploy your
    workloads and don’t see the
    cluster.
    Cloud Run on Anthos
    Deploy into Anthos, run
    serverless side-by-side with
    your existing workloads.
    Knative everywhere
    Use the same APIs and
    tooling anywhere you run
    Kubernetes with Knative.

    View full-size slide

  7. Proprietary + Confidential
    Containers
    Any language Rich ecosystem
    Any library of base images
    Any binary
    .js .rb .go
    .py .sh …
    0 1 0
    1 0 0
    1 1 1
    Containers
    Flexibility
    Serverless
    Velocity

    View full-size slide

  8. Proprietary + Confidential
    Container contract... and resources
    ● Listen on 0.0.0.0 on port $PORT
    (default 8080)
    ● HTTP server must start < 4 min
    (timeout → 504)
    ● Request time < 15 min
    (default → 5 min, up to 60 min)
    ● Stateless
    (in-memory file system)
    ● Computation only within request
    (No background activity)
    ● 1 vCPU per container instance
    (configurable to 4 vCPU)
    ● 256 MiB of memory up to a max of 8 GiB
    (configurable, soon 16 GiB)
    ● 80 concurrent requests per container
    (configurable 1-80, soon 250)
    ● 100 max containers by default
    (configurable 1-1000, support overridable)
    ● Sandboxed by gVisor

    View full-size slide

  9. Proprietary + Confidential
    Pay per use
    CPU / Memory / Requests 100ms

    View full-size slide

  10. Proprietary + Confidential
    Billable time
    Instance
    Billable Time
    Request 1 Start Request 1 End
    Request 2 Start Request 2 End
    Instance Time
    Billable
    Non-billable

    View full-size slide

  11. Proprietary + Confidential
    Concurrency model
    concurrency = 1
    concurrency = 80

    View full-size slide

  12. Cloud Run is available
    Planned
    Future GCP region

    View full-size slide

  13. Gradual
    rollouts &
    Rollbacks
    Specify % traffic
    between revisions
    Blue / Green deployments
    Get URLs for specific
    revisions
    # Gradual rollout
    $ gcloud beta run deploy myservice \
    --image gcr.io/project/image:f5bd774 \
    --no-traffic \
    --tag green
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=1
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=10
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=50
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=100
    # Rollback
    $ gcloud run services update-traffic myservice
    --to-revisions my-service-0002-joy=100
    $ curl https://green---myservice-12345-us.a.run.app

    View full-size slide

  14. GCP Project
    VPC
    Access
    Connect to Cloud
    Memorystore Redis and
    Memcached
    Connect to private IPs
    Shared VPC
    Cloud Run
    Serverless
    VPC
    Connector
    Cloud
    Memorystore
    VM
    Compute
    Engine
    Virtual Private Cloud
    Private IP
    Private IP

    View full-size slide

  15. Cloud
    Workflows
    Orchestrate serverless tasks
    Fully managed.
    Use cases:
    ● Process events
    ● Chaining API calls
    ● Automate infra management
    ● Implement retry policies
    GCP Project
    Start
    Every day at
    6pm
    Get list
    of all
    dev VMs
    End
    For each VM:
    Is it
    running
    ?
    Extract
    status
    Stop VM
    Email
    owner

    View full-size slide

  16. Min
    instances
    Keep a number of
    instances warm
    Use to avoid cold starts
    Lower instance price
    when not in use
    Container Instances
    min = 2
    0

    View full-size slide

  17. Graceful
    instance
    termination
    Receive a SIGTERM signal
    before container instance
    is terminated.
    If handled, CPU is
    allocated for 10s max
    $ cat index.js
    const process = require('process');
    process.on('SIGTERM', () => {
    console.log('Container is shutting down...');
    // TODO: Close database connections
    // TODO: Send any buffered telemetry data
    });
    ...

    View full-size slide

  18. gRPC
    support
    $ cat main.go

    func main() {
    port := os.Getenv("PORT")
    listener, err := net.Listen("tcp", ":"+port)
    grpcServer := grpc.NewServer()
    pb.RegisterPingServiceServer(grpcServer, &pingService{})
    if err = grpcServer.Serve(listener); err != nil {
    log.Fatal(err)
    }
    }

    View full-size slide

  19. Server-side
    streaming
    Server-Sent Events,
    WebSockets.
    Stream HTTP or gRPC
    responses
    Responses no more buffered
    and limited to 32MB
    $ cat main.go

    http.HandleFunc("/", func(w http.ResponseWriter, r
    *http.Request) {
    w.Header().Set("Content-Type", "video/mp4")
    w.Header().Set("Transfer-Encoding", "chunked")
    f, _ := os.Open("videos/demo.mp4")
    io.Copy(w, f)
    })
    port := os.Getenv("PORT")
    log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
    (coming soon)

    View full-size slide

  20. Proprietary + Confidential
    Cloud Run Use cases
    More at https://cloud.run
    Serverless Use Cases Cloud Run
    Build a web or mobile app
    HTTP web app
    Bidirectional streaming
    WebSockets & gRPC
    Developing APIs
    Web & mobile backends
    Internal APIs and services
    Data Processing
    Automation
    Event driven reactive automation
    Workflow & Orchestration
    Event driven reactive automation
    Connecting Cloud Services
    Stateful Workloads ?
    Healthcare Gaming
    Retail Banking

    View full-size slide

  21. Proprietary + Confidential
    Attached
    clusters
    Anthos on Azure
    Anthos on AWS
    Google Anthos and Jfrog Partnership

    View full-size slide

  22. Cloud Run with Jfrog Demo Stack
    JFrog Cloud Platform on GCP
    - Jfrog Pipelines
    - Jfrog Docker Artifactory
    - Jfrog Xray
    Pull request
    Trigger
    Docker Repository
    CI CD
    Update Image tag
    Cloud Run Fully
    Managed
    Cloud Run for
    Anthos
    Thanks @jenn_viau for helping to build this amazing demo!
    Code Quality
    Build Image
    Scan Image

    View full-size slide

  23. Online Store application

    View full-size slide

  24. Amazing Cloud Run Links
    ● Slides - https://speakerdeck.com/cncfcanada
    ● Awesome Cloud Run - github.com/steren/awesome-cloudrun
    ● Cloud Run FAQ - github.com/ahmetb/cloud-run-faq
    ● Vault on Cloud Run - gh/kelseyhightower/serverless-vault-with-cloud-run
    Guillaume Laforge
    Developer Advocate @ Google
    Ayrat Khayretdinov
    CNCF Ambassador,
    Hybrid Cloud Specialist @ Google
    @glaforge
    @archyufa

    View full-size slide

  25. THANK YOU!
    Guillaume Laforge
    Developer Advocate @ Google
    Ayrat Khayretdinov
    CNCF Ambassador, GDG, GDE
    Hybrid Cloud Specialist @ Google
    @glaforge
    @archyufa

    View full-size slide

  26. STEREN’S
    CLOUD RUN
    DECK

    View full-size slide

  27. Cloud Run
    2021-04

    View full-size slide

  28. 33
    Proprietary + Confidential
    Application
    Development
    Performance
    Monitoring
    Serverless
    Physical
    Servers
    Network
    Hardware
    Physical
    Security
    Internet
    Connectivity
    Scaling
    VM
    Provisioning
    Ops & Security
    Management
    The Responsibility
    Pyramid
    Managed by
    customer
    Fully Managed
    by Google

    View full-size slide

  29. 34
    Risk
    Simplicity
    Velocity
    Elasticity
    Key values of developing, deploying and
    scaling apps in a fully managed environment

    View full-size slide

  30. 35
    Serverless
    Compute
    Deploy and scale applications fast and
    securely in a fully managed environment
    No Infra
    Management
    Speed to
    Market
    Auto-scaling

    View full-size slide

  31. “We can’t be locked in.”
    “How can we use
    existing binaries?”
    “Why do I have to choose between
    containers and serverless?”
    “Can you support language ____ ?”
    But… customers ask us:

    View full-size slide

  32. Containers
    • Any Language
    • Any Library
    • Any Binary
    • Ecosystem of base images
    .js .rb .go
    .py .sh …
    0 1 0
    1 0 0
    1 1 1

    View full-size slide

  33. Proprietary + Confidential
    Cloud Run
    Deploy in seconds
    Automatic HTTPS, Custom domains
    Any language, any library
    Portability
    No cluster management
    Run containers on a fully managed environment

    View full-size slide

  34. Deploy in
    seconds
    Demo

    View full-size slide

  35. Proprietary + Confidential
    Build a web app
    Web app
    Request-based services
    Developing APIs
    Web & mobile backends
    Internal APIs and services
    Data Processing
    Automation
    Workflow & Orchestration
    Event driven reactive automation
    Connecting Cloud Services
    Use cases
    More at https://cloud.run

    View full-size slide

  36. Get Started
    Setup Compute
    CICD
    Run
    Other
    Create
    Project
    Setup
    Compute
    Core
    Infra
    Load
    Balancing
    Service
    Mesh
    Specify
    Volumes NAT
    Shared
    VPC
    Workload
    YAML
    Auto-Scale
    YAML
    Cloud Build
    YAML
    Deploy
    Time Test
    Image
    Rep.
    Source Version
    Control
    Run
    8x faster time to market
    with Cloud Run!
    Shared
    VPC
    Create
    Project
    Deploy
    Time &
    Test
    Source
    Version
    Control
    Image
    Rep
    Cloud
    Build
    YAML
    Run

    View full-size slide

  37. Proprietary + Confidential
    Engineers
    3
    Days
    2
    Containerized
    app
    1
    5.5M
    Requests per day
    700ms
    Latency
    €500
    a month
    40% cost reduction and 80% less
    effort

    View full-size slide

  38. News and Roadmap

    View full-size slide

  39. Cloud Run is available
    Planned
    Future GCP region

    View full-size slide

  40. Delight developers
    ● Deploy from YAML / Export to YAML
    ● Easy Continuous Deployment set up from Git
    ● Develop and run locally
    ● Graceful instance termination
    ● Trigger from 60+ GCP sources (Eventarc)
    ● Cloud Workflows
    Expand addressable workloads
    ● Min instances
    ● Up to 16GB RAM
    ● Up to 4 CPU
    ● bidirectional gRPC streaming Preview
    ● websockets Preview
    ● HTTP/2 Preview
    ● Long running instances (1hr)
    What's new?
    Enterprise Class
    ● VPC connectors: Shared VPC support
    ● Ingress = internal
    ● Egress controls
    ● VPC-SC
    ● Secret Manager integration
    ● Customer Managed Encryption Keys
    ● Binary Authorization
    ● API Gateway support
    ● Identity Aware Proxy support
    ● Cloud Armor support
    ● Committed Use Discounts

    View full-size slide

  41. Keep delighting developers
    Enterprise-class
    Expand addressable workloads
    ● Access and Transparency
    ● Liveness and Readiness checks
    ● Always on CPU
    ● Autoscaling improvements
    ● Deterministic URLs, Service Discovery and Service Mesh

    Current
    Current
    🠻
    🠻
    🠻
    🠻
    ● Local development experience (with Cloud Code)
    ● Trigger with events (with Eventarc)
    ● Recommendations and insights (with Recommendation Hub)
    ● Better observability: new metrics, platform errors, tracing (with Cloud Ops)
    ● Infrastructure as code via Config Connector
    1. Expand the Cloud Run Service:
    ● Instances: filesystem access, more CPU, more RAM…
    ● Spec: multiple container
    2. New resource types: Jobs, non-HTTP services...

    View full-size slide

  42. Docs: https://cloud.run
    PM hotline: [email protected]
    Q&A Help

    View full-size slide

  43. Deep dive:
    Pricing

    View full-size slide

  44. Cloud Run (fully managed): Pay-per-use
    CPU / Memory / Requests 100ms

    View full-size slide

  45. Billable time
    Instance
    Billable Time
    Request 1 Start Request 1 End
    Request 2 Start Request 2 End
    Instance Time
    Billable
    Non-billable

    View full-size slide

  46. Deep dive:
    Advanced Features

    View full-size slide

  47. Gradual
    rollouts &
    Rollbacks
    Specify % traffic between
    revisions
    Blue / Green deployments
    Get URLs for specific
    revisions
    # Gradual rollout
    $ gcloud beta run deploy myservice \
    --image gcr.io/project/image:f5bd774 \
    --no-traffic \
    --tag green
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=1
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=10
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=50
    $ gcloud beta run services update-traffic myservice \
    --to-tags green=100
    # Rollback
    $ gcloud run services update-traffic myservice
    --to-revisions my-service-0002-joy=100
    $ curl https://green---myservice-12345-us.a.run.app

    View full-size slide

  48. GCP Project
    VPC
    Access
    Connect to Cloud
    Memorystore Redis and
    Memcached
    Connect to private IPs
    Shared VPC
    Cloud Run
    Serverless
    VPC
    Connector
    Cloud
    Memorystore
    VM
    Compute
    Engine
    Virtual Private Cloud
    Private IP
    Private IP

    View full-size slide

  49. Cloud
    Workflows
    Orchestrate serverless tasks.
    Fully managed.
    Use cases:
    ● Process events
    ● Chaining API calls
    ● Automate infra management
    ● Implement retry policies
    GCP Project
    Start
    Every day at
    6pm
    Get list
    of all
    dev VMs
    End
    For each VM:
    Is it
    running
    ?
    Extract
    status
    Stop VM
    Email
    owner

    View full-size slide

  50. Min
    instances
    Keep a number of instances
    warm.
    Use to avoid cold starts.
    Lower instance price when
    not in use.
    Container Instances
    min = 2
    0

    View full-size slide

  51. Graceful
    instance
    termination
    Receive a SIGTERM signal
    before
    container instance is
    terminated.
    If handled, CPU is allocated
    for 10s max
    $ cat index.js
    const process = require('process');
    process.on('SIGTERM', () => {
    console.log('Container is shutting down...');
    // TODO: Close database connections
    // TODO: Send any buffered telemetry data
    });
    ...

    View full-size slide

  52. gRPC
    support
    $ cat main.go

    func main() {
    port := os.Getenv("PORT")
    listener, err := net.Listen("tcp", ":"+port)
    grpcServer := grpc.NewServer()
    pb.RegisterPingServiceServer(grpcServer, &pingService{})
    if err = grpcServer.Serve(listener); err != nil {
    log.Fatal(err)
    }
    }

    View full-size slide

  53. Server-side
    streaming
    Stream HTTP or gRPC
    responses.
    Responses no more buffered
    Responses no more limited
    to 32MB
    $ cat main.go

    http.HandleFunc("/", func(w http.ResponseWriter, r
    *http.Request) {
    w.Header().Set("Content-Type", "video/mp4")
    w.Header().Set("Transfer-Encoding", "chunked")
    f, _ := os.Open("videos/demo.mp4")
    io.Copy(w, f)
    })
    port := os.Getenv("PORT")
    log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
    (coming soon)

    View full-size slide

  54. Deep dive:
    GCLB

    View full-size slide

  55. Multi-region
    load
    balancing
    via External HTTP(S) Load Balancing
    Expose a global endpoint that
    routes requests to the closest
    service.
    Be resilient to regional outages
    GCP Project
    Cloud
    Load
    Balancing
    Cloud Run
    europe-west1
    Cloud Run
    us-central1
    Cloud Run
    asia-northeast1
    San Francisco
    Paris
    https://example.com
    https://example.com

    View full-size slide

  56. GCP Project
    Cloud CDN
    via External HTTP(S) Load Balancing
    Reduce the load on your service
    and improve its performance
    by returning Cache-Control headers
    Cloud Run
    Cache-Control: public, max-age=30, s-maxage=300
    1
    2
    Cloud
    CDN
    Cloud
    Load
    Balancing

    View full-size slide

  57. GCP Project
    Identity
    Aware
    Proxy
    via External HTTP(S) Load
    Balancing
    Create internal web apps:
    ● automatic login screen
    ● grant access to users in
    your org
    (coming soon)
    Cloud
    Run
    Identity
    Aware
    Proxy
    Cloud
    Load
    Balancing

    x

    View full-size slide

  58. Cloud
    Armor
    via External HTTP(S) Load Balancing
    DDoS defense
    Filter traffic by:
    ● IP address
    ● geography
    Firewall rules
    Cloud Run
    Cloud
    Armor

    x
    Cloud
    Load
    Balancing
    ip-ranges: "198.51.100.0/24"
    action: "allow"
    198.51.100.1
    192.0.2.0
    GCP Project

    View full-size slide

  59. Deep dive:
    IAM invoker

    View full-size slide

  60. GCP
    Invoker permissions
    Service
    IAM
    Requests
    Auth check:
    "allUsers"
    "user:[email protected]"
    "serviceAccount:..."

    View full-size slide

  61. No authentication required
    Public service
    Frontend
    IAM:
    role: "roles/run.invoker"
    member: "allUsers"

    View full-size slide

  62. Leverage "Invoker" IAM role and
    service identity.
    Private
    service to service
    Frontend
    Backend
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:frontend@..."
    header:"Authorization: Bearer ID_TOKEN"

    View full-size slide

  63. Push Events with
    Pub/Sub
    Pub/Sub push to Cloud Run URL
    with authentication token.
    Leverage "Invoker" IAM role to
    authorize push.
    No need to validate URL.
    Cloud Run
    Service
    Cloud Pub/Sub
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:pubsub@..."
    gcloud alpha pubsub subscriptions create my-sub
    --topic my-topic
    --push-endpoint=https://service.run.app
    --push-auth-service-account=pubsub@...

    View full-size slide

  64. Async tasks
    Cloud Tasks HTTP targets (Beta soon)
    push to Cloud Run URL
    with authentication token
    Leverage "Invoker" IAM role.
    Service
    Cloud Tasks
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:tasks@..."
    HTTP target

    View full-size slide

  65. Scheduled services
    Cloud Scheduler
    with authentication token
    Leverage "Invoker" IAM role.
    Service
    Cloud Scheduler
    IAM:
    role: "roles/run.invoker"
    member: "serviceAccount:scheduler@..."

    View full-size slide

  66. Deep dive:
    Concurrency

    View full-size slide

  67. Concurrency
    Each Service is autoscaled to many container instances.
    Concurrency = "maximum number of requests that can be sent at the
    same time to a given container instance"
    AWS Lambda or Google Cloud Functions:
    only one request at a time to each instance, "concurrency = 1".
    With Cloud Run: set concurrency value from 1 to 250 (default: 80)
    → optimized resource consumption
    → optimized costs
    concurrency = 1
    concurrency = 80

    View full-size slide

  68. 400 Clients
    Targeting 3
    Req / S
    Concurrency = 1

    View full-size slide

  69. 400 Clients
    Targeting 3
    Req / S
    Concurrency = 80

    View full-size slide

  70. Deep dive:
    Choosing a serverless
    product

    View full-size slide

  71. Proprietary + Confidential
    Google Cloud Serverless Compute
    Product Portfolio
    App Engine
    Cloud Run
    Cloud Functions Event-driven Functions-as-a-Service
    Run containers on a fully managed
    environment
    Run source-based web applications on a
    fully managed environment

    View full-size slide

  72. Proprietary + Confidential
    Serverless Use Cases App Engine Cloud Run
    Cloud
    Functions
    Build a web app
    Web app
    Request-based services
    Developing APIs
    Web & mobile backends
    Internal APIs and services
    Data Processing
    Automation
    Workflow & Orchestration
    Event driven reactive automation
    Connecting Cloud Services
    Best use cases for Serverless compute products
    More at https://cloud.run

    View full-size slide

  73. Docs: https://cloud.run
    PM hotline: [email protected]
    Help

    View full-size slide