Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Google Cloud Serverless for .NET developers

Google Cloud Serverless for .NET developers

Ce3e7802c882fca3f9160768f57ae116?s=128

Mete Atamel

October 01, 2021
Tweet

Transcript

  1. Google Cloud Serverless for .NET developers Mete Atamel Developer Advocate

    at Google @meteatamel atamel.dev speakerdeck.com/meteatamel
  2. Legacy architecture for .NET apps End Users On-Prem Data Center

    Load Balancer IIS (.NET) Backend (.NET) MSMQ SQL Server Active Directory
  3. None
  4. cloud.google.com/architecture/modernization-path-dotnet-applications-google-cloud

  5. None
  6. Compute Engine Windows Server App Engine Flex* Kubernetes Engine Linux

    containers .NET on Windows .NET (Core) on Linux Cloud Run Where should I run my .NET stuff? Cloud Functions Kubernetes Engine Windows containers Serverless * Not really serverless!
  7. Serverless functions Serverless web applications Serverless HTTP containers Source code-based

    event driven functions Source code-based web apps and API backends HTTP containers fully managed HTTP containers on GKE cluster Cloud Functions App Engine Cloud Run Serverless options
  8. Backend functions that trigger in response to an event Microservices

    Cloud "glue" Certain versions of Node.js, Python, Go, Java, .NET, PHP, Ruby Cloud Functions
  9. Cloud Functions as cloud glue Access Google services Cloud Storage

    Cloud Pub/Sub HTTPS Firebase Cloud Scheduler Cloud Tasks
  10. Versions supported by Cloud Functions Node.js 10, 12, 14 Python

    3.7, 3.8, 3.9 Go 1.11, 1.13 Java 11 .NET Core 3.1 PHP 7.4 Ruby 2.6, 2.7
  11. What to do when Cloud Functions runtime is not enough?

    Create your own runtime!
  12. Containers Any language Any library Ecosystem around containers .js .rb

    .go .py .sh … 0 1 0 1 0 0 1 1 1
  13. Containers Flexibility Serverless Velocity

  14. Cloud Run Fully managed, deploy your workloads and don’t see

    the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative Everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative. Serverless containers with Knative and Cloud Run
  15. HTTPS Endpoint Public • Website • API endpoint Private •

    Internal services • Async tasks • Mobile backend • Webhook
  16. Container contract Listen on 0.0.0.0 on port $PORT (default 8080)

    HTTP server must start < 4 min (timeout → 504) Request time < 60 min (default → 5 min) Stateless (in-memory file system, doesn’t persist) Computation only within request (No background activity)
  17. Container resources 1 vCPU per container instance (configurable to 4

    vCPU) 256 MiB of memory up to a max of 8 GiB (configurable) 80 concurrent requests per container (configurable 1-1000) 100 max containers by default (configurable 1-1000) Access to a Metadata Server Sandboxed by gVisor
  18. Pay per use CPU / Memory / Requests 100ms

  19. Billable time Instance Billable Time Request 1 Start Request 1

    End Request 2 Start Request 2 End Instance Time Billable Non-billable
  20. Concurrency concurrency = 1 concurrency = 80 (default) - 1000

    (max)
  21. Pub/Sub triggered internal services Cloud Run Cloud Pub/Sub Queue Queue

    Queue
  22. Storage triggered internal services Cloud Run Cloud Pub/Sub Queue Queue

    Queue Cloud Storage
  23. Scheduled services Command Line Interface (CLI) User Interface (UI) Scheduler

    API Cloud Run Cloud Scheduler
  24. Custom Sources And more ... Workflows Orchestrated Cloud Functions Cloud

    Scheduler Eventarc Event-driven Cloud Run Cloud Run for Anthos GKE Compute Engine Pub/Sub Sources BigQuery Cloud Storage Google Sources Targets Serverless integration
  25. Asynchronously deliver events from a wide selection of sources to

    many targets Private alpha since early 2020 as Events for Cloud Run Public GA as Eventarc in Jan 2021 Eventarc
  26. Google Cloud GA Your own apps GA Cloud Run GA

    Cloud Functions Preview Workflows Preview Targets Sources Cloud Run for Anthos on GKE Preview Eventarc Triggers to filter events CloudEvents format
  27. New message in a Pub/Sub topic → Cloud Run gcloud

    eventarc triggers create trigger-pubsub \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION \ --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" --transport-topic=projects/$PROJECT_ID/topics/$TOPIC_ID Pub/Sub Trigger
  28. Create a trigger for new object creation in Cloud Storage

    → Cloud Run gcloud eventarc triggers create trigger-gcs \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION --event-filters="type=google.cloud.storage.object.v1.finalized" \ --event-filters="bucket=$BUCKET_NAME" \ --service-account=$PROJECT_NO-compute@developer.gserviceaccount.com Cloud Storage Trigger
  29. New Compute Engine VM creation → Cloud Run gcloud eventarc

    triggers create trigger-auditlog \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION --event-filters="type=google.cloud.audit.log.v1.written" \ --event-filters="serviceName=compute.googleapis.com" \ --event-filters="methodName=beta.compute.instances.insert" \ --service-account=$PROJECT_NO-compute@developer.gserviceaccount.com Audit Log Trigger
  30. Cloud Console

  31. POST / HTTP/1.1 Content-Type: application/json; charset=utf-8 Content-Length: 33 ce-specversion: 1.0

    ce-type: google.cloud.pubsub.topic.publish ce-time: 2020-09-05T03:56:24Z ce-id: 1234-1234-1234 ce-source: mycontext/subcontext custom-attr: 42 { "message": "Hello Cloud Next!" } CloudEvents cloudevents.io "Data" "Context"
  32. Cloud Run Service CloudEvent SDK HTTP request → CloudEvent in

    your language cloudevents.io Google Events Library Type library for CloudEvent#data Event parsing libraries HTTP POST to Cloud Run URL HTTP body is a CloudEvent with event data ("binary" CloudEvent V1) (optional) (optional) github.com/googleapis/google-cloudevents
  33. const { HTTP } = require("cloudevents"); const {toLogEntryData} = require('@google/events/cloud/storage/v1/StorageObjectData')

    app.post('/', async (req, res) => { // Read CloudEvent using CloudEvents SDK const cloudEvent = HTTP.toEvent({ headers: req.headers, body: req.body }); // Read Cloud Storage event using Google.Events library for Node.js const storageObjectData = toStorageObjectData(cloudEvent.data); // Extract bucket and objectName const bucket = storageObjectData.bucket; const objectName = storageObjectData.name;
  34. using CloudNative.CloudEvents; using Google.Events; using Google.Events.Protobuf.Cloud.PubSub.V1; public async Task<CloudEvent> Read(HttpContext

    context) { var formatter = CloudEventFormatterAttribute.CreateFormatter(typeof(MessagePublishedData)); // Read CloudEvent using CloudEvents SDK var cloudEvent = await context.Request.ToCloudEventAsync(formatter); // Read Pub/Sub message using Google.Events library for .NET var messagePublishedData = (MessagePublishedData)cloudEvent.Data; // Extract the Pub/Sub message var pubSubMessage = messagePublishedData.Message;
  35. Serverless Compute External API’s Google API’s etc... Workflows - orchestrate

    & integrate SaaS API’s Private API’s Other Clouds
  36. - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails}

    result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: input: ${processResult.body} result: shipResult - notifyUser: call: http.post ... Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items YAML or JSON syntax
  37. Payment Processor Cloud Run Authorize & charge CC Notifier Cloud

    Run Notify user Shipper Cloud Functions Prepare & ship items Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items WAIT Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items shipmentDetails userDetails Step Sequencing Serverless Pause Variable passing JSON Parsing Steps
  38. Errors and retries Payment Processor Cloud Run Authorize & charge

    CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items MAX: 5 times BACKOFF Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items Pager Cloud Run Escalate to support SUCCESS ERROR Configurable retries Configurable exception handling
  39. Conditionals and 3rd party calls Notifier Cloud Run Notify user

    Shipper Cloud Functions Prepare & ship items Pager Cloud Run Escalate to support SUCCESS ERROR Out of Stock? No Request from the supplier Yes Read inventory Inventory DB Update inventory Inventory DB Supplier API
  40. Other useful features Subworkflows to encapsulate common reusable flows Connectors

    to connect to other Google Cloud services & APIs More iterations, callbacks (preview)
  41. Deploy, execute, manage workflows # Deploy a workflow gcloud workflows

    deploy my-workflow --source=workflow.yaml # Execute a workflow gcloud workflows execute my-workflow # See the result gcloud workflows executions describe <your-execution-id> --workflow my-workflow
  42. None
  43. Image Processing Pipeline - Eventarc and Workflows End Users Images

    Input Cloud Storage Images Output Cloud Storage Filter Cloud Run Cloud Storage Trigger Eventarc Image Processing Workflows Watermarker Cloud Functions Resizer Cloud Functions Labeler Cloud Functions github.com/GoogleCloudPlatform/eventarc-samples/tree/main/processing-pipelines/image-workflows
  44. @meteatamel atamel.dev speakerdeck.com/meteatamel cloud.google.com/dotnet cloud.google.com/functions cloud.google.com/run cloud.google.com/eventarc cloud.google.com/workflows cloud.google.com/architecture/modernization-path-dotnet-applications-google-cloud Thank

    you!