Google Cloud Serverless for .NET developers

Transcript

1. Google Cloud Serverless for .NET developers Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel
2. Legacy architecture for .NET apps End Users On-Prem Data Center Load Balancer IIS (.NET) Backend (.NET) MSMQ SQL Server Active Directory
4. cloud.google.com/architecture/modernization-path-dotnet-applications-google-cloud
6. Compute Engine Windows Server App Engine Flex* Kubernetes Engine Linux containers .NET on Windows .NET (Core) on Linux Cloud Run Where should I run my .NET stuff? Cloud Functions Kubernetes Engine Windows containers Serverless * Not really serverless!
7. Serverless functions Serverless web applications Serverless HTTP containers Source code-based event driven functions Source code-based web apps and API backends HTTP containers fully managed HTTP containers on GKE cluster Cloud Functions App Engine Cloud Run Serverless options
8. Backend functions that trigger in response to an event Microservices Cloud "glue" Certain versions of Node.js, Python, Go, Java, .NET, PHP, Ruby Cloud Functions
9. Cloud Functions as cloud glue Access Google services Cloud Storage Cloud Pub/Sub HTTPS Firebase Cloud Scheduler Cloud Tasks
10. Versions supported by Cloud Functions Node.js 10, 12, 14 Python 3.7, 3.8, 3.9 Go 1.11, 1.13 Java 11 .NET Core 3.1 PHP 7.4 Ruby 2.6, 2.7
11. What to do when Cloud Functions runtime is not enough? Create your own runtime!
12. Containers Any language Any library Ecosystem around containers .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1
13. Containers Flexibility Serverless Velocity
14. Cloud Run Fully managed, deploy your workloads and don’t see the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative Everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative. Serverless containers with Knative and Cloud Run
15. HTTPS Endpoint Public • Website • API endpoint Private • Internal services • Async tasks • Mobile backend • Webhook
16. Container contract Listen on 0.0.0.0 on port $PORT (default 8080) HTTP server must start < 4 min (timeout → 504) Request time < 60 min (default → 5 min) Stateless (in-memory file system, doesn’t persist) Computation only within request (No background activity)
17. Container resources 1 vCPU per container instance (configurable to 4 vCPU) 256 MiB of memory up to a max of 8 GiB (configurable) 80 concurrent requests per container (configurable 1-1000) 100 max containers by default (configurable 1-1000) Access to a Metadata Server Sandboxed by gVisor
18. Pay per use CPU / Memory / Requests 100ms
19. Billable time Instance Billable Time Request 1 Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable
20. Concurrency concurrency = 1 concurrency = 80 (default) - 1000 (max)
21. Pub/Sub triggered internal services Cloud Run Cloud Pub/Sub Queue Queue Queue
22. Storage triggered internal services Cloud Run Cloud Pub/Sub Queue Queue Queue Cloud Storage
23. Scheduled services Command Line Interface (CLI) User Interface (UI) Scheduler API Cloud Run Cloud Scheduler
24. Custom Sources And more ... Workflows Orchestrated Cloud Functions Cloud Scheduler Eventarc Event-driven Cloud Run Cloud Run for Anthos GKE Compute Engine Pub/Sub Sources BigQuery Cloud Storage Google Sources Targets Serverless integration
25. Asynchronously deliver events from a wide selection of sources to many targets Private alpha since early 2020 as Events for Cloud Run Public GA as Eventarc in Jan 2021 Eventarc
26. Google Cloud GA Your own apps GA Cloud Run GA Cloud Functions Preview Workflows Preview Targets Sources Cloud Run for Anthos on GKE Preview Eventarc Triggers to filter events CloudEvents format
27. New message in a Pub/Sub topic → Cloud Run gcloud eventarc triggers create trigger-pubsub \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION \ --event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" --transport-topic=projects/$PROJECT_ID/topics/$TOPIC_ID Pub/Sub Trigger
28. Create a trigger for new object creation in Cloud Storage → Cloud Run gcloud eventarc triggers create trigger-gcs \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION --event-filters="type=google.cloud.storage.object.v1.finalized" \ --event-filters="bucket=$BUCKET_NAME" \ --service-account=$PROJECT_NO-compute@developer.gserviceaccount.com Cloud Storage Trigger
29. New Compute Engine VM creation → Cloud Run gcloud eventarc triggers create trigger-auditlog \ --destination-run-service=$SERVICE_NAME \ --destination-run-region=$REGION --event-filters="type=google.cloud.audit.log.v1.written" \ --event-filters="serviceName=compute.googleapis.com" \ --event-filters="methodName=beta.compute.instances.insert" \ --service-account=$PROJECT_NO-compute@developer.gserviceaccount.com Audit Log Trigger
30. Cloud Console
31. POST / HTTP/1.1 Content-Type: application/json; charset=utf-8 Content-Length: 33 ce-specversion: 1.0 ce-type: google.cloud.pubsub.topic.publish ce-time: 2020-09-05T03:56:24Z ce-id: 1234-1234-1234 ce-source: mycontext/subcontext custom-attr: 42 { "message": "Hello Cloud Next!" } CloudEvents cloudevents.io "Data" "Context"
32. Cloud Run Service CloudEvent SDK HTTP request → CloudEvent in your language cloudevents.io Google Events Library Type library for CloudEvent#data Event parsing libraries HTTP POST to Cloud Run URL HTTP body is a CloudEvent with event data ("binary" CloudEvent V1) (optional) (optional) github.com/googleapis/google-cloudevents
33. const { HTTP } = require("cloudevents"); const {toLogEntryData} = require('@google/events/cloud/storage/v1/StorageObjectData') app.post('/', async (req, res) => { // Read CloudEvent using CloudEvents SDK const cloudEvent = HTTP.toEvent({ headers: req.headers, body: req.body }); // Read Cloud Storage event using Google.Events library for Node.js const storageObjectData = toStorageObjectData(cloudEvent.data); // Extract bucket and objectName const bucket = storageObjectData.bucket; const objectName = storageObjectData.name;
34. using CloudNative.CloudEvents; using Google.Events; using Google.Events.Protobuf.Cloud.PubSub.V1; public async Task<CloudEvent> Read(HttpContext context) { var formatter = CloudEventFormatterAttribute.CreateFormatter(typeof(MessagePublishedData)); // Read CloudEvent using CloudEvents SDK var cloudEvent = await context.Request.ToCloudEventAsync(formatter); // Read Pub/Sub message using Google.Events library for .NET var messagePublishedData = (MessagePublishedData)cloudEvent.Data; // Extract the Pub/Sub message var pubSubMessage = messagePublishedData.Message;
35. Serverless Compute External API’s Google API’s etc... Workflows - orchestrate & integrate SaaS API’s Private API’s Other Clouds
36. - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: input: ${processResult.body} result: shipResult - notifyUser: call: http.post ... Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items YAML or JSON syntax
37. Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items WAIT Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items shipmentDetails userDetails Step Sequencing Serverless Pause Variable passing JSON Parsing Steps
38. Errors and retries Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items MAX: 5 times BACKOFF Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items Pager Cloud Run Escalate to support SUCCESS ERROR Configurable retries Configurable exception handling
39. Conditionals and 3rd party calls Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items Pager Cloud Run Escalate to support SUCCESS ERROR Out of Stock? No Request from the supplier Yes Read inventory Inventory DB Update inventory Inventory DB Supplier API
40. Other useful features Subworkflows to encapsulate common reusable flows Connectors to connect to other Google Cloud services & APIs More iterations, callbacks (preview)
41. Deploy, execute, manage workflows # Deploy a workflow gcloud workflows deploy my-workflow --source=workflow.yaml # Execute a workflow gcloud workflows execute my-workflow # See the result gcloud workflows executions describe <your-execution-id> --workflow my-workflow
43. Image Processing Pipeline - Eventarc and Workflows End Users Images Input Cloud Storage Images Output Cloud Storage Filter Cloud Run Cloud Storage Trigger Eventarc Image Processing Workflows Watermarker Cloud Functions Resizer Cloud Functions Labeler Cloud Functions github.com/GoogleCloudPlatform/eventarc-samples/tree/main/processing-pipelines/image-workflows
44. @meteatamel atamel.dev speakerdeck.com/meteatamel cloud.google.com/dotnet cloud.google.com/functions cloud.google.com/run cloud.google.com/eventarc cloud.google.com/workflows cloud.google.com/architecture/modernization-path-dotnet-applications-google-cloud Thank you!