Choreography vs Orchestration in microservices and best practices

We went from a single monolith to a set of microservices that are small, lightweight, and easy to implement. Microservices enable reusability, make it easier to change and scale apps on demand but they also introduce new problems. How do microservices interact with each other toward a common goal? How do you figure out what went wrong when a business process composed of several microservices fails? Should there be a central orchestrator controlling all interactions between services or should each service work independently, in a loosely coupled way, and only interact through shared events? In this talk, we’ll explore the Choreography vs Orchestration question and see demos of some of the tools that can help. And we'll explore some best practices and patterns to apply when adopting an orchestration approach.

1. Choreography vs Orchestration in microservices & best practices Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel Guillaume Laforge Developer Advocate at Google @glaforge glaforge.appspot.com speakerdeck.com/glaforge
2. Choreography vs Orchestration
3. How do you organize a group of microservices to cooperate towards a common goal?
4. Option 1: Direct service-to-service calls Services calling each other directly Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user
5. Direct service-to-service calls Pros ➕ Easy to implement: Services simply call each other Cons ➖ Too much coupling ➖ Each service can be a single point of failure ➖ Each service needs its own error / retry / timeout logic ➖ Who ensures the whole transaction is successful? (hint: saga pattern)
6. Option 2: Indirect via events (choreography) Event-driven services Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Message Broker Google Cloud: Pub/Sub, Eventarc AWS: SQS, SNS, EventBridge Azure: Event Grid, Event Hubs, Service Bus Other: Kafka, Pulsar, Solace PubSub+, RabbitMQ, NATS...
7. Indirect via events Pros ➕ Services are loosely coupled ➕ Services can be changed/scaled independently ➕ No single point of failure ➕ Events are useful to extend the system Cons ➖ Difficult to monitor ➖ Errors / retries / timeouts are hard ➖ The business flow is not captured explicitly ➖ Who ensures the whole transaction is successful?
8. Imagine a more complex scenario
9. Option 3: A central orchestrator Orchestrated services Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Orchestrator Google Cloud: Workflows, Cloud Composer AWS: Step Functions Azure: Logic Apps Other: CNCF Serverless Workflow, Apache Airflow, Camel, Camunda…
10. A central orchestrator Pros ➕ Business flow captured centrally, source controlled, versioned etc. ➕ Each step can be monitored ➕ Errors / retries / timeouts can be centralized ➕ Services are still independent Cons ➖ A new orchestrator service to learn and maintain ➖ Orchestrator could be a single point of failure ➖ Loss of eventing flexibility ➖ How do you compensate for failed steps? (hint: saga pattern)
11. Choreography or Orchestration? It depends! (We’ll revisit this question later)
12. Orchestration
13. CNCF Serverless Workflow serverlessworkflow.io Defines a vendor-neutral, open-source, and fully community-driven ecosystem for defining and running DSL-based workflows that target the Serverless technology domain • Specification for defining DSL-based workflows • Developer SDKs for different programming languages • Workflow runtimes supporting the specification • Developer tooling support for writing DSL-based workflows
14. CNCF Serverless Workflow serverlessworkflow.io Workflow projects need to implement & support the spec Spec doesn’t necessarily cover all aspects of a product and not all products cover the whole specification Services need to be described with OpenAPI, events with CloudEvents Product Spec
15. Serverless Compute External API’s Google API’s etc... Workflows - orchestrate & integrate SaaS API’s Private API’s Other Clouds Google Cloud Workflows
16. UI Overview
23. Syntax elements
24. Proprietary + Confidential Sequences of steps Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: address: ${processResult.body.address} result: shipResult - notifyUser: call: http.post ...
25. Proprietary + Confidential Variable passing & JSON parsing Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: address: ${processResult.body.address} result: shipResult - notifyUser: call: http.post ...
26. Proprietary + Confidential Calling HTTP APIs Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: address: ${processResult.body.address} result: shipResult - notifyUser: call: http.post ...
27. Proprietary + Confidential Authentication (OAuth2 | OIDC) Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} auth: type: OIDC result: processResult ... AUTHENTICATION
28. Proprietary + Confidential Pause Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items WAIT - pause: call: sys.sleep args: seconds: 60
29. Proprietary + Confidential Logging Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items LOG - log-processed: call: sys.log args: text: "Payment processed" severity: INFO
30. Proprietary + Confidential base64 • encode • decode text • encode • decode • find_all • find_all_regex • match_regex • replace_all • replace_all_regex • split • substring • to_lower • to_upper • url_encode • url_encode_plus Built-in functions http • get • post • put • patch • delete • request • default_retry_* sys • get_env • sleep • slee_until • now • log time • fomart • parse retry • always • default_backoff • never errors • type_error • value_error • index_error • key_error • not_implemented_error • recursion_error • zero_division_error • system_error • timeout_error • resource_limit_error json • encode • encode_to_string • decode events • await_callback • Create_callback_endpoint math • abs • max • min list • concat map • get
31. Proprietary + Confidential Error handling, conditionals, jumps Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: try: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult except: as: e steps: - known_errors: switch: - condition: ${not("HttpError" in e.tags)} next: connectionError - condition: ${e.code == 404} return: "Sorry, URL wasn't found." - unhandled_exception: raise: ${e} ERROR CHECKING
32. Proprietary + Confidential Retry & backoff Payment Processor Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: try: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult retry: max_retries: 5 backoff: initial_delay: 1 max_delay: 60 multiplier: 2 MAX: 5 times BACKOFF
33. Proprietary + Confidential Like programming language subroutines or functions Subworkflows main: steps: - call_fullname: call: get_fullname args: first_name: "Sherlock" last_name: "Holmes" result: output - return_message: return: ${output} get_fullname: params: [first_name, last_name] steps: - prepMessage: return: ${first_name + " " + last_name}
34. Proprietary + Confidential Connectors - runQuery: call: googleapis.bigquery.v2.jobs.query args: projectId: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")} body: useLegacySql: false useQueryCache: false timeoutMs: 30000 # Find top 100 titles with most views on Wikipedia query: ${ "SELECT TITLE, SUM(views) FROM `bigquery-samples.wikipedia_pageviews."+table+"` WHERE LENGTH(TITLE) > 10 GROUP BY TITLE ORDER BY SUM(VIEWS) DESC LIMIT 100" } result: queryResult Make it easier to work with certain Google Cloud services and APIs, like transparently waiting for long-running operations.
35. Proprietary + Confidential Dataflow Document AI Firestore Google Forms ßeta Natural Language Machine Learning ßeta Pub/Sub Cloud Run Secret Manager BigQuery BigQuery Data Transfer ßeta Cloud Build Cloud Functions Cloud Resource Manager ßeta Cloud Scheduler Cloud Tasks Compute Container Connectors: currently available, but more to come… Google Sheets ßeta Spanner SQL Admin Cloud Storage Cloud Storage Transfer Translate Workflows Executions Workflows
36. Proprietary + Confidential Iterations - init: assign: - results: {} - tables: - 202202h - 202203h - 202204h - 202205h - runQueries: for: value: table index: table_idx in: ${tables} steps: - runQuery: call: googleapis.bigquery.v2.jobs.query args: … Loop over lists, maps, and over a range of values. Optional index. Use break and continue to short circuit the loop, with a next jump.
37. Proprietary + Confidential Parallel iterations - init: assign: - results: {} - tables: - 202202h - 202203h - 202204h - 202205h - runQueries: parallel: shared: [results] for: value: table index: table_idx in: ${tables} steps: - runQuery: call: googleapis.bigquery.v2.jobs.query args: … Add a parallel block to parallelize iterations. Specify shared to list the variables that can be accessed with write-access in parallel.
38. Proprietary + Confidential Parallel steps - parallelSteps: parallel: shared: [metadataResp, indexResp, collageResp] branches: - storeMetadataBranch: steps: - storeMetadata: call: http.request args: … result: metadataResp - indexPictureMetadataBranch: steps: - indexPictureMetadata: call: http.post args: … result: indexResp - collageCallBranch: steps: - collageCall: call: http.get args: … result: collageResp Branches of steps can be run in parallel, with the parallel keyword, and variables can be shared with write access with the shared keyword.
39. Proprietary + Confidential Gcloud commands # Deploy a workflow gcloud workflows deploy my-workflow \ --source=workflow.yaml # Execute a workflow gcloud workflows execute my-workflow # See the result gcloud workflows executions \ describe <your-execution-id> \ --workflow my-workflow Deploy and execute a workflow. Inspect the result of the execution of a workflow.
40. Patterns & Best Practices
41. Make a conscious choice Event-driven architecture ➔ Services are not closely related ➔ Services are not executed in parallel or in no certain order ➔ Services can exist in different bounded contexts Central Orchestrator ➔ Services are closely related ➔ Services are usually deployed and executed in the same order ➔ Can you describe the architecture in a flow chart? Direct Calls ➔ A simple architecture with a handful of services that do not change that often
42. Event-driven orchestration github.com/GoogleCloudPlatform/eventarc-samples/tree/main/processing-pipelines/image-v3
43. Handle errors with retries and saga pattern github.com/GoogleCloudPlatform/workflows-demos/tree/master/retries-and-saga
44. Wait for HTTP/event callbacks instead of polling github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-translation github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-event
45. Parallelize when you can github.com/GoogleCloudPlatform/workflows-demos/tree/master/bigquery-parallel Orchestration usually involves steps run sequentially one after another. Try to parallelize those steps when you can. Example: running BigQuery jobs against Wikipedia dataset with Workflows: • Serial: 5 queries run sequentially each 20 seconds: Total 1 min • Parallel: 5 queries run in parallel: Total 20 seconds
46. Combine serverful workloads with serverless orchestration Sometimes you can’t use serverless due to some limitation (time, memory, CPU) Instead you use a Virtual Machine (VM) with the configuration you need Automate the VM lifecycle with an orchestrator to have a serverless experience github.com/GoogleCloudPlatform/workflows-demos/tree/master/long-running-container
47. Manage long running batch jobs with serverless orchestration github.com/GoogleCloudPlatform/workflows-demos/tree/master/screenshot-jobs github.com/GoogleCloudPlatform/batch-samples/tree/main/primegen
48. Use GitOps to manage orchestration lifecycle github.com/GoogleCloudPlatform/workflows-demos/tree/master/gitops
49. Plan for multi-environment deployments github.com/GoogleCloudPlatform/workflows-demos/tree/master/multi-env-deployment
50. Workflows cloud.google.com/workflows Workflows Demos github.com/GoogleCloudPlatform/workflows-demos Codelab: Intro to serverless orchestration with Workflows codelabs.developers.google.com/codelabs/cloud-workflows-intro Thank you! Guillaume Laforge @glaforge glaforge.appspot.com speakerdeck.com/glaforge Mete Atamel @meteatamel atamel.dev speakerdeck.com/meteatamel