Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Choreography vs Orchestration in microservices and best practices

Choreography vs Orchestration in microservices and best practices

We went from a single monolith to a set of microservices that are small, lightweight, and easy to implement. Microservices enable reusability, make it easier to change and scale apps on demand but they also introduce new problems. How do microservices interact with each other toward a common goal? How do you figure out what went wrong when a business process composed of several microservices fails? Should there be a central orchestrator controlling all interactions between services or should each service work independently, in a loosely coupled way, and only interact through shared events? In this talk, we’ll explore the Choreography vs Orchestration question and see demos of some of the tools that can help. And we'll explore some best practices and patterns to apply when adopting an orchestration approach.

Guillaume Laforge

October 20, 2022
Tweet

More Decks by Guillaume Laforge

Other Decks in Technology

Transcript

  1. Choreography vs Orchestration in microservices & best practices Mete Atamel

    Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel Guillaume Laforge Developer Advocate at Google @glaforge glaforge.appspot.com speakerdeck.com/glaforge
  2. Choreography vs Orchestration

  3. How do you organize a group of microservices to cooperate

    towards a common goal?
  4. Option 1: Direct service-to-service calls Services calling each other directly

    Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user
  5. Direct service-to-service calls Pros ➕ Easy to implement: Services simply

    call each other Cons ➖ Too much coupling ➖ Each service can be a single point of failure ➖ Each service needs its own error / retry / timeout logic ➖ Who ensures the whole transaction is successful? (hint: saga pattern)
  6. Option 2: Indirect via events (choreography) Event-driven services Frontend App

    Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Message Broker Google Cloud: Pub/Sub, Eventarc AWS: SQS, SNS, EventBridge Azure: Event Grid, Event Hubs, Service Bus Other: Kafka, Pulsar, Solace PubSub+, RabbitMQ, NATS...
  7. Indirect via events Pros ➕ Services are loosely coupled ➕

    Services can be changed/scaled independently ➕ No single point of failure ➕ Events are useful to extend the system Cons ➖ Difficult to monitor ➖ Errors / retries / timeouts are hard ➖ The business flow is not captured explicitly ➖ Who ensures the whole transaction is successful?
  8. Imagine a more complex scenario

  9. Option 3: A central orchestrator Orchestrated services Frontend App Engine

    Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Orchestrator Google Cloud: Workflows, Cloud Composer AWS: Step Functions Azure: Logic Apps Other: CNCF Serverless Workflow, Apache Airflow, Camel, Camunda…
  10. A central orchestrator Pros ➕ Business flow captured centrally, source

    controlled, versioned etc. ➕ Each step can be monitored ➕ Errors / retries / timeouts can be centralized ➕ Services are still independent Cons ➖ A new orchestrator service to learn and maintain ➖ Orchestrator could be a single point of failure ➖ Loss of eventing flexibility ➖ How do you compensate for failed steps? (hint: saga pattern)
  11. Choreography or Orchestration? It depends! (We’ll revisit this question later)

  12. Orchestration

  13. CNCF Serverless Workflow serverlessworkflow.io Defines a vendor-neutral, open-source, and fully

    community-driven ecosystem for defining and running DSL-based workflows that target the Serverless technology domain • Specification for defining DSL-based workflows • Developer SDKs for different programming languages • Workflow runtimes supporting the specification • Developer tooling support for writing DSL-based workflows
  14. CNCF Serverless Workflow serverlessworkflow.io Workflow projects need to implement &

    support the spec Spec doesn’t necessarily cover all aspects of a product and not all products cover the whole specification Services need to be described with OpenAPI, events with CloudEvents Product Spec
  15. Serverless Compute External API’s Google API’s etc... Workflows - orchestrate

    & integrate SaaS API’s Private API’s Other Clouds Google Cloud Workflows
  16. UI Overview

  17. None
  18. None
  19. None
  20. None
  21. None
  22. None
  23. Syntax elements

  24. Proprietary + Confidential Sequences of steps Payment Processor Cloud Run

    Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: address: ${processResult.body.address} result: shipResult - notifyUser: call: http.post ...
  25. Proprietary + Confidential Variable passing & JSON parsing Payment Processor

    Cloud Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: address: ${processResult.body.address} result: shipResult - notifyUser: call: http.post ...
  26. Proprietary + Confidential Calling HTTP APIs Payment Processor Cloud Run

    Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult - shipItems: call: http.post args: url: https://.../cloudfunctions.net/ship body: address: ${processResult.body.address} result: shipResult - notifyUser: call: http.post ...
  27. Proprietary + Confidential Authentication (OAuth2 | OIDC) Payment Processor Cloud

    Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} auth: type: OIDC result: processResult ... AUTHENTICATION
  28. Proprietary + Confidential Pause Payment Processor Cloud Run Authorize &

    charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items WAIT - pause: call: sys.sleep args: seconds: 60
  29. Proprietary + Confidential Logging Payment Processor Cloud Run Authorize &

    charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items LOG - log-processed: call: sys.log args: text: "Payment processed" severity: INFO
  30. Proprietary + Confidential base64 • encode • decode text •

    encode • decode • find_all • find_all_regex • match_regex • replace_all • replace_all_regex • split • substring • to_lower • to_upper • url_encode • url_encode_plus Built-in functions http • get • post • put • patch • delete • request • default_retry_* sys • get_env • sleep • slee_until • now • log time • fomart • parse retry • always • default_backoff • never errors • type_error • value_error • index_error • key_error • not_implemented_error • recursion_error • zero_division_error • system_error • timeout_error • resource_limit_error json • encode • encode_to_string • decode events • await_callback • Create_callback_endpoint math • abs • max • min list • concat map • get
  31. Proprietary + Confidential Error handling, conditionals, jumps Payment Processor Cloud

    Run Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: try: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult except: as: e steps: - known_errors: switch: - condition: ${not("HttpError" in e.tags)} next: connectionError - condition: ${e.code == 404} return: "Sorry, URL wasn't found." - unhandled_exception: raise: ${e} ERROR CHECKING
  32. Proprietary + Confidential Retry & backoff Payment Processor Cloud Run

    Authorize & charge CC Notifier Cloud Run Notify user Shipper Cloud Functions Prepare & ship items - processPayment: try: call: http.post args: url: https://payment-processor.run.app/... body: input: ${paymentDetails} result: processResult retry: max_retries: 5 backoff: initial_delay: 1 max_delay: 60 multiplier: 2 MAX: 5 times BACKOFF
  33. Proprietary + Confidential Like programming language subroutines or functions Subworkflows

    main: steps: - call_fullname: call: get_fullname args: first_name: "Sherlock" last_name: "Holmes" result: output - return_message: return: ${output} get_fullname: params: [first_name, last_name] steps: - prepMessage: return: ${first_name + " " + last_name}
  34. Proprietary + Confidential Connectors - runQuery: call: googleapis.bigquery.v2.jobs.query args: projectId:

    ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")} body: useLegacySql: false useQueryCache: false timeoutMs: 30000 # Find top 100 titles with most views on Wikipedia query: ${ "SELECT TITLE, SUM(views) FROM `bigquery-samples.wikipedia_pageviews."+table+"` WHERE LENGTH(TITLE) > 10 GROUP BY TITLE ORDER BY SUM(VIEWS) DESC LIMIT 100" } result: queryResult Make it easier to work with certain Google Cloud services and APIs, like transparently waiting for long-running operations.
  35. Proprietary + Confidential Dataflow Document AI Firestore Google Forms ßeta

    Natural Language Machine Learning ßeta Pub/Sub Cloud Run Secret Manager BigQuery BigQuery Data Transfer ßeta Cloud Build Cloud Functions Cloud Resource Manager ßeta Cloud Scheduler Cloud Tasks Compute Container Connectors: currently available, but more to come… Google Sheets ßeta Spanner SQL Admin Cloud Storage Cloud Storage Transfer Translate Workflows Executions Workflows
  36. Proprietary + Confidential Iterations - init: assign: - results: {}

    - tables: - 202202h - 202203h - 202204h - 202205h - runQueries: for: value: table index: table_idx in: ${tables} steps: - runQuery: call: googleapis.bigquery.v2.jobs.query args: … Loop over lists, maps, and over a range of values. Optional index. Use break and continue to short circuit the loop, with a next jump.
  37. Proprietary + Confidential Parallel iterations - init: assign: - results:

    {} - tables: - 202202h - 202203h - 202204h - 202205h - runQueries: parallel: shared: [results] for: value: table index: table_idx in: ${tables} steps: - runQuery: call: googleapis.bigquery.v2.jobs.query args: … Add a parallel block to parallelize iterations. Specify shared to list the variables that can be accessed with write-access in parallel.
  38. Proprietary + Confidential Parallel steps - parallelSteps: parallel: shared: [metadataResp,

    indexResp, collageResp] branches: - storeMetadataBranch: steps: - storeMetadata: call: http.request args: … result: metadataResp - indexPictureMetadataBranch: steps: - indexPictureMetadata: call: http.post args: … result: indexResp - collageCallBranch: steps: - collageCall: call: http.get args: … result: collageResp Branches of steps can be run in parallel, with the parallel keyword, and variables can be shared with write access with the shared keyword.
  39. Proprietary + Confidential Gcloud commands # Deploy a workflow gcloud

    workflows deploy my-workflow \ --source=workflow.yaml # Execute a workflow gcloud workflows execute my-workflow # See the result gcloud workflows executions \ describe <your-execution-id> \ --workflow my-workflow Deploy and execute a workflow. Inspect the result of the execution of a workflow.
  40. Patterns & Best Practices

  41. Make a conscious choice Event-driven architecture ➔ Services are not

    closely related ➔ Services are not executed in parallel or in no certain order ➔ Services can exist in different bounded contexts Central Orchestrator ➔ Services are closely related ➔ Services are usually deployed and executed in the same order ➔ Can you describe the architecture in a flow chart? Direct Calls ➔ A simple architecture with a handful of services that do not change that often
  42. Event-driven orchestration github.com/GoogleCloudPlatform/eventarc-samples/tree/main/processing-pipelines/image-v3

  43. Handle errors with retries and saga pattern github.com/GoogleCloudPlatform/workflows-demos/tree/master/retries-and-saga

  44. Wait for HTTP/event callbacks instead of polling github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-translation github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-event

  45. Parallelize when you can github.com/GoogleCloudPlatform/workflows-demos/tree/master/bigquery-parallel Orchestration usually involves steps run

    sequentially one after another. Try to parallelize those steps when you can. Example: running BigQuery jobs against Wikipedia dataset with Workflows: • Serial: 5 queries run sequentially each 20 seconds: Total 1 min • Parallel: 5 queries run in parallel: Total 20 seconds
  46. Combine serverful workloads with serverless orchestration Sometimes you can’t use

    serverless due to some limitation (time, memory, CPU) Instead you use a Virtual Machine (VM) with the configuration you need Automate the VM lifecycle with an orchestrator to have a serverless experience github.com/GoogleCloudPlatform/workflows-demos/tree/master/long-running-container
  47. Manage long running batch jobs with serverless orchestration github.com/GoogleCloudPlatform/workflows-demos/tree/master/screenshot-jobs github.com/GoogleCloudPlatform/batch-samples/tree/main/primegen

  48. Use GitOps to manage orchestration lifecycle github.com/GoogleCloudPlatform/workflows-demos/tree/master/gitops

  49. Plan for multi-environment deployments github.com/GoogleCloudPlatform/workflows-demos/tree/master/multi-env-deployment

  50. Workflows cloud.google.com/workflows Workflows Demos github.com/GoogleCloudPlatform/workflows-demos Codelab: Intro to serverless orchestration

    with Workflows codelabs.developers.google.com/codelabs/cloud-workflows-intro Thank you! Guillaume Laforge @glaforge glaforge.appspot.com speakerdeck.com/glaforge Mete Atamel @meteatamel atamel.dev speakerdeck.com/meteatamel