Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Choreography vs Orchestration in microservices and best practices

Choreography vs Orchestration in microservices and best practices

We went from a single monolith to a set of microservices that are small, lightweight, and easy to implement. Microservices enable reusability, make it easier to change and scale apps on demand but they also introduce new problems. How do microservices interact with each other toward a common goal? How do you figure out what went wrong when a business process composed of several microservices fails? Should there be a central orchestrator controlling all interactions between services or should each service work independently, in a loosely coupled way, and only interact through shared events? In this talk, we’ll explore the Choreography vs Orchestration question and see demos of some of the tools that can help. And we'll explore some best practices and patterns to apply when adopting an orchestration approach.

Guillaume Laforge

October 20, 2022
Tweet

More Decks by Guillaume Laforge

Other Decks in Technology

Transcript

  1. Choreography vs
    Orchestration
    in microservices & best practices
    Mete Atamel
    Developer Advocate at Google
    @meteatamel
    atamel.dev
    speakerdeck.com/meteatamel
    Guillaume Laforge
    Developer Advocate at Google
    @glaforge
    glaforge.appspot.com
    speakerdeck.com/glaforge

    View full-size slide

  2. Choreography
    vs Orchestration

    View full-size slide

  3. How do you organize
    a group of microservices
    to cooperate towards
    a common goal?

    View full-size slide

  4. Option 1: Direct service-to-service calls
    Services calling each other directly
    Frontend
    App Engine
    Order request
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Shipper
    Cloud Functions
    Prepare & ship items
    Notifier
    Cloud Run
    Notify user

    View full-size slide

  5. Direct service-to-service calls
    Pros
    ➕ Easy to implement: Services
    simply call each other
    Cons
    ➖ Too much coupling
    ➖ Each service can be a single point
    of failure
    ➖ Each service needs its own error /
    retry / timeout logic
    ➖ Who ensures the whole
    transaction is successful? (hint:
    saga pattern)

    View full-size slide

  6. Option 2: Indirect via events (choreography)
    Event-driven services
    Frontend
    App Engine
    Order request
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Shipper
    Cloud Functions
    Prepare & ship items
    Notifier
    Cloud Run
    Notify user
    Message Broker
    Google Cloud: Pub/Sub, Eventarc
    AWS: SQS, SNS, EventBridge
    Azure: Event Grid, Event Hubs, Service Bus
    Other: Kafka, Pulsar, Solace PubSub+, RabbitMQ, NATS...

    View full-size slide

  7. Indirect via events
    Pros
    ➕ Services are loosely coupled
    ➕ Services can be changed/scaled
    independently
    ➕ No single point of failure
    ➕ Events are useful to extend the
    system
    Cons
    ➖ Difficult to monitor
    ➖ Errors / retries / timeouts are hard
    ➖ The business flow is not captured
    explicitly
    ➖ Who ensures the whole transaction
    is successful?

    View full-size slide

  8. Imagine a more complex scenario

    View full-size slide

  9. Option 3: A central orchestrator
    Orchestrated services
    Frontend
    App Engine
    Order request
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Shipper
    Cloud Functions
    Prepare & ship items
    Notifier
    Cloud Run
    Notify user
    Orchestrator
    Google Cloud: Workflows, Cloud Composer
    AWS: Step Functions
    Azure: Logic Apps
    Other: CNCF Serverless Workflow, Apache Airflow, Camel, Camunda…

    View full-size slide

  10. A central orchestrator
    Pros
    ➕ Business flow captured centrally,
    source controlled, versioned etc.
    ➕ Each step can be monitored
    ➕ Errors / retries / timeouts can be
    centralized
    ➕ Services are still independent
    Cons
    ➖ A new orchestrator service to learn
    and maintain
    ➖ Orchestrator could be a single point
    of failure
    ➖ Loss of eventing flexibility
    ➖ How do you compensate for failed
    steps? (hint: saga pattern)

    View full-size slide

  11. Choreography or Orchestration?
    It depends!
    (We’ll revisit this question later)

    View full-size slide

  12. Orchestration

    View full-size slide

  13. CNCF Serverless Workflow
    serverlessworkflow.io
    Defines a vendor-neutral, open-source, and fully community-driven
    ecosystem for defining and running DSL-based workflows that
    target the Serverless technology domain
    ● Specification for defining DSL-based workflows
    ● Developer SDKs for different programming languages
    ● Workflow runtimes supporting the specification
    ● Developer tooling support for writing DSL-based workflows

    View full-size slide

  14. CNCF Serverless Workflow
    serverlessworkflow.io
    Workflow projects need to implement
    & support the spec
    Spec doesn’t necessarily cover all
    aspects of a product and not all
    products cover the whole specification
    Services need to be described with
    OpenAPI, events with CloudEvents
    Product Spec

    View full-size slide

  15. Serverless
    Compute
    External
    API’s
    Google
    API’s
    etc...
    Workflows - orchestrate & integrate
    SaaS
    API’s
    Private
    API’s
    Other
    Clouds
    Google Cloud
    Workflows

    View full-size slide

  16. Syntax
    elements

    View full-size slide

  17. Proprietary + Confidential
    Sequences of steps
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    - processPayment:
    call: http.post
    args:
    url: https://payment-processor.run.app/...
    body:
    input: ${paymentDetails}
    result: processResult
    - shipItems:
    call: http.post
    args:
    url: https://.../cloudfunctions.net/ship
    body:
    address: ${processResult.body.address}
    result: shipResult
    - notifyUser:
    call: http.post
    ...

    View full-size slide

  18. Proprietary + Confidential
    Variable passing &
    JSON parsing
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    - processPayment:
    call: http.post
    args:
    url: https://payment-processor.run.app/...
    body:
    input: ${paymentDetails}
    result: processResult
    - shipItems:
    call: http.post
    args:
    url: https://.../cloudfunctions.net/ship
    body:
    address: ${processResult.body.address}
    result: shipResult
    - notifyUser:
    call: http.post
    ...

    View full-size slide

  19. Proprietary + Confidential
    Calling HTTP APIs
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    - processPayment:
    call: http.post
    args:
    url: https://payment-processor.run.app/...
    body:
    input: ${paymentDetails}
    result: processResult
    - shipItems:
    call: http.post
    args:
    url: https://.../cloudfunctions.net/ship
    body:
    address: ${processResult.body.address}
    result: shipResult
    - notifyUser:
    call: http.post
    ...

    View full-size slide

  20. Proprietary + Confidential
    Authentication
    (OAuth2 | OIDC)
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    - processPayment:
    call: http.post
    args:
    url: https://payment-processor.run.app/...
    body:
    input: ${paymentDetails}
    auth:
    type: OIDC
    result: processResult
    ...
    AUTHENTICATION

    View full-size slide

  21. Proprietary + Confidential
    Pause
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    WAIT
    - pause:
    call: sys.sleep
    args:
    seconds: 60

    View full-size slide

  22. Proprietary + Confidential
    Logging
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    LOG
    - log-processed:
    call: sys.log
    args:
    text: "Payment processed"
    severity: INFO

    View full-size slide

  23. Proprietary + Confidential
    base64
    ● encode
    ● decode
    text
    ● encode
    ● decode
    ● find_all
    ● find_all_regex
    ● match_regex
    ● replace_all
    ● replace_all_regex
    ● split
    ● substring
    ● to_lower
    ● to_upper
    ● url_encode
    ● url_encode_plus
    Built-in functions
    http
    ● get
    ● post
    ● put
    ● patch
    ● delete
    ● request
    ● default_retry_*
    sys
    ● get_env
    ● sleep
    ● slee_until
    ● now
    ● log
    time
    ● fomart
    ● parse
    retry
    ● always
    ● default_backoff
    ● never
    errors
    ● type_error
    ● value_error
    ● index_error
    ● key_error
    ● not_implemented_error
    ● recursion_error
    ● zero_division_error
    ● system_error
    ● timeout_error
    ● resource_limit_error
    json
    ● encode
    ● encode_to_string
    ● decode
    events
    ● await_callback
    ● Create_callback_endpoint
    math
    ● abs
    ● max
    ● min
    list
    ● concat
    map
    ● get

    View full-size slide

  24. Proprietary + Confidential
    Error handling,
    conditionals, jumps
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    - processPayment:
    try:
    call: http.post
    args:
    url: https://payment-processor.run.app/...
    body:
    input: ${paymentDetails}
    result: processResult
    except:
    as: e
    steps:
    - known_errors:
    switch:
    - condition: ${not("HttpError" in e.tags)}
    next: connectionError
    - condition: ${e.code == 404}
    return: "Sorry, URL wasn't found."
    - unhandled_exception:
    raise: ${e}
    ERROR
    CHECKING

    View full-size slide

  25. Proprietary + Confidential
    Retry & backoff
    Payment Processor
    Cloud Run
    Authorize & charge CC
    Notifier
    Cloud Run
    Notify user
    Shipper
    Cloud Functions
    Prepare & ship items
    - processPayment:
    try:
    call: http.post
    args:
    url: https://payment-processor.run.app/...
    body:
    input: ${paymentDetails}
    result: processResult
    retry:
    max_retries: 5
    backoff:
    initial_delay: 1
    max_delay: 60
    multiplier: 2
    MAX: 5 times
    BACKOFF

    View full-size slide

  26. Proprietary + Confidential
    Like programming language
    subroutines or functions
    Subworkflows main:
    steps:
    - call_fullname:
    call: get_fullname
    args:
    first_name: "Sherlock"
    last_name: "Holmes"
    result: output
    - return_message:
    return: ${output}
    get_fullname:
    params: [first_name, last_name]
    steps:
    - prepMessage:
    return: ${first_name + " " + last_name}

    View full-size slide

  27. Proprietary + Confidential
    Connectors - runQuery:
    call: googleapis.bigquery.v2.jobs.query
    args:
    projectId: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
    body:
    useLegacySql: false
    useQueryCache: false
    timeoutMs: 30000
    # Find top 100 titles with most views on Wikipedia
    query: ${
    "SELECT TITLE, SUM(views)
    FROM `bigquery-samples.wikipedia_pageviews."+table+"`
    WHERE LENGTH(TITLE) > 10
    GROUP BY TITLE
    ORDER BY SUM(VIEWS) DESC
    LIMIT 100"
    }
    result: queryResult
    Make it easier to work with
    certain Google Cloud services
    and APIs, like transparently
    waiting for long-running
    operations.

    View full-size slide

  28. Proprietary + Confidential
    Dataflow
    Document AI
    Firestore
    Google Forms ßeta
    Natural Language
    Machine Learning ßeta
    Pub/Sub
    Cloud Run
    Secret Manager
    BigQuery
    BigQuery Data Transfer ßeta
    Cloud Build
    Cloud Functions
    Cloud Resource Manager ßeta
    Cloud Scheduler
    Cloud Tasks
    Compute
    Container
    Connectors: currently available, but more to come…
    Google Sheets ßeta
    Spanner
    SQL Admin
    Cloud Storage
    Cloud Storage Transfer
    Translate
    Workflows Executions
    Workflows

    View full-size slide

  29. Proprietary + Confidential
    Iterations
    - init:
    assign:
    - results: {}
    - tables:
    - 202202h
    - 202203h
    - 202204h
    - 202205h
    - runQueries:
    for:
    value: table
    index: table_idx
    in: ${tables}
    steps:
    - runQuery:
    call: googleapis.bigquery.v2.jobs.query
    args: …
    Loop over lists, maps, and over
    a range of values.
    Optional index.
    Use break and continue to
    short circuit the loop, with a
    next jump.

    View full-size slide

  30. Proprietary + Confidential
    Parallel iterations
    - init:
    assign:
    - results: {}
    - tables:
    - 202202h
    - 202203h
    - 202204h
    - 202205h
    - runQueries:
    parallel:
    shared: [results]
    for:
    value: table
    index: table_idx
    in: ${tables}
    steps:
    - runQuery:
    call: googleapis.bigquery.v2.jobs.query
    args: …
    Add a parallel block to
    parallelize iterations.
    Specify shared to list the
    variables that can be accessed
    with write-access in parallel.

    View full-size slide

  31. Proprietary + Confidential
    Parallel steps - parallelSteps:
    parallel:
    shared: [metadataResp, indexResp, collageResp]
    branches:
    - storeMetadataBranch:
    steps:
    - storeMetadata:
    call: http.request
    args: …
    result: metadataResp
    - indexPictureMetadataBranch:
    steps:
    - indexPictureMetadata:
    call: http.post
    args: …
    result: indexResp
    - collageCallBranch:
    steps:
    - collageCall:
    call: http.get
    args: …
    result: collageResp
    Branches of steps can be run
    in parallel, with the parallel
    keyword, and variables can be
    shared with write access with
    the shared keyword.

    View full-size slide

  32. Proprietary + Confidential
    Gcloud commands
    # Deploy a workflow
    gcloud workflows deploy my-workflow \
    --source=workflow.yaml
    # Execute a workflow
    gcloud workflows execute my-workflow
    # See the result
    gcloud workflows executions \
    describe \
    --workflow my-workflow
    Deploy and execute
    a workflow.
    Inspect the result of
    the execution
    of a workflow.

    View full-size slide

  33. Patterns &
    Best Practices

    View full-size slide

  34. Make a conscious choice
    Event-driven architecture
    ➔ Services are not closely
    related
    ➔ Services are not executed
    in parallel or in no certain
    order
    ➔ Services can exist in
    different bounded
    contexts
    Central Orchestrator
    ➔ Services are closely related
    ➔ Services are usually deployed
    and executed in the same
    order
    ➔ Can you describe the
    architecture in a flow chart?
    Direct Calls
    ➔ A simple architecture
    with a handful of
    services that do not
    change that often

    View full-size slide

  35. Event-driven orchestration
    github.com/GoogleCloudPlatform/eventarc-samples/tree/main/processing-pipelines/image-v3

    View full-size slide

  36. Handle errors with retries and saga pattern
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/retries-and-saga

    View full-size slide

  37. Wait for HTTP/event callbacks instead of polling
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-translation
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-event

    View full-size slide

  38. Parallelize when you can
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/bigquery-parallel
    Orchestration usually involves steps run sequentially one after
    another. Try to parallelize those steps when you can.
    Example: running BigQuery jobs against Wikipedia dataset with
    Workflows:
    ● Serial: 5 queries run sequentially each 20 seconds: Total 1 min
    ● Parallel: 5 queries run in parallel: Total 20 seconds

    View full-size slide

  39. Combine serverful workloads
    with serverless orchestration
    Sometimes you can’t use serverless due to some limitation
    (time, memory, CPU)
    Instead you use a Virtual Machine (VM)
    with the configuration you need
    Automate the VM lifecycle with an orchestrator
    to have a serverless experience
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/long-running-container

    View full-size slide

  40. Manage long running batch jobs
    with serverless orchestration
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/screenshot-jobs
    github.com/GoogleCloudPlatform/batch-samples/tree/main/primegen

    View full-size slide

  41. Use GitOps to manage orchestration lifecycle
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/gitops

    View full-size slide

  42. Plan for multi-environment
    deployments
    github.com/GoogleCloudPlatform/workflows-demos/tree/master/multi-env-deployment

    View full-size slide

  43. Workflows
    cloud.google.com/workflows
    Workflows Demos
    github.com/GoogleCloudPlatform/workflows-demos
    Codelab: Intro to serverless
    orchestration with Workflows
    codelabs.developers.google.com/codelabs/cloud-workflows-intro
    Thank you!
    Guillaume Laforge
    @glaforge
    glaforge.appspot.com
    speakerdeck.com/glaforge
    Mete Atamel
    @meteatamel
    atamel.dev
    speakerdeck.com/meteatamel

    View full-size slide