Workflows: orchestrate & automate API services with serverless workflows
- Workflows at a glance, benefits, key features, use cases
- UI interface in Google Cloud console
- Deep dive into the Workflows syntax
- Workflows connectors
- Demos
Proprietary + Confidential Orchestrate work across any services & APIs you use Easy-to-use workflow orchestration managing the work across Google Cloud products or any HTTP-based APIs, including SaaS or private APIs. Workflows — Benefits Serverless scalability and managed infrastructure Focus on modeling your workflow logic and let Workflows completely manage the infrastructure with rapid scaling. Pay-per-use pricing model Pay only if your workflows run: scale your costs down to zero during times of inactivity.
Proprietary + Confidential Define workflows with a YAML or JSON syntax. Visual representation of your workflows. Expression formulas supporting decision points, conditional step executions, and operations on variables. Passing information between steps with built-in JSON parsing and expression-based variable manipulations. Workflow definition and visualisation Built-in decisions and conditional step executions Passing variable values between workflow steps Workflows — Features
Proprietary + Confidential Reliable workflow execution Low latency of execution Workflows — Features Execute workflows with reliability required for enterprise and line-of-business applications. Fast scheduling of workflow executions and transitions between tasks. Predictable performance with no cold starts.
Proprietary + Confidential Built-in authentication for Google Cloud products Support for external API calls Built-in error handling Workflows — Features Orchestrate work of any Google Cloud product without worrying about authentication. Use a proper service account and let Workflows do the rest. Out-of-the-box support for calls to API endpoints outside of Google Cloud. Out of the box error handling for your workflow steps with configurable retry policies.
Proprietary + Confidential Workflows — Use cases Reliable transactions Low-latency, conditional processes with 3rd party integration IT infrastructure automation
Proprietary + Confidential Gcloud commands # Deploy a workflow gcloud workflows deploy my-workflow \ --source=workflow.yaml # Execute a workflow gcloud workflows execute my-workflow # See the result gcloud workflows executions \ describe \ --workflow my-workflow Deploy and execute a workflow. Inspect the result of the execution of a workflow.
Proprietary + Confidential Simplifies access to Google Cloud products within a workflow: ● No need to tweak the URLs to call, or specify authentication ● Transparent handling of errors and retries (improves reliability and service SLA through retries) ● Handles long-running operations (transparent polling till result is ready, using a backoff) Connectors
Proprietary + Confidential Stopping a Compute Engine VM with the connector No need to poll: the connector waits for the end of a “long running operation” Connector example - stop_machine: call: googleapis.compute.v1.instances.stop args: instance: ${instanceName} project: ${project} zone: ${zone} # Optional connector parameters connector_params: timeout: 100 polling_policy: initial_delay: 1 multiplier: 1.25
Proprietary + Confidential ● How to invoke a workflow execution programmatically ○ Using the multi-language client libraries ● How to schedule a workflow execution ○ Thanks to Cloud Scheduler ● How to use functions to palliate the lack of expressiveness ○ Taking advantage of Cloud Functions for logic impossible with Workflows ● How to access secrets from Secret Manager ○ Avoid hard-coding secrets ● How to send a Pub/Sub message Workflows in action!
Proprietary + Confidential Márton Kodok (Google Developer Expert) shared concrete use cases with Workflows https://martonkodok.medium.com/ ● Automate the execution of BigQuery queries with Cloud Workflows ● Firestore backups the easy way with Cloud Workflows ● Run shell commands and orchestrate Compute Engine VMs with Cloud Workflows ● Using Cloud Workflows to load Cloud Storage files into BigQuery Concrete examples from the Community