Xebia-Deploying_Multi-Agent_Systems_on_Google_Cloud.pdf

Transcript

1. Deploying Multi-Agent Systems on Google Cloud Timothy van der Werf GCP & AI Consultant
2. 2 Introduction Timothy van der Werf GCP & AI Consultant
3. 3 Agenda 1. The Agentic Shift 2. Google’s Agentic Ecosystem 3. Building a Multi-Agent System 4. Deployment & Governance 5. Live Demo 6. Q&A
4. 4 Chapter 1 Agentspace & ADK
5. 5 What is an Agent? Workflow Workflows are systems ​ where LLMs and tools are orchestrated through predefined code paths.​ Agent Agents are systems where ​ LLMs dynamically direct their own processes and tool usage,​ maintaining control over how​ they accomplish tasks.​
6. 6 What is a Multi-Agent System? A system where multiple, specialized AI agents collaborate to solve complex problems. Autonomy Agents operate independently, making their own decisions to achieve goals. Specialization Each agent is an expert in a specific domain or task, like a researcher or an analyst. Collaboration Agents communicate, delegate tasks, and work together to solve problems that are beyond any single agent’s capability. Analogy: Think of an expert human team, where a project manager orchestrates the work of specialists.
7. 7 GCP’s Managed Features for Production Agents Core Infrastructure Serverless Scaling: Automatic scaling with Cloud Run and the Vertex AI Agent Engine. Managed Memory: Persistent, scalable memory for your agents with Vertex AI Memory Bank. Layered Enterprise Security: Protect against data exfiltration and threats with IAM, VPC Service Controls, and Model Armor. Enterprise Governance: Discover, manage, and govern your agents at scale with Agentspace and the Agent Gallery. Agent-Specific Tooling Built-in Observability: Track cost, performance, and quality with OpenTelemetry and Google Cloud’s operations suite. Factual Grounding: Improve factual accuracy and reduce hallucinations with Vertex AI Search grounding. Agnostic Evaluation: Evaluate agent quality and trajectory, including tool use, with the ADK’s framework. Simplified Deployments: Easily deploy your agents to GCP with the adk deploy CLI.
8. 8 Google Agentspace Agentspace is a unified, secure platform to build, manage, and adopt AI agents at scale. It’s a central hub for your enterprise’s knowledge and AI capabilities, with tools for everyone from business users to pro developers. Key Capabilities AI-Powered Chat & Search Get answers grounded in your enterprise data with a Gemini-powered chat experience. No-Code Agent Designer Empower business users to create custom agents without writing any code. Agent Gallery Discover, manage, and deploy pre-built and custom agents from a central marketplace. Enterprise Security & Compliance Benefit from Google Cloud’s secure infrastructure and compliance with standards like ISO 27001, HIPAA, and SOC.
9. 9 The Agent Development Kit (ADK) ADK is a flexible and modular framework for developing and deploying AI agents. While optimized for the Google ecosystem, it is model-agnostic, deployment-agnostic, and compatible with other frameworks. It’s designed to make agent development feel more like traditional software development, making it easier to create everything from simple single-agent tasks to complex, multi-agent workflows. Core Pillars Build: Define agent instructions and tools. Interact: Test agents in a CLI or web UI. Evaluate: Measure agent performance. Observe: Monitor cost, performance, and quality. Deploy: Package agents for the cloud.
10. 10 Chapter 2 Building a Custom Agent
11. 11 Scenario: "Personalized Content Curation Agent" A user wants a personalized digest of news and articles from their favorite online sources. Our Multi-Agent Design: 1. Coordinator Agent: The primary agent that manages the workflow and interacts with the user. 2. Content Fetcher Agent: A specialist for fetching the latest articles from a list of websites and RSS feeds. 3. Summarization & Ranking Agent: A specialist that summarizes and then ranks the articles based on the user’s preferences. Agents User Coordinator Agent For each source Content Fetcher Agent For each article Summarization & Ranking Agent Synthesize Results
12. 12 Step 1: The Content Fetcher Agent This agent’s only job is to use a tool to fetch content from a given URL. It’s a highly specialized and simple component. from adk.tools import tool from adk.agents import LlmAgent, Instruc import requests @tool def fetch_content(url: str) -> str: ''' Fetches the content from a given URL. ''' response = requests.get(url) return response.text fetcher_instruction = "You are " \ "a web content fetcher. " \ "Your only purpose is to use the " \ "available tools to fetch content" \ "from a given URL." fetcher_agent = LlmAgent( instruction=fetcher_instruction, tools=[fetch_content] )
13. 13 Step 2: The Summarization & Ranking Agent This agent takes the content of an article, summarizes it, and gives it a relevance score based on the user’s interests. summarize_and_rank_instruction = "You are a content analyst. "\ "Summarize the provided article and rank its " \ "relevance to a user interested in 'AI and Python'. " \ "Output in JSON format with 'summary' and 'relevance_score' fields." summarization_agent = LlmAgent( instruction=summarize_and_rank_instruction )
14. 14 Step 3: The Coordinator Agent This root agent orchestrates the process by delegating tasks to its specialist agents. It uses them as tools via ADK’s AgentTool class. from adk.tools import AgentTool # fetcher_agent and summarization_agent # are defined in previous steps coordinator_agent = LlmAgent( tools=[ AgentTool(agent=fetcher_agent), AgentTool(agent=summarization_agent), ] )
15. 15 Deploying Your Agent From Local to Production The ADK provides a seamless experience for testing your agents locally and deploying them to the cloud. Use adk web for a local UI to test and debug your agent. GCP Deployment Targets Vertex AI Agent Engine: A fully managed, serverless runtime optimized for agents. Google Cloud Run: A versatile, serverless platform for any containerized application. Google Kubernetes Engine (GKE): For maximum control and portability, deploy your agents to a GKE cluster.
16. 16 Deploying to Vertex AI Agent Engine Vertex AI Agent Engine is a fully managed, serverless runtime that is optimized for AI agents. It’s the recommended way to deploy your ADK agents to production, as it handles state, memory, and scalability for you. Deploying to Agent Engine is a simple, one-step process with the adk CLI. # 1. Set your project and location export GOOGLE_CLOUD_PROJECT="your-projec export GOOGLE_CLOUD_LOCATION="us-central # 2. Define the path to your agent code export AGENT_PATH="./agents/url_summariz # 3. Deploy the agent adk deploy agent_engine \ --project=$GOOGLE_CLOUD_PROJECT \ --region=$GOOGLE_CLOUD_LOCATION \ --service_name=url_summarization_agent $AGENT_PATH
17. 17 Adding the agent to Agenspace read -r -d '' HEADERS << EOF -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ -H "X-Goog-User-Project: ${PROJECT_NUMBER}" EOF export URL='https://discoveryengine.googleapis.com/v1alpha/projects/${PROJECT_NUMBER}/loca
18. 18 Adding the agent to Agenspace curl -X POST $HEADERS \ https://discoveryengine.googleapis.com/v1alpha/projects/${PROJECT_NUMBER}/locations/globa -d '{ "displayName": "'"${DISPLAY_NAME}"'", "description": "'"${DESCRIPTION}"'", "adk_agent_definition": { "tool_settings": { "tool_description": "'"${TOOL_DESCRIPTION}"'" }, "provisioned_reasoning_engine": { "reasoning_engine": "'"${REASONING_ENGINE}"'" }, "authorizations": [ "projects/'"${PROJECT_NUMBER}"'/locations/global/authorizations/'"${AUTH_ID}"'" ]
19. 19 Demo Show don’t tell
20. Agentspace query
21. Agentspace result
22. Agentspace result
23. Tracing
24. Thank You & Q&A Timothy van der Werf timothy.vanderwerf@xebia.com xebia.com
25. Xebia