Agents with Prompty and Azure AI Foundry Bethany Jepchumba Cloud Advocate, Microsoft @bethanyjep 

Agentic AI 

AI Agents Unlike existing automation, AI Agents can automate requests as soon as they come in by… Reasoning over the request Proactively retrieving more context Performing required actions with APIs 

What are agents? AI designed to perform a task Tasks can vary in level of complexity and capabilities depending on your need Simple Advanced Generation Generate summaries, images, audio, and more with an AI model and inputs. Retrieval Retrieve information from grounding data, reason, summarize, and answer user questions Action Take actions to automate workflows, and replace repetitive tasks for users 

With AI agents, you can make your processes faster than ever VALUE Yesterday – manual Forms over data Other apps HR manager New account New employee form Automation HR manager New employee API Automation Today – RAG Chatbots Tomorrow Automated agents Approve? New account Agent HR manager New account New employee form Automation Chatbot Improve employee productivity Improve process efficiency 

Agentic AI capabilities Agent Planning Memory Tools Action Web Search Code Execution Calculator Calendar ...more ReAct Self-critique Collaboration Subgoal decomposition ...many more design patterns 

Common Design Patterns RAG Agent Code Generation Agent Multi-agent Systems Multi-domain Agent Systems 

Retrieval Augmented Generation Agent Search Tools discussion D Intelligent RAG Agent Plan Query Observe Update plan Compile answer Knowledge Graph Translates questions into a research problem with human in the loop to produce high quality answers to complex questions within the scope of its domain question final answer 

Session and memory management Dynamic context look-up Planning & tracking Toolsets & coding interface Human interaction Coordinator Multi-turn reasoning and action (ReAct) Code Generation Agent Generates code based on natural language requirements, leveraging existing code base, templates, guidelines, libraries to match policies and best practices while interacting with humans to clarify, validate and deliver functionality as intended. Code Executor Constraints Existing Codebase Coding Guidelines (docs) Internal SDKs Dev Task (spec / bug / feature) Code w/ Tests, Doc, DevOps code, etc 

MS #3 MS #2 MS #1 Multi-Agent System A complex problem is decomposed into smaller, manageable parts, each addressed by specialized agents, effectively a micro-service (MS). These agents work together in a coordinated manner within a workflow to efficiently solve the overall problem. conversation data query docker Coding Guidelines history of work Critical Design Elements Adaptive planning within scope of existing tightly scoped skills (agents) Handles ambiguity by discussing and refining requirements with human Memory to handle complex long running execution of a plan Effective inter agent communications Test, monitor, release & maintain each agent independently to quickly handle quality & safety issues 

Multi-Domain Agents System Multiple domain-specific agents are orchestrated by an Agent Runner to scale across multiple domains while appearing as a single agent to users. Agent pool Agent Runner Active Agent Agent 1 Revaluate agent assignment Run Transfer Run + Back-off Role/goals + skills Agent 2 Run + Back-off Role/goals + skills Agent n Run + Back-off Role/goals + skills Shared context memory Critical Design Elements Agents capability descriptors Scalable Agent Runner able to manage 10s to 100s of agents Ability to manage domain switching with proper memory management Avoid single interceptor problem as individual agents maintain direct communication with user and can hand off when needed 

How does your agent work? User Travel Booking Agent “Help me book a trip to New York for a client meeting? I need to fly out next Monday and return on Friday.” Knowledge Sources (search, files, databases, storage etc.) Models (Azure OpenAI Service, Models-as-a- Service) Actions (Pre-built or custom tools to automate processes) “I’ve booked your trip to New York as requested. Here are details:…” 

How does your agent work? Step 1: Create an Agent Step 2: Create a Thread Step 3: Run the Agent Step 5: Check the Run status Step 6: Display the Agent’s Response Agent Travel Planning Agent Instructions You are a travel booking and expense management assistant designed to help employees plan, book, and manage business travel. Run 2 Model Tools (optional) File Search Code Interpreter Function Calling Bing Search Microsoft SharePoint (coming soon) Microsoft Fabric (coming soon) Azure Logic Apps (coming soon) Azure Functions OpenAPI 3.0 specified tools User’s message I need to book a hotel in New York for 2 stays. Agent’s message Here are some suggestions: Run 1 1 Use Tripadvisor API to search the nearest hotel Create message 2 Your data (optional) User’s message What’s the daily meal allowance for the business trip? Agent’s message The daily allowance for your business trip is $75, as per company policy. Use Microsoft SharePoint to query the company travel policy Create message 2 1 Thread Travel Planning Azure AI Search Files (local or Azure Blob) 

Model Catalog GitHub Models Use GitHub Models to find and experiment with AI models for free. Once you are ready to bring your application to production, you can switch to a token from a paid Azure account. Azure AI Foundry Model Catalog Discover and use a wide range of models for building generative AI applications. The model catalog features hundreds of models across model providers such as Azure OpenAI Service, Mistral, Meta, Cohere, NVIDIA, and Hugging Face, including models that Microsoft trained. 

Prompty as a language-agnostic prompt asset Easy to start Code first Intuitive to develop Multiple programming language will be supported Simplification deployment Agency with Observability Choose the framework 

Scenario A creative writing assistant that will help write well researched, product specific articles for Contoso Outdoor Company. 

Multi-agent creative writing assistant Processing services User input Azure Monitor Researcher Agent Azure OpenAI Service Azure OpenAI Service Azure OpenAI Service Grounding with Bing Azure AI Search Azure Managed Identity Azure Application Container (ACA) Response Azure AI Foundry Product Agent Writer Agent Editor Agent Azure AI Search Azure OpenAI Service https://aka.ms/ai-apps 

Tracing and debugging with Prompty benefits Provides a detailed view of the execution flow of the application Support both local and cloud runs Detail every LLM call including message, prompt, parameters, token, and latency Collect & attach user feedback using OpenTelemetry's semantic conventions 

AI Agent workflows can enhance efficiency, accuracy, and customer satisfaction Key Use Cases Across Industries • Assists employees in booking business trips • Integrates with Tripadvisor, Outlook, and SharePoint • Books via Teams chat or email • Uses OCR to gather receipts • Automates expense report submission and tracking Travel Booking & Expense Management • Personalized onboarding assistant for new hires • Uses LLMs grounded in HR data from SharePoint • Provide relevant training materials • Schedule orientations and set up software accounts • Monitor task completion and ensure efficient onboarding Employee Onboarding • Diagnoses issues by referencing history and product manuals • Provides tailored solutions or escalates through automated workflows • Creates tickets and schedules follow-ups • Updates CRM records, enhancing future support Personalized Customer Support • Analytics data from data lake and data warehouse • Responds to user requests in natural language • Generates insights, visualization, and sends via Teams or email • Automates data handling for real-time, effortless decision-making Data Analytics and Reporting 

Thank you Resources https://aka.ms/GenAI/LLMOps https://aka.ms/ai-apps https://aka.ms/azureai/discord Check out the workshop! Build your first agent with Azure AI Agent Service 

No content