Building AI Apps with the Azure AI Agent Service

This is the 15-minute "intro talk" used to set the stage for the 45-minute instructor-guided version of the workshop found here:
https://aka.ms/aitour/WRK552

This workshop helped educate the audience on the capabilities of the Azure AI Agent Service with a hands-on workshop that helped them learn how to build a sales agent application with a rich set of supported tools (and related instructions)
---

Delivered as the signature workshop for this Microsoft event:
NYC | Architecture Recipes for AI-Powered Applications
https://developer.microsoft.com/en-us/reactor/events/24784/?wt.mc_id=eventspg_24784_webpage_reactor

Transcript

1. 1 5 - M I N I N T R O D U C T I O N T O B U I L D I N G A I A P P S W I T H T H E A Z U R E A I A G E N T S E R V I C E NITYA NARASIMHAN, PHD MARCH 20, 2025 https://aka.ms/aitour-agents-theater-session
2. 1. What is an AI Agent? 2. Why use the Azure AI Agent Service? 3. Scenario: Contoso Sales Assistant 4. Setup: Create My First Agent 5. Action: Function Calling (SQL query) 6. Action: Code Interpreter (Python) 7. Knowledge: Document (File Search) 8. Knowledge: Real-Time (Bing Search)
3. Build Your First agent with Azure AI Agent Service A 75-minute self-guided workshop
4. What is an AI Agent? What Can I Use Them For? An AI Agent is semi-autonomous software designed to achieve a given goal without requiring predefined steps or processes. Instead, it determines what to do dynamically. In the Azure AI Foundry platform, an AI Agent acts as a "smart" microservice that can be used to answer questions (RAG), perform actions, or completely automate workflows. Show: Foundry Portal https://aka.ms/aitour/ai-agents-theater-session
5. What is the Azure AI Agent Service? Why use the Azure AI Agent Service? It is a fully managed service for developers to securely build, deploy, and scale high-quality, and extensible AI agents without having to worry about compute & storage. Streamlined and secure way to build AI agents with automatic tool calling (server-side), securely managed data (conversation state) and rich built-in tools for data, actions, RAG. Show: Documentation https://aka.ms/aitour/ai-agents-theater-session
6. Use case Working for an e-commerce selling camping equipment Want to analyze sales data
7. Scenario: Contoso Sales Assistant Development: Code-First or Low Code A conversational agent designed to answer questions about sales data, generate charts for understanding, and download data files for reporting and analysis – for an enterprise. Create tailored AI agents by using custom instructions, custom functions, and tool-driven augmentations – using either the UI (portal) or the SDK (Python, C#, JS/TS) Show: AI Tour Workshop https://aka.ms/aitour/ai-agents-theater-session
8. Provision: An Azure AI Foundry Project Create: My Contoso Sales Assistant Configure project client with connection string. Then use it to create agent with name, model, tools and instructions and specify any tool-related resources required. Show: Azure AI Project A standard agent using Managed Identity for authentication – with key vault, storage, and AI Services (e.g., AI model deployment) along with AI hub and project resources. https://aka.ms/aitour/ai-agents-theater-session
9. Action Tool: Function Calling Use Case: Query My SQLite Database Function calling enables LLMs to interact with external systems, execute tasks, and integrate with APIs. Define a function, add it Function Tool toolset. It gets invoked for you. Enable function logic to execute dynamic SQL queries against a SQLite database in 3 steps. Define function, add function to agent toolset, update agent instructions to trigger tool use. Demo 1: Answer Query https://aka.ms/aitour/ai-agents-theater-session
10. Action Tool: Code Interpreter (Python) Use Case: Visualize result as a chart Code interpreter allows agents to write and run Python code in a sandboxed execution environment. Agents can be configured to try different iterations (on failure) till it succeeds. ”Show sales by region as a pie chart” – first runs basic function call to get SQL results, then writes Python code to visualize data and store it in a local file for user viewing. Demo 2: Create Charts https://aka.ms/aitour/ai-agents-theater-session
11. Knowledge Tool: File Search Use Case: Grounding with Documents ”Chat with your data” – the agent can answer questions on information found in uploaded files. It uses built-in vector store capability to parse and index information for later retrieval. Uses the File Search tool to retrieve relevant information to enable RAG-style search on a PDF file with product data. Tool uses a vector store with built-in semantic search capability. Demo 3: Use PDF Data https://aka.ms/aitour/ai-agents-theater-session
12. Knowledge Tool: … Bing Search Use Case: Grounding in Real Time data The Grounding with Bing Search tool allows you to incorporate real-time public web data in returned responses. Create the resource & connect it. Agent decides if & when to use it Provide instructions to agent to use tool for competitive insights and present concisely. We can’t access raw content but use and display requirements include query link & citations. Demo 4: Search Results https://aka.ms/aitour/ai-agents-theater-session
13. AI Agent Service in Action Step 1: Create an Agent Step 2: Create a Thread Step 3: Run the Agent Step 5: Check the Run status Step 6: Display the Agent’s Response Agent Contoso Sales Agent Instructions: You are an advanced sales analysis agent for Contoso, specializing in assisting users with sales data inquiries Run 2 Model Tools (optional) File Search Code Interpreter Function Calling Bing Search Microsoft SharePoint Microsoft Fabric Azure AI Search Azure Logic Apps Azure Functions OpenAPI 3.0 specified tools User’s message Tell me the total sales by region Agent’s message Here is the sales: Europe: $15478.00 America: $78792.00 Run 1 1 Function Calling Tool Query SQLite DB Create message 2 Your data (optional) User’s message Show as a pie chart Agent’s message Code Interpreter Tool Create a pie chart Create message 2 1 Thread Sales analysis Azure AI Search Files (local or Azure Blob)
14. B U I L D I N G A I A P P S W I T H T H E A Z U R E A I A G E N T S E R V I C E STAR THIS REPO Found this useful? Star the repo and check back for the slides, visual and more learning resources!
15. Open-Source Curriculum Self-Guided Workshop https://aka.ms/aitour/wrk552 https://aka.ms/ai-agents-beginners Self-Guided Learning