Ctrl + Alt + Assist: AI Agents Edition Bethany Jepchumba | @bethanyjep 

• Cloud Advocate (DevRel) at Microsoft • Works with NGOs & Communities, currently ICT Director at TOFA • I am a hobbist, I enjoy trying out new hobbies Today’s session: https://bethany-jep.com/events/agentcon-accra-2025/ Bethany Jepchumba Microsoft confidential 2 

Question: Have you built an agent before? 

The concept of LLMs…. 4 

How language models work Natural language input Model Tokens Probability distribution Natural language output Decoding + Post-processing Get results Pre-processing 

RAG, Agents and MCP Giving your LLMs some powers, sort of…. 6 

BYOD -> Retrieval-Augmented Generation (RAG) 

What is an AI agent? LLM Instructions Tools Agent + + An AI agent is a micro-service that takes unstructured messages, optionally invokes other APIs and returns messages/action 1 2 3 Input System events User messages Agent messages 1 Tool calls Knowledge Actions Memory 2 Output Agent messages Tool results 3 

Unlike previous tools, AI Agent can… reason, act and learn 

Showcasing Agent Framework 

No content

Foundry Local Design, customize and manage AI applications and agents on-device Preview Microsoft confidential 12 Installation Privacy and Security Latency Cost Efficient Low Bandwidth 

Showcasing Foundry Local + Edge AI for Beginners 

No content

What is Model Context Protocol (MCP)? Ship once – reach every MCP-enabled client in your organization A secure, open standard that lets AI agents access tools, APIs, and data through a consistent interface — without custom integrations. In Azure AI Foundry Agent Service, MCP connects agents to your enterprise and external systems with a single, reusable protocol — secured, governed, and observable by Azure. Clients • Foundry Agents • Copilot Studio • GitHub Copilot • VS Code Servers/ Tools • Azure Services (Search, Cosmos DB, Power Platform, etc.) • Third-party APIs (Salesforce, ServiceNow, REST APIs) • On-prem data MCP Protocol • One standard interface • Security: Entra ID • Observability: Azure Monitor 

MCP Model Context Protocol - Easier to give context to models - Optimized communication between LLMs, external tools, data sources and applications - Uses a client-server model for interactions 

Model Context Protocol At Microsoft Model Context Protocol MCP enables seamless integration between LLM Apps and external data sources Azure API Management Build Local & Remote MCPs MCP SDK Microsoft Connectors Azure Functions Tools Etc. … Community Windows Registry VS Code BYO Enterprise or MCP Clients & Host Copilot Studio Foundry agents Visual Studio Windows VS Code GitHub Copilot Entra ID 

Enhancing Security for MCP Servers Use credential manager to authorize access to your backend MCP servers Protect your remote MCP servers with OAuth 2.0 and API Management Blog aka.ms/remote-mcp-apim-auth-blog Generally Available Azure API Management Safety Policies Azure OpenAI MCP enabled Tools Azure Monitor Logs & Metrics … Credential Manager Agents … 

Azure AI Foundry Security • Identity • Management Foundry Models Foundry Agent Service Azure AI Search Foundry Observability Azure AI Services Azure Machine Learning Azure AI Content Safety Copilot Studio Visual Studio GitHub Foundry SDK Serverless Control Azure Kubernetes Service Azure Container Apps Azure App Service Azure Functions Cloud Azure Azure Arc Foundry Local Edge 

Showcasing MCP Demo 

AI Toolkit 

MCP Servers AI Toolkit 

No content

No content

No content

Security, monitoring and evaluations. Agents in production…. 27 

Generative AI Development Lifecycle Hypo thesis Find LLMs Try prompts Manage Customize Design Managing PREPARE FOR APP DEPLOYMENT ADVANCE PROJECT BUSINESS NEED Deploy LLM App/UI Quo ta and cost management REVERT PROJECT C ontent Filtering M onitoring SEND FEEDBACK Prompt Engine ering or Fine-tuning Evalua tion Exceptio n Handling Retrieval Augmente d Generation Prote ct & govern 

How do you typically test if a model or application is working as intended? 

Mitigation layers Application Platform 

• Violence • Sexual • Self-harm • Hate and Unfairness • Ungrounded Attributes • Code Vulnerability • Protected Materials • Content Safety Evaluators in Azure AI Foundry Safety & Security • Fluency • QA • Coherence • Intent Resolution • Task Adherence • Tool call accuracy General Purpose • Retrieval • Document Retrieval • Groundedness • Relevance • Response Completeness RAG Agents 

Evaluate agentic workflows User query “Weather tomorrow.” Intent resolution User wants to know the local weather and the time to forecast. Tool calls Call location and time API Call weather API Final response “The temperature will be 30 degrees. Rain will…” Intent resolution evaluation • User intent identification • Clarification for ambiguity Tool call accuracy evaluation • Correct tool call selection • Correct parameter extraction Final response evaluation • Task adherence • Response completeness Quality and Safety evaluation • RAG quality (Groundedness/Relevance/Retrieval) • Risk and Safety (Jailbreak/Code Vulnerabilities/Ungrounded Attributes 

Showcasing Demo: AI Evaluators 

No content

Agents Use Cases 

From english to xx language in a single click 

No content

No content

AI Agent workflows can enhance efficiency, accuracy, and customer satisfaction Key Use Cases Across Industries • Assists employees in booking business trips • Integrates with Tripadvisor, Outlook, and SharePoint • Books via Teams chat or email • Uses OCR to gather receipts • Automates expense report submission and tracking Travel Booking & Expense Management • Personalized onboarding assistant for new hires • Uses LLMs grounded in HR data from SharePoint • Provide relevant training materials • Schedule orientations and set up software accounts • Monitor task completion and ensure efficient onboarding Employee Onboarding • Diagnoses issues by referencing history and product manuals • Provides tailored solutions or escalates through automated workflows • Creates tickets and schedules follow-ups • Updates CRM records, enhancing future support Personalized Customer Support • Analytics data from data lake and data warehouse • Responds to user requests in natural language • Generates insights, visualization, and sends via Teams or email • Automates data handling for real-time, effortless decision-making Data Analytics and Reporting 

Resources - Learning resources on GenAI, agents and MCP - Getting started with Azure AI - Links to the different tools – foundry local, ai toolkit on VSCode, GitHub Models and GitHub Copilot https://bethany- jep.com/events/agentcon-accra-2025/ 

Thank You