Prompty, Azure AI Foundry and Agentic AI

In this session, I’ll share how I built a lightweight AI agent using Prompty and Azure OpenAI to automate GitHub issue tagging, one of the many small but time-consuming tasks in open source projects. You’ll get a behind-the-scenes look at how I use prompt engineering to streamline contributor experience, enhance repo hygiene, and turn repetitive maintenance work into intelligent automation. Whether you're managing community-driven projects or just exploring AI tools in DevRel, this talk will show you how agents can help you focus on what truly matters: empowering developers.

Transcript

1. Prompty, Azure AI Foundry & Agentic AI Bethany Jepchumba | @bethanyjep
2. And how AI agents can help… Your processes today
3. It looks like there might be three different issues that could lead to too much noise: Unbalanced Load If the clothes inside are unevenly distributed, especially during the spin cycle, the drum can wobble and cause loud banging or thumping sounds. Worn Drum Bearings Over time, the bearings that help the drum spin smoothly can wear out, leading to grinding or rumbling noises, particularly during spinning. Loose or Damaged Parts Items like shock absorbers, suspension springs, or drive belts can become loose or damaged, resulting in clunking, squealing, or rattling sounds. using generative AI for knowledge retrieval (aka RAG)… Before AI Agents Search: W100 manual A customer is having an issue with their washing machine W100 making too much noise; can you help me diagnose the issue? You should create a ticket for a technician to take a look Customer Customer Support Emails, calls, or messages Searches internal knowledge base
4. But most downstream processes are still brittle and require meaningful human involvement Before AI Agents Customer Customer Support Emails, calls, or messages Ticket form Creates ticket Database API RPA Automation Sends notification Searches internal knowledge base
5. But most downstream processes are still brittle and require meaningful human involvement Before AI Agents 6 Automation cannot self- heal or handle novel cases RPA breaks whenever UI is updated or changed wasted time structuring data Teams must constantly monitor existing RPA scripts to ensure they do not break. Nearly all automation today starts with form over data which is slow and error prone. Complex logic is needed to handle unexpected edge cases within automation systems.
6. LLMs update RPAs if the UI updates or changes Tool calling LLMs can self- heal or handle novel cases Here you go! Would you like me to submit the JSON to the API? With AI Agents, you can make your automation more resilient while removing the drudgery from work After AI Agents 7 { "name" : "John Clar "issue": ”Washing m } Convert the customer conversation into a JSON that looks like this… Transcript Click on the submit button for the scheduling form Looks like the ID of the order form is submit-order, I’ll go ahead and click it now. Submit form The form has been successfully submitted, is there anything else you need? Assign technician LLMs spend little time structuring data Update work order Let me try again with an available technician. Update work order Alex declined because they're on leave Successfully assigned Lindsay
7. Agentic AI
8. AI Agents Unlike existing automation, AI Agents can automate requests as soon as they come in by… Reasoning over the request Proactively retrieving more context Performing required actions with APIs
9. What is an AI agent? LLM Instructions Tools Agent + + An AI agent is a micro-service that takes unstructured messages, optionally invokes other APIs and returns messages 1 2 3 Input System events User messages Agent messages 1 Tool calls Knowledge Actions Memory 2 Output Agent messages Tool results 3
10. Unlike previous tools, AI Agent can… reason, act and learn
11. Agentic patterns Acting aka tool calling Learning aka memory Reasoning aka planning I’ve captured the underlying issue in the details { "name" : "John Clar "issue": ”Washing m Create a JSON describing the user issues Check logs Machine is past due for maintenance Go beyond transcriptions and slot filling by allowing agents to deduce the why I’ll look at the logs to see what happened Transcript Fill out and submit the workorder Populate field: name The form has been successfully Agents can take a simple request and chain multiple tool calls to complete the ask Only make a mistake once. With memory, agents can recall prior experiences Populate field: issue Name field populated Issue field populated Submit form Update work order Let me try again with an available technician. Update work order Alex is already booked at this same time Successfully assigned technician Assign technician Saved memory: Alex is OOF next week
12. An AI agent in action… LLM Instructions Tools Agent User: My washer machine is making weird noises Agent: Recall user washing machine Tool: User has w100 with sn: 142352 Agent: I’ll see what could be causing your w100 issues Agent: Search manuals Agent: It looks like there are three common reasons Agent: I’ll check your machine if it has any notices Agent: Check logs for 142352 Agent: It looks like your machine is due for maintenance and is still under warranty. Want me to schedule an appointment? User: Sure! Let’s target early next week Agent: Look for available times Agent: Would Monday at 1:00pm work? User: Let’s do it! Agent: Create workorder Agent: Done! Is there anything else I could help you with? GPT-4o You’re responsible for helping customers with their appliances Appliance manuals Check appliance logs Create workorder Recall user details
13. Azure AI Foundry Security • Identity • Management Foundry Models Foundry Agent Service Azure AI Search Foundry Observability Azure AI Services Azure Machine Learning Azure AI Content Safety Copilot Studio Visual Studio GitHub Foundry SDK Serverless Control Azure Kubernetes Service Azure Container Apps Azure App Service Azure Functions Cloud Azure Azure Arc Foundry Local Edge
14. Foundry Agent Service Combine the best models, services, and tools in Azure AI Foundry into reusable, testable agentic components. Azure AI Foundry Foundry Models Hosted & sold by Microsoft & others Fine-tuned models BYO-models Azure AI Content Safety Content filters Cost management Governance Enterprise Security Development frameworks and toolkits Foundry SDK Semantic Kernel AutoGen LangGraph LangChain LlamaIndex CrewAI And more… Agent integrations Assistants API Activity protocol A2A Agent protocol Knowledge & Tools OpenAPI tools MCP tools Bing Azure AI Search SharePoint Microsoft Fabric Logic Apps Azure Functions Foundry Observability Tracing Logging Monitoring Evaluation Experimentation Foundry Agent Service Managed orchestration Managed threads Network isolation OBO-auth integration
15. The Agent Factory Trust Customization Observability Orchestration 2 5 4 6 3 Knowledge & Tools Models 1 Azure AI Foundry
16. 1 Models Model choice Don’t lock yourself into a single model provider Azure OpenAI Service o3-mini, o3, o4-mini, GPT- 4.1, 4o, etc Models-as-a-Service Llama 3.1-405B-Instruct Mistral Large Cohere-Command-R-Plus
17. 1 2 Customization Models Customization Use the data in your threads to fine-tune and distill models
18. Knowledge and tools 1 2 3 Customization Models AI tools Give your agents the skills to retrieve data and act Knowledge Actions Logic Apps OpenAPI Azure functions MCP* Connected agents A2A* Agent service Copilot Studio
19. Knowledge and tools Orchestration 1 2 3 4 Customization Orchestration We’ll automatically invoke tools and store results in your thread Agent Tool calls Knowledge Actions Memory
20. Knowledge and tools Orchestration Trust 1 2 3 4 5 Trust Reduce risk by controlling your data and by leveraging RAI ▪ BYO-thread storage ▪ BYO-file storage ▪ BYO-search index ▪ BYO-virtual network ▪ OBO authentication ▪ Content filtering
21. Knowledge and tools Orchestration Trust Observability 1 2 3 4 5 6 Observability Watch everything your agent does within a thread
22. Create a code generator Agent
23. Scenario An open-source maintainer for Prompty looking for a way to simplify tagging issues.
25. Introduction to Prompty
26. Prompty as a language-agnostic prompt asset Specification Prompty is intended to be a language agnostic asset class for creating prompts and managing the responses. The goal is to simplify your workflow by creating a standard that can be used by any language, any framework, and any tool to create a prompt and manage the response. Prompty can be imported to/exported from Azure AI Studio for smooth transition between local and cloud. Prompty assets can be also shared across organizations.
27. Prompty Tooling in Visual Studio Code Tooling Visual Studio Code Extension • Frontmatter Autocompletion • Colorization / Syntax Highlighting • Validation • Quick Run • Code Generation • Evaluations – Future • ASK US
28. Prompty Runtime Library Runtime Prompty runtime is the whatever engine that understands and can execute the format. • Azure AI Foundry • Python – LangChain • C# – Semantic Kernel • TypeScript/JavaScript – Extension • Your Language – Implement your own!
29. Create your first Prompty
30. Multi-Agent Orchestration
31. Typical Multi-Agent Architectures Network Maker-Checker worker reviewer Supervisor Custom Hierarchical Hand-Off
32. Connected Agents Multi Agent Workflow Give one agent the abilities of another… Declaratively orchestrate multiple agents together Powered by Semantic Kernel, Deployed in Foundry Agent Service Multi-agent capabilities in Foundry Agent Service In Foundry Agent Service
33. And you can integrate agents into your existing apps with our Agent frameworks * Available with Semantic Kernel ✧ Coming soon Your app Foundry SDK Semantic Kernel AutoGen Azure AI Foundry Agent Service
34. https://aka.ms/ai-agents-prompty Resources and code!
35. https://aka.ms/aitour/WRK552 Resources and code!
36. https://aka.ms/ai-agents-beginners Resources and code!
37. https://aka.ms/azureaifoundry/discord Resources and code!
38. https://aka.ms/azureaifoundry/forum Let’s chat!
39. Thank you. Bethany Jepchumba | @bethanyjep
40. © Copyright Microsoft Corporation. All rights reserved.