Scaling Open-Source documentation with AI.

Maintaining high-quality documentation is essential for any open-source project, but keeping it accurate, up-to-date, and efficient can be challenging. In this session, I will share insights from revamping the documentation for Prompty.ai, an open-source project, and how we leveraged GitHub Copilot to streamline the process. I’ll walk through practical ways AI-assisted documentation improved accuracy, accelerated content creation, and helped us prepare for Version 2 of our docs. Whether you're a developer, technical writer, or open-source contributor, this talk will provide actionable strategies for integrating AI into your documentation workflow.

Transcript

1. Scaling Open-Source documentation with AI Bethany Jepchumba | @bethanyjep
2. Would you trust an AI to …?
3. Pick a movie or book you’d enjoy?
4. Drive you in busy Thika Road Traffic?
5. Recommend your next career move?
6. Assist you in coding?
7. Write and send emails?
8. Write documentation?
9. Documentation in OSS
10. - Documentation as a Service - Agents as a Service - Translation as a Service - Questions?
11. What we want - Clear, accurate, and up-to-date documentation The struggle? - maintaining robust documentation at scale is often resource-intensive, manual, and challenging
12. Automating and streamlining workflows… How can we integrate AI to how we create, maintain & translate OSS documentation?
13. Documentation as a Service
14. Prompty  https://github.com/microsoft /prompty
15. What we want - Clear, accurate, and up-to-date documentation The struggle? - One maintainer - Widely used across presentations - Not enough technical depth on the project
16. Prompty as a language-agnostic prompt asset Specification Prompty is intended to be a language agnostic asset class for creating prompts and managing the responses. The goal is to simplify your workflow by creating a standard that can be used by any language, any framework, and any tool to create a prompt and manage the response. Prompty can be imported to/exported from Azure AI Studio for smooth transition between local and cloud. Prompty assets can be also shared across organizations.
17. Prompty Tooling in Visual Studio Code Tooling Visual Studio Code Extension • Frontmatter Autocompletion • Colorization / Syntax Highlighting • Validation • Quick Run • Code Generation • Evaluations – Future • ASK US
18. Prompty Runtime Library Runtime Prompty runtime is the whatever engine that understands and can execute the format. • Azure AI Foundry • Python – LangChain • C# – Semantic Kernel • TypeScript/JavaScript – Extension • Your Language – Implement your own!
19. Scenario A ton of empty doc pages with some understanding of the codebase, how could we make it to 100%
20. Agents as a Service
21. What we want - Streamlined and well-maintained docs The struggle? - One maintainer - Lots of issues & discussions
22. AI Agents Unlike existing automation, AI Agents can automate requests as soon as they come in by… Reasoning over the request Proactively retrieving more context Performing required actions with APIs
23. What is an AI agent? LLM Instructions Tools Agent + + An AI agent is a micro-service that takes unstructured messages, optionally invokes other APIs and returns messages 1 2 3 Input System events User messages Agent messages 1 Tool calls Knowledge Actions Memory 2 Output Agent messages Tool results 3
24. Unlike previous tools, AI Agent can… reason, act and learn
25. Agentic patterns Acting aka tool calling Learning aka memory Reasoning aka planning I’ve captured the underlying issue in the details { "name" : "John Clar "issue": ”Washing m Create a JSON describing the user issues Check logs Machine is past due for maintenance Go beyond transcriptions and slot filling by allowing agents to deduce the why I’ll look at the logs to see what happened Transcript Fill out and submit the workorder Populate field: name The form has been successfully Agents can take a simple request and chain multiple tool calls to complete the ask Only make a mistake once. With memory, agents can recall prior experiences Populate field: issue Name field populated Issue field populated Submit form Update work order Let me try again with an available technician. Update work order Alex is already booked at this same time Successfully assigned technician Assign technician Saved memory: Alex is OOF next week
27. Scenario An open-source maintainer for Prompty looking for a way to simplify tagging issues.
28. Auto-tagging issues & discussions
29. Translation as a Service
30. What we want - Localized docs The struggle? - One maintainer - Lots of issues & discussions
31. https://github.com/Azure/co-op-translator
32. From en to xx in a single click
34. Scenario How can a global educational curriculum with >2M views be made accessible?
35. Auto-translation for ml for beginners curriculum https://github.com/microsoft/ML-For-Beginners
36. https://aka.ms/ai-agents-prompty Resources and code!
37. https://aka.ms/azureaifoundry/discord Resources and code!
38. https://aka.ms/azureaifoundry/forum Let’s chat!
39. Thank you. Bethany Jepchumba | @bethanyjep
40. © Copyright Microsoft Corporation. All rights reserved.