Designing for developers who've changed how they build: lessons from Azure Developer CLI data

Talk from apidays Toronto 2026, Sept 9-10

The way developers interact with APIs has changed in a way that's not going back. Not because developers are gone (they're not) but because their relationship to your API is increasingly mediated by a coding agent. And DX has to catch up to that reality.

This talk maps two new developer personas that are already in your telemetry, whether you've named them or not: the developer building an agent that consumes APIs autonomously, and the developer building anything, assisted by coding agents making calls on their behalf.

Drawing on growth data from the Azure Developer CLI and experience leading DX for some of the world's largest API platforms, this is a practitioner's view of what these two personas need today, and what breaks when we design for neither of them.

https://www.apidays.global/events/toronto

Transcript

1. Designing for developers who’ve changed how they build Lessons from Azure Developer CLI data | Kristen Womack apidays Toronto | Sept 9-10
2. Kristen Womack Principal Product Manager Azure Developer CLI @kristenwomack
3. A developer’s relationship to your API is increasingly mediated by a coding agent. @kristenwomack
4. Two patterns to design for The developer shipping agents The developer building with agents What they are building is changing. How they build is changing. Coding agents Agents in their apps call your API read documentation, choose tools, write on behalf of end users. integrations, and help deploy software. @kristenwomack
5. Developers shipping agents Illustrative scenarios Coaching App Customer Service Medical Transcription An agent coaching athletes, creating motivation from data A small business assistant handling queries autonomously A clinical assistant drafts summaries for clinician review
6. @kristenwomack
7. 01 Building AI features with chat and RAG @kristenwomack
8. 02 Build an agent from the CLI Announcing the azd AI agent extension: Publish Microsoft Foundry agents directly from your development environment The Declarative Model `azure.yaml` connects the Foundry project, model deployments, agents, tools, services, and dependencies while referencing application code and environmentspecific values. @kristenwomack
9. 03 Run and debug an agent locally azd ai agent run azd ai agent invoke Run Locally Send a request Start the agent server using the project configuration. Remote models and services may still incur charges. Invoke the deployed agent (or use `--local` for the local server to test the agent locally). azd ai agent show azd ai agent monitor Inspect deployment status Stream logs View the hosted agent's status, version, and configured endpoints. Stream console or system logs for an agent session. @kristenwomack
10. 04 Modular Foundry capabilities in the CLI Independently versioned extensions composed under `azd ai`: agents, models, toolboxes, runtime skills, connections, routines, training, and inspection. azure.ai.agents azure.ai.models Core agent lifecycle: init, run, deploy, monitor Model deployment and capacity management azure.ai.toolboxes azure.ai.skills Reusable tool collections for agent capabilities Structured knowledge skills for coding agents azure.ai.connections azure.ai.routines Data source and service connection management Automated agentic routine orchestration azure.ai.training azure.ai.inspector Model fine-tuning and training workflows Agent debugging and inspection tools @kristenwomack
11. Steep PR increase AI-assisted development changed the volume of pull requests we needed to review. @kristenwomack
12. The cost of producing a change collapsed More generated code and integrations create more work for review, testing, security, and release systems. LLM App @kristenwomack
13. Increase shipping frequency Building an agentic harness allows us to respond to change more quickly @kristenwomack
14. Developers building with agents https://milvus.io/blog/harness-engineering-ai-agents.md
15. Coding agents work best when the task is clear and verifiable Excel At Struggle With Writing integration code Ambiguity Calling endpoints, parsing responses Makes confident judgment calls that are wrong Stitching APIs together Context they can’t access Connecting services when contracts are explicit Can't access what it can't see Your API is exactly the surface Plausible but incorrect output they'll be very good at (or very bad at) using Looks right, compiles, does the wrong thing @kristenwomack
16. @kristenwomack
17. The design gap 89% 24% developers surveyed use design APIs with generative AI in their daily work AI agents in mind Source: Postman, 2025 State of the API Report (5,700+ developers, architects, and executives) @kristenwomack
18. @kristenwomack
19. 01 Detect the agent, change interaction model # azd detects coding agents via env vars: CLAUDE_CODE GITHUB_COPILOT_CLI GEMINI_CLI OPENCODE # Then that detection is used to auto-enable noninteractive behavior --no-prompt --output raw # structured, not friendly prose Instrument agent-assisted usage. Find where those workflows fail. Create an explicit automation path without bypassing safeguards. @kristenwomack
20. 02 Build structured surfaces for agents AGENTS.md + Copilot instructions guide behavior Written for agents to read — conventions, docs locations, output formats. Path-scoped rules for different file types. Tools expose capabilities An agent calls `validate_azure_yaml` and receives structured validation results it can act on with bounded permissions. Skills teach procedures Preflight checks, changelog generation, weekly reports. Structured-instruction surface for internal and external workflows. @kristenwomack
21. 03 Test whether agents can complete the task safely Measure: • Discovery • Verified task completion • Tool selection • Recovery behavior • Input accuracy • Latency and cost • Execution sequence • Tool reliability Agent usability is a testable quality dimension now like accessibility was ten years ago. @kristenwomack
22. 04 Make failures actionable and machine-readable Don’t make the agent guess whether to retry, repair, ask, or stop. @kristenwomack
23. Agent changes Changes we made for azd to adapt to agentic calls to our CLI 01 02 03 04 Detect the agent Structured Agent usability Actionable failures surfaces @kristenwomack
24. Mobile Cloud Accessibility Agents Source: Sylwia Laskowska https://dev.to/sylwia-lask/is-this-how-well-build-websites-soon-webmcp-live-demo--2e33 @kristenwomack
25. Source: Sylwia Laskowska https://dev.to/sylwia-lask/is-this-how-well-build-websites-soon-webmcp-live-demo--2e33 @kristenwomack
26. Source: Sylwia Laskowska https://dev.to/sylwia-lask/is-this-how-well-build-websites-soon-webmcp-live-demo--2e33 @kristenwomack
27. Evolve to measure the workflow, not just the API call Metrics We Have • Time to first call • Activation rate • Docs page views • Support ticket volume • … Metrics We Need • • • • • • • Is your API agent-consumable? Does a coding agent recommend you? Was the correct API discovered? Did the agent select the right operation? Were inputs valid, appropriately scoped? Did it recover safely from failure? How many retries, unnecessary calls, tokens, time, and cost were required? @kristenwomack
28. The agent discovery test > Ask a coding agent to recommend the best API > for something in your domain. Does your API come up? Is it described accurately? Is it clear to the agent how to use it? Developers are discovering APIs through coding agents. If your content is not structured for agents, is your API discoverable to them? Source: Adam DuVander / EveryDeveloper — llmrank.fyi @kristenwomack
29. Three things you can do today! 01 Ask an agent to complete a real task with your API 02 Rewrite one common failure for machine actionability 03 Instrument the workflow with telemetry Watch discovery, authentication, execution, and recover Include a stable code, state, safe retry guidance, and the next valid step Track agent-assisted attempts, completion, retries, and failure points. @kristenwomack
30. 83 Source: Kin Lane, API Evangelist, May 2026 A dev portal is not proof of agent discoverability @kristenwomack
31. APIs increasingly serve human intent mediated through agents Design for the human who sets the goal, the agent that executes, and the system that must keep both safe. @kristenwomack
32. Thank you kristenwomack.io @kristenwomack aka.ms/azd https://github.com/Azure/azure-dev