Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Agentic AI in Practice: A Clear Definition and ...

Avatar for Emery Fertic Emery Fertic
December 11, 2025
36

Agentic AI in Practice: A Clear Definition and the Real Barriers to Implementation

Avatar for Emery Fertic

Emery Fertic

December 11, 2025

More Decks by Emery Fertic

Transcript

  1. IT SLOAN SCHOO F MANAGEMENT By Emery Fertic ‘28 Professor

    Melissa Webster Fall 2025 UROP Agentic AI in Practice: A Clear Definition and the Real Barriers to Implementation
  2. IT SLOAN SCHOO F MANAGEMENT Acknowledgements: Melissa Webster Interviewees My

    research group: Deeksha Gupta, Coco Zhang, and Taylor McAndrew, Ben Fertic 3
  3. How do we define agentic AI and what does its

    emergence reveal about expectations, perceptions, and early implementation practices? 6
  4. Research Methodology Secondary sources: • We supplemented our interviews with

    additional information from: ◦ Academic papers ◦ News articles ◦ Companies’ market research reports Primary sources: • Conducted 5 interviews
  5. 8 Summer Fall Agentic workflows Difficulty defining AI categories Agentic

    workflows Same challenge amplified as implementation takes place This fall, attempts to implement agentic workflows (semi-autonomous and autonomous) Media and companies use “agentic AI” frequently Despite popularity → no consensus definition exists 1
  6. - AI that “takes things off my plate” - Tools

    that “keep going” after a single prompt - Delegation at a workflow level (note-taking, research, drafting) Their definitions were broad, intuitive, and behavior-based, not technical. Summer interviewees described early autonomy without calling it “agentic AI” 1
  7. Senior AI Strategist in Capital Markets Agents require: Autonomy, Decision-making,

    Tool use + context Emphasizes levels of agentic capability Fall interviewees gave tight, technical criteria because they build or evaluate agentic systems LLM = brain; agent = brain + hands + legs Must: call tools, access data, execute workflows Agents = automate multi-step workflows Reduce manual sequences across tools/humans Agents can plan and execute without constant prompting Global AI & Agentic Lead at Firm Silicon Valley Product & AI Leader AI Research Director/Big Tech Perspective 1
  8. OpenAI (2024–25) — Goal-Directed Autonomy Secondary Sources Anthropic — Delegation

    + Safety Google DeepMind — Memory, Planning, and Behavior McKinsey — Enterprise Orchestration Systems - Agents take high-level goals, generate plans, use tools, and act toward completion. - “Agents break down tasks, choose tools, and execute.” Emphasis: planning + tool use + goal pursuit - Agents act on behalf of the user, with reversibility and safeguards. - “Agents must be controllable and safe as they act.” Emphasis: safe action-taking + bounded autonomy - Agentic LLMs” can remember, plan, and act over time. - Autonomy is a spectrum. Emphasis: memory + planning loops - Agents operate across multiple enterprise tools, requiring guardrails and observability. - “Agentic AI demands orchestration, monitoring, and governance.” Emphasis: workflow orchestration + governance 1
  9. • Devin-style coding agents (multi-step → plan → act →

    verify) • V0 (web actions, tool calling) • Workflow orchestrators (Adept ACT-1, OpenAI’s new agent framework) • Multi-step retrieval-planning agents • Internal enterprise automations (e.g., fraud detection agents) Agentic • ChatGPT (standard chat mode) • Simple RPA bots (UiPath, Automation Anywhere) • Notebook LM summarization • Copilot drafting tools • One-off SQL generators (no planning loop) Not Agentic Agentic vs. not agentic 1
  10. Discipline Divide Capability Confusion Different Mental Models Term Inflation Not

    standardized: how much autonomy counts, what qualifies as agentic behavior, or what boundaries define each level Product, engineering, academia, executives, etc. each use the term differently.
 People conflate tool calling, workflow automation, and true autonomous behavior. They’re not the same, but are talked about as if they are.
 Summer users → “agentic” = “more helpful AI”
 Fall experts → “agentic” = structured autonomy + orchestration
 Literature: mixes autonomy, planning, tool use, memory
 Vendors call simple automation “agentic” to make it sound more advanced.
 1
  11. Unrealistic expectations - Interviewees across industries emphasized significant hype -

    Companies rebrand older tools as “agentic”. - Leaders struggle to distinguish true agentic systems from rebranded automation. Most “agents” on the market today: 1 Simple RPA workflows They follow fixed, rule-based scripts and cannot adapt or make decisions on their own. 2 Have existed for decades These tools are essentially older automation systems that have been rebranded as “agentic.” 3 Lack autonomy, reasoning, workflow execution They cannot plan, reason, or complete multi-step tasks without constant human prompting. 2
  12. Competitive pressure to appear “AI-first" Companies want to signal innovation

    Popularity of the term Lowers the bar For using it Low technical understanding → everything gets labeled “agentic” Contributors to Inflated Expectations of Agentic 2
  13. Effects of inflated expectations Leadership confusion: • Unclear understanding of

    agentic maturity leads to misaligned strategy. Misguided implementations: • Projects are scoped around imagined features, not actual ones Disappointment cycle: • Underperformance erodes trust and slows meaningful adoption Organizations expect autonomy that today’s tools can’t provide 2
  14. 2 Findings ➔ Definitions of Agentic AI ➔ Why the

    Concept Is Currently Overhyped ➔ Implementation Challenges
  15. - The struggle to implement agentic AI is not just

    technical — it is the moment where reality meets inflated expectations. - The term “agentic AI” entered the Peak of Inflated Expectations unusually fast - Enterprise leaders expect digital employees, not tool-assisted workflows - Implementation reveals the true limitations, forcing organizations into the Trough of Disillusionment - If agentic systems were easy to implement, there would be no hype problem → expectations and reality would already match. Why Implementation Is Difficult 3
  16. Structural Causes 1 Data Fragmentation: Data scattered across 10–15 systems.

    Legacy architecture slows orchestration and tool-chaining. 2 Workflow Complexity: Real enterprise workflows involve branching logic, exceptions, multi-person dependencies. Agents collapse under large processes. 3 Lack of Ontologies/Process Mapping: Agents fail when they don’t know the “rules of the domain” Companies have not formally defined their processes. 4 Governance, Risk, and Compliance: Highly regulated industries (finance, healthcare) require strict controls. Hard to give systems autonomy without violating security or privacy. 3
  17. Leaders pull back investment - Move from ambitious end-to-end automation

    → small pilot projects. - Loss of confidence in vendor claims “Agentic” becomes a diluted term” - Companies scale down their definition to fit what they can actually deploy - More hype → more disappointment → deeper trough Misalignment between technical and business terms - Business expects autonomy - Engineering teams deliver glorified workflow builders Slow path to the Slope of Enlightenment - Organizations slowly relearn what agentic systems can do - They shift from unrealistic expectations → realistic capabilities (tart implementing small, dependable multi-step workflows) Common effects of implementing during inflated expectations 3
  18. Based on Interview Insights: Trough of Disillusionment - Consolidation of

    agents: fewer, smarter, more reliable workflows - Better enterprise architectures & unfragmented data - Stronger guardrails, evals, and observability - AI-native workforce + leadership alignment - Less hype → clearer definitions → achievable goals Expectations fall back to reality as early agentic systems struggle with real-world complexity 3
  19. Based on Interview Insights: Slope of Enlightenment - Consolidation of

    agents: fewer, smarter, more reliable workflows - Better enterprise architectures & unfragmented data - Stronger guardrails, evals, and observability - AI-native workforce + leadership alignment - Less hype → clearer definitions → achievable goals Organizations begin to redefine realistic expectations. 3
  20. Based on Interview Insights: The Plateau of Productivity - Consolidation

    of agents: fewer, smarter, more reliable workflows - Better enterprise architectures & unfragmented data - Stronger guardrails, evals, and observability - AI-native workforce + leadership alignment - Less hype → clearer definitions → achievable goals True digital employees require unified data + well-defined processes. 3
  21. 3 Key Takeaways from our summer 2025 research 28 •

    Many legacy tools are being rebranded as “agents,” raising expectations beyond reality. • People confuse tool-calling, automation, and true autonomy, driving skepticism when tools fall short. • Mirrors the Gartner Hype Cycle: rising expectations → fatigue despite real technical gains. Overhype and Terminology Drift Are Masking Real Progress • Agentic systems require orchestration, guardrails, and clean data that many orgs don’t yet have. • Low-risk tasks automate easily; high-risk workflows stall due to oversight and liability concerns. • The gap between what tools can do and what environments support explains slow adoption. Implementation Is Hard Because Workflows Are Complex • Interviews and literature use “agentic AI” to mean different capabilities, from automation to full autonomy. • This definitional mismatch fuels confusion, hype inflation, and misaligned expectations. • Without a shared vocabulary, organizations struggle to evaluate tools or plan adoption. Agentic AI Lacks a Shared Definition, Causing Fragmentation
  22. 29 Implications of our findings Give preview of 3 buckets

    before 3 slides Without a shared definition of agentic AI, organizations risk misalignment, stalled adoption, and investing in tools that don’t match their needs. Firms that succeed with agentic AI will build the infrastructure—guardrails, orchestration, monitoring—to support safe, scalable semi-autonomous workflows. The gap between hype and real-world constraints will push companies toward more grounded, evidence-based evaluation of agentic tools, accelerating maturity and reducing over-reliance.
  23. Future research directions Standardizing Definitions and Taxonomies: 1. Can the

    field agree on a formal definition of “agentic AI”? 2. What objective criteria should classify systems as Level 0, 1, or 2 autonomy? 3. How do we measure “agency” across different tools and industries? Evaluating Real-World Agent Performance: 1. What benchmarks best capture multi-step planning, error recovery, and tool reliability? 2. How can researchers compare agentic performance across different platforms (OpenAI, Anthropic, Devin, etc.)? 3. What new failure modes emerge only in long-horizon tasks? Designing Infrastructure for Safe Autonomy: 1. What orchestration frameworks, guardrails, and monitoring tools are required before enterprises can trust agents? 2. How should agents signal uncertainty, ask for human input, or safely stop? 3. What architectures support reversible or auditable actions? 30
  24. IT SLOAN SCHOO F MANAGEMENT Special thank you to Melissa

    Webster and our Fall 2025 Interviewees! 31
  25. Based on Interview Insights: The Peak of Inflated Expectations Agentic

    AI hype accelerates due to vendor branding + media coverage. - Consolidation of agents: fewer, smarter, more reliable workflows - Better enterprise architectures & unfragmented data - Stronger guardrails, evals, and observability - AI-native workforce + leadership alignment - Less hype → clearer definitions → achievable goals
  26. 2 Findings ➔ Definitions of Agentic AI ➔ Why the

    Concept Is Currently Overhyped ➔ Implementation Challenges 1
  27. - No more asking for single outputs → professionals delegate

    multi-step processes to AI - “summarize this doc” → “listen to this meeting, generate notes, share with team, draft follow-up.” - Reflects a higher level of abstraction: AI as orchestrator, not just assistant 36 From Summer Findings: AI is moving beyond prompt-and-response into goal-driven orchestration Prompt Response Goal Actions Notes Listen Significances 1
  28. 37 Procedural Agent High Oversight Executes predefined chains of reasoning

    (multi-step planners, task decomposers). → Requires human start and end supervision. Goal-Oriented Agent Multi-Agent System Meta- Agent Moderate Oversight Given a goal, the agent decides how to reach it (e.g., self-prompting, using APIs, re-planning). → Moderate human oversight. Distributed Oversight Several agents coordinate or critique each other to reach complex outcomes. → Oversight becomes distributed rather than direct. Minimal Oversight Supervises or creates other agents; learns to manage workflows independently. → Minimal or no human oversight. Agentic AI: Emerging Forms and Oversight Evolution Sources: https://arxiv.org/abs/2210.03629 https://cdn.openai.com/papers/gpt-4-system-card.pdf https://hbr.org/2018/07/collaborative-intelligence-humans-and-ai-are-joining-forces 1
  29. Common threads across interviews: - Agentic AI systems perform actions,

    not just assist - Must involve tool use, data access, and workflow execution - True agents operate with some degree of autonomy - Still no shared industry definition → contributes to confusion and overhype Synthesis: What is Agentic AI? 1