AI That Delivers: Scaling An Enterprise Conversation Platform

Transcript

1. AI THAT DELIVERS SCALING AN ENTERPRISE CONVERSATIONAL PLATFORM MAYANK KUMAR STRATEGIC PLANNING ADVISOR, FEDEX
2. From Disjointed Support to an Enterprise Conversational Platform Customer Expectations Service Landscape Instant and always available More channels Personalized and Higher expectations context-aware for continuity Seamless across Greater operational every interaction complexity Transformation Our Destination Summary Automation → Conversational AI Single Bot → Multi- One Intelligent Customer Experience Agent Platform As customer expectations evolved, delivering exceptional experiences required more than Answering → Resolving Trusted, Scalable Enterprise AI automation. Our journey is about transforming every interaction into an intelligent, connected, and outcome-driven experience.
5. Solution Evaluation Criteria Product Cost Capability Alignment Vendor Fit Contract Quality Proof of Concept Vendor Risk
6. RISKS & CHALLENGES 01 02 03 04 Platform Maturity Groundedness Tool calls Agent Stability Early Adopter Risk Manual & systemic evaluation of responses Client function timing out for largesized tool responses Unstable agent behavior 05 06 07 08 Feature Gap Hallucination Latency Issues Security Out-of-the-box voice capabilities support Responses base on source docs Some ingestions taking ~12+ hours Cookie propagation in Request Headers
7. We Didn’t Build Another Chatbot RAG Performance Auth & Identity Chunking Strategy Conversation Archive Session & Conv. IDs Conversation Flow Agent Evaluation Data Governance Availability & Scalability User Interface DevOps Caching Intent Recognition Grounded Response Knowledge Base User Experience Hallucination Observability KPIs Session Management Insights Dashboard Conversational Guidelines NBA Mapping Data Masking & PII
8. One Conversation - Multiple Agents DISCOVER Sell, win customers, cross sell JOIN Resolve account creation issues SEND TRACK RECEIVE Resolve pickup issues Resolve missing document issues Address/ delivery time change issues Resolve shipment creation issues Resolve D&T issues Resolve shipment prepayment issues Resolve wrong address issues PAY GET SUPPORT INSIGHTS Resolve invoice disputes Measure KPIs Resolve lost package issues Analyze issues to find preventive measures Analyze scores Resolve damaged package issues Analyze issues to find proactive measures Resolve invoice disputes Conversation Orchestrator Onboarding Agent Shipping Agent Tracking Agent Clearance Agent Claims Agent PickUp Agent Doc Resolution Agent CDO Agent Billing Agent Knowledge Agent Developed Agent Potential Agent Escalation Agent Analytics Agent
11. Learnings Building Agents Prompts Are Software Multi-Agent Architecture Offload Logic to Tools Version, test & deploy prompts with the same rigor as code. Decompose into specialised subagents. Small agents, one job. Controller Pattern: keep reasoning lean, push compute to tools. Prompt → Agent Stability Explicit contracts on data & context at every agent boundary. Optimize Token & Reasoning Loops Hard iteration caps + lazy context loading cuts cost & latency. Root-Agent Routing Governance Strict intent classification before delegation — no ambiguity. Observability Graceful Resource Recovery Compressed snapshots over full replay — minimise bootstrap payload. Resource exhaustion requires graceful recovery handling
12. Don’t Forget Non-Functional Requirements Data Privacy Audit logging Token caching Observability High Availability Session management Auto-Scaling API Gateway Distributed Tracing Rate Limits Secret Management PII Masking DB Versioning Regression Validation Session Persistence Performance Testing
13. What We Learned Scaling AI • Data quality beats model quality. Always. • Architecture matters even more as AI scales. • Think platform, not project. • Governance must be designed into the product. • Evaluation is a continuous engineering discipline.
15. THANK YOU!