The Journey to Intelligent Conversational Agents

AI was supposed to give us time back. Instead, many of us now spend our days writing prompts, waiting, and checking what comes back. This talk asks whether conversational agents can change that. We speak faster than we type and read faster than we listen, so we argue for multimodal voice agents: you talk to them, and they show you. We trace the road from Turing and ELIZA to Alexa and ChatGPT, and compare a voice agent's parts to the centers of a human brain. Then we demo Ultron, the multimodal voice agent we built, and walk through its architecture. For each part (speaker diarization, interruptions, speech to text, its own voice and memory) we show why we chose it. We close with Wardley Maps, a way to decide what to build and what to rent.

Transcript