Testing AI Agents — Automated Evaluation for Chatbots & RAG Systems

AI agents, chatbots, and RAG systems are easy to prototype — but hard to test reliably. A tweaked prompt, a new model version, or a re-indexed knowledge base can silently change behavior, and classic assertions (string matching, snapshots) can't capture what actually matters: correctness, relevance, and grounded answers.

This talk walks from real-world failures (the $1 Chevy Tahoe, Air Canada's chatbot) to a practical evaluation workflow you can adopt on Monday:

• Why chatbots have become agentic systems — and where prompt injection hides
• The AI Agent Testing Pyramid: deterministic checks → metrics → LLM-as-a-Judge → humans
• LLM-as-a-Judge with DeepEval — evaluating meaning, not exact matches, right inside pytest
• Faithfulness, thresholds, and quality gates for RAG answers
• Running evals in CI and on a daily schedule to catch drift
• Wrapping evaluations in BDD/Gherkin so product owners and content editors can write tests too

You'll leave with a reusable blueprint for automated AI evaluation — from local runs to CI pipelines.
Slides & demo project: https://github.com/messeb/py-deepeval-behave-bdd-testing-example
Talk given at WeAreDevelopers World Congress 2026 (Europe) by Sebastian Messingfeld, Eurowings Digital.

Transcript

1. Testing AI Agents Automated Evaluation for Chatbots & RAG Systems Sebastian Messingfeld 10.07.2026 | WeAreDevelopers World Congress 2026 - Europe
2. Slides & Code https://github.com/messeb/py-deepeval-behave-bdd-testing-example 10.07.2026 Testing AI Agents 2 Demo Project on GitHub
3. Chatbot Examples GM Dealer Chat Bot Agrees To Sell 2024 Chevy Tahoe For $1 Source: GMAuthority 10.07.2026 Testing AI Agents 3 AI coding tool wipes production database, fabricates 4,000 users, and lies to cover its tracks Source: cybernews Air Canada ordered to pay customer who was misled by airline’s chatbot Source: The Guardian Manipulation Destruction Liability What could go wrong!?
4. Chatbots Become Agentic Systems • Beyond answering — bots that act, not just answer • Wired into internal systems — live data and operations via APIs • New capabilities emerge — bridging domains creates what didn't exist before • More moving parts — every layer below is a new place to fail 10.07.2026 Testing AI Agents 4 More capability, more autonomy, more that has to be tested Memory Large Language Model Tools Knowledgebase Guardrails
5. Prompt Injection Sneaks In Lifecycle of an Injection • Entry — hides in input, a doc, or an API reply • Bypass — talks past the guardrails • Action — lets the model call a tool it shouldn't: send, delete, pay, leak • Persistence — written to memory, outlives the chat • Re-trigger — next conversation reads memory and runs it again 10.07.2026 Testing AI Agents 5 Every capability you added is also a way in Prompt injection is when instructions trick an AI into following them instead of its given ruleset, like social engineering, but aimed at the machine. Entry Bypass Action Persistence Re-trigger
6. AI Agent Testing Pyramid 10.07.2026 Testing AI Agents 6 Cheap tests at the bottom, human at the top, judge in between. Human / red-team Schema valid? Tool called right? Guardrail rule fired? Exact/regex match Final judgment, adversarial probing Similarity to expected answer, retrieval hit-rate Is the open-ended answer correct, faithful, on-tone, safe? cheap expensive
7. Let an LLM Do the Judging • Judges meaning — tone, correctness, faithfulness • Scales — grade thousands of answers in minutes • Frees your people — humans concentrate on edge cases & errors • Consistent & repeatable — same rubric every time • Explains why — not just pass/fail • Cheap enough for CI — catches regressions early 10.07.2026 Testing AI Agents 7 Evaluate meaning, not just exact matches – at scale An LLM-as-a-Judge is an AI that scores another AI's answers against a rubric as an automated quality review at scale. Example: FAIL — Score 0.15 / 1.0 Q: "What's the capital of Germany?" A: "Bonn." Reason: "Incorrect — the capital is Berlin. Bonn was the West German capital until 1990, but not since reunification."
8. DeepEval — a Framework for Testing LLM Applications • Open-source Python framework — runs in pytest, assert_test in the suite you already have • https://github.com/confident-ai/deepeval • Judging is just prompts — each metric is a proven, research-backed prompt that scores the answer • … but a framework wraps them — datasets, thresholds, retries, CI reporting • Every score comes with a reason — not a bare number: why it passed or failed • Built-in metrics — Faithfulness, Relevancy, Hallucination, Bias, Toxicity • Bring your own rubric — G-Eval turns plain-English criteria into a scored metric 10.07.2026 Testing AI Agents 8 LLM-as-a-Judge, as a pytest you already know You pick what "good" means. DeepEval brings the proven prompt — and everything around it.
9. One Test, Start to Finish 10.07.2026 Testing AI Agents 9 Hand the answer + policy to the judge
10. What Is a Metric — and What's 0.7? • A metric is one question — here Faithfulness: "is the answer true to the retrieved context?" • It splits the answer into claims — "5–7 days", "Mon–Fri", "own flights only" • It checks each claim against the context — supported or invented • The score is the share that's supported — all grounded → 1.0, half made up → 0.5 • The threshold is the pass line — 0.7 means 70% must be grounded; higher = stricter • You calibrate it — start at the default, then tune against answers a human already judged 10.07.2026 Testing AI Agents 10 The metric decides what to check; the threshold decides how strict The metric is the question. The threshold is how good is good enough — calibrated against real judgments, not a gut guess.
11. Ship It to CI — Then Watch It Daily • The model shifts — a new version quietly changes answers • Data moves — the knowledge base gets re-indexed, docs go stale • Tools & APIs drift — a field disappears, a format changes • Memory accumulates — grows, skews answers — and can carry a prompt injection 10.07.2026 Testing AI Agents 11 Your code didn't change, but everything around it did On every commit (mocked) Nightly / daily (live) Fast, deterministic, offline Slower, hits real systems — model, tools, data Catches your regressions Catches drift you didn't cause Runs in pytest / CI Runs on a schedule + alerts Normal tests break when you change your code. Agents break when the world changes around them — test on a schedule, not just on commit.
12. Everyone Shapes the Agent — Only Devs Can Test It • Product & Business Owners — decide what the agent should do and promise • Content Editors — write the knowledge base, the policies, the wording • Prompt Engineer — shape how it thinks and responds • Developers — build the tools, the memory, the plumbing 10.07.2026 Testing AI Agents 12 The people who define "good" can't check for it Prompt Engineer Product Owner Content Editor Business Owner Agent Tests → Developers only
13. Enable The People Who Own It To Test It • BDD — tests as plain-language examples: Given a situation, When it acts, Then expect a result. 10.07.2026 Testing AI Agents 13 Behavior Driven Development — the policy owner writes the test • Content editor owns the policy — so they write the Given (the real refund rules) • Product owner owns what "good" means — so they write the Then (the expectations) • No Python required — developers wire it up once
14. What to Take Home • Agents fail expensively — and now they act, remember, and loop • You can't assert quality — open-ended answers need a judge, not exact matches • LLM-as-a-Judge fills the gap — scores meaning, explains why, at CI scale • Test on a schedule — the model, tools, data, and memory drift under you • Quality is everyone's job — BDD lets the policy owner write the test 10.07.2026 Testing AI Agents 14 From "what could go wrong" to a test the whole team owns The people who define "good" should be the ones who can check for it — and now they can.
15. Slides & Code https://github.com/messeb/py-deepeval-behave-bdd-testing-example 10.07.2026 Testing AI Agents 15 Demo Project on GitHub
16. Thank you