Who Gets to Decide? Designing Agency Back Into AI Search - Haystack EU 2026 @Berlin

A talk from Haystack Europe 2026 on a shift most AI search interfaces haven't caught up with: search used to hand you options, now it hands you one answer already assembled. Using a UN Women finding that one in five AI responses about gender come back sexist, I look at why fluent answers are harder to question than wrong ones, and walk through five interface principles for keeping human judgment in the loop: making reasoning visible, designing for uncertainty, keeping alternative perspectives visible, creating thresholds for questioning and correction, and treating the answer as a door to the source rather than a wall. I strongly believe UX & Product Design aspect requires an immediate attention in AI Search Agents to create successful product stories.

Transcript

1. HAYSTACK EU 2026 · WOMEN OF SEARCH Who Gets to Decide? Designing Agency Back Into AI Search Öykü Doğru Product Designer · Berlin
3. WHO GETS TO DECIDE? · HAYSTACK EU 2026 ÖYKÜ DOĞRU THE SHIFT Search used to retrieve. Now it decides. THEN AI Bias in Search Results How Ranking Reinforces Stereotypes NOW unesco.org acm.org Gender and Language Models, a Review nature.com Search Neutrality Is a Myth wired.com The user scans, compares, decides. → AI systems show measurable bias in how they describe gender, largely inherited from training data. Recent evaluations found this pattern in roughly one in five responses. The system scans, compares, decides, and hands over one answer. 39
4. WHO GETS TO DECIDE? · HAYSTACK EU 2026 ÖYKÜ DOĞRU CONFIDENCE ISN'T CORRECTNESS 1 in 5 “...AI responses about gender came back sexist.” UNESCO, 2024. Cited in UN Women’s 2026 AI and gender advisory. COMPLETE THE SENTENCE “Bias and discriminatory algorithms are not a glitch in AI.” “She works as a ___.” “…a nurse, warm and detail-oriented.” — UN Women, 2026 That is not a glitch. It is what happens when critical human intelligence gets pushed out of the loop. ! Confident answer. Unexamined assumption. No sources shown Illustrative example, not a verbatim system output 40
5. WHO GETS TO DECIDE? · HAYSTACK EU 2026 ÖYKÜ DOĞRU THE REAL PROBLEM This is also a design problem, not solely a data problem. MODEL OUTPUT One confident answer ✓ USER JUDGEMENT THE DESIGN LAYER → What gets shown? What gets hidden? What can be challenged? → Accept, inspect, or challenge ? FRAMING · DEFAULTS · FRICTION The smoother the system feels, the less it gets questioned. Design decides whether people stay in the loop. 41
6. WHO GETS TO DECIDE? · HAYSTACK EU 2026 ÖYKÜ DOĞRU THE QUESTION IS BIGGER THAN BIAS Who gets to decide? AI search doesn't just decide what to show. MODEL Generates It increasingly decides: RETRIEVAL Selects RANKING Orders INTERFACE Reveals What is relevant. What is important. What is credible. 1 ONE ANSWER Most of the chain is no longer visible. What is left out. A chain of product decisions When decisions become one answer, the interface can make them invisible. What users see is not simply model output. 42
7. WHO GETS TO DECIDE? · HAYSTACK EU 2026 ÖYKÜ DOĞRU THE NEW PRODUCT PROBLEM When the answer is already decided THEN Less cognitive effort Search Explore Compare Decide Less inspection NOW Search Synthesize Decide Less visible disagreement The system is making part of the judgment before the user does. More reliance on the answer Convenience can quietly become dependence. 43
8. WHO GETS TO DECIDE? · HAYSTACK EU 2026 ÖYKÜ DOĞRU WHAT ARE WE ACTUALLY DESIGNING? Not just answers. We're designing the relationship between: 01 SYSTEM 02 What it knows, assumes and prioritizes INTERFACE What it reveals, hides and makes possible 03 USER What they trust, question and decide How much judgment should remain human? The goal isn't to keep humans in the loop. It's to keep humans meaningful in the loop. 44
9. WHO GETS TO DECIDE? · HAYSTACK EU 2026 ÖYKÜ DOĞRU CASE IN POINT: BLOOMIE Designing for the bad day of the user Most habit trackers turn a missed day into a broken streak and a reason to quit. Not Bloomie. A lapse changes the plant not the person’s worth. Recovery remains part of the experience. More about Bloomie →
10. WHO GETS TO DECIDE? · HAYSTACK EU 2026 CASE IN POINT: BLOOMIE Support, not shame. Bloomie responds without judgement. It offers one gentle next step and leaves room to choose. 01 WAIT for struggle 02 OFFER a smaller step 03 LEAVE the choice human ÖYKÜ DOĞRU
11. WHO GETS TO DECIDE? · HAYSTACK EU 2026 01 ÖYKÜ DOĞRU Make the reasoning visible Show WHY the system landed here, not only what it landed on. Recruit your users as co-producers of your AI agent. Where this already exists AI-powered search increasingly decides what counts as the answer, shifting judgment away from the user. 1 Cited answers Inline source links next to every claim, not buried in a footnote. SOURCES (3) 2 Visible chain of steps A short summary of what the system checked before answering. UNESCO, Bias in Large Language Models (2024) ACM, Fairness in Retrieval-Augmented Systems Nature, Bias Patterns in Language Models 3 Confidence flags A marker when the system is inferring rather than retrieving. Reasoning trace, expanded by default 47
12. WHO GETS TO DECIDE? · HAYSTACK EU 2026 02 ÖYKÜ DOĞRU Show uncertainty instead of hiding it Every answer sounds equally sure. Most systems don't actually feel that way internally Uncertainty is not a failure state.. Design moves Search engines are shifting from retrieval to synthesis, changing what “trustworthy” means for a result. Confidence indicators A visible scale, not a single confident tone for everything. Disagreement signals Flag when sources conflict instead of picking one silently. Honest hedging Language that admits partial knowledge, in plain words. CONFIDENCE Moderate confidence, based on partial sources “I’m not fully certain here the sources disagree on this point.” 48
13. WHO GETS TO DECIDE? · HAYSTACK EU 2026 03 ÖYKÜ DOĞRU Put alternative perspectives side by side One answer reads as the truth. Three answers read as a judgment call, which is closer to reality. A Perspective A Fix bias at the training data level. Curate more representative datasets before deployment. ALTERNATE SOURCE B Perspective B Fix bias at the interface layer. Show uncertainty and sources so people can catch what data can’t. MOST CITED C Perspective C Fix bias at the evaluation layer. Build benchmarks that catch these patterns before shipping. ALTERNATE SOURCE 49
14. WHO GETS TO DECIDE? · HAYSTACK EU 2026 04 ÖYKÜ DOĞRU Leave room to challenge and correct In architecture, a threshold makes people pause before they enter. Interfaces need the same kind of deliberate friction. Design moves AI RESPONSE In many datasets, a doctor is more commonly described as male and a nurse as female. This answer relies on patterns that may reflect historical bias. A real correction path Not a hidden feedback form. A visible, one-tap disagreement. Deliberate friction A brief pause on high-stakes answers instead of instant certainty. Editable results Looks right Challenge this Let people adjust the query’s assumptions, not just re-ask. 50
15. WHO GETS TO DECIDE? · HAYSTACK EU 2026 05 ÖYKÜ DOĞRU Make the answer a door, not a wall The summary should never be the last stop. It should be the way back to the original source. AI ANSWER ORIGINAL SOURCE Search results are increasingly synthesized into a single confident answer. UNESCO (2024) → Bias in large language models. Cited in UN Women’s 2026 advisory. unesdoc.unesco.org ↗ SUMMARY A door invites you through. A wall just tells you the answer and stops there. 51
16. AI should reduce effort, not curiosity. A healthier relationship between humans and intelligent systems keeps people in the room when the deciding happens. Öykü Doğru · Product Designer · Berlin
17. Thank you. Let’s connect. Öykü Doğru UI & UX Product Designer · Berlin (open to full time product design roles) Scan to connect