Talking to Stakeholders, With AI in the Loop

Presented at Working Products, Hamburg 10 Sept, 2026

https://working-products.de/en

Building got cheap. Anyone can ship a data step, a scoring rule or a small model without an expert in the room, and the questions that expert would have asked get skipped. Alignment used to come as a by-product of moving slowly. Now it needs a step of its own, and structured stakeholder interviews are that step. They used to be expensive: every stakeholder, every question, the notes, the aggregating, the extracting. With AI and structure, they are not anymore.

This talk shows how I ran a round of structured stakeholder interviews for a multi-stakeholder data product, and what changed when I did it with AI in the loop instead of putting myself in the loop as the person who checks everything. The interviews have two jobs: collect what you need, and let stakeholders find their own gaps, so they own the fix instead of following someone else's agenda. Then a four-step structure: prepare with intent and goals, listen and take your own notes, tell AI exactly how to extract and analyse, and write the process down so it can be repeated and refined.

You leave with the questionnaire, the prompts and the skill file, and a clear line on which decisions stay yours. For product managers, product owners, tech leads and anyone who spends too much time convincing people.

Transcript

1. TEREZA IOFCIU TALKING TO STAKEHOLDERS, WITH AI IN THE LOOP Working Products #11 · Hamburg · 10 Sept 2026
2. 20 YEARS DATA 2 YEARS PM 7 YEARS LEAD HI, I’M TEREZA, I’VE NEVER BEEN IN REAL STAKEHOLDER INTERVIEWS.
3. WHO HERE LOVES STAKEHOLDER MANAGEMENT?
4. THIS YEAR I TRIED IT TEACHING AI PRODUCT MANAGEMENT A CLIENT. MANY STAKEHOLDERS. We were surprised. They were impressed. So why had I never done this?
5. STAKEHOLDER MANAGEMENT Having good questions Talking to everybody. Taking notes. Extracting information. Communication. REQUIRES PLANNING & COORDINATING WITH OTHER PEOPLE BUT A LOT OF THE WORK IS ABOUT PROCESSING INFORMATION .. WHICH AI IS GOOD AT
6. HOW WE WORK IS CHANGING THEN Talking to people. Someone in the room had done it before & talking to them was normal & over time a shared understanding was reached NOW Talking to AI agents, We ask our AI agent more often than our colleagues.. some even communicate via AI written messages only
7. ALIGNMENT USED TO BE CHEAPER We were slower so we had more time to realize we were on the wrong path Have I ever had with clear goals and alignment? Fast in the wrong direction costs more than slow in the right direction. Having clear goals and alignment are more important than ever. STAKEHOLDER INTERVIEWS ARE A GOOD TOOL FOR ALIGNMENT
8. AI IN THE LOOP. NOT HUMAN IN THE LOOP. You drive. AI does the work you don't want to do. Three things keep you driving: GOALS STRUCTURE REFLECTION know why you ask, and what you want AI to find one kind of work per step, told exactly how do it, look back, refine. Every time.
9. 1 HAVE A CLEAR GOAL BEHIND YOUR STAKEHOLDER INTERVIEWS
10. GOALS Understand the product and how people see the product → Surface disagreement → Prepare the scene for compromising Based on the initial calls we noticed some issues A CLIENT. MANY STAKEHOLDERS.
11. EDUCATING VS COACHING Give your questions two jobs: collecting what you need, and letting the other person find the gaps themselves. YOU TELL THEY DISCOVER YOU OWN THEY OWN GOOD QUESTIONS MAKE YOUR LIFE EASIER: LESS EXPLAINING. LESS CONVINCING.
12. HARDWARE OUT IN THE WORLD → A PIPELINE THAT GUESSES WHERE IT ENDED UP AN OPS TEAM THAT → TRIES TO GET IT BACK A small data step in the middle. Which was not truly evaluated. But it was on everybody’s mind A CLIENT. MANY STAKEHOLDERS.
13. THEIR PART "HOW WOULD YOU KNOW THAT IT IS WORKING?" THE WHOLE PRODUCT
14. Everybody was very focused on defining good for the whole product, … but not for its parts. Mostly due to building stretched thin. The questions surfaced this issue
15. EVERY QUESTION HAS AN ADDITIONAL GOAL HOW WOULD YOU KNOW YOUR PART IS WORKING? The gap surfaces itself. HOW INVOLVED DO YOU WANT TO BE, AND IN WHAT? Ask. More people wanted in than I assumed. IF THE IDEAL IS NOT FEASIBLE, WHAT WOULD YOU STILL ACCEPT? Gets people ready to compromise. WHAT WAS MOST EXCITING ABOUT THE ORIGINAL IDEA? How "honestly, I'm scared of this" gets said. THE GOALS OF YOUR QUESTIONS DEPEND ON YOUR SPECIFIC SITUATION
16. EXTRACTING INFORMATION FROM THE INTERVIEWS ALSO HAS AN INTENT COO Success in 6 to 12 months What ops should see DATA PM OPS MKT LEAD agree disagree silent Cadence Acceptable fallback Who approves WE WANT TO SURFACE BOTH THE COMMON GROUND AND THE DISAGREEMENT
17. 2 THE STRUCTURE, WITH AI IN THE LOOP
18. TURN TO THE PERSON ON YOUR LEFT In your stakeholder work today: what do you hand to AI? And what do you keep for yourself? 1min each
19. FOUR STEPS. ONE KIND OF WORK EACH. 1 2 PREPARE INTERVIEW Define the intent and the goals. Then draft the questions with AI. Listen. Take your own notes. AI shouldn’t be the source of truth on note taking. 3 EXTRACT AND ANALYSE Tell AI exactly how: quotes per question, not answered, agree and disagree with names. STRUCTURE 4 WRITE IT DOWN Turn the run you liked into a skill. Refine it every time. ONE KIND OF WORK PER CONVERSATION. NEVER GIVE AI A HUGE CHUNK OF WORK IN ONE GO
20. 1 PREPARE: INTENT FIRST, THEN DRAFT Before any prompt, write down: • what you need to learn • what the interview should do for the other person • the second-line goal (mine: get people ready to compromise) GOALS Context: [3 lines] Stakeholders: [role, owns, what I think they want] I need to learn: [...] Second goal: [...] Draft the questionnaire. Every question must also do one of: name a not-perfect outcome they'd accept / tell me how involved they want to be / let them notice a gap themselves. Write the goal next to each question.
21. 1 PREPARE: BE CRITICAL OF THE DRAFT arXiv 2505.13995 · 2025 (preprint) The model agrees with you. LLMs validated the user 72% of the time. Humans: 22%. Be the most sceptical person on this list. Which questions would you dodge? Which would get a diplomatic answer? Rewrite the three worst. YOU DECIDE WHICH QUESTIONS GO IN
22. 2 INTERVIEW: LISTEN LISTEN. RECORD THE TRANSCRIPT. AND TAKE YOUR OWN NOTES. DON’T RELY ON AI TO READ BETWEEN THE LINES OR ASK FOLLOW UP QUESTIONS
23. STRUCTURE EXTRACT. DON'T SUMMARISE. 3 For each question: quote the answer verbatim, with timestamp. If not answered, write "not answered" and note what they talked about instead. Do not summarise. Do not merge. Do not fill gaps. QUESTION VERBATIM REF How would you know it works? "Haven't thought about it..." 31:40 Constraints we should know? NOT ANSWERED talked about hiring instead AI SUMMARIES HAVE THE INTENT OF THE TOOL YOU USE.. AND THAT INTENT CAN CHANGE
24. 3 ANALYSE: TELL IT WHAT TO LOOK FOR COO DATA PM OPS MKT Question by stakeholder: agree / disagree / silent. Quote both sides, with names. List what nobody mentioned. Then: what did I not ask? Success in 6 to 12 months What ops should see Cadence Acceptable fallback STRUCTURE "hourly" = hours a person spends, not data refresh TREAT IT AS A DATA EXPERIMENT, WHERE YOU GO IN WITH YOUR OWN QUESTIONS
25. THE AI TRAP We tend to check the AI output instead of doing our own thinking. And the more confident we are in AI, the less we think. SO ASK IT TO HELP YOU SEE MORE INSTEAD OF GIVING YOU THINGS TO CHECK arxiv.org/html/2510.17575 · Lee et al., CHI 2025 · Zhu et al., Front. Psychol. 2026 · arxiv.org/abs/2603.11821
26. "AGENTS WILL DO WHAT YOU ASK. BUT YOU NEED TO UNDERSTAND WHERE AND HOW IT MIGHT FAIL, AND KEEP IT HONEST WITH CHECKLISTS, FOLLOWUP QUESTIONS, AND A VERY TIGHT DEVELOPMENT ENVIRONMENT." Reuven Lerner, 8 Sept 2026, on writing with Claude Code CHECKLISTS AND FOLLOW-UP QUESTIONS WORK OUTSIDE CODE TOO. lernerpython.com/2026/09/08/claude-code-always-produces-something
27. 3 DO IT ONCE. THEN MAKE IT A REPEATABLE PROCESS.
28. HOW DO YOU FEEL ABOUT PROMPTING?
29. 4 WRITE IT DOWN REFLECTION Next time, you re-explain the project, the roles, the rules you discovered. Or you keep the conversation and ask: Look back over this conversation. Turn what we did into reusable skills: the steps, the final templates and prompts, what a good output looks like, and every correction I made, as a rule. Ideally each task has its own skill. A SKILL: A DOCUMENTED DESCRIPTION OF A TASK AND HOW IT IS DONE, SO WE CAN DO IT AGAIN. https://anthropic.skilljar.com/introduction-to-agent-skills
30. SKILLS WILL GO STALE THE MODEL + THE PROVIDER'S RULES REFLECTION + YOUR SKILL A new model version. A Claude desktop update. The same skill on ChatGPT. Same text, different behaviour. IF YOU HAD TO CORRECT THE OUTPUT, FIX THE SKILLS
31. GIVE IT YOUR DEFINITION OF GOOD REFLECTION From the run you liked, ask AI to pull out: GOOD QUESTIONS A GOOD EXTRACTION A GOOD SYNTHESIS for this goal, these are the questions I expect this is what a well-processed interview looks like this is what I send to the room THE DEFINITION OF GOOD IS THE BEST LEVER YOU HAVE FOR SKILL QUALITY CONTROL
32. YOU OWN THE DECISIONS • Which questions get asked • What counts as information worth extracting • What a good output looks like • What gets shared with others HAND THOSE OVER, AND YOU ARE THE HUMAN IN THE (TIRED) LOOP AGAIN. Laura Summers, The Human in the Loop is Tired https://www.youtube.com/watch?v=-KxwfHoOZDk
33. DELEGATING THE THINKING TRANSFERS AGENCY TO THE AI, KEEPING THE JUDGMENT DOES NOT. BOTH FEEL EQUALLY USEFUL IN THE MOMENT. Not all cognitive offloading is equal .. Zhu et al., Frontiers in Psychology, July 2026
34. GOALS. STRUCTURE. REFLECTION. GOALS Know why you ask. Tell AI what to find, don’t just summarise. STRUCTURE Four steps, one kind of work each. Listen yourself. Tell it exactly how. REFLECTION Write it down. Give it your definition of good. Refine it every time. AI IN THE LOOP. HUMAN AT THE WHEEL.
35. THANK YOU! terezaiofciu.com linkedin.com/in/terezaiofciu TEMPLATES AND PROMPTS ARE IN THE APPENDIX. SLIDES WILL BE ON HTTPS://SPEAKERDECK.COM/TEREZAIF
36. REFERENCES • Torres, Stakeholder management: show your work (Product Talk, Mar 2026) • Torres, Customer interview analysis: where AI helps and hurts (Oct 2025) • Torres, AI evals: a hands-on guide for product teams (Sept 2026) • Moran & Rosala, Accelerating research with AI (NN/g, 2024/2026) • Cheng et al., ELEPHANT: social sycophancy in LLMs (arXiv 2505.13995, 2025) • Sharma, Cochrane, Wallace, DeTAILS (arXiv 2510.17575, 2025) • Lee et al., GenAI and critical thinking (CHI 2025) • Lerner, Claude Code always produces something (lernerpython.com, 8 Sept 2026) • Böckeler, Harness engineering for coding agent users (martinfowler.com, Apr 2026) • Schluntz & Zhang, Building effective agents (Anthropic, Dec 2024) • Szapar, The personal AI operating system (2026) • Laura Summers, The Human in the Loop is Tired https:// www.youtube.com/watch?v=-KxwfHoOZDk • Illustrations from: https://motionarray.com/browse/producer/ alina-kolyuka/ •
37. APPENDIX: THE TEMPLATES Slides will be shared. Questionnaire, permission round, six prompts, the table, the synthesis, after you’ve adapted and run this for yourself ask your AI Agent to transform the steps into skills, ideally one task per skill.
38. A1 · QUESTIONNAIRE, PART 1 # QUESTION WHY WE ASK 1 What is the core problem [INITIATIVE] is meant to solve for you or your team? Their interpretation of the purpose, in their words. 2 Why is this the right moment to invest in it? Urgency, business drivers, who is pushing. 3 What outcome would you call a success in the next 6 to 12 months? Concrete success criteria per person. 4 Assuming it works perfectly, what does [PRIMARY USER] see? The output they imagine: format, detail, cadence. 5 What actions should [PRIMARY USER] be able to take from it? Expected workflow and responsibility boundaries. 6 Which pieces of information are essential for those decisions? MVP fields. Reveals who does not know, which is fine. 7 How often would someone work with it? Cadence. Watch for "time spent" vs "refresh rate". 8 Which metrics matter most to you for this? Where the missing metric shows up. Ask again for the step they own. 9 Who else should be involved or informed? Missing stakeholders, communication dependencies. 10 Whose support or input is critical for this to work? Cross-team dependency risk. 11 What constraints should we know about: data, process, people, contracts? Blockers early. Contract limits appear here. • gives something back
39. A2 · QUESTIONNAIRE, PART 2 # QUESTION WHY WE ASK 12 When this was first conceived, what part was most exciting or meaningful to you? Attachment. A feelings question; it is how fear gets said. 13 Which parts are essential, and which could evolve as we learn more? Where pivoting is acceptable, per person. 14 How open are you to adjusting if the investigation points somewhere else? Openness, asked without triggering defensiveness. 15 What is the ideal outcome you personally hope for? Personal goal, separate from the official one. 16 If it works exactly as you imagine, what changes in the company? Expected impact. Compare the size of the answers. 17 From where you sit, what is the current state? Gap between perception and status. 18 What are the main challenges or limitations right now? Constraints, and blind spots by omission. 19 What other approaches could reach your goal? Introduces alternatives gently. 20 If parts turn out less feasible, what would you still consider acceptable? The compromise question. The range tells you who needs work later. 21 How would you like to be involved as we refine this? Involvement and decision rights. Ask, do not assume. 22 What would make you confident we are moving in the right direction? Expectations for updates, evidence, trust. 23 Anything else we should understand about your expectations? Space for the unexpected. • gives something back
40. A3 · BEFORE THE INTERVIEWS STAKEHOLDER (ROLE) OWNS WHAT I THINK THEY WANT WHAT I THINK THEY WILL AVOID COO Data lead PM Ops / supply chain Marketing Sponsor Rules • The Goal column is not optional. If you cannot write why you ask, cut the question. • Keep at least one "gives back" question per block. • Two interviewers: split by stakeholder, agree the goals before the first interview. • Record. Leave "not answered" visible. Compare column three with what they said: that delta is your first blind-spot list, and it is yours.
41. A4 · ROUND TWO: THE PERMISSION QUESTIONNAIRE "One important assumption behind [INITIATIVE] is that [ASSUMPTION]. To validate this early, and to support your vision, we want to start with [ONE OR TWO SMALL EXPERIMENTS] this [period]. We are not executing yet, just preparing. Can we ask a few focused questions so we can move quickly?" 1 Are you comfortable with us designing [experiment] this [period]? Explicit permission. Surfaces prior attempts. 2 What tone should it have: official, friendly, urgent? Preferences that otherwise appear as late rewrites. 3 What must be included? Mandatory content per person. 4 What must we avoid saying or doing? Legal, contractual, reputational limits. The contract clause nobody mentioned in round one. 5 Should [channel A] and [channel B] follow the same content? Scope of the work. 6 Which [segment] should we start with? Expect a split (small and friendly vs large and valuable). Name it back. 7 Who should this appear to come from? Credibility and ownership. 8 Can we start with a small number before [deadline]? Feasibility of a quick pilot. 9 Who must approve before we proceed? Ask everyone. The disagreement is the approval bottleneck of week three. 10 What is the success metric for this experiment? One line per person. Compare. 11 Where did the last attempt fail, if there was one? Cheap history.
42. A5 · PROMPTS: BEFORE P1 DRAFT THE QUESTIONNAIRE P2 ATTACK IT Context: [initiative in three lines: what it is, who it is for, what stage it is at] Stakeholders: [role, what they own, what I think they want, one line each] I need to learn: [list] Why: language models agree with the user most of the time (ELEPHANT 2025: 72% validation vs 22% for humans). Ask for the fight explicitly. Draft an interview questionnaire. Every question must also do one of these: - get the person to name a not-perfect outcome they would still accept - tell me how involved they want to be, and in what - let them notice a gap in their own area without me pointing at it For each question write the goal in one line: why we ask, what we do with the answer. Then list which questions are leading, and which will get a polite answer instead of a real one. Output as a table: block, question, goal, gives-back (yes/no). Here is the questionnaire: [paste] Here are the stakeholders: [roles, what they own] Be the most sceptical person on this list. Read every question as they would. - Which questions would you dodge, and what would you actually be thinking while giving a polite answer? - Which questions assume something about you that might be wrong? - Which question would make you decide the interviewer has already made up their mind? Then rewrite the three worst questions.
43. A6 · PROMPTS: AFTER P4 EXTRACT, DON'T SUMMARISE P5 AGREEMENT AND DISAGREEMENT Here is a transcript: [paste or attach] Here is the question list: [paste] Here are the answer tables for [N] stakeholders: [paste P4 outputs] For each question: - quote the answer verbatim, with the timestamp or line reference - if the question was not asked or not answered, write "not answered" and note in one line what they talked about instead Do not summarise. Do not merge answers. Do not fill gaps. Output as a table: question, verbatim answer, reference, not-answered note. Build one table: rows are questions, columns are stakeholders. Cell values: agree / disagree / silent, relative to the majority answer. For every "disagree", quote both sides in one line each, with names. Then three lists: 1. What nobody mentioned, of the things I asked. 2. What I did not ask that the answers suggest I should have. 3. Where someone answered a different question than the one asked. Do not group into themes. Keep names on everything.
44. A7 · PROMPTS: REHEARSE, AND WRITE THE SKILL P3 REHEARSE (OPTIONAL) P6 TURN IT INTO A SKILL Play [role]. You own [thing]. You have said publicly that success is [X]. You have not thought about [Y]. Answer my questions as that person: brief, a little guarded, realistic. I will ask one question at a time. After each answer, in one separate line marked [coach], tell me what a better follow-up question would have been. Look back over this whole conversation. Turn what we did into a reusable skill for [task name]: - when to use it (and when not to) - what it needs from me before starting - the steps, in order - the final versions of every template and prompt we ended up with (not the first versions) - a checklist for what a good output looks like - every correction I made during this conversation, written as a rule Output as a single markdown file I can save. Keep it under 80 lines. IF YOU CORRECTED THE OUTPUT, THE TEMPLATE HAS A BUG.
45. A8 · THE AGREEMENT TABLE R1 Success in 6 to 12 months What the primary user sees Essential information Cadence Metrics that matter Acceptable fallback Desired involvement Who approves R2 R3 R4 R5 R6 Cells: agree / disagree / silent, relative to the majority answer per row. Two extra columns in the real sheet: "where they differ (both sides, quoted)" and "not answered by". Example rows (anonymised): • Cadence: "hourly" vs "weekly would be realistic". Human note: hourly meant hours of a person's time, not data refresh. Keep a notes column for exactly this. • Acceptable fallback: "a more manual way, not ideal, but fine" vs "can't think of any alternative". • Desired involvement: "very involved, this is my project" vs "updates and report back" vs "only on the messaging".
46. A9 · THE ONE-PAGE SYNTHESIS # [INITIATIVE] - Stakeholder interview summary [N] interviews, [dates]. Roles: [...]. Recorded; quotes are verbatim. ## 1. Where there is strong agreement Purpose and success · Operational reality · Validation and feedback (each bullet names the roles who said it) ## 2. Where perspectives differ For each: the question, the positions, who holds each, one quote per position. "This suggests [N] possible futures, not all of which need deciding now." ## 3. What nobody said ## 4. What we did not ask ## 5. Desired involvement ## 6. Next step (asked, no answer from most people) (goes into round two) (table: role | involved in | updates on) (the one decision the group has to make) Rule for section 2: quote both sides or neither.