Retrospect your agents

Your agent setup is a system, and so are you. Most of us tune the first and ignore the second. This talk treats working with coding agents like a team running retrospectives: collect what happened, find where each problem should have been caught, act on the setup or on your own habits, and check whether the fix held. Based on retros from three real projects, including one where cutting the setup back was the improvement. The appendix has the prompts I use to run my own retros.

code.talks 2026, Hamburg

Transcript

1. TEREZA IOFCIU RETROSPECT YOUR AGENTS Why did that happen? How you and your agents get better together. CODE.TALKS 2 02 6 · HAMBURG TOK
2. Conversations at work today W O H D E IS R P R U S I AM G IS N I D O C E IB V H C U M DONE AT WOR K .
3. Conversations at work today OW H D E IS R P R U S M IA NG IS MU C H V IB E COD I DONE AT WOR K . I A M SU R P R I S ED H OW M UCH VI B E COD I NG I AM DO I NG.
4. 20 YEARS DATA COACH & LEAD TEACHING AI.* HI, I'M TEREZA. I'M NOT SURPRISED HOW MUCH VIBE CODING I'M DOING… Tereza Iofciu · terezaiofciu.com
5. THE PROBLEM WITH VIBE CODING YOUR STACK WHAT YOU SAID YESTERDAY YOUR DEFINITION OF DONE YOUR CONVENTIONS CONVENIENT BUT TIRING TO REPEAT YOURSELF Tereza Iofciu · terezaiofciu.com
6. SO YOU BUILD A SYSTEM AROUND YOUR SDLC AG ENT FILE the base: CLAUDE.md, AGENTS.md PLAN DEPLOY SPEC TEMPL ATES plan and spec SKILLS how-tos, used at every step SUBAG ENTS test and review REVIEW BUILD TEST HOOKS gates between steps OR START WI TH A POPU L A R ONE superpowers · OpenSpec · Matt Pocock's skills · BMAD ONE SYSTEM PER REPO. AND EVERY REPO'S IS DIFFERENT Tereza Iofciu · terezaiofciu.com
7. ONE AUTOMATION STEP AWAY FROM A SOFTWARE FACTORY. Automate everything between a signal (a chat, an issue, a bug) and the pull request. Tereza Iofciu · terezaiofciu.com
8. TURN TO YOUR NEIGHBOR What’s in your setup, and how do you know it works? 1 min each Tereza Iofciu · terezaiofciu.com
9. HOW DO YOU KNOW YOUR AGENTIC SYSTEM WORKS FOR YOU?
10. WHY RETROSPECTIVE? Take a general feeling about working with your agents and look at the specific situations You, your team or org are also part of your retro, not just your agentic setup You already know how to do it… it’s a well documented agile process IT ’S BOTH ABOUT IMPROVING THE SYSTEM AND YOUR BEHAVIOR Tereza Iofciu · terezaiofciu.com
11. THREE DIFFERENT PROJECTS SMALL TEAM Started with a whole factory, built up front. Tereza Iofciu · terezaiofciu.com ENTERPRISE CLIENT Ran retros so skills and hooks delivered code I could trust. SOLO Started small. Collected data, built the system as I needed it.
12. SMALL TEAM THE FACTORY We built everything we had read about.
13. SMALL TEAM GOAL: ANYBODY CAN BUILD FEATURES ROLE AG ENTS A LEDG ER SIX REVIEWERS A COST DASHBOARD HOOKS SKILLS TO SELF IMPROVE TOO MUCH FRICTION. I COULDN’ T REALLY ONE SHOT BIGG ER CHUNKS OF WORK Tereza Iofciu · terezaiofciu.com
14. SMALL TEAM FIRST RETRO "ARE WE OVER-ENGINEERING THE SETUP?" 59% of all commits up to then touched the setup, not the product. SPENDING MORE TIME BUILDING THE SYSTEM THAN THE PRODUCT Tereza Iofciu · terezaiofciu.com
15. SMALL TEAM SO WE CUT IT Reviewers six to two. Hooks off. Ledger app deleted. 16% of commits touched the setup in July, down from 59%. 84% went to the product. NOT A REAL EXPERIMENT: BET TER SETUP OR BET TER MODELS? WE CAN' T TELL Tereza Iofciu · terezaiofciu.com
16. ANYBODY CAN BUILD BECAME EVERYONE DOES WHAT THEY’RE GOOD AT We never converged on one way to ask. Now one of us refines in Claude Design, the other builds. Not everyone needs to become a git expert. Tereza Iofciu · terezaiofciu.com
17. ENTERPRISE CLIENT THE HABIT ML project with human in the loop. Incremental improvements.
18. TOKEN-BASED LICENSE TOK Tereza Iofciu · terezaiofciu.com small, focused chats strict context plan apart from build
19. CLIENT MOST ASKED QUESTION: WHY DID YOU DO THAT? EP IC → PL AN → SLIC E SPEC → TESTS FIRST → IM PLEM EN T → CHECK → REVIE W After each epic.. trace errors and improve setup Tereza Iofciu · terezaiofciu.com
20. CLIENT DECISIONS VS ASSUMPTIONS Us: The more questions we get, the less we read them. Especially when we don’t have a strong opinion. Agent: It tries not to bother us. So it decides for us what to assume and what to ask. VISIBLE ASSUMPTIONS SHOW YOU WHEN YOU SHOULD G ET AN OPINION Tereza Iofciu · terezaiofciu.com
21. I DIDN’ T AUTOMATE IT BECAUSE I NEEDED TO UNDERSTAND THE CODE Tereza Iofciu · terezaiofciu.com
22. EVERY CORRECTION GOES INTO A DATED LOG. WHAT IT BECOMES DEPENDS ON HOW OFTEN IT COMES BACK. A line in the agent file, a change to a skill, or a hard rule in a hook. The log tells me which. Tereza Iofciu · terezaiofciu.com
23. S O LO COLLECT FROM DAY ONE A small app: from prototype to app
24. S O LO WEEK ONE Lots of escaped defects A quick prototype, no process. I judged it by one test: do I like it? Every defect was found by me, while using it. WITHOUT A PROCESS, YOU ARE THE ONLY EVAL Tereza Iofciu · terezaiofciu.com
25. S O LO "WHO MADE THIS RULE?" Vanilla JS was right for a quick prototype. Weeks later I asked for TypeScript and Claude pushed back: there is a rule. I never gave that rule. DON’ T LET YOUR AG ENT GO YOLO ON YOUR CL AUDE.MD Tereza Iofciu · terezaiofciu.com
26. S O LO HAVE YOU TRIED /INSIGHTS IN CLAUDE CODE? 23 wrong approach 16 misunderstood request 11 excessive changes We all interact with our system in different ways, so we will have different point of frictions So my fixes target my problems. RUN YOUR OWN RETRO. CHECK /INSIGHTS FROM TIME TO TIME Tereza Iofciu · terezaiofciu.com
27. S O LO WHY COLLECT YOUR OWN DATA ONLY W HAT YOU NEED NO C EN TRAL PL AC E Not everything, just in case. Collect what answers your questions. And those depend on how you work. More than one AI tool means more than one history, each with its own format and retention. Nobody joins them for you. IT CAN BE A SMALL LOG IN YOUR REPO Tereza Iofciu · terezaiofciu.com
28. S O LO WHAT I COLLECT A HANDOVER at the end of every session: model, skills, what I decided, what it assumed A LEDG ER every defect: who found it, where it should have been caught, decided by whom A FRICTION LOG in the moment: every time I repeat myself, stop it, send it back Tereza Iofciu · terezaiofciu.com
29. YOU ARE PART OF THE SYSTEM YOU ARE TUNING. We ask for a better model. A better harness. Maybe we need to communicate better. Be more specific about what we want and what the context is for the task. Tereza Iofciu · terezaiofciu.com Bottom Liners, Eric & Bill Teitelbaum, © 2014 Tribune Content Agency
30. THE PROCESS, AS IT IS TODAY 1 2 3 4 COLLECT CLUSTER ACT RECHECK What happened? The data, then the post-its: start, stop, continue. Group the post-its. Pick the top ones. Where should each have been caught? Action items for the agents or for you. Each with a pass or fail check. Did it stick? Did it come back? Did the fix overshoot? BORROWED FROM AGILE METHODOLOGIES Tereza Iofciu · terezaiofciu.com
31. ACTIONS FOR THE AGENTS, AND FOR YOU THE AG ENTS A rule, a skill, a hook, a check. If it comes back, make it harder to ignore. Up to code. YOU Say what is out of scope up front. Ask it to flag assumptions before it concludes. Audit its rules for ones you never set. A JUDG EMENT CALL NEVER HOLDS AS A NOTE. KEEP A PERSON ON IT Tereza Iofciu · terezaiofciu.com
32. DO YOUR RETRO ACTIONS STICK? 1 Your usual problems stop appearing 2 You catch new problems before they get annoying 3 You feel less frustrated with your agents the ledger's "came back?" column stays at no they show up once in the friction log, not five times fewer friction rows, less "I had to repeat myself", /insights counts going down AND BACK EACH ONE WITH DATA … BE A DATA DRIVEN EM OF YOUR AG ENTS Tereza Iofciu · terezaiofciu.com
33. SOME USEFUL HABITS WHEN WORKING WITH AI 1 End each session with a handover 2 Keep one ledger 3 Run one retro after a body of work model, skills, what you decided, what you had to repeat who found it, where it should have been caught not on a calendar: after a feature, a phase, a big cut. Actions for the agents and for you Tereza Iofciu · terezaiofciu.com
34. THANK YOU Slides will be uploaded to speakerdeck.com/terezaif See you on Feb 27, 2027 Hamburg.Python.Pizza terezaiofciu.com linkedin.com/in/terezaiofciu
35. APPENDIX PROMPTS TO RUN YOUR OWN RETRO Prompts you can start with. Keep adapting them to your style and grow your process
36. FIVE PROMPTS Each states what you want to be able to answer and what must not happen. Paste one into the agent you code with; it works out the files for your repo, you decide. P1 START TR ACKI NG collect · once per project P4 R ECH ECK recheck · on check-back day Tereza Iofciu · terezaiofciu.com P2 HAN DOVER collect · end of every session P5 WHAT C OULD I DO BET TER? you · after every third retro P3 RETRO look back + act · after a body of work
37. P1 START TRACKING C O L L E CT · O N C E P E R P R O J E CT I want to start tracking how I work with AI, so I can improve either how I interact with you or the setup itself. Propose the smallest set of logs in this repo that would let us look back later and see: what I intended, what we built, what I decided and why, what you decided without asking me, where I had to correct or stop you, which steps we didn't need, and when we check each change again. Log it in the moment, not reconstructed. Keep it in the repo, not in one tool. Show me the files and columns before you create anything. Tereza Iofciu · terezaiofciu.com
38. P2 HANDOVER C O L L E CT · E N D O F E V E R Y S E S S I O N Before I close this session, write a handover for a future retro. Only what will be gone once this chat is: which model and skills ran, what I decided, what you assumed without asking me, where I had to correct or repeat myself, what is still open, and whether the last changes to the setup helped or got in the way. Short lines. Quote me in my own words. Put it in .retro/handovers/ in this repo, kept out of git (.gitignore, or .git/info/exclude on a client repo). Tereza Iofciu · terezaiofciu.com
39. P3 RETRO LO O K B AC K + ACT · A F T E R A B O DY O F WO R K Look back at this repo since <point>. Change nothing. Did we build what I intended, in the most direct way? Compare the result with the stated intent. If the intent isn't written down, ask me first; don't infer it from the code. Count rework (commits that redo or revert earlier work); show how. Start with any retro logs; check them against git, issues, PRs, instruction files and transcripts. Show me: steps we didn't need, decisions you made without asking, where I corrected or repeated myself, defects and who found them first as the logs say it (no evidence if not logged), and which earlier fixes held. Cite a commit, issue, file or log row for every claim. Separate what the files show from your interpretation. Say "no evidence" instead of guessing. Propose at most three changes, each at the lowest rung that would have caught it (memory note < rule < checklist or template step < test or check). Don't make them. After I pick, make only those. A complaint is not a rule. Each change names its log row and when we check it again. Use the same tables every time: intent, numbers, findings, fixes held, proposals (layout below). Tereza Iofciu · terezaiofciu.com
40. P4 RECHECK R E C H E C K · O N C H E C K- B AC K DAY These changes are due: <list, or find them>. A change is due when its check-back point has come and it has no result for that point yet. List every due change, with the count, before you judge any. For each: pass or fail? Did the problem come back? Did the fix cost more than it saved, or grow past what it was for? Answer keep, climb a rung, loosen, or remove, with the rows as evidence. Also: what grew in the instruction files since last time, and which rules no longer have a reason? Loosening needs evidence too. Use the same table every time (layout below). Tereza Iofciu · terezaiofciu.com
41. P5 WHAT COULD I DO BETTER? YO U · A F T E R E V E R Y T H I R D R E T R O Look at how I work with you, not at the code. Change nothing. First the picks from last time: did they help? (None the first time: leave that table empty.) Then read the retro logs and my own messages since the last time we did this (or since the start): friction, decisions (also the ones I reversed later), where I corrected or repeated myself. Suggest at most three changes to how I work with you: how I phrase things, when I step in, what I approve without reading, how decisions get revisited. For each: what you saw, cited (a log row, or my words quoted), what it would cost me, and what we would see if it helped. Separate what the logs show from your interpretation. Suggest only. I pick what to try; nothing becomes a rule unless I pick it. Put the report next to the retro reports, kept out of git: it quotes me. Use the same tables every time (layout below). Tereza Iofciu · terezaiofciu.com
42. SAME TABLES EVERY TIME P3, P4 and P5 end with "use the same tables every time". The report reads the same each run, so you know where to look. P3 RETRO 1 Intent: met or not 2 Numbers: commits, rework, defects 3 Findings, with evidence 4 Did earlier fixes hold? 5 Proposals (max 3) Tereza Iofciu · terezaiofciu.com P4 RECHECK Per due change: pass or fail, came back?, cost vs. saved, verdict Then: what grew, and rules without a reason P5 YOU 1 Did last picks help? 2 Suggestions (max 3): what the logs show, cost to me, what we'd see if it helped
43. START SMALL & FIND YOUR OWN PROCESS P1 + P2 Set up the logs once. Hand over at the end of every session. P3 After your first body of work: look back, pick up to three changes. P4 On the check-back day: did it stick? P5 After every third retro: what could you do better? In Claude Code, /insights covers part of the look-back and of P5. These add your intent, keep the logs in the repo across tools, and close the loop with the recheck. Tereza Iofciu · terezaiofciu.com