2026-02 Dev With AI - No Vibes Allowed (20') - Getting AI to Work in Complex Codebases

July 2025: Stanford University presents a study based on 100k commits demonstrating the ineffectiveness of AI-driven development on complex codebases.
August 2025: Dexter Horthy, who coined the term Context Engineering, shares a retrospective on a new approach that made it possible to deliver two feature PRs in 7 hours on a 300k-LOC open-source codebase, validated by the maintainers, who had estimated the effort at 3–5 days each if they had done it themselves.
This talk presents that approach.

Transcript

1. “No Vibes Allowed” Advanced Context/Harness Engineering Getting AI to Work in Complex Codebases The Foundations Emmanuel Sciara February 03, 2026
2. The broken promise…
3. Updated @ AIE Code Summit (november) 2025 : https://www.youtube.com/watch?v=cMSprbJ95jg&t=9705s Yegor Denisov-Blanch
6. Until new approaches emerged Summer ‘25!
9. The Goals 🗆 AI Works well in Brownfield Codebases 🗆 Solves Complex Problems 🗆 No Slop 🗆 Helps handle PR reviews of the shear size of code generated
10. Talk Summary 1. Context/Harness Engineering: Principles and tools 2. Advanced Workflow: Overview of the RPI (Research-Plan- Implement) approach 3. Checking against the goals. 4. How do I learn & What’s next?
11. Part 1: Context/Harness Engineering Principles
12. LLMs: Meet Dory the Fish 🐟 LLMs are stateless => No memory of the current context but what you give them… … for each call.
13. Giving LLMs Memory Two Types: • Session memory: Previous prompts/answers, discarded after session • Long-term memory: Persistent instructions/info across sessions
14. What is a Context Window made of? Components: • System instructions • Long-term memory (CLAUDE.md and other loaded .md files) • Tools – claude built-in tools – MCP tools (includes calls to RAG, etc.) • Session conversation history – User message – Assistant actions (tool calls, LLM calls) – Assistant responses • Current task/query
15. The curse of growing context: Context Rot
16. https://research.trychroma.com/context-rot
18. The Dumb Zone “The name of the game is that you only have approximately 170k of context window to work with. So it's essential to use as little of it as possible. The more you use the context window, the worse the outcomes you'll get.” https://ghuntley.com/ralph
20. Guidelines on Dumb Zone frontier • Depends on task complexity, and… • … 40% (up to 60%) is a good guideline.
22. Context Engineering = “Cleverly avoiding the Dumb Zone”
24. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents Harness
25. https://x.com/dexhorthy/status/1985699548153467120
26. Part 2: Context/Harness Engineering for Complex Codebases
27. Context Engineering (or lack of): the Naive Way (Using Context Until It Dries Out)
28. Slightly Smarter: Start Over When should you start over? “You are absolutely right!” =>
29. Slightly Smarter: Intentional Compaction
30. What could we be compacting? => What takes up space in your context window?! Could be: • Looking for files • Understanding code flow • Edits to files • Test/Build outputs • JSON tool reponses
31. Smart Move #2: Managing Context with Sub-Agents Agents, wielded correctly, manage context well.
32. Even better: Frequent Intentional Compaction Frequent Intentional Compaction is about building your Entire Workflow around Context Management.
33. Frequent Intentional Compaction Three phases: Research Plan Implement One Goal: Stay in the smart zone (40%)
34. RPI (Research-Plan-Implement) Workflow
35. Research • Understand how the system works • Find All Relevant Files • Stay Objective - No Bug Hunting Compression of Truth: Prevents Agents to make stuff up!
36. Approach to Compression of Truth Amount of Lies
37. Prefer On-Demand Compressed Context => files are the only source of truth.
38. Research: Keep Things Objective • Discourage opinions • Avoid implementation plan • Research = Compression of Truth
39. Research Output
40. Plan • Outline the Exact Implementation Steps • Include Filenames, Lines and Snippets • Explicit about testing steps Leverage/ Compression of intent
43. Implement • Go write the code • "If properly planned, the implementation is easy and expected." • Keep Context under 40%
44. Does this always work? NO! => Sometime need to go back to drawing board DO NOT OUTSOURCE THE THINKING! "The hard work of thinking can't be outsourced to AI, only amplified by it" https://www.arthropod.software/p/vibe-coding-our-way-to-disaster
45. You MUST Read the Outputs (No Magic)
46. Part 3: How did this fulfill The Goals? 🗆 AI Works well in Brownfield Codebases 🗆 Solves Complex Problems 🗆 No Slop 🗆 Handle PR reviews of the shear size of code generated
48. https://hlyr.dev/he-gh
49. How did this fulfill The Goals? ✅ AI Works well in Brownfield Codebases (300k LOC rust project) 🗆 Solves Complex Problems ✅ No Slop (Merged PR, no notes) 🗆 Handle PR reviews of the shear size of code generated
52. How did this fulfill The Goals? ✅ AI Works well in Brownfield Codebases ✅ Solves Complex Problems ✅ No Slop 🗆 Handle PR reviews of the shear size of code generated
53. How did this fulfill The Goals? ✅ AI Works well in Brownfield Codebases ✅ Solves Complex Problems ✅ No Slop 🗆 Handle PR reviews of the shear size of code generated
54. Handling Reviews of AI Generated Code • Claude Code initially sucked. (Lots of slop) • Eventually figured something out => ended up shipping 3x more quality code than before. • => Can’t review that much code… • => No choice but to change the way we collaborated.
55. What are Code Reviews for? https://blakesmith.me/2015/02/09/code-review-essentials-for-software-teams.html
56. How did this fulfill The Goals? ✅ AI Works well in Brownfield Codebases ✅ Solves Complex Problems ✅ No Slop 🗆 Handle PR reviews of the shear size of code generated => Mental Alignment…
57. How can we maintain Mental Alignment without having to review all the code? • Reading thousands of lines of code every week is hard… • … But reading 300 lines of code of implementation plans is feasible. • If the plans are good, that’s enough! => Catch Problems Early => Maintain System Understanding ⚠ Does NOT mean the code is never read ⚠
58. Plans for Mental Alignment • Your Goal is Leverage • You want high confidence the model will do the right thing => Checkboxes are not enough
59. Plans for Mental Alignment • Your Goal is Leverage • You want compression of intent AND reliable execution
63. Impact Hierarchy for Coding Agents
65. How did this fulfill The Goals? ✅ AI Works well in Brownfield Codebases ✅ Solves Complex Problems ✅ No Slop ✅ Handle PR reviews of the shear size of code generated => Mental Alignment
66. DO NOT OUTSOURCE THE THINKING Remember: YOU STILL HAVE TO READ THE PLAN https://www.arthropod.software/p/vibe-coding-our-way-to-disaster
67. Part 4: How do I learn how much Context Engineering to use? • It takes repetitions => Build your experience • Pick ONE tool and get some repetitions • You won't develop intuition if you're constantly min-maxing across modules and tools
68. Don’t underestimate the change and effort to learn • Real Engineering & Craftsmanship, but at a higher level • Dex REX: Completely rewired how we build/communicate/ review – 8 weeks for a team of 3 – Really f**** hard – Never going back
70. What’s Next • Coding Agents will be commoditized. • Team/Workflow Transformation will be the hard part. • “If you can't figure it out, you're hosed.”
71. The Growing Rift • Junior/Mid-level engineers use a lot of AI • It fills in some skill gaps • But it also produces some slop
73. How to Go with it? • Deep cultural change is impossible without buy-in and guidance from the top • If you're a technical leader at your company • Pick ONE tool and get some repetitions
74. THANKS! https://www.linkedin.com/in/emmanuelsciara/