Harness Engineering or How to Keep Agents on Track

Agentic coding works perfectly until it doesn't. That's the nature of it. There's a ton of things you can do to improve it, but to get there you need to go one layer deeper.

Agent harnesses are not magic black boxes and the key to understanding how to leverage agentic coding is understanding how it works under the hood. How does a harness work? What can you do to steer the agent?

In this talk I'll give you an intuition of how coding agents work. And what you can do to steer, to build guard rails, and to effectively reshape what has become the main way of producing code in this day and age.

Practical guidelines on giving agents senses, containing the blast radius, context and engineering, and building deterministic checks around nondeterministic behavior.

The harness matters more than the model.

Transcript

1. @iurysza iurysouza.dev Harness Engineering How to Keep Agents On Track or
2. About me •Google Dev Expert •Co-host @ Fragmented-AI •Platform Engineering @ SumUp @iurysza iurysouza.dev
3. Non deterministic Coding
4. Non deterministic Coding “They’ve done studies, you know? 60% of the time, it works every time” - Brian Fantana (Anchorman)
5. Non deterministic Coding •Agents are writing more and more code
6. Non deterministic Coding •Agents are writing more and more code •They write solid code, until they don’t
7. Non deterministic Coding •Agents are writing more and more code •They write solid code, until they don’t •New bottlenecks • Keeping code quality • Steering • Containing blast radius
8. Non deterministic Coding •Platform Teams attempts to keep quality standards for developers
9. Non deterministic Coding •Platform Teams attempts to keep quality standards for developers •Harness Engineering does that for agents
10. Non deterministic Coding •Prompt Engineering •Context Engineering •Harness Engineering
11. Non deterministic Coding Made up word
12. Non deterministic Coding The Agentic Coding Ladder
13. Non deterministic Coding The Agentic Coding Ladder
14. How we got Here
15. How we got Here The Anthropic Moment
16. The Anthropic Moment •Claude Code preview: Feb 25 •Claude 3.7 Sonnet launch •CLI-first agentic coding Feb 2025: Claude Code
17. The Anthropic Moment •Claude Code preview: Feb 25 Feb 2025: Claude Code
18. The Anthropic Moment •Claude Code preview: Feb 25 •Claude 3.7 Sonnet launch Feb 2025: Claude Code
19. The Anthropic Moment •Claude Code preview: Feb 25 •Claude 3.7 Sonnet launch •TUI / CLI based agent Feb 2025: Claude Code
20. •Opus 4.5 Sep 2025: Opus The Anthropic Moment
21. •Opus 4.5 •Step change in quality Sep 2025: Opus The Anthropic Moment
22. •Opus 4.5 •Step change in quality •Model can prompt itself Sep 2025: Opus The Anthropic Moment
23. •Opus 4.5 •Step change in quality •Model can prompt itself •Improved planning mode Sep 2025: Opus The Anthropic Moment
24. •The IDE starts losing ground Holidays: Claude Code + Opus The Anthropic Moment
25. •The IDE starts losing ground •The harness was the main improvement Holidays: Claude Code + Opus The Anthropic Moment
26. •The IDE starts losing ground •The harness was the main improvement •Competition shows up • Codex • Opencode • Amp • Pi Holidays: Claude Code + Opus The Anthropic Moment
27. •The IDE starts losing ground •The harness was the main improvement •Competition shows up • Codex • Opencode • Amp • Pi •It’s not magic Holidays: Claude Code + Opus The Anthropic Moment
28. •The IDE starts losing ground •The harness was the main improvement •Competition shows up • Codex • Opencode • Amp • Pi •It’s not magic •The harness matters more than the model Holidays: Claude Code + Opus The Anthropic Moment
29. What is a harness anyway?
30. What is a harness anyway? It’s all about steering
31. What is a harness (…) a set of straps and fittings used to control an animal (…) •Control
32. What is a harness (…) a set of straps and fittings used to control an animal (…) •Control •Direction
33. What is a harness (…) a set of straps and fittings used to control an animal (…) •Control •Direction •Constraints
34. What is a harness (…) a set of straps and fittings used to control an animal (…) •Control •Direction •Constraints A horse Harness
35. What is an agent harness It's what the LLM uses to navigate the world and complete tasks. An Agent Harness
36. What is an agent harness It is what the LLM uses to navigate the world and complete tasks. •Makes a model useful for real work •From text replies to action An Agent Harness
37. Should I build a Harness? Short answer: no •You can customize it •You can extend it •You can build infrastructure for it
38. A minimal Agent Agent: an LLM running inside a harness •Instruction Layering A typical agent definition
39. A minimal Agent Agent: an LLM running inside a harness •Instruction Layering •Tools A typical agent definition
40. A minimal Agent Agent: an LLM running inside a harness •Instruction Layering •Tools •Permissions A typical agent definition
41. A minimal Agent Agent: an LLM running inside a harness •Instruction Layering •Tools •Permissions •Context Management A typical agent definition
42. A minimal Agent Agent: an LLM running inside a harness •Instruction Layering •Tools •Permissions •Context Management •Loop control A typical agent definition
43. Make it your own Adapt it to your workflows •AGENTS.md •Skills •Commands •MCPs •Event Hooks
44. Make it your own Adapt it to your workflows Pi Agent Extensions API •AGENTS.md •Skills •Commands •MCPs •Event Hooks
45. Stacking Loops What agents are made of
46. What agents are made of The ReAct Architecture Stacking Loops
47. The Inner Loop Next Token Generation Stacking Loops
48. The agentic loop Stacking Loops The Outer Loop
49. Stacking Loops How agents get things done
50. Stacking Loops Why this works? •Non deterministic -> deterministic •Code is verifiable •Feedback loop
51. Stacking Loops Why this works? Tool Calling Feedback Loop
52. Stacking Loops Why this works? E.G.: Opencode’s AskUserQuestion Tool Error
53. Closing the Loop How to leverage the harness to make the agent smarter
54. Closing the Loop Give the agent senses •Observability enables self correction
55. Closing the Loop Give the agent senses •Observability enables self correction •Design agent friendly tools, CLIs/MCPs
56. Closing the Loop Give the agent senses •Observability enables self correction •Design agent friendly tools, CLIs/MCPs •Favor them over raw text instructions
57. Closing the Loop Give the agent senses •Machine Readable Output
58. Closing the Loop Give the agent senses •Machine Readable Output •Self describing schemas
59. Closing the Loop Give the agent senses •Machine Readable Output •Self describing schemas •Safety rails against hallucinations
60. Closing the Loop Give the agent senses •Machine Readable Output •Self describing schemas •Safety rails against hallucinations •Instrument the repository for agents
61. Closing the Loop Give the agent senses •Token Efficiency
62. Closing the Loop Give the agent senses •Token Efficiency •Reliability
63. Closing the Loop Give the agent senses •Token Efficiency •Reliability •Reproducibility
64. Closing the Loop Give the agent senses •Token Efficiency •Reliability •Reproducibility •Auditable
65. Give the agent senses Closing the Loop
66. Context Management Give the agent a memory
67. Context Management Give the agent a memory •Context is a scarce resource
68. Context Management Give the agent a memory •Context is a scarce resource •A well crafted AGENTS.md + docs goes a long way
69. Context Management Give the agent a memory •Context is a scarce resource •A well crafted AGENTS.md + docs goes a long way •Progressive disclosure
70. Context Management Give the agent a memory •Context is a scarce resource •A well crafted AGENTS.md + docs goes a long way •Progressive disclosure •Agents ❤ file system + find + grep
71. Context Management Give the agent a memory •Context is a scarce resource •A well crafted AGENTS.md + docs goes a long way •Progressive disclosure •Agents ❤ file system + find + grep
72. Context Management Give the agent a memory
73. Context Management Give the agent a memory
74. Platform Engineering And the tragedy of the commons
75. Platform Engineering The tragedy of the commons An economic and ecological concept describing how individuals, acting strictly in their own self-interest, deplete or spoil a shared resource, ultimately ruining it for everyone
76. Platform Engineering Enables Speed and Cohesion •Company Scaling Implications • Shared standards • Tooling • Guardrails
77. Platform Engineering Dedicated Agent Infra •Fast moving environment •Agent Evals and benchmarking •Skills versioning and distribution •Safely expose company systems for agents •Research and Development
78. Platform Engineering Dedicated Agent Infra •Fast moving environment •Agent Evals and benchmarking
79. Platform Engineering Dedicated Agent Infra •Fast moving environment •Agent Evals and benchmarking •Skills versioning and distribution
80. Platform Engineering Dedicated Agent Infra •Fast moving environment •Agent Evals and benchmarking •Skills versioning and distribution •Safely expose company systems for agents
81. Platform Engineering Dedicated Agent Infra •Fast moving environment •Agent Evals and benchmarking •Skills versioning and distribution •Safely expose company systems for agents •Research and Development
82. Platform Engineering Dedicated Agent Infra Bounded Work Agent
83. Conclusion Experiment & Iterate
84. •Mastering Harness engineering • Enables shipping faster • Fewer regressions • Lower Organizational drift •No one-size-fits-all •Start experimenting and find out what works best for your team/org Conclusion Experiment & Iterate
85. @iurysza iurysouza.dev Harness Engineering How to Keep Agents On Track or
86. Thank you! @iurysza iurysouza.dev
87. Questions? @iurysza iurysouza.dev