Harness & Loop Engineering: How We Let AI Agents Build Production Code Safely, Fast, and On Repeat

A session at LINE THAILAND DEVELOPER CONFERENCE 2026

Transcript

1. Harness & Loop Engineering How we let AI agents build production code safely, fast, and on repeat
2. Have you ever encountered this?
3. Engineer designed, AI delivered
4. Agenda • Harness — what the agent knows • Loop — how it checks itself • Challenges — what it struggles
5. What we shipped 23 services · 11 Kafka topics · 4 db · 2 gateways · 2 clients
6. Writing code got fast. Checking it did not.
7. Why AI coding gets stuck The agent is quick. Trusting it is slow. It doesn't know our rules Hexagonal layering, Ginkgo BDD, design tokens — so we review style, not logic. It can't run our stack APISIX, Kafka, MongoDB, k3d. “It works” stays a guess until a person checks. It doesn't know when it broke something We find out in review, or in QA, days later.
8. 01 HARNESS What the agent knows before it writes
9. Harness is the system framework designed for agents to operate, access tools, and execute safely.
10. Three things we give it Written once. Read by every agent, every session. AGENTS.md Skills MCP Rules, stack and A procedure it loads Jira, Confluence, Figma, conventions — one file. when the task matches. GitHub
11. AGENTS.md — the core of harness first thing every agent reads
12. The rules, as the agent reads them An excerpt from AGENTS.md — decidable, not aspirational. ## Core rules - **Inspect before edit.** Read existing code first. - **Smallest safe diff.** No refactor beyond the ask. - **Tests with changes.** Ship them together. - **Pre-commit gates.** Never bypass; fix the cause. ## Coding standards — Go - Hexagonal architecture; dependencies flow inward. - Every external dependency behind an interface. - Testing — REQUIRED — Ginkgo V2 + Gomega, gomock. - Wrap errors with context: fmt.Errorf("...: %w", err) - Never log full payloads or sensitive fields.
13. Skills — one skill, end to end A procedure, written once — and found by every tool. --name: clickhouse-cluster --name: create-new-service shard or replicated cluster. description: Use when user needs to create new microservices with scaffolding tool, requests service scaffolding, or mentions consumer/producer/worker service setup --- --- # Engineering conclusion # Creating Services with Scaffolding description: Use when designing, creating, migrating, querying, or troubleshooting ClickHouse databases and tables on a multi- Without `ON CLUSTER`, the DDL runs only on the node receiving the request. `MergeTree` stores data only on that node and does not replicate it. This creates inconsistent schema, no replica failover, and incomplete reads. Use three layers instead: 1. A `Replicated` database for metadata replication. 2. A shard-local `ReplicatedMergeTree` table for durable replicated storage. 3. A `Distributed` table for application reads and writes across shards. Replicas contain copies by design. If a Distributed table contains 1100 logical scaffolds microservices with a 5-step meta-values workflow. ## Naming Rule: No `example-` prefix Service directories in this monorepo use **short names without the `example-` prefix**. Pass the short name directly to `create-meta-values` so the directory is created correctly from the start. | ❌ Wrong | ✅ Correct | |----------|------------| | `example-server` | `server` | | `example-consumer-import` | `consumer-import` | | `example-server-chat` | `server-chat` |
14. A skill is a procedure it can load The knowledge that used to live in one person's head. Written once, read by every tool One folder, symlinked wherever each tool looks for it. /skills symlinked /.claude/skills Using for Claude /.agent/skills Using for Codex /.github/skills Using for GitHub Copilot
15. Our design system, as skills Jansson (JDS) — a plugin that brings its own Figma connection. DESIGN.md color, font, spacing etc. Figma SKILL.md Rules to keep agent to generate UI stays consistent Keep design system details as COMPONENTS.json Keep mapping component on Figma
16. Our design system, as skills Jansson (JDS) — a plugin that brings its own Figma connection. --name: core-foundation description: Use for anything related to the Jansson Design System (JDS), including foundations, components, tokens, DESIGN.md updates, Figma/MCP design checks, and synchronization of design decisions into implementation. --# Core Foundation ## Goal Implement and evolve Jansson core styling from a single design contract so agent-generated UI stays consistent. ## Primary source Figma { "paginations": { "label": "Paginations", "nodeId": "118:2219", "type": "page", "description": "Pagination controls for multi-page content" }, ... - Treat `DESIGN.md` in this skill as the design contract. - Use `COMPONENTS.json` to locate component node IDs before fetching Figma context. - When applying updates, read the Figma source through MCP before making changes. - Treat these Figma files as general reference defaults (use when no task-specific Figma URL is provided): - Component reference: `https://www.figma.com/design/ oYV4QfLuCVz9jpI3diFTlb/JDS-Component-v1.0`. - Foundation reference: `https://www.figma.com/design/ 9yJJNkly0ocAUwPsDI6W00/JDS-Foundation-v1.0`. - If the caller provides a specific Figma URL, use that URL as the primary design source for the task and keep the defaults as secondary reference only. - Select exactly one MCP endpoint for the session: - `https://mcp.figma.com/mcp` (hosted MCP) - `http://127.0.0.1:3845/mcp` (local MCP via Figma Desktop App) - Keep `DESIGN.md` self-contained. Do not reference repository paths inside the file. - Preserve the two-layer structure: - YAML frontmatter for machine-readable tokens. - Markdown sections for intent, rationale, and constraints. --name: Jansson Design System description: Neutral-first operational product UI foundation with semantic accents, consistent hierarchy, and accessibility-focused token usage. colors: primary: '#3C41FF' brand: '#00D782' white: '#FFF' black: '#000' navy: 4: '#F7F7F8' 6: ‘#F2F3F4' ... typography: bp-park-700-32: fontFamily: bp-park fontSize: 2rem fontWeight: 700 lineHeight: 2.625rem bp-park-500-32: fontFamily: bp-park fontSize: 2rem fontWeight: 500 lineHeight: 2.625rem ... components: button-primary: typography: '{typography.bp-park-500-16}' textColor: '{colors.white}' backgroundColor: '{colors.indigo.50}' rounded: '{rounded.4}' padding: '{spacing.12} {spacing.16}' button-primary-hover: backgroundColor: '{colors.indigo.20}' button-primary-active: backgroundColor: '{colors.indigo.90}' button-primary-disabled: backgroundColor: '{colors.navy.20}' } COMPONENTS.json SKILL.md DESIGN.md
17. MCP — It reads the source Most bad output is a missing-input problem. Jira tickets and acceptance criteria no more pasting them in Confluence specs and API contracts no more re-explaining Figma tokens, variables, screenshots no more reading pixels GitHub PRs, CI status, review threads no more copying output
18. Where the harness lives One repository. Every agent reads the same things. example-app—monorepo/ ├── AGENTS.md rules · stack · test gates ├── .pre-commit-config.yaml the gate ├── .mcp.json Jira · wiki · docs ├── skills/ source of truth │ ├── create-new-service/ │ └── clickhouse-cluster/ | └── k3d-local-test/ ├── .claude/skills → skills/ Claude Code ├── .github/skills → skills/ Copilot ├── .agents/skills → skills/ Codex ├── backend/ Go · hexagonal · Ginkgo ├── frontend/ Vue 3 · Vitest · Playwright ├── database/ MongoDB · ClickHouse └── script/ k3d · APISIX · coverage
19. 02 LOOP How it finds out it was wrong
20. Every failure goes back into the loop And the agent is the one who picks it up. .pre-commit-hook-config.json The agent The check The commit writes runs lands PASS FAIL It reads the error and fixes it Test results It goes round this loop many times before anyone sees the change.
21. A git hook. Not a CI job. The agent commits. The hook fails. The agent reads the error and fixes it. Nobody waits.
22. Four checks, in order The agent runs the first three itself. 01 The agent's own run go test ./… · npm run test 02 pre-commit hook lint · types · unit · coverage · integration 03 pre-push hook build · Playwright E2E · screenshots 04 CI the full suite, as a required check A failure at any step goes straight back to the agent — not into a queue.
23. What has to pass before a commit Each check catches a different kind of mistake. 01 02 03 04 Format Unit Integration E2E tests & lint tests tests before push Catches style, Checks that each Checks that Runs real user flows syntax, type errors function or services work with in the browser and and leaked secrets. component logic real MongoDB, compares works on its own. Kafka and APISIX screenshots. routes. Each step covers a gap the one before it leaves. Fast checks fail first, and only fully verified code gets committed.
24. The gate is a file in the repo Not a pipeline the agent has to wait for. fail_fast: true repos: - repo: local hooks: - id: go-lint-and-unit-tests files: ^backend/.*\.go$ stages: [pre-commit] entry: ./script/go-incremental-test-lint-fmt.sh - id: coverage-95-percent stages: [pre-commit] entry: ./script/pre-commit-backend-coverage.sh - id: backend-service-integration-test stages: [pre-commit] entry: ./script/pre-commit-backend-integration-test.sh - id: playwright-screenshots-e2e stages: [pre-push] entry: ./script/pre-commit-frontend.sh .pre-commit-hook-config.json
25. 03 Challenges What makes this harder than it looks
26. UI Development with AI Agents Lack of Visual Feedback Loops Frontend requires real visual inspection — which text only models struggle to evaluate. Accessibility & Semantic Failures AI-generated UI frequently misses proper ARIA attributes and keyboard navigation.
27. How it checks its own UI Figma is the source. A screenshot is the proof. Figma with MCP Storybook Playwright Different? The push is blocked.
28. How it checks its own UI Figma is the source. A screenshot is the proof. import { expect, type Page, test } from '@playwright/test'; test('renders the redesigned default page', async ({ page }) => { await prepareVisualPage(page, defaultStory); await expect(page.getByRole('heading', { name: 'Chat assignment' })).toBeVisible(); await expect(page.getByText('Recent activity')).toBeVisible(); await expect(page).toHaveScreenshot('unassigned-queuepage-default.png', visualOptions); }); Storybook Page toHaveScreenshot() vs the baseline
29. Real ENV testing Problem Tests pass locally, but the things break in real environments Infrastructure is hard to test before deploying Some infrastructure are set up through K8s manifest files, so mistakes only show up after we apply them to a real cluster.
30. We mirror the system, then test it The same gateway, the same manifests, a real database. K3D — Run rancher’s minimal Kubernetes cluster inside containers. Developer Focused — Ideal for rapid prototyping and local testing. CI/CD ready — Integrates seamless into automated build pipelines.
31. We mirror the system, then test it The same gateway, the same manifests, a real database. K3d live in docker Real k8s cluster Apply same K8s manifest Run Integration test suite on clone env
32. And then the bot reviews it Passing the hook is not the same as being right.
33. One last concern: the bill A tighter loop means many more runs. One proxy in front of all of them. Every tool through one prompt cache proxy The stable prefix — rules, skills, repo map — is cached One developer, one month: all of the saving came from caching
34. We Built In 3 months From approximately 6-8 months Functional and non-functional — infrastructures, load tests, CI/CD, analytics pipeline, visual regression.
35. Start with three things 1 Write your rules in one file 2 Add one pre-commit hook 3 Never skip it AGENTS.md lint and tests, on changed files no --no-verify, for anyone Skills, MCP and screenshots come after those three work.
36. The agent is not the bottleneck. The feedback loop is. None of this needed a better model. All of it is available in your repo today.
37. BUILD THE NEXT FRONTIER