Anybody Can Generate Code. Can Your AI Ship It?

AI coding agents can generate entire features, refactor applications and write tests in minutes. But producing code is not the same as engineering software we can confidently ship.

What happens when we stop treating AI as a coding assistant and start treating it as an engineering team?

In this session, we'll build a custom AI engineering harness using Google's Gemini, Gemma and Antigravity ecosystem, combining local, open-source and cloud models based on task complexity, cost and privacy requirements.

We'll create specialized agents for architecture, implementation, testing, code review and performance engineering. Using repository-level rules, reusable agent skills, structured handoffs and model routing, we'll orchestrate these agents into a coordinated engineering swarm.

But the real challenge starts when the agents say they're done.

We'll build independent verification layers around AI-generated code: compilation, static analysis, automated tests, UI validation, performance benchmarks and runtime telemetry. Instead of trusting an agent's claim that a task is complete, our harness will collect real engineering evidence.

When an agent introduces a regression, the verification system will identify the failure, feed measurements back into the agent workflow and require another implementation-and-validation cycle before accepting the change.

We'll also explore how local Gemma models enable private and offline development workflows, when to escalate tasks to more capable cloud models, and how multilingual voice interfaces can make agent orchestration more accessible.

The result is more than an AI coding assistant. It's the foundation of a verifiable AI engineering system, with specialized agents, automated quality gates and measurable outcomes.

Anybody can generate code now. The harder question is: can your AI actually ship it?

Transcript

1. Anybody Can Generate Code. Can Your AI Ship It? Building a personalised coding harness that earns permission to ship Rivu Chakraborty Google Developer Expert for Android & Kotlin Rivu Chakraborty: hps://rivu.dev 1
2. WHO AM I? • GDE (Google Developer Expert) for Android • Previously India’s first GDE for Kotlin • More than 14 years in the Industry • Staff Engineer @ Sarvam AI (Indus, Sarvam Code) • Worked With ◦ JioCinema/JioHotstar, Byju’s, Paytm, Gojek, Meesho, Touchlab • Author (wrote multiple Kotlin books) • Speaker • Mentor • Community Person (Started KotlinKolkata) • YouTuber (http://youtube.com/@RivuTalks) • Guitarist
3. One growing AGENTS.md carries too many jobs Every correction adds another rule The file now mixes dierent scopes Use Kotlin for the examples. Keep UI imports out of domain code. Never reset my personal phone. Run the release checks. Remember that the old API changed. Use a dierent model for reviews. Personal preferences and team policy Permanent rules and temporary facts Task procedures and tool permissions Advice and acceptance requirements The problem is mixing scope, lifecycle and authority in one always-loaded document. Rivu Chakraborty: hps://rivu.dev 3
4. More context can still miss the point Load the task’s dependency slice and the rules that govern it. Preserve decisions and evidence outside chat. Rivu Chakraborty: hps://rivu.dev 4
5. Why AGENTS.md is not enough What repository instructions can do What needs another mechanism Describe project conventions Select the procedure for this task Point to ownership and architecture Validate tool arguments and permissions Name verification commands Run checks and retain actual results Explain known traps and expected behaviour Block acceptance when evidence is missing Keep the project map. Add selection, execution and enforcement around it. Rivu Chakraborty: hps://rivu.dev 5
6. AGENTS.md as a map and contract # Repository contract Architecture: docs/architecture.md Module ownership: docs/owners.md Verification: ./scripts/verify Existing violations are backlog, not precedent. New layer violations fail architecture tests. Visible changes need platform screenshots. Short enough to read Specific enough to check Module rules stay beside the module Before editing, name the owning layer. Before finishing, map each criterion to evidence. Missing or skipped required checks block finish. Rivu Chakraborty: hps://rivu.dev 6
7. Context needs an address AGENTS.md A short repository map. Link to architecture and verification commands. Module rules Keep a module’s invariants and accepted examples beside its code. Task context Load the relevant files, contracts, incidents and acceptance fixtures. Evidence store Persist reports, screenshots and artifact digests outside transient chat. Retrieve what the current decision needs. Anthropic, Eective context engineering · OpenAI, Harness engineering Rivu Chakraborty: hps://rivu.dev 7
8. The talk at a glance Section What you will see Personalisation Context, skills, Android guidance and Agent Mode Shipping gap Generated code, accepted behaviour and scoped authority Verification Receipts, Compose UI evidence and trace analysis Worked examples Kotlin pricing, module boundaries and gate decisions Measurement Evaluation, performance claims and operating cost Discussion Implementation choices, tradeoffs and questions Rivu Chakraborty: hps://rivu.dev 8
9. 01 A harness that knows how you work Personal preferences shape the workflow. Rules and typed tools enforce policy. Rivu Chakraborty: hps://rivu.dev 9
10. What a profile is A named, versioned configuration Resolved by the harness for this session Preferences for how I work Who is working? Which repository and task? What capabilities are available? Which checks does this change need? Which policy constrains the choices? References to project instructions Task-relevant skills and tool choices Verification and routing defaults Profile is a design concept here, not a universal agent file format. Runtime policy remains authoritative. Rivu Chakraborty: hps://rivu.dev 10
11. Why a profile is useful The same engineer changes tasks Keep stable preferences, select the rest Backend API change Contract tests and migration checks Kotlin examples stay consistent Compose UI change Preview and device journey Performance investigation Trace capture and benchmark procedure Load only the relevant procedure Use the owning repository’s commands Apply the permied model and tool policy Reuse the way you work across projects without copying every project rule into a global instruction file. Rivu Chakraborty: hps://rivu.dev 11
12. AGENTS.md, profile and skill have separate jobs A concrete Kotlin UI change What the harness actually does AGENTS.md Domain cannot depend on the UI module Resolve the selected configuration Profile Kotlin examples, UI skill, repository verifier Skill Render the changed state and test retry Load the UI procedure when relevant Run the repository’s checks and tools Require current evidence before acceptance Moving a rule into a profile does not enforce it. The runtime and required checks enforce the boundary. Rivu Chakraborty: hps://rivu.dev 12
13. A personal profile has several layers You Example: concise explanations, Kotlin snippets and accessible feedback. Repository Example: owning layer, module rules, test commands and reviewer ownership. Task Example: fix tax order, preserve existing behaviour and prove acceptance. Organisation Example: local-only source, brokered credentials and an explicit release owner. Convenience seings never expand authority. Rivu Chakraborty: hps://rivu.dev 13
14. A small, explicit profile profile: mobile-engineer preferences: explanations: concise examples: kotlin project: instructions: AGENTS.md verify: [format, architecture, unit, smoke] policy: source_egress: local_only writes: worktree_only release: human_owner budgets: retries_per_failure: 2 wall_clock_minutes: 20 Rivu Chakraborty: hps://rivu.dev Illustrative schema The policy layer wins over preferences. Use repository facts and measured failures to route work. 14
15. A preference cannot grant permission Personal preference Repository policy “Use cloud when it is faster.” sourceEgress = LOCAL_ONLY A profile describes a convenient workflow. It does not approve source egress. Run locally when the task fits. Otherwise stop and ask the owner. Eective authority depends on user approval, repository policy and runtime permissions. Rivu Chakraborty: hps://rivu.dev 15
16. Corrections become maintained assets Correction Durable home Proof “Put this logic in the domain layer” Module rule + example Dependency test “Don’t wipe my test phone” Device lease + typed install tool Blocked destructive fixture “Show the actual screen” UI verification skill Screenshot + interaction probe “That advice is obsolete” Known-drift note + expiry Current source or contract test Rivu Chakraborty: hps://rivu.dev 16
17. Rules, skills, tools and extensions Mechanism What it contributes Example Rule An invariant to follow Keep UI imports in ui/ Skill A repeatable procedure Validate an Android release Tool / MCP A typed action or data connection Read a trace, run a test Extension Runtime behaviour and UI Block publish without a receipt The runtime must enforce any boundary the model can otherwise bypass. Pi, Extensions reference · MCP, Security best practices Rivu Chakraborty: hps://rivu.dev 17
18. A skill packages an operational procedure --name: verify-mobile-change description: Use for UI changes or before a mobile PR. --Read AGENTS.md. Do not restate its rules. 1. Locate the owning module and existing tests. 2. Run ./scripts/verify with the affected targets. 3. Exercise the changed journey on each platform. 4. Return verdict, commands and artifact links. Metadata first Procedure when selected References at the step that needs them Traps: missing selectors must fail the run. Load references/performance.md for perf changes. Google Antigravity, Agent Skills Rivu Chakraborty: hps://rivu.dev 18
19. The harness is the environment around the model Context Permissions Model Tools Evidence Task + project facts Allowed eects Proposed actions Scoped execution Acceptance decision The harness loads context, controls actions, observes results and decides whether work is complete. Rivu Chakraborty: hps://rivu.dev 19
20. Guidance and enforcement do different jobs Mechanism Before the action After the action Instructions Architecture notes, rules, examples Review rubric, explanation of failures Executable controls Typed tools, permission checks, codemods Compiler, contracts, tests, runtime probes A useful rule has a nearby signal that tells us whether it held. Birgia Böckeler, Harness engineering for coding agent users Rivu Chakraborty: hps://rivu.dev 20
21. The current Google tooling path Antigravity Repository rules and Agent Skills package repeatable development procedures. Android CLI Discover oicial Android tools and install the skills relevant to the change. Gemini API Use provider adapters for structured calls, model controls and validation. Gemma locally Use local inference only when the hardware, task and privacy policy fit. Legacy Antigravity workflows ended October 19, 2026. Migrate them to skills. Google Antigravity, Rules · Google Antigravity, Workflows to skills migration · Android Developers, Android CLI · Google, Gemma function calling Rivu Chakraborty: hps://rivu.dev 21
22. Useful Android and Kotlin starting points Oicial Android skills XML to Compose, AGP upgrades, Navigation, edge-to-edge and R8 Android CLI Discover tools and skills with android init and android skills list Kotlin agent skills Kotlin and multiplatform migration procedures Rivu Chakraborty: hps://rivu.dev 22
23. Android guidance becomes a repository contract Context the agent needs Evidence the harness requires Owning modules and dependency direction The aected build variants compile Design system and supported API versions Required tests actually execute Repository instructions for team rules Changed journeys run on a device A named skill for a specialised task Results identify the tested snapshot Android Developers guides explain the harness, context selection, tool feedback and trusted skills. Rivu Chakraborty: hps://rivu.dev 23
24. Agent Mode connects an intent to device feedback Input to Android Studio Expected acceptance bundle “Add retry to the checkout error state. Plan and di across aected files Keep the existing design system. Preserve the entered address. Verify empty, error and success paths.” Build and focused test results Deployed app, screenshots and Logcat Retry journey and preserved-state check Agent Mode can plan, edit, build and inspect a connected device. Your acceptance criteria still need evidence. Rivu Chakraborty: hps://rivu.dev 24
25. The extension lifecycle Session starts Before tool Tool runs After tool Finish request Load profile Authorise eect Record outcome Update evidence Validate receipt Custom tools, commands, status UI and reload hooks belong at explicit lifecycle points. Pi, Extensions reference · Anthropic, Steering Claude Code Rivu Chakraborty: hps://rivu.dev 25
26. Effects are the enforcement unit A fragile command rule A protected capability Block the word “uninstall”. A broker owns installation. A lease owns the device. The request declares possible data loss. The runtime checks authority before execution. A Gradle task or shell script can still erase app data. The eect survives a command rename. Regex checks help with known mistakes. Isolation and typed capabilities cover more paths. Rivu Chakraborty: hps://rivu.dev 26
27. A protected install checks the effect Request Lease Authorise Execute Verify Install the app Own the device Declare data loss Use the broker Launch + data check A Gradle task that wipes data crosses the same protected eect as an uninstall command. Rivu Chakraborty: hps://rivu.dev 27
28. The trust boundary surrounds the tools Workspace isolation Credential broker One checkout per task. Resolve canonical paths and contain symlink escapes. Keep long-lived secrets outside code that the implementation agent can modify. Extension supply chain Network policy Review extension code, pin versions and inventory its granted capabilities. Allow intended destinations and inspect source or credential egress. Repository text is untrusted input. OS and service boundaries control credentials and data egress. OpenAI, Sandbox security · MCP, Security best practices Rivu Chakraborty: hps://rivu.dev 28
29. Routing follows constraints and observed failure Lane Candidate jobs Escalation Local model Private code, indexing, small bounded edits Human if policy forbids cloud Implementation model Changes in familiar modules Stronger lane after repeated failure Strong reviewer Contracts, edge cases, high-risk dis Human for unresolved risk Speech model Transcription and language handling Confirm ambiguous engineering intent Evaluate the lane on your tasks. Self-reported confidence is weak evidence. Google, Gemini structured outputs · Google, Gemini thinking · Google, Gemma function calling Rivu Chakraborty: hps://rivu.dev 29
30. A policy router with a hard privacy ceiling fun route(task: Task, policy: Policy): Lane { if (policy.sourceEgress == LOCAL_ONLY) { return if (canRunLocally(task)) LOCAL else HUMAN } return when { task.isReview -> INDEPENDENT_REVIEWER task.gateFailures >= 2 -> STRONG_IMPLEMENTER task.crossesModuleBoundary -> STRONG_IMPLEMENTER else -> DEFAULT_IMPLEMENTER } } // Illustrative domain types and lanes. // Policy applies before capability and cost. Rivu Chakraborty: hps://rivu.dev Illustrative adapter code Policy first Capability next Cost within the permied set 30
31. Roles create useful separation Explorer Implementer Verifier Reviewer Read-only exploration. Returns the owning module, contracts and relevant files. Owns one scoped patch. Returns the di and an explanation of changed behaviour. Runs checks independently. Returns evidence without editing the patch to pass. Challenges logic and compatibility. Returns reproducible findings or a verdict. Context map Scoped patch Evidence receipt Review verdict Parallel editors need disjoint ownership and an integration check. Rivu Chakraborty: hps://rivu.dev 31
32. Coordination has a bill Add a specialist when its independent evidence is worth the hando cost. Rivu Chakraborty: hps://rivu.dev 32
33. One harness I work with: sarvam-code Internal coding harness I work on and use sarvam-code across product work at Sarvam AI. Built by a wider team The examples reflect shared engineering work. My perspective Lessons from backend, harness, web, mobile and desktop work. Your application The paerns transfer to the coding tools you use. I use these lessons across tools and projects. Rivu Chakraborty: hps://rivu.dev 33
34. The most optimistic code reviewer The implementation agent should never be the only source of acceptance evidence. Rivu Chakraborty: hps://rivu.dev 34
35. “Done” has several meanings Compiles Passes existing tests Implements the request Ships safely The compiler accepts the code. A missing feature can still compile. Existing assertions pass. They may never exercise the requested behaviour. Each requirement has a specific implementation and independent evidence. The installed package works with real data, permissions and runtime limits. A compiler result, a test report, requirement coverage and an installed-app check answer dierent questions. Rivu Chakraborty: hps://rivu.dev 35
36. Two green builds, two broken outcomes The installed app failed The benchmark measured nothing The sandbox passed because its setup supplied a resource path. The installed desktop app did not have that path and failed to launch. A missing selector silently skipped the interaction. The benchmark returned 0 ms because the journey never ran. Rivu Chakraborty: hps://rivu.dev 36
37. The harness owns the path to acceptance Intent Scoped work Goal + constraints Context + policy Tool execution Isolated environment Evidence Acceptance Trusted checks CI + owner The model proposes actions. The harness controls eects and acceptance. Birgia Böckeler, Harness engineering for coding agent users Rivu Chakraborty: hps://rivu.dev 37
38. 02 Verification earns the verdict Acceptance needs fresh, complete evidence from a trusted producer. Rivu Chakraborty: hps://rivu.dev 38
39. A completion state machine Changed Checked Reviewed Accepted Released Receipt invalid Required checks pass Fresh verdict Policy satisfied Artifact verified Any source, test, config or dependency change invalidates aected evidence. Rivu Chakraborty: hps://rivu.dev 39
40. The acceptance contract { "snapshot": "sha256:<inputs>", "policy_version": "v1", "requirements": { "R1": ["contract-test:tax-order"], "R2": ["smoke:installed-launch"] }, "checks": {"unit": "PASS", "smoke": "NOT_RUN"}, "review": {"snapshot": "sha256:<inputs>", "verdict": "PASS"}, "artifact": {"digest": "sha256:<bytes>"} Snapshot Matches this code Required smoke NOT_RUN Decision BLOCK } // Required smoke is NOT_RUN: completion is blocked. Google, Gemini structured outputs Rivu Chakraborty: hps://rivu.dev 40
41. 03 A tiny harness you can break Discount before tax. Prove 8,400 paise, then reject stale evidence. Rivu Chakraborty: hps://rivu.dev 41
42. The demo contract Feature For this demo, apply the discount before tax. Use integer paise and basis points. Evidence A literal 8,400-paise fixture and edge cases challenge the implementation. Gate The verifier binds its result to code, tests, policy and the current input snapshot. Aack Edit after green, replay the old receipt and watch acceptance return BLOCK. The dependency-free demo runs oine. Voice and model calls are optional adapters. Rivu Chakraborty: hps://rivu.dev 42
43. First, a plausible bug fun total(subtotal: Long, discount: Long, taxBps: Int): Long { // Bug: tax uses the undiscounted subtotal. val tax = (subtotal * taxBps + 5_000) / 10_000 return subtotal - discount + tax } Tax: 500 10,000 × 5% // Amounts in paise; tax in basis points. // total(10_000L, 2_000L, 500) // Expected: 8_400L. Actual: 8_500L. Expected: 8,400 Bug: +100 paise Rivu Chakraborty: hps://rivu.dev Payable: 8,500 8,000 + 500 43
44. Then, a scoped fix with evidence fun total(subtotal: Long, discount: Long, taxBps: Int): Long { val discounted = subtotal - discount val tax = (discounted * taxBps + 5_000) / 10_000 return discounted + tax } Tax: 400 8,000 × 5% // check(total(10_000L, 2_000L, 500) == 8_400L) // verify: PASS; finish: ACCEPT // Edit a source file after verify. // finish: BLOCK, snapshot changed verify: PASS finish: ACCEPT Rivu Chakraborty: hps://rivu.dev Payable: 8,400 8,000 + 400 Edit after green: BLOCK stale receipt 44
45. A finish tool consumes the gate verdict suspend fun finishTask(ctx: HarnessContext): FinishResult { val decision = ctx.verifier.accept( snapshot = ctx.workspace.snapshot(), policyVersion = ctx.policy.version ) if (decision.status != ACCEPT) { return Blocked(decision.reason) } return Ready(decision.artifacts) } // Illustrative host adapter, not a vendor API. // CI and release consume the same trusted verdict. Rivu Chakraborty: hps://rivu.dev Illustrative host API Tool names and event shapes vary. A finish hook alone cannot prevent a bypass. 45
46. Worked example: a receipt earns acceptance data class Receipt( val snapshot: String, val tests: Int, val passed: Boolean ) fun finish(now: String, r: Receipt?): String = when { r == null -> "BLOCK: missing receipt" r.snapshot != now -> "BLOCK: stale snapshot" r.tests == 0 -> "BLOCK: zero tests" !r.passed -> "BLOCK: failed checks" else -> "ACCEPT" } // Simplified decision logic. Production also validates // test identities, policy, provenance and artifacts. Current A + receipt A 5 tests passed ACCEPT Current B + receipt A BLOCK stale snapshot Current A + 0 tests BLOCK zero tests No receipt BLOCK missing receipt Expected outcomes from the simplified Kotlin function. A local receipt alone does not authenticate its producer. Rivu Chakraborty: hps://rivu.dev 46
47. Each requirement needs its own witness R1: discount before tax R2: installed app launches Implementation: Pricing.kt total() Artifact: release package digest Fixture: total(10_000L, 2_000L, 500) Expected result: 8_400L Evidence: independent acceptance test Journey: install, launch and interact Evidence: smoke result + screenshot Inputs: matching package and environment A unit test can prove R1 while R2 remains NOT_RUN. Required missing evidence blocks acceptance. Rivu Chakraborty: hps://rivu.dev 47
48. Requirement coverage needs a witness Criterion Implementation Independent evidence Discount before tax Pricing.kt: total() Literal example + boundary values No lost existing schedules scheduler registration Upgrade scenario from old data Every new screen reports load screen instrumentation Coverage test + trace span Works from the installed app packaging + resources Clean install and launch A checked box without implementation and evidence remains open. Rivu Chakraborty: hps://rivu.dev 48
49. Different checks catch different failures Evidence Strong at Common blind spot Compiler + static analysis Types and known structural rules Missing product behaviour Unit + property tests Specified invariants and edge cases A wrong shared assumption Contracts + integration Compatibility across boundaries Real installation conditions UI + accessibility Rendered behaviour and interaction Unobserved device states AI + human review Intent, logic and tradeos Variance, time and missing context Rivu Chakraborty: hps://rivu.dev 49
50. Risk chooses the required evidence Change Required locally Required before merge Text or styling Build + screenshot Visual and accessibility review Business rule Unit + edge cases Independent acceptance examples Public API / storage Contract + migration test Compatibility and rollback review Permissions / secrets Negative policy tests Security owner and trusted CI Performance Deterministic budget test Controlled benchmark + trace An approved exception records the owner, reason, expiry and compensating check. Rivu Chakraborty: hps://rivu.dev 50
51. Freshness depends on the input snapshot val snapshot = hashInputs( baseCommit, sourceTree, tests, lockfiles, toolchainVersion, policyVersion ) val receipt = trustedRunner.verify(snapshot) check(receipt.snapshot == currentSnapshot()) check(receipt.requiredChecksAllPass) check(receipt.requirementsHaveEvidence) check(receipt.producer == trustedRunner.identity) A branch name is mutable. A timestamp is insuicient. A changed working tree needs new evidence. // Freeze inputs while checking. // Recheck the snapshot before acceptance. Rivu Chakraborty: hps://rivu.dev 51
52. The old green receipt is now stale Changing code, tests, policy or dependencies changes the input snapshot. Reverify before acceptance. Rivu Chakraborty: hps://rivu.dev 52
53. A red job must block the merge Required checks Every required guard must feed a status that branch protection requires. Skipped jobs A skipped or cancelled job needs an explicit outcome. Missing evidence blocks. Bypass paths Audit admin bypass, alternate push routes and direct publishing permissions. Gate outage If the verifier crashes or times out, protected merge and release stay blocked. A badge is a display. A branch rule is enforcement. GitHub, Protected branches Rivu Chakraborty: hps://rivu.dev 53
54. A gate earns trust when it can fail Known bad fixture Reintroduce a known bug. The gate must return FAIL with a useful reason. Missing evidence Delete a report, omit a budget or run zero tests. Completion must BLOCK. Stale evidence Edit code or change the base after green. Reject the previous receipt. Unavailable verifier Crash or time out the verifier. Its missing result must never become PASS. A negative control protects the check’s meaning. Rivu Chakraborty: hps://rivu.dev 54
55. Worked example: a forbidden dependency data class Edge(val from: String, val to: String) fun violations(edges: List<Edge>): List<Edge> = edges.filter { it.from == "domain" && it.to == "ui" } val bad = listOf(Edge("domain", "ui")) val safe = listOf(Edge("ui", "domain")) check(violations(bad).size == 1) check(violations(safe).isEmpty()) // Teaching input: an already-resolved module graph. // A real scanner obtains edges from the build model. Bad input domain depends on ui BLOCK 1 forbidden edge Safe input ui depends on domain PASS 0 forbidden edges The verifier reports resolved module edges, policy version and the tested snapshot. CI requires that result. Rivu Chakraborty: hps://rivu.dev 55
56. A green review is one observation Review practice Why it maers Repeat identical reviews Reveal variance Compare reviewer lanes Find missed bugs Combine findings Widen coverage Validate the validator Check false rejections Replay small, labelled PRs Measure useful recall Treat a green review as one observation. Require independent checks. Rivu Chakraborty: hps://rivu.dev 56
57. A reviewer needs an evaluation set Replay Replay the patch from before a human found a bug. Keep the bug label hidden. Controls Include clean changes so a reviewer that flags everything cannot score well. Repetitions Repeat identical inputs. Report per-task variation and missed bugs. Metrics Measure recall, precision, false rejections and findings humans accept. Keep a held-out set. Validate rejected findings as well as accepted ones. Anthropic, Demystifying evals for AI agents · OpenAI, Custom Code Review rules Rivu Chakraborty: hps://rivu.dev 57
58. The harness needs regression tests Area Adversarial fixture Provider adapter Unsupported role, malformed arguments, empty output Context Dropped instructions, stale index, failed compaction Tool control Symlink escape, indirect mutation, duplicate call Evidence Changed tree, forged producer, missing required check Workflow Hook not registered, skipped loop item, unrequired job Record outcomes and traces. Change one component at a time. Anthropic, Harness design for long-running apps Rivu Chakraborty: hps://rivu.dev 58
59. A passing status needs a denominator Record the actual test count and required test identities. An empty selection must block completion. Rivu Chakraborty: hps://rivu.dev 59
60. Turn recurring failures into controls Boundaries Guard data-changing eects and coordinate ownership of shared devices. Coverage Exercise nulls, existing jobs, release shrinking and removed fallback paths. Completion Review the exact input snapshot and prove every promised requirement. Cost Count cache reads, output tokens, repair aempts and accepted changes. Repeated repair rounds signal a design problem. Revisit the workflow. Rivu Chakraborty: hps://rivu.dev 60
61. Compose skill gives the agent source context aldefy / compose-skill How I would use it Guides for state, lists, eects and navigation Input: item state breaks after reordering Compose Multiplatform references Read list and state guidance Check source against project versions Propose a narrow Kotlin change Run a reorder regression on the UI Linked AndroidX and CMP source excerpts Task routing into relevant references Reference source supports a claim. A test or trace supports the behaviour of your app. Rivu Chakraborty: hps://rivu.dev 61
62. Stable keys preserve the identity of a row // Before: identity follows the item position. LazyColumn { items(people) { person -> PersonRow(person) } } // After: identity follows a unique person ID. LazyColumn { items(people, key = { it.id }) { person -> PersonRow(person) } } // PersonRow owns remember { mutableStateOf(false) } // Test input: expand B, then prepend X. Input IDs [A, B, C] B expanded After insert [X, A, B, C] Expected result B stays expanded A stays collapsed A row-identity regression checks behaviour. It does not prove that scrolling is faster. Rivu Chakraborty: hps://rivu.dev 62
63. A changed screen needs runtime evidence Build and inspect Exercise the journey Compile shared and platform code. Run the owning module’s tests. Check architecture and semantics. Capture the actual rendered screen. Tap through the installed app. Exercise empty and error states. Check font scale and accessibility. Bind evidence to the tested build. Choose evidence that can reveal the failures the change could introduce. Rivu Chakraborty: hps://rivu.dev 63
64. ComposeProof adds UI observations to the loop Observations through MCP Where the evidence applies Headless preview rendering Preview: inspect a controlled screen state Golden comparison and reports Golden: compare an approved baseline Device interaction and UI inspection Device: test navigation and interaction Runtime state and recomposition tools Runtime: inspect the installed app state Some capabilities need Pro or an embedded SDK. Headless output does not establish Android runtime parity. Rivu Chakraborty: hps://rivu.dev 64
65. A preview fixture makes the UI claim inspectable @Preview(widthDp = 320, fontScale = 1.5f) @Composable fun CheckoutErrorPreview() { AppTheme { CheckoutScreen( error = "Payment failed. Try again.", onRetry = {} ) } } // AppTheme and CheckoutScreen are app APIs. // Input: compact width, large text, error state. Expected checks Message is visible Retry is reachable No clipped text On-device check Retry changes state Address survives Illustrative fixture and expected results. Retain screenshots and assertion outcomes, then verify the live journey. Rivu Chakraborty: hps://rivu.dev 65
66. 0 4 Rivu Chakraborty: hps://rivu.dev Performance and the harness itself Validate the measurement, test the gates and track cost per accepted change. 66
67. Compose performance skills structure diagnosis skydoves / compose-performance-skills An evidence-producing procedure Stability and recomposition Input: dropped frames during fast scroll Lazy layouts and modifiers Capture a release-build baseline Read compiler and runtime evidence Change one supported hypothesis Repeat the same measured journey Eects, build configuration and measurement Focused SKILL.md procedures Skippability and recomposition counts are diagnostic signals. Frame timing tests the user-visible claim. Rivu Chakraborty: hps://rivu.dev 67
68. Performance evidence has different clocks Signal Where it runs How to use it Frame / recomposition budget Focused PR test Hard gate in a controlled test Wall-clock ceiling Suitable CI worker Detect large regressions with known noise Macrobenchmark + Perfeo Controlled device lane Compare repeated release-like runs Runtime telemetry Production Detect real-world regressions and investigate Every measurement needs a valid journey, sample count and environment identity. Android Developers, Benchmark your app · Android Developers, Measuring performance Rivu Chakraborty: hps://rivu.dev 68
69. A benchmark must prove the journey ran False green Valid measurement Selector missing Interaction skipped Sample count: 0 Duration: 0 ms Selector found Journey completed Expected sample count recorded Environment identity recorded Invalid run: BLOCK Only then compare the budget Missing selectors and empty samples are failures. Zero duration alone cannot establish speed. Rivu Chakraborty: hps://rivu.dev 69
70. Missing measurements need their own state enum class BudgetStatus { PASS, OVER, NO_DATA, NO_BUDGET } fun evaluate(marker: Marker, samples: List<Long>): BudgetStatus { val cap = budgets[marker] ?: return BudgetStatus.NO_BUDGET if (samples.isEmpty()) return BudgetStatus.NO_DATA return if (samples.median() <= cap) BudgetStatus.PASS else BudgetStatus.OVER } Illustrative Kotlin Separate absence from success. Capture the trace beside the result. check(device.waitForHome()) { "Home never appeared" } // Required selector misses abort the journey. Rivu Chakraborty: hps://rivu.dev 70
71. Zero can be a missing-data bug A successful run needs proof that the measured journey actually happened. Rivu Chakraborty: hps://rivu.dev 71
72. Perfetto skills ground a diagnosis in a trace Input to the investigation Output I can review A trace and the aected app or process Queries against the discovered schema A specific slow interval or user journey Relevant events, durations and intervals Device, build and capture configuration A hypothesis tied to those observations A question the trace can answer A repeatable before-and-after capture google/perfeo ai/skills is source material. Use the documented packaged installation, then record and analyse. Rivu Chakraborty: hps://rivu.dev 72
73. The query and its result belong beside the claim -- Illustrative slice rows, durations in ms: -- demo/Checkout: 20, 15 -- demo/Decode: 8 -- demo/Layout: 4 SELECT name, ROUND(SUM(dur) / 1e6, 1) AS total_ms FROM slice WHERE dur > 0 AND name GLOB 'demo/*' GROUP BY name ORDER BY total_ms DESC; Calculated output Checkout 35.0 ms Decode 8.0 ms Layout 4.0 ms Next question Which thread and interval explain the slow frame? Teaching data, not app benchmark results. Slice durations can overlap and do not equal CPU time or prove causality. Rivu Chakraborty: hps://rivu.dev 73
74. The user runs an artifact Development path Acceptance path Build from the checkout. Run unit and integration tests. Use injected paths or fixtures. Install the actual release package. Launch through its normal entry point. Exercise the user journey. Capture evidence and the package digest. These prove behaviour in that environment. Real launch conditions expose assumptions the test environment supplied. Rivu Chakraborty: hps://rivu.dev 74
75. Release keeps the tested bytes Build Verify Aest Promote Observe All targets Installed artifact Digest + provenance Same bytes Health + rollback Promotion uses the verified artifact. Runtime signals can stop the rollout. SLSA v1.2, Provenance Rivu Chakraborty: hps://rivu.dev 75
76. The agent needs observable consequences Tool span Record tool, argument class, duration, result and retry reason for each call. Verification span Record snapshot, selected checks, real test count, exit status and artifacts. Product span Record whether the actual user journey completed and where it failed. Feedback Turn a failure trace into a reproducer, then set a bounded repair task. Redact source, secrets and personal data before telemetry leaves the environment. OpenTelemetry, Traces Rivu Chakraborty: hps://rivu.dev 76
77. Cost per accepted change Cost driver Useful control Repeated input and cache misses Stable prefix, small retrievals, scoped summaries Duplicate specialist context Limit fan-out and return evidence links Repeated repair loops Failure taxonomy, escalation and redesign Human review time Small dis, clear coverage and reproducible findings Tool and environment time Reusable fixtures, measured setup, reliable cleanup Track total cost and elapsed time against accepted outcomes. Rivu Chakraborty: hps://rivu.dev 77
78. Retries can erase the saving per call Route A: cheaper aempts Route B: fewer repairs Illustrative comparison ₹1 per aempt × 6 aempts Accepted changes: 1 Illustrative comparison ₹3 per aempt × 1 aempt Accepted changes: 1 ₹6 ₹3 per accepted change per accepted change Include human review eort, tool cost and environment time in the real comparison. Rivu Chakraborty: hps://rivu.dev 78
79. Autonomy needs a stopping policy Retry budget Set a repair limit for each reproduced failure. Example policy: two aempts. Escalation Escalate with the failing evidence when a dierent model or owner can help. Redesign Repeated failures in one subsystem warrant a design review before another retry. Stop condition Finish only when required checks pass, coverage is complete and work has stopped. Removing an obsolete harness component is an improvement too. Anthropic, Harness design for long-running apps · Anthropic, Eective harnesses for long-running agents Rivu Chakraborty: hps://rivu.dev 79
80. Another retry needs a reason Bound repairs for a reproduced failure. Repeated failures in one subsystem call for a design review. Rivu Chakraborty: hps://rivu.dev 80
81. Voice changes the input, not the acceptance bar Speech Transcript Structured intent Engineering loop Spoken result Indic + English Saaras Goal + criteria Same permissions Gate verdict Proposed integration. Confirm ambiguous paths, numbers and destructive intent. Sarvam, Saaras · Sarvam, Choosing a speech API Rivu Chakraborty: hps://rivu.dev 81
82. What changed how I use coding agents Dropped instruction role An adapter dropped instructions. Test request bytes and one behaviour probe. Unregistered completion tool The agent could not call finish. Assert registration and run a real end-to-end turn. Confident wrong blame A mismatched environment caused a 401. Reproduce locally before blaming a provider. Verification that never ends Choose a focused check, a relevant sweep and a clear stopping condition. I turn recurring failures into checks I can reuse. Rivu Chakraborty: hps://rivu.dev 82
83. A 200 response can still lose the instructions Wire contract Behaviour probe Required instructions reach the provider. Tool names and schemas match. Arguments parse into the expected call. Empty output produces an explicit error. Read a fixture file. Edit a disposable copy. Run the focused check. Call the registered finish tool. Validate the request bytes, then exercise a real turn through the adapter. Rivu Chakraborty: hps://rivu.dev 83
84. Your first week of harness engineering Day 1 Day 2 Day 3 Day 4 Day 5 Choose one real failure. Record the triggering input and expected outcome. Add an enforcing check with one failing fixture and one safe counterexample. Make completion consume the verdict and make CI require the same check. Replay known bugs and clean changes. Inspect missed bugs and false positives. Measure accepted outcomes, total cost, elapsed time and human review eort. Keep the parts that improve outcomes. Retire the parts that no longer help. Rivu Chakraborty: hps://rivu.dev 84
85. Anybody can generate code. Shipping still needs engineering. Questions? sessionize.com/rivu-chakraborty speakerdeck.com/rivuchk Rivu Chakraborty: hps://rivu.dev 85
86. A Implementation reference Appendix and source library Rivu Chakraborty: hps://rivu.dev 86 / appendix
87. Harness implementation map Engineering concern Where to implement it Rules with consequences Corrections, typed eects, trust boundaries Independent verification Reviewer replay, acceptance, requirement coverage Context and cost Scoped retrieval, indexes, routing and budgets Automation reliability Harness regression tests and bounded repair loops Continuous improvement Evaluation fixtures and the first-week plan Use this map to place each recurring failure in an enforceable control. Rivu Chakraborty: hps://rivu.dev 87 / appendix
88. A repository layout that stays navigable AGENTS.md docs/ architecture.md known-drift.md .agents/skills/verify-mobile/SKILL.md harness/ policy.json adapters/ checks/ fixtures/ schemas/ receipts/ # generated, not authority by itself scripts/verify scripts/finish Rivu Chakraborty: hps://rivu.dev Suggested layout Keep source policy versioned. Protect acceptance credentials outside the checkout. 88 / appendix
89. Receipt validation checklist Identity Validate the trusted producer, repository, policy version and intended task. Freshness Match base, source tree, tests, dependencies and environment to current inputs. Completeness Require every mandated check and a witness for each acceptance criterion. Integrity Verify artifact digests and immutable reports. Keep signing keys outside the worker. Decision Accept explicit PASS or an authorised waiver. FAIL, NOT_RUN and missing data block. Do not infer acceptance from a missing field or a model’s prose. Rivu Chakraborty: hps://rivu.dev 89 / appendix
90. Pi as a programmable host Interception Intercept the proposed tool call before the eect. Block it when policy denies it. Tools and commands Expose a narrow capability with validated arguments instead of arbitrary shell. UI and reload Show evidence state and release resources on shutdown or extension reload. Packaging Share reviewed, versioned extensions and skills. Treat extension code as trusted. Use current documented event contracts. Runtime extension code is trusted code. Pi, Extensions reference Rivu Chakraborty: hps://rivu.dev 90 / appendix
91. Contract tests protect provider seams check(outbound.messages.first().role in supportedRoles) check(requiredRule in flatten(outbound.messages)) check(outbound.tools == expectedToolSchema) check(parseToolCall(fixture) == expectedCall) check(emptyOutputProducesExplicitError(response)) // Then exercise one real turn on a disposable fixture: // read, edit a copy, run a focused check, call finish. // Illustrative adapter domain types. Rivu Chakraborty: hps://rivu.dev Test both bytes and behaviour. Protocol compatibility needs more than a 200 response. 91 / appendix
92. A shrinking architecture baseline val actual = scanLayerViolations() val baseline = loadBaseline() val added = actual - baseline val stale = baseline - actual check(added.isEmpty()) { "New dependency violations: $added" } check(stale.isEmpty()) { "Delete resolved baseline entries: $stale" } Rivu Chakraborty: hps://rivu.dev Illustrative Kotlin Existing debt is explicit. New debt fails. Fixed debt leaves the baseline. 92 / appendix
93. Android and KMP acceptance examples Risk Scenario Release shrinking Exercise code that depends on names after R8 Upgrade semantics Run existing data and background-job registrations Shared code Compile and run aected Android, iOS and desktop paths UI behaviour Dierent font scales, locale, theme and empty states Performance Release-like artifact, controlled device, valid journey Packaging Install and launch through the ordinary entry point Android Developers, Measuring performance · Android Developers, Android skills Rivu Chakraborty: hps://rivu.dev 93 / appendix
94. Voice-specific failure modes Names and paths Display the transcript and ask for clarification when a path or name is ambiguous. Numbers and negation Confirm amounts, versions and negation before a destructive or expensive action. Language conversion Keep original words beside normalised criteria so the user can catch meaning drift. Authority Interpret the requested task. A transcript cannot approve unrelated destructive work. Text fallback and accessible feedback remain part of the workflow. Rivu Chakraborty: hps://rivu.dev 94 / appendix
95. Provider adaptation checklist Contract Probe Instruction roles Required instructions survive normalisation Tool schema Supported types, names and argument validation Reasoning budget Room remains for a visible action or final result Context recovery Overflow, compaction and retry keep task identity Streaming Reconnect and duplicate-event handling Model selection Unavailable route produces an explicit error Rivu Chakraborty: hps://rivu.dev 95 / appendix
96. Security tests for the outer harness Aempt Expected result README asks to upload credentials Treat content as data and reject egress Shell script bypasses a tool restriction OS / service boundary still blocks the eect Worker edits acceptance policy Protected policy version does not change Extension loads unreviewed code Trust and package policy blocks activation Hook crashes during publish Publish remains blocked Two sessions install on one phone Second operation cannot acquire the lease MCP, Security best practices · OpenAI, Sandbox security Rivu Chakraborty: hps://rivu.dev 96 / appendix
97. Evaluation metrics with denominators Question Metric How often is a task solved? Successful tasks / all aempted tasks How many known bugs are found? True bugs found / labelled bugs How noisy is the reviewer? True findings / all reported findings How repeatable is the result? Per-task success across repeated trials What does acceptance cost? Total cost / accepted changes Who catches escaped bugs? Escapes by stage, severity and change type Anthropic, Demystifying evals for AI agents Rivu Chakraborty: hps://rivu.dev 97 / appendix
98. A handoff that survives a fresh session Task: <goal and acceptance criteria> Snapshot: <base, head, working-tree identity> Decisions: <choice, evidence, rejected alternative> Completed: <requirement -> artifact or test> Open: <failure, exact reproduction, next action> Processes: <running jobs and resource leases> Risks: <owner and decision needed> Next check: <smallest command that resolves uncertainty> Proposed hando format Facts and decisions first. Links replace giant log dumps. Anthropic, Eective harnesses for long-running agents · Anthropic, Eective context engineering Rivu Chakraborty: hps://rivu.dev 98 / appendix
99. Configuration, extension or fork? Approach Use when Maintenance burden Configure Rules, skills and existing tools cover the need Low Extend You need a new capability or runtime hook Moderate Embed / SDK A product needs its own interface and lifecycle Higher Fork A required core behaviour cannot be exposed safely Highest Own the loop only when the missing control justifies maintaining it. Rivu Chakraborty: hps://rivu.dev 99 / appendix
100. Further resources hps://speakerdeck.com/rivuchk/anybody-can-generate-code-can-your-ai-ship-it-devfest-hyderabad-2026 Engineering Verification Tools & agents Mobile & models Anthropic: Agent evals Pi: Extensions Android: Agent skills OpenAI: Code review rules Kotlin: Agent skills OpenAI: Sandbox security Claude Code: Steering agents Antigravity: Rules MCP: Security practices Antigravity: Agent skills Gemini: Thinking GitHub: Protected branches Antigravity: Skills migration Gemma: Function calling SLSA: Provenance Android: CLI Android: Performance Android: Agent guides ComposeProof: UI evidence Compose: Performance skills Sarvam: Saaras Android Studio: Agent Mode Perfeo: Agent skills Compose: Source guidance OpenTelemetry: Traces Adit Lal: Harness engineering Böckeler: Harness engineering OpenAI: Harness engineering Anthropic: Context engineering Anthropic: Long-running agents Anthropic: Harness design Rivu Chakraborty: hps://rivu.dev Gemini: Structured outputs 100 / appendix