Back to Research

Agent harnesses need guardrails

Standardize agent harness changes, test regressions, and keep tool scope tight across coding tools.

Editorial illustration for Agent harnesses need guardrails. The situation Agentic coding tools are no longer just chat with code.
Rogier MullerMay 4, 20265 min read

The situation

Agentic coding tools are no longer just chat with code. They are harnesses: they read repo instructions, decide which skills or rules to load, call tools, and try to repair their own mistakes. That makes the harness part of the product surface, not just the model behind it.

The useful question is not whether a vendor says the harness is faster or more token-efficient. It is how you test it, how you catch regressions, and how you keep it from drifting across repos and models.

This matters for teams standardizing across Codex, Claude Code, and Codex. Each tool exposes a different control plane, but the governance problem is the same: define instructions clearly, scope tools tightly, and make reviewable artifacts the default. If you are building a team standard, start from agentic coding governance.

Walkthrough

Treat the harness as a versioned team asset.

Stop hiding rules in chat history. Put durable instructions in the file the tool reads at startup, then keep local exceptions close to the code. In practice that means Codex project rules in .cursor/rules/*.mdc, Claude Code memory in CLAUDE.md, and Codex instructions in AGENTS.md or override files.

A small shared pattern helps teams compare behavior across tools: one repo-level policy, one directory-level exception, one short review checklist.

Keep the artifact small enough to inspect.

The best harness changes are boring. A rule file should say what to do, where it applies, and when it should load. A memory file should capture durable repo conventions, not a transcript of every preference. A skill should describe the task clearly enough that the model can decide when to load it without flooding context.

---
description: Enforce API change review for backend routes
globs: ["src/api/**/*.ts"]
alwaysApply: false
---
- Update tests when route behavior changes.
- Flag any new external connector or permission.
- Ask for review if the change touches auth, billing, or data export.

Map one concrete artifact to each tool.

  • Codex: use scoped .mdc rules and, where needed, AGENTS.md for repo conventions. If a team is still on a single monolithic rule file, split it into smaller scoped rules first.
  • Claude Code: use CLAUDE.md for persistent project memory, then add skills, hooks, or MCP boundaries when the workflow needs progressive disclosure or deterministic checks.
  • Codex: use AGENTS.md plus the CLI verification loop so headless runs can be checked before merge. If you need temporary exceptions, keep them in an override file with an expiry mindset.

The point is not to make the files identical. The point is to make the governance intent legible in each product’s native surface.

Test harness changes like code changes.

When a rule, skill, or connector changes, run a short regression loop: one happy-path task, one boundary case, one failure case. For example, ask the agent to edit a known file, then ask it to touch a file outside scope, then ask it to explain which instruction governed the decision.

This is where model-specific tuning belongs. If one model needs a shorter instruction or a different tool boundary, record that explicitly instead of assuming the same harness works everywhere.

Review the boundary, not just the diff.

A good review checks whether the agent had too much power. Did the MCP connector have least privilege? Did the hook run outside the model loop? Did the skill or rule load only when relevant? Did the agent produce a summary that a human can verify quickly?

That is the practical version of governance: make the agent’s operating envelope visible.

Use the methodology step that fits the change.

For harness work, the most useful step is usually Review: inspect the instruction chain, connector scope, and fallback behavior before you let the change spread. Our methodology page is a good reminder that the review step should end in a concrete artifact, not a vague approval note: our methodology.

Tradeoffs and limits

This approach breaks down when teams try to centralize everything in one giant policy file. Large instructions are harder to load, harder to test, and easier to ignore. They also make it difficult to tell whether a failure came from the model, the rule, or the connector.

It also breaks down when MCP or other external tools are granted broad access. The more the agent can do, the more your evaluation needs to include permission review, logging, and rollback paths. A harness that is smarter but opaque is usually worse than a narrower one that is easy to audit.

Another limit is model variance. The same instruction set can behave differently across tools or model families, so teams should avoid claiming a universal best setup. Instead, keep a small compatibility matrix: which repo rules load, which skills are available, which hooks fire, and which verification step proves the change.

Finally, do not confuse automation with governance. A background agent or CLI loop can speed up work, but it does not replace code review, especially for auth, data access, or connector changes. The safest pattern is still: narrow scope, explicit instructions, short verification, human review.

Further reading

Related training topics

Related research

Ready to start?

Transform how your team builds software.

Get in touch