Codex autoreview settings worth changing

The failure you are trying to avoid

Codex autoreview goes bad in a predictable way. Week one, everyone reads the comments. Week two, half of them are style opinions nobody asked for. Week three, people resolve the whole thread without reading. Week four, a real finding gets resolved along with the rest and ships.

Nothing about the model caused that. The team never decided what a review comment is for. So the defaults decided, and the defaults are deliberately broad because the vendor does not know your codebase.

Narrow the mandate first

Before touching any toggle, write down what an automated reviewer is allowed to raise. Ours, in most client repos, is four categories: correctness, security, data loss, and breaking changes to a public interface. Everything else is out of scope.

Then put it in the repo instructions where the reviewer will read it. A block in AGENTS.md along these lines does more than any configuration screen:

Review scope: flag correctness bugs, security issues, data loss risks, and breaking API changes. Do not comment on naming, formatting, file organisation, or test style. If you have nothing in scope to say, say nothing.

That last sentence matters. Without permission to stay quiet, a reviewer finds something to say on every diff.

Where to point Codex autoreview, and where not to

Coverage should be uneven, because your risk is uneven.

• On: payment, authentication, permissions, migrations, anything writing to production data.
• On: pull requests from people new to the repository, where a second pass earns its cost.
• Off: automated dependency bumps. Hundreds of diffs a month, near-zero useful findings.
• Off: generated code and lockfiles. Add them to the ignore rules the same way you would for a linter.
• Off: draft pull requests, until the author marks them ready.

The dependency bot exclusion alone usually removes the largest slice of both the cost and the noise, and we have never seen a team regret it.

Make it run before the human, not instead of one

The highest-value moment for a review pass is before you open the pull request, on your own machine, against your own uncommitted work. You fix the two real findings privately and the reviewer never spends attention on them. That is a quiet productivity gain that never shows up in a dashboard.

What you should not do is make it a required check that blocks merge. Model output varies between runs, so a blocking gate produces flaky merges and teaches people to bypass the gate. Advisory comments plus a human approver is the arrangement that holds up. If you want a hard gate, gate on the deterministic things: tests, types, and the linter.

Be clear-eyed about what it will not catch. It reads the diff, so a missing feature, a wrong requirement, or an approach that should have been rejected at design time all pass through untouched. Every team we have trained that treated autoreview as a substitute for design review found this out through an incident rather than a document.

What to do this week

Pull the last 50 autoreview comments in your busiest repository and mark each one as acted on or ignored. If fewer than a fifth were acted on, write the scope block, exclude the bots, and measure again in two weeks. The number is the only argument that will convince your team either way.

If you want help putting this into practice, talk to us.