Back to Research

Codex GitHub reviews without the noise

Codex GitHub integration puts an agent inside your pull requests. What to hand it, what to keep on your laptop, and how to stop the comment flood.

La Rochelle, landscape painting by Paul Signac (1911).
Rogier MullerAugust 15, 20264 min read

Two different things share the name

Codex GitHub usage splits into review and delegation, and teams that conflate them get a bad first month.

Review is the agent commenting on a diff someone already wrote. Delegation is handing the agent a task and getting a branch or a pull request back. The first is low risk and immediately useful. The second is high variance and depends almost entirely on how well the task was written.

Start with review. It fails cheaply. A bad comment costs someone ten seconds. A bad delegated pull request costs a reviewer twenty minutes of reading code nobody on the team has thought about.

Making review land instead of annoy

Left unconstrained, a model reviewing a diff will find something to say about every hunk, because that is what being helpful looks like from the inside. You get one real race condition buried under thirty naming suggestions, and within two weeks your engineers scroll past all of it.

Constrain the scope in the repo instructions file. Put a short block in AGENTS.md saying what a review comment is permitted to be about. Correctness. Data loss. Security. Public API changes. Style belongs to the formatter and naming belongs to the author.

Then measure one thing for a month: what fraction of agent comments led to a code change. Under one in five means your scope rules are wrong, not the model.

What to delegate to Codex GitHub runs, and what not to

Delegation works when the task is specified well enough that a competent contractor with no context could do it. That rules out most of your backlog.

  • Good: a dependency bump with a known breaking change and a test suite that proves it.
  • Good: mechanical migration of a deprecated call across many files.
  • Good: a bug with a failing test already written.
  • Bad: anything phrased as "improve" or "clean up".
  • Bad: work touching a module whose conventions live only in a senior engineer's head.

Cloud runs have a harder constraint than laptop runs: the agent cannot ask you a question mid-task. On your machine, an ambiguous instruction produces a clarifying question. In a delegated run, it produces a guess, and you find out at review time.

Repo hygiene decides the outcome

The agent's output quality tracks how runnable your repo is from a cold start. If setup requires three undocumented environment variables and a VPN, a cloud run fails or works around it in ways you will not like.

Two fixes carry most of the weight. Make the test command work with no arguments from a clean clone. And write down in the instructions file what the agent is not allowed to touch: generated files, lockfiles you manage by hand, vendored code. Agents reformat things nobody asked them to reformat, and the diff noise makes the real change unreviewable.

Also cap the blast radius. Branch protection and required checks apply to agent-authored pull requests exactly as they do to human ones. If they do not currently apply, that is a gap that existed before the agent showed up.

What to do next

Turn on review only, with the scope rules written first, and leave delegation off for a month. Track the comment-to-change ratio. Once that number is healthy and your team trusts the comments, pick the single most mechanical ticket in your backlog and delegate exactly one task. Judge the whole idea on how much editing that pull request needed, not on how fast it arrived.

If you want help putting this into practice, talk to us.

Related training topics

Related research

Ready to start?

Transform how your team builds software.

Book a 15-minute sync