Back to Research

Codex AI training that changes what a team merges

What Codex AI training should cover for a working engineering team, how to structure it, and the measures that show whether it worked.

Study of a Hill Top and Sky, landscape painting by James Ward.
Rogier MullerAugust 15, 20263 min read

Start from the failure, not the feature

Good Codex AI training opens with a broken run. We give the room a task the agent handles badly, usually something that requires knowledge living in a config file nobody put in context, and let it fail in front of everyone. Then we fix the context and run it again.

That order matters. Teams who see the polished demo first assume the tool is magic and get quietly disappointed by Tuesday. Teams who see the failure first learn the actual skill, which is managing what the model knows.

The four things worth a full day

  • Repository priming. A committed AGENTS.md naming the build command, the test command, the directories that are generated, and the ones nobody may touch.
  • Task shaping. Turning a vague ticket into a brief with a definition of done that the agent can verify itself, usually by running tests.
  • Interruption. Knowing within two minutes that a run is going wrong, and killing it rather than watching hopefully.
  • Reviewing code you did not type. This is a distinct skill from reviewing a colleague's work, because there is no author to ask.

Everything else, the keyboard shortcuts, the model picker, the settings, can be read in the documentation. Do not spend a trainer's day on it.

Interruption is the underrated one

Most wasted time with agents comes from letting a bad run continue. The engineer has already spent six minutes, feels invested, and keeps nudging. Twenty minutes later they have a branch they do not understand and throw away.

We teach a hard rule during training and it survives contact with real work: if the agent has made two corrections in a row that did not move toward a passing test, stop. Reset the branch with git checkout ., rewrite the brief with the thing that was missing, and start again. Restarting from a better brief is almost always faster than salvaging.

Measuring whether the training worked

Vanity measures are easy and useless. Lines of AI-generated code tells you nothing. Number of agent sessions tells you less.

Three things we ask teams to track for six weeks, starting the week before training so there is a baseline:

  • Time from ticket picked up to pull request opened, on a comparable class of ticket.
  • Review round trips per pull request. This usually goes up first, then down, and the shape of that curve is informative.
  • Rollbacks and hotfixes. If this rises, the team is merging faster than it is reviewing, and that is a governance problem rather than a tooling one.

Be honest about what training cannot fix. If your test suite takes forty minutes and fails intermittently, agents will not help much, because the verification loop they depend on is broken. That is infrastructure work, and no amount of prompting technique substitutes for it.

What to do next

Before booking anything, spend an afternoon writing AGENTS.md for your main repository and get the test suite green and fast. Teams who arrive with that done get roughly double the value from the day, because the first three hours are not spent on plumbing.

If you want help putting this into practice, talk to us.

Related training topics

Related research

Ready to start?

Transform how your team builds software.

Book a 15-minute sync