Rudder Measures Your Input on AI Code

Rudder is an open-source TypeScript project from RudderCode that plugs into Claude Code, Anthropic's coding agent, and OpenAI Codex, OpenAI's coding agent. It deals with a very specific worry: after a long agent session, how much of the resulting code came from your intent rather than the model filling in the blanks? The useful takeaway is simple: treat prompt-derived tests as an authorship check, not as proof that the software is correct. It sits near the codex cli github workflow people already use: local repo, branch context, agent session, then verification.

Watch the prompt, not just the diff

Rudder is a local plugin that records which repository and branch an agent is working in, then uses the session history to generate tests. Prompt-based testing is a way to turn the developer's written intent into executable checks against the code an agent produced.

That is why the Show HN post landed with developers who have been letting agents write larger chunks of code. The uncomfortable feeling is not that agents are useless. It is that the diff can look plausible while the developer's own decisions become hard to find.

A normal review asks whether the patch compiles, passes tests, and looks sane. Rudder asks a narrower question first: can the behavior be traced back to something the human actually asked for?

The trap is treating that question as moral purity. Agent-generated code can still be good when it includes helpful inference. Rudder is more interesting when it highlights the gap between intent and implementation, not when it tries to shame the tool.

Make tests prove intent, not the agent's memory

The project's trick is to repurpose unit tests. Instead of letting the same agent write tests from the implementation it just created, Rudder tells the agent to rewrite tests using only prompts recorded for the working branch.

That changes the meaning of coverage. Coverage stops being only a measure of which lines ran. It becomes a rough proxy for whether the generated code matches the session's explicit instructions.

A concrete example: you ask a Codex agent to add a rate limit to a password reset endpoint, preserve existing audit logging, and keep the error response generic. A prompt-derived test suite should check those three requested behaviors. If it also tests a caching layer you never asked for, that is a review smell.

The trap is believing tests can capture every human judgment. They cannot. Naming, architecture, API shape, and maintainability still need a human diff review.

The objection is fair: coverage can lie

The strongest objection is that an agent can game tests, especially when it writes both the code and the tests. Rudder does not make that impossible. It narrows the agent's source material to the prompts for the branch, which makes gaming less invisible.

That is a useful constraint, but not a magic wall. If your prompts were vague, the tests will be vague. If the code changes touch security or data loss paths, prompt-derived tests are only one signal.

This is where Codex users should keep the old loop alive: run the repo's normal test command, inspect the diff, and ask the agent to explain any behavior not tied to a prompt. For more CLI-oriented examples, keep this alongside the related training topic, not instead of it.

The trap is replacing review with a new number. A Rudder coverage result is a conversation starter. It is not a release stamp.

Try Rudder when the question is authorship

Rudder is worth trying when an agent session produced more code than you can comfortably reconstruct from memory. It is especially relevant after broad refactors, generated test scaffolds, or feature work where the prompt carried business rules.

It is overkill for tiny edits, mechanical formatting, or a one-line bug fix you can reason about directly. It is also a poor fit when the repo has no meaningful test runner yet, because Rudder relies on the repository's own test and coverage tools.

For a Codex CLI workflow, keep the experiment small: one branch, one feature, one verification pass. If someone searches codex cli github and lands in the OpenAI repository first, the practical next question is not how to install every helper. It is how to prove the agent's patch still reflects the developer's prompt.

A useful AGENTS.md note for this experiment is short:

# Verification rule
When generating tests for an agent-produced change, tie each test to a user prompt from this branch. Do not add tests for behavior that was only inferred from the implementation.

The trap is putting this rule everywhere before you know whether the signal is useful. Try it on one branch where the authorship question already feels real.

Try it safely checklist

Use this checklist when you want to test Rudder without turning the project into ceremony.

• Pick a branch with one agent-written feature, not a week of mixed work.
• Confirm the repo has a real test command and coverage command.
• Keep Node.js 24 or newer, npm, and Git available, matching Rudder's stated requirements.
• Run the agent session normally before invoking Rudder at the end.
• Review tests for prompt traceability: each important assertion should map back to something you asked for.
• Mark any test that validates agent-inferred behavior rather than user intent.
• Run the normal test suite again after generated tests are rewritten.
• Keep the final decision in the diff review, not in the coverage number.

A nice side benefit: this checklist also makes handoff cleaner. If another developer opens the branch later, they can see which behaviors came from the prompt and which ones still need review.

For a related story about making agent work visible in a different way, see Claude MIDI Twister Shows Agent State.

One methodology lens

One useful way to read this through our methodology is the Plan step: delegate first-pass decomposition and dependency mapping, review the sequencing and assumptions, and keep ownership of scope and priorities. If that split is still fuzzy, the workflow usually is too.

Practical starter checklist

- [ ] Name the Codex artifact first: an AGENTS.md instruction, a Codex CLI verification loop, an MCP boundary note, or a skills handoff.
- [ ] Write the review checklist before generation starts: scope, owner, tests, rollback.
- [ ] Keep the first step small enough that a reviewer can inspect the receipt without replaying the whole chat.

Common questions

• What should teams know about codex cli github?

  Start by writing down one visible team rule for Codex, not a loose preference. That usually means a short repository convention, a review checklist, and one owner who can reject agent output when the evidence is missing.

• Which Codex artifact should teams standardize first?

  Standardize the smallest artifact that reviewers already touch: a AGENTS.md instruction, MCP note, or verification checklist. The point is not documentation volume; it is a shared place where scope, allowed tools, expected tests, and rollback notes are visible before generated code reaches review.

• How do teams know the convention is working?

  The convention is working when reviewers can approve or reject agent output from the artifact and evidence alone. Track whether pull requests name the rule used, include the promised checks, and avoid replaying long sessions just to understand what changed.

Best ways to use this research

• Best for: Codex teams deciding which AGENTS.md instruction, CLI workflow, MCP boundary, or verification loop to standardize next around “Rudder Measures Your Input on AI Code.”
• Best first artifact: turn the named fix into an AGENTS.md rule, verification checklist, MCP note, or review receipt before the next automated run.
• Best comparison angle: compare the workflow against the current Codex CLI review loop, shell boundary, and evidence trail; keep the path that leaves the shortest auditable trail.

Further reading

• Rudder — source
• OpenAI Developers — Codex CLI
• GitHub — openai/codex
• OpenAI developers: codex quickstart

Next move

Take this into the related training topic and test whether a new reviewer can defend the merge without replaying the chat.