9lives Heals Playwright Tests Safely

9lives is QualityMax's open-source Python test runner for self-healing Playwright tests. It deals with a very specific failure mode: a coding agent renames or reshapes UI, a selector breaks, and someone rewrites a test that was still telling the truth. The useful takeaway for Codex users is not to let agents make tests green at any cost; it is to separate harmless selector drift from real product failures. That is also the heart of openai codex agents.md best practices: give OpenAI Codex, OpenAI's coding agent, a narrow repair lane and a verification loop it cannot skip.

Self-healing testing is a test workflow that repairs low-risk test breakage, such as stale locators, while preserving the assertion that caught the original behavior. In 9lives, that idea shows up as a tiered repair path: deterministic selector healing first, coding-agent reasoning only when the structure changed, and a diff before anything lands.

Watch the failure before asking an agent

The 9lives pitch is small enough to understand in one command: 9l heal login.spec.ts.

The runner executes the Playwright test, classifies the failure, and tries to repair the test in tiers. Tier 1 is the interesting part because it is offline and deterministic. If a locator stopped matching, 9lives looks at the page snapshot captured at failure time and searches for a better surviving anchor.

The author's example is the classic agent-era paper cut. A button label or locator changes, the test goes red, and the human or coding agent rewrites a test that was not conceptually wrong. 9lives tries to avoid that by preferring stable anchors such as data-testid, then id, then aria-label, then visible text, then class names.

That priority order matters. A changed CSS class is often noise. A missing checkout button is not. The trap is treating both as the same kind of red build.

As of July 10, 2026, the repository was tiny: MIT licensed, mainly Python, and newly pushed. That makes it a live Show HN experiment, not a settled testing platform. The idea is still worth studying because it names a problem many agent-heavy repos are starting to feel.

Notice the clever boundary: selectors first, semantics second

The best design choice in 9lives is that it does not start with an LLM.

Tier 1 uses the failure-time Playwright snapshot and rewrites the locator when it can find the same element again. No network call. No API key. No model guessing whether the test should still exist.

Tier 2 is for bigger structural changes. The README says 9lives can shell out to installed coding-agent CLIs, including Anthropic's Claude Code, OpenAI Codex through codex exec, or other local agent CLIs, so the subscription you already use does the reasoning. It can also use raw API keys when configured.

That boundary is why developers cared. The project is not saying agents should auto-fix all test failures. It is saying most locator drift should not burn human attention, and the harder cases should still produce a visible patch.

The trap is accepting a green run as proof. A self-healed test that changed an assertion, skipped a wait, or loosened the user journey may have hidden the bug you needed. 9lives' promise lives or dies on the diff.

Keep Codex instructions narrower than the healer

For Codex CLI workflows, the lesson is to make the repository rule smaller than the tool.

Do not write an instruction that says Codex may fix failing tests. That is too broad. Write an instruction that says Codex may attempt selector-only healing first, must preserve assertions, and must paste the diff and rerun command in the handoff.

Here is the practical openai codex agents.md best practices version: let AGENTS.md define the repair boundary, not the agent's mood. The rule should say which changes are allowed, which are suspicious, and what verification proves the patch.

# AGENTS.md

## Playwright test healing
- When a Playwright spec fails because a locator no longer matches, run `9l heal path/to/spec.ts` before rewriting the test by hand.
- Accept selector-only patches only when the diff changes locators, not assertions, fixtures, waits, or product code.
- If 9lives invokes Codex through `codex exec`, keep the task scoped to the failing spec, the captured snapshot, and the smallest related component.
- Never change an assertion to make the test pass unless the handoff explains the product behavior that changed.
- Verification: rerun the healed spec and one nearest affected spec. Include the unified diff and commands run in the final note.

## MCP boundary
- MCP servers used during test healing may read issue context, test artifacts, and repository files.
- Do not grant write access to trackers, deployment systems, or production data for this task.

This is a good place to use nested AGENTS.md files if your repo has separate app surfaces. A billing UI can have stricter locator and assertion rules than a marketing page. The Codex docs on custom instructions are the right starting point, and our Codex CLI workflows notes cover the broader command-and-verification pattern.

The trap is putting philosophy in the file instead of operational rules. Codex does not need a speech about quality. It needs permission boundaries, commands to run, and the exact evidence you expect back.

Try 9lives safely on one flaky spec

9lives is a good fit when your Playwright suite breaks from UI naming churn more often than from true behavior changes. It is overkill if your tests are mostly unit tests, your app lacks stable DOM anchors, or your current pain is slow infrastructure rather than locator drift.

A safe first experiment is one spec, one branch, and one diff. Pick a test that fails from renamed buttons or moved labels, not a checkout or permissions flow where semantics are already in question.

A small command workflow is enough:

# Start with a known failing spec
git checkout -b try-9lives-login-heal
9l heal tests/e2e/login.spec.ts

# Review the patch before trusting the result
git diff -- tests/e2e/login.spec.ts
npx playwright test tests/e2e/login.spec.ts
npx playwright test tests/e2e/auth-nearby.spec.ts

The review checklist is short. Did the patch change only selectors? Did it preserve assertions? Did it avoid adding arbitrary sleeps? Did it rerun the failing spec from a clean state? Did the final note include the diff?

This is adjacent to a larger agent-control question: how much should repo tooling enforce before the agent acts? We covered a policy-first version in Kastra Enforces Policies for Claude Code, Codex, and Codex, but 9lives is the lighter version: fix the narrow test pain first.

Common questions

• Will 9lives hide real bugs?

  It can if you accept a patch without reading the diff. The project tries to reduce that risk by healing selector drift first and showing a unified diff before the change is accepted. Treat assertion edits, skipped checks, and broad wait changes as warning signs, not successful healing.

• How do openai codex agents.md best practices apply to 9lives?

  They apply by making the repair boundary explicit in AGENTS.md. Tell Codex when it may run 9l heal, what files it may change, and what verification it must return. The citable rule is simple: selector-only patches are reviewable automation; assertion changes require a product explanation.

• Does 9lives require an API key?

  Not for Tier 1 selector healing. The README describes Tier 1 as offline, deterministic, and account-free because it uses the Playwright failure snapshot. Tier 2 can use an installed coding-agent CLI such as codex exec, or raw API keys if you prefer that path.

• When should I not use self-healing tests?

  Do not use them to resolve unclear product behavior. If the failure says a permission check changed, a payment state disappeared, or an accessibility contract broke, keep the test red until someone explains the intended behavior. Self-healing is best for locator drift, not for negotiating product truth.

• Is this a replacement for better Playwright selectors?

  No. It is more like a smoke alarm for weak selectors. If 9lives keeps replacing class-based locators, the better fix is usually to add stable data-testid values or accessible names in the app. The healed diff should teach the repo how to become less fragile.

Best ways to use this research

• Best for: Repos where Playwright failures often come from renamed controls, moved markup, or agent-made UI edits rather than broken business logic.
• Best first artifact: A small AGENTS.md rule that allows selector-only healing, blocks assertion changes, and requires a rerun command plus diff in the handoff.
• Best comparison angle: Compare Tier 1 deterministic healing against agent-generated test rewrites. The former should be boring and narrow; the latter needs stronger review.
• Best Codex workflow: Use Codex CLI after the deterministic pass, not before it. Let 9l heal gather the failure context, then ask Codex for the smallest patch only when structure changed.

Further reading

• OpenAI Developers — AGENTS.md guide
• OpenAI Developers — Codex quickstart
• 9lives — source

Try one healed selector

Pick one non-critical Playwright failure and run 9lives on a branch. If the patch changes only the locator and the test still proves the same behavior, you have found a useful boundary for your next Codex workflow.

One methodology lens

One useful way to read this through our methodology is the Plan step: delegate first-pass decomposition and dependency mapping, review the sequencing and assumptions, and keep ownership of scope and priorities. If that split is still fuzzy, the workflow usually is too.