Write and run tests

Use caseIntermediate14 min

A coding agent in a local checkout already writes and runs tests well. The hard parts are elsewhere: knowing which tests in other repositories exercise the behaviour you are changing, writing a test from the input that actually failed in production rather than an invented one, and knowing whether a red build is your change or a test that has been failing for a week. CodeZero gives the agent the organisation's code and a plan that declares which test commands may run; Vörr gives the assistant real errors, traces and CI results. Neither product runs your tests: the agent does, with your test runner.

When you'd do this

You are about to change a function or an endpoint and want to know which tests protect it, including tests that live in the repositories that call it.
A production error has a stack trace and a failing request, and you want a test that fails for the same reason before you write the fix.
CI went red on a branch and you need a regression test for the real failure, not the first error line in a long log.
You are adding tests to a service and suspect the team already has factories, fixtures or fake clients for the same models in a shared package.
A change spans several repositories and each one has its own test command, and you want those commands, and only those, to run while the agent works or while you are away.
You shipped a fix and want to confirm in production that the error has gone, not just that the tests pass.

Before you start

CodeZero installed in VS Code with Claude Code, Codex, GitHub Copilot or Cursor as the agent, signed in to your organisation, with the relevant repositories indexed. Repositories you have checked out are read from the working tree; others are read from the index of the default branch.
The repositories you will edit or run tests in checked out locally, with their dependencies installed. CodeZero offers to clone a repository when a plan needs one that is missing.
For production inputs: Vörr connected to Claude, ChatGPT or another MCP assistant, with your error tracker (for example Sentry or Bugsnag) and, ideally, tracing and logging tools connected, and the service mapped so names resolve across tools. ChatGPT needs a paid plan with Developer mode turned on.
For CI failures: your CI connected in Vörr. GitHub Actions, GitLab CI and Bitbucket Pipelines come through the code connection; Jenkins, CircleCI, Buildkite, Drone and others are connected separately.
Roles: Vörr answers from the organisation's connected sources, and workspaces label what comes back; an admin connects the tools and maintains the service map.

Operations

The operations below cover the testing work end to end: finding existing tests, finding gaps, reusing helpers, writing tests from real failures, bounding and running the test commands, and checking CI and production afterwards. Each one names which product, or the agent's own tools, does each step.

Find the tests that protect a behaviour

In CodeZero, find the definition of the function, class or route you plan to change. Symbol search covers TypeScript, JavaScript, Python, Go, Java, Ruby and C#; other languages are searched as text.
Ask CodeZero for references and callers across all indexed repositories, then filter to test files (by path, such as test, spec or __tests__, and by naming convention).
Ask Chat which tests protect the behaviour. Chat answers with file references, so each claimed test can be opened and read.
The agent reads each test with its own tools (or through CodeZero for repositories not checked out) and sorts them: tests that assert the behaviour directly, tests that only reach it indirectly, and tests that mock it out entirely.

In CodeZero Chat

Which tests protect the behaviour of calculateRefundAmount? Search every indexed repository, include tests that call it indirectly through the refunds endpoint, and say which ones mock it out.

Before a change

I am about to change how applyDiscount rounds. List every test, in any repository, that asserts on its output, with the file and the assertion.

A good result lists tests with file references, says which repository each is in, and separates direct assertions from tests that mock the function. References are matched by name, so check that a hit really refers to your symbol and not a same-named one in another package. A short list is not proof that nothing else depends on the behaviour: callers over HTTP or queues do not appear as references, which is the next operation.

Find test gaps for callers in other repositories

In CodeZero's Graph, find the inferred edges into the service you are changing: HTTP calls in other repositories matched to its routes, services named in Kubernetes, Helm or Compose config, and queue producers and consumers.
Use Verify on the HTTP edges that matter. The agent checks the call site and records a verdict (confirmed, wrong or retargeted) that the whole organisation then sees.
For each confirmed consumer, the agent searches that repository for tests that exercise the call: contract tests, client tests, recorded fixtures or mocks of your response shape.
The agent reasons over what it found: which consumers have a test that would fail if your response changed, and which have none. This is reasoning over edges and test files, not a coverage measurement.

In CodeZero Chat

Which repositories call POST /v2/refunds? For each caller, is there a test that would fail if the response stopped including refund_id? Show the call site and the test, or say there is none.

Queue consumers

Who consumes the order.cancelled event? Which consumers have tests that build that payload, and does the payload in those tests match what the producer sends today?

A good result names each consumer with its call site, the confidence of the edge if it is an HTTP or config edge, and whether you verified it, and for each one either a test file or an explicit "no test found". Low-confidence edges, where a path matched routes in several repositories, need Verify before you plan work on them. Fixtures that hard-code an old response shape are worth listing too: they pass today and will hide a break.

Find existing test helpers, fixtures and factories

Before the agent writes a new factory or fake, ask CodeZero to search across repositories for existing ones: by symbol (for example a factory or builder for the model), by text (fixture file names, fake client classes) and by topic with Go deep.
Check the per-repository dependencies view to see whether the shared test package is already a dependency of the repository you are working in.
Read the helper and its last-commit provenance to judge whether it is maintained, and the owners view to see who to ask.

In CodeZero Chat

Is there already a test factory or fixture for Order or Refund objects anywhere in the organisation? I want to reuse it rather than write a new one. Show where each lives and which repositories already import it.

Fake clients

Do we have a fake or stub for the payments provider client that other services use in tests? Show its file, who last changed it and how it is configured.

A good result points to a helper the repository can already import, or tells you why it cannot (a different language, not a dependency, unmaintained). Without this step agents tend to write a near-copy of a factory that already exists in a shared package they cannot see.

Write a test that reproduces a production error

In Claude or ChatGPT with Vörr, ask for the top issues for the service. Each comes with a count, the culprit, a redacted sample and the stack.
If the error is tied to a request, ask for a trace. Vörr returns the trace waterfall and the bottleneck span, which shows which call failed and with what timing.
Ask Vörr which repository and file the culprit frame points to, through the same code index.
Take the redacted sample, the stack and the relevant spans into CodeZero. The agent writes a failing test that feeds the same shape of input into the same code path, using existing fixtures where possible.
The agent runs the test and confirms it fails for the same reason as production, with the same exception and frame, before any fix is written.

In Vörr

What are the top errors for the refunds service in production over the last 24 hours? For the most frequent one, give me the redacted sample, the full stack and one trace that shows the failing request.

In CodeZero

Here is a production error from Vörr: TypeError in calculateRefundAmount when line_items is empty, stack attached. Write a failing test that reproduces it with our existing order factory, run it, and show me that it fails with the same error before changing any code.

A good result is a test that fails on the current code with the production exception and passes only after the fix. Samples are redacted, so the test uses the shape and edge-case values of the real input, not personal data. If the error cannot be reproduced locally, the test or the sample is missing something; say so rather than adjusting the test until it passes.

Write a regression test for a CI failure

Ask Vörr why the latest build of the branch or service failed. The CI build-failure diagnosis covers the most recent few failed builds and gives the category, the phase, the reason, the relevant log excerpt, and the failing test names where the CI source reports them (Jenkins, CircleCI, Buildkite and Drone).
Decide with the diagnosis whether this is a code failure, an environment or dependency failure, or a test that was already failing elsewhere.
For a genuine code failure, open the failing test and the code it covers in CodeZero. Use Git Tree to compare the branch with the last green one.
The agent writes or adjusts a regression test that pins the behaviour that broke, runs it locally, and then fixes the code.

In Vörr

Why did the latest failed CI build for the refunds service fail? If Vörr can identify the build for the feature/refund-rounding branch, use that one. Give me the phase, the reason, the failing tests and the log excerpt, and say if anything in the log suggests this is environmental rather than a code failure.

In CodeZero

CI reports test_partial_refund_rounding failing with expected 10.00, got 9.99. Find the code it covers, compare this branch with main, and write a regression test for the rounding case before fixing it.

A good result names the failing tests and quotes the log excerpt it relied on, so the agent is working from the real error rather than a paraphrase of a log you pasted.

Check whether failing or flaky tests are new

Ask Vörr for recent failed builds of the service across your connected CI sources and the diagnosis of each, where the CI source records them.
The assistant compares the failing tests across those builds: a test that fails on main and on unrelated branches is not caused by your change; one that fails and passes on the same commit is a flakiness candidate.
Take the conclusion into your plan: fix the flaky test as separate work, or leave it out of the verification commands with a note, rather than letting it block the change.

In Vörr

Using Vörr, for the recent failed CI builds of the refunds service that Vörr can see, which tests failed in each? Say which CI sources answered and how far back they go.

Vörr reports what the CI tools recorded; the judgement that a test is flaky is the assistant's reasoning over those builds, so check the builds it cites. If a CI source is not connected, Vörr says so rather than reporting a clean history.

Declare the verification commands in a plan

Describe the change in CodeZero in Planned or Auto mode. The agent investigates and asks you about any decisions the code cannot answer.
The plan names the repositories, the phases and the verification commands: the exact test, lint or build commands for each repository.
Review those commands as part of the plan. The agent cannot approve its own plan.
After approval, only the declared verification commands run without asking. Any other command still needs your confirmation, and an edit outside the repositories the plan names asks first.

In CodeZero

/plan Fix the empty line_items crash in calculateRefundAmount. Start with the failing test, then the fix. Verification: the refunds unit tests and the type check only.

A good plan uses commands the repository actually has (check its package scripts, Makefile or CI config), scopes them to what the change touches, and avoids commands that need production credentials or mutate shared environments.

Run tests per repository in a cross-repo plan

For a change across a provider and its consumers, the plan lists each repository with its own verification commands, for example a pytest run in one and an npm test run in another.
Tasks shows which repository each task belongs to and the dependencies between tasks, so consumer tests run after the provider change they depend on.
If a consumer repository is not checked out, CodeZero offers to clone it before its tasks run.
The agent runs each repository's commands in that repository and reports the outcome per task.

In CodeZero

Plan adding refund_reason to the refunds API response and updating the two consumers that read it. Declare the test command for each repository separately and run consumer tests only after the provider task.

A good result reports the command and the outcome for each repository, not a single "tests pass" for the whole change.

Let Coffee Mode run them unattended

Once a plan is approved and every clarification question is answered, start Coffee Mode.
CodeZero works through the remaining tasks and runs the plan's verification. Normal edits and recoverable commands proceed without asking; where a question comes up, the agent picks the safest option and records the decision.
It never commits, pushes, switches branches, rewrites or merges history, publishes or runs destructive commands. Changes stay in the working tree.
When you return, read the summary, the test results and the decisions it recorded, then review the diff in Git Tree.

Before you step away

The plan is approved. Before I start Coffee Mode, confirm the verification commands for each repository and tell me which tests you expect to fail first and pass after the fix.

Verify after release

Once the fix is deployed, ask Vörr for the deploy in the organisation-wide feed, so you know exactly when it reached the environment.
Ask for the error rate and the specific issue count after the deploy against a comparison window, for example the same period seven days earlier. Each source is reported separately, and disagreement between sources is flagged.
Check release health and any regressions reported for the deploy.

In Vörr

The refunds fix was deployed to production this morning. Has the TypeError in calculateRefundAmount stopped since that deploy? Compare error rate and issue count with the same window seven days ago, per source.

A good result shows the deploy time, the before and after numbers per source and the match quality of the service name. A drop in one tool with no data from another is not confirmation; check the coverage Vörr reports.

Without CodeZero or Vörr

Claude Code in a checkout is a capable test writer. It can grep for tests, read them, write new ones, run the test runner, read the output and iterate until they pass. With the gh CLI it can read GitHub Actions logs; with a Sentry or Datadog MCP server configured and credentialed it can fetch errors. ChatGPT or Claude chat can reason well over a stack trace or CI log you paste. The gaps are in what each can see and what bounds it.

Operation	Claude Code / ChatGPT on their own	With CodeZero and Vörr
Find tests for a symbol	Good within cloned repositories; literal search, so tests in repositories you have not cloned are invisible	References and Chat across every indexed repository, with file references
Find tests for HTTP or queue consumers	Only if you know which repositories call you, clone them and search for the path string yourself	Graph infers the consumers with file:line evidence, and a confidence on HTTP and config edges; Verify confirms them; the agent then checks each for tests
Reuse fixtures and factories	Finds helpers in the open repository; a shared test package elsewhere looks absent, so it writes a new one	Search by symbol, text and topic across repositories, plus each repository's declared dependencies
Reproduce a production error	Works from what you paste, or from one error tracker if you set up its MCP server with credentials	Vörr returns the redacted sample, stack, culprit and a trace from the connected tools, with resolved service names
Diagnose a CI failure	Possible with gh for GitHub Actions; other CI systems need their own CLI or server, and long logs strain context	Vörr's build-failure diagnosis gives phase, reason and excerpt for the most recent few failed builds, plus failing test names from Jenkins, CircleCI, Buildkite and Drone
Check whether a failure is new	Manual: query each CI system's run history and compare	Ask Vörr for the diagnoses of recent failed builds, where the CI source exposes them; missing sources are reported, not treated as clean
Bound which commands run	Per-command permission prompts or broad allow-lists you configure yourself	The approved plan declares the verification commands; only those run without asking, and the agent asks before editing outside the plan's repositories
Run tests unattended	Possible with permissive settings, with no built-in limit on git or destructive commands	Coffee Mode runs the approved plan's verification and never commits, pushes or runs destructive commands
Confirm the fix in production	Requires access to each monitoring tool and a manual before/after comparison	Vörr compares windows per source after the deploy in its feed

With CodeZero and Vörr

Tests beyond the open folder (CodeZero): Search, references and Chat across the organisation's indexed repositories, so tests in a shared package or another service are part of the answer.
Consumers you cannot see as references (CodeZero): Inferred HTTP, config and queue edges in Graph, with file:line evidence, a confidence on HTTP and config edges, and Verify verdicts kept for the team.
Helpers to reuse (CodeZero): Existing factories, fixtures and fakes found by symbol, text or topic, with provenance and owners.
Real failing inputs (Vörr): Redacted error samples, stacks, culprits and trace waterfalls from the error-tracking and tracing tools you already run.
Real CI results (Vörr): Build-failure diagnosis with the log excerpt across connected CI sources, and failing test names from Jenkins, CircleCI, Buildkite and Drone, so a regression test targets the actual failure.
A command boundary (CodeZero): Verification commands declared in an approved plan, run per repository, and run unattended by Coffee Mode within its safety limits.
Confirmation in production (Vörr): Comparison windows, release health and deploy regressions after the fix ships.

Together: Vörr supplies the failing input and the CI or production evidence; CodeZero finds where the code and its tests live, plans the test-first change and bounds what runs. You carry the finding from one to the other: Vörr does not run inside CodeZero, and CodeZero does not call Vörr.

Illustrative example

Worked example: a refund crash becomes a test-first fix

The service names, files, counts and output below are examples, not output from a real run.

The refunds service starts throwing errors after a partner integration begins sending orders with no line items. The developer starts in Claude with Vörr.

Vörr: top errors, refunds-api (production), last 24h
Source: Sentry  Match: explicit  Environment: production
1. TypeError: Cannot read properties of undefined (reading 'amount')
   Count: 412   Culprit: src/refunds/calculate.ts in calculateRefundAmount
   Sample (redacted): { order_id: "[redacted]", line_items: [], currency: "EUR", partial: true }
   Stack: calculate.ts:48 calculateRefundAmount
          refunds.controller.ts:31 RefundsController.create
Trace: POST /v2/refunds  320 ms  bottleneck span: refunds.calculate (error)
Code: frigga-example/refunds-api  src/refunds/calculate.ts:48

Illustrative example

In CodeZero, the developer asks which tests protect calculateRefundAmount and who consumes the endpoint.

Tests that reach calculateRefundAmount
  refunds-api  test/refunds/calculate.spec.ts    direct, 6 cases, none with empty line_items
  refunds-api  test/refunds/controller.spec.ts   mocks calculateRefundAmount
Consumers of POST /v2/refunds (Graph)
  support-console  src/api/refunds.ts:22   high confidence  verified: confirmed
     test: src/api/refunds.test.ts  uses recorded fixture, no error-path case
  billing-worker   jobs/refund_sync.py:57  medium confidence  verified: confirmed
     test: none found
Existing helpers
  shared-test-utils  factories/order.ts  buildOrder()  imported by refunds-api

Illustrative example

The developer asks for a plan. The approved plan has three tasks: a failing test in refunds-api built with buildOrder and the redacted sample's shape; the fix, which returns a 422 for an order with no line items; and a new test in billing-worker for the 422 path. Verification: npm test -- test/refunds in refunds-api and pytest tests/jobs in billing-worker. billing-worker is not checked out, so CodeZero offers to clone it. The developer starts Coffee Mode.

Coffee Mode summary (illustrative)
Task 1 refunds-api: added 'rejects an order with no line items'
  npm test -- test/refunds   1 failed (TypeError at calculate.ts:48, matches production)
Task 2 refunds-api: guard empty line_items, return 422
  npm test -- test/refunds   23 passed
Task 3 billing-worker: added test for 422 response in refund_sync
  pytest tests/jobs          14 passed
Decisions recorded: used error code REFUND_NO_ITEMS (matches existing 422 codes in refunds-api)
Not done: no commit, no push. Changes are in the working trees.

Illustrative example

Conclusion: the crash came from an input shape the existing tests never covered, and one consumer had no test for the error path. Verification: the developer reviews the diff in Git Tree, commits and pushes, and CI passes. After the deploy, Vörr shows the TypeError count at zero since the deploy time against 412 in the same window a week earlier, from Sentry with an explicit match. Follow-ups: add an error-path case to the support-console fixture test, and ask the partner integration owner whether empty orders are expected.

Verify and follow up

Provenance: open every test Chat or the agent cites. For repositories read from the index, check the last-commit provenance; a local working tree may differ.
Match quality: references are matched by name and callers are heuristic. Confirm that each hit is your symbol, and Verify the inferred consumer edges you rely on.
Vörr sources: check the source, match quality and environment on each error, trace and CI result, and the coverage Vörr reports before treating an absence as real.
Test first: confirm the new test failed before the fix and for the same reason as production or CI, not just that it passes afterwards.
Plan verification: read the per-repository results of the declared commands, and in Coffee Mode the decisions it recorded.
After release: compare windows per source and check release health.
Next: carry consumer gaps into the owning team's plan, and fix flaky tests as their own work.

Limits

CodeZero does not run tests. The agent runs them with your test runner, in your checkout, with your local dependencies and environment.
Neither product measures test coverage. Finding tests and gaps is search and reasoning over references, edges and test files; use your coverage tool for line or branch coverage.
References are name-based and callers are heuristic, not compiler-accurate. Tests that reach a function through reflection, dependency injection or generated code can be missed.
Inferred cross-repository edges can be wrong or missing, particularly for dynamically built URLs. Low-confidence edges need Verify.
The index covers committed code on the default branch. Tests on unmerged branches in repositories you have not checked out are not visible.
Error samples and log lines are redacted. A reproduction may need realistic placeholder values, and some failures depend on state the sample does not carry.
Vörr only sees connected tools. Without a connected CI source there is no build-failure diagnosis for it, and Vörr reports that rather than guessing. Flakiness is a judgement the assistant makes from the builds, not a label Vörr assigns.
Vörr is read-only towards your systems: it cannot re-run a pipeline or change a test.
Coffee Mode will not commit, push or switch branches, so the results always wait for your review.

When you'd do this

Before you start

Operations

Find the tests that protect a behaviour

Find test gaps for callers in other repositories

Find existing test helpers, fixtures and factories

Write a test that reproduces a production error

Write a regression test for a CI failure

Check whether failing or flaky tests are new

Declare the verification commands in a plan

Run tests per repository in a cross-repo plan

Let Coffee Mode run them unattended

Verify after release

Without CodeZero or Vörr

With CodeZero and Vörr

Worked example: a refund crash becomes a test-first fix

Verify and follow up

Limits

Related