tutti-test-audit

# Tutti Test Audit

Make every changed test earn its maintenance cost. A green test, a coverage
increase, or a larger test count is not a quality verdict.

Read the root and closest scoped `AGENTS.md`, then read
`docs/conventions/unit-testing.md` before a non-trivial test change or review.
Follow `docs/conventions/testing.md#validation-selection` for commands and
validation scope.

## Establish the evidence map

Before editing, inspect:

1. the production owner, public entry point, relevant caller and callee;
2. sibling implementations that share the same invariant;
3. existing tests, fixtures, conformance suites, and repository checks;
4. the changed-aware and platform lane that will select the evidence;
5. relevant history for a regression or a suspicious existing test.

For Agent Host lifecycle semantics, also read
`packages/agent/host/README.md` and start with the conformance contract. Do not
reimplement lifecycle tests in an adapter.

## Pass the authoring gate

Answer all six questions before adding or materially rewriting a test:

1. **Protected contract:** What observable behavior, invariant, compatibility
   promise, or prior failure matters?
2. **Credible failure:** What plausible faulty implementation must make the
   test fail?
3. **Coverage gap:** Why would existing evidence not catch that failure? Can an
   existing table, scenario, or fixture express it without duplication?
4. **Owner and observer:** Which module owns the decision, and what is the
   lowest boundary that can observe the risk without mocking it away?
5. **Production seam:** Does the test require an export, flag, wrapper, global,
   or injection hook that no production caller needs? If so, move to the real
   boundary or redesign the owner.
6. **Execution:** Which blocking lane selects the test, and which native OS must
   execute it?

A missing answer means the test is not ready. “No new test” is a valid outcome
when stronger existing evidence already protects the contract or the proposed
assertion only restates a type, literal, implementation detail, or presentation
choice.

## Choose the proof shape

- Extend an existing table or conformance scenario when it owns the same
  contract and fixture.
- Add a new scenario when the behavior, failure mechanism, setup, or execution
  lane is independently meaningful.
- Move upward to a component, narrow integration, conformance, or E2E boundary
  when a unit double would replace the semantics under test.
- Use a repository check, not a unit test, when source structure, imports,
  generated artifacts, manifests, or architecture are the actual contract.
- Keep visual styling in visual evidence unless styling changes native behavior
  such as pointer ownership or window dragging.

Prefer one vertical proof through the owning boundary plus narrow tests for
important decision partitions over many stub-backed branch tests.

## Build sensitive evidence

For a confirmed bug, capture the failing reproduction before the fix whenever
possible. The regression test must fail on pre-fix behavior for the intended
reason and pass after the owner-boundary repair. If the old revision cannot be
run safely, temporarily inject or restore the faulty decision as a local
negative control and report that limitation; never commit the mutation.

For new behavior, first demonstrate that the focused test fails because the
contract is absent, not because setup, imports, or fixtures are broken.

For a read-only audit, do not edit production code merely to manufacture a
negative control. Name the smallest plausible faulty implementation, determine
whether the current assertions would detect it, and label that sensitivity
unverified when it cannot be executed safely.

Then:

- assert stable outcomes and important non-effects;
- include neighboring valid, rejection, cleanup, retry, or preservation cases
  only when they address a credible risk;
- use real isolated SQLite, filesystem, Git, HTTP, IPC, parser, or process
  boundaries when their semantics are the subject;
- keep external doubles narrow and fail closed on unexpected calls;
- coordinate concurrency with events, channels, barriers, or latches rather
  than sleeps or lucky elapsed-time bounds;
- control clocks, IDs, randomness, ports, user state, credentials, globals,
  listeners, timers, processes, and temporary resources;
- assert call counts only when count is itself a contract such as idempotency,
  single-flight, or at-most-once dispatch.

## Apply Tutti routing

- Session, Turn, Goal, runtime-operation, and recovery semantics start in
  `packages/agent/host/conformance`; consumers provide drivers.
- Adapter tests own transport, DTO, authorization, query, presentation, and
  product-policy behavior only.
- SQLite, filesystem, path, shell, executable, process, signal, permission, and
  native API behavior cross the real isolated receiver boundary.
- UI behavior drives accessible keyboard, pointer, focus, and state transitions
  and observes callbacks or user-visible state.
- HTTP, IPC, and provider protocol tests cross the real parser, middleware, or
  codec boundary when receiver behavior is the risk.
- Platform-dependent behavior runs on the native platform. A serialized
  request shape or a skipped Linux test is not Windows proof.

## Audit existing tests

Treat these as review candidates, not automatic deletions:

- no meaningful assertion, self-comparison, or only “does not throw”;
- copied production algorithm, catalog, defaults, or current inventory;
- source strings or regexes presented as runtime behavior proof;
- private call-shape or mocked call-graph assertions;
- static render, class, style, SVG path, or incidental DOM assertions;
- duplicate coverage weaker than an existing boundary scenario;
- sleeps, polling delays, pass-on-retry, silent skips, or zero-selected lanes;
- test-only production exports, wrappers, flags, globals, or dead code.

Before weakening or removing a test, identify the failure it can actually catch