manual-testing

# Manual Testing for Relaticle

Adversarial post-task QA. Pretend 1000s of users in production are about to exercise this change from every angle. Find what they would find. Fix what's safe to fix. Report what isn't.

## When to invoke

Self-invoke immediately after finishing any code change in this codebase, before reporting work as complete. The skill decides for itself whether the change is testable; do not pre-filter.

## Skip conditions

Run `.github/skills/manual-testing/bin/should-skip.sh` first. If it exits 0, exit the skill with one line: the reason from the script. If it exits 1, proceed.

Skip rules (encoded in the script):

1. `MANUAL_TESTING_SKIP=1` env var, a hard override.
2. Branch is `main`, `master`, or `release/*`.
3. Diff is empty.
4. Every path in the diff matches a trivial pattern: `*.md`, `*.txt`, `*.lock`, `*.gitignore`, `composer.lock`, `package-lock.json`, `pnpm-lock.yaml`, `yarn.lock`, `tests/**`.

Additionally, **the skill itself** (not the script) must check the user's message in the current turn for `[skip-qa]`, `skip qa`, `no test`, or `dont test` BEFORE invoking the script. If found, exit with `SKIP user-request:<phrase>` logged to `.context/testing/reports/skipped.log`. The script handles env-var, branch, empty-diff, and trivial-path cases; the user-message gate is the skill's responsibility because the script has no access to chat context.

## Workflow phases

### 1. DETECT

Run `.github/skills/manual-testing/bin/should-skip.sh`. If exit 0, exit the skill with the one-line reason. If exit 1, capture the diff: `git diff HEAD --name-only` for paths, `git diff HEAD --stat` for the size summary.

### 2. CLASSIFY

Load `references/risk-rubric.md`. Score Likelihood and Impact against the diff:

1. **Likelihood**: pick the anchor (1-5) that best matches the most-impactful change in the diff. When in doubt, round up.
2. **Impact**: start with the rubric anchor. Then grep the diff for the high-impact tag patterns. Any match forces Impact = 5. Record which pattern fired.
3. **Total**: `L × I`, classify into smoke/light/medium/deep tier.
4. **Surfaces**: walk the surface scoping table; any matching path adds its surfaces to the matrix.

Print the classification at the top of the working notes:

```
Risk: 4×5=20 → deep
- Likelihood 4: cross-cutting refactor of TaskPolicy + 2 callers in Filament and MCP.
- Impact 5: high-impact tag matched (`app/Policies/TaskPolicy.php`).
- Surfaces: Filament UI, REST API, MCP.
- Auto-fix ceiling: High only, never in auth/tenancy/migration code.
```

### 3. CHARTER

Load `references/personas.md`, `references/oracles.md`, `references/tours.md`. Compose the charter using exactly this grammar:

> **Explore** `<changed area>` **with** `<personas, tours, surfaces, time budget>` **to discover** `<information goal>`.
> **Definition of Done:** machine-checkable list of assertions.

Definition of Done items must be **measurable**. Bad: "form looks good." Good: "Filament Tasks list view loads in <2s for 1000 records."

Use `templates/charter.md` as the structural starting point and fill in all `<placeholders>`.

**Time budgets per tier:** smoke 2 min, light 5 min, medium 10 min, deep up to 60 min.

If the charter has fewer than 3 DoD items, regenerate. DoD is what prevents the LLM from declaring success on incomplete work.

### 4. PLAN

Build the candidate matrix (Persona × Surface × Tour × Data-state). Reduce in this order:

1. Filter by surface scoping (from CLASSIFY phase).
2. Filter by tour-relevance per surface (per `references/tours.md` selection table).
3. Apply pairwise: every (Persona, Surface) pair appears at least once; every (Surface, Tour) pair at least once; every (Persona, Data-state) at least once.
4. **Hard-add** the multi-tenant checklist row whenever any model/policy/scope path was touched (see `references/multi-tenant-checklist.md`). Never reduced away.
5. Boundary-value analysis: for each text input on the diff, add 2-4 nasty-data cells from `references/data-nasties.md`.

Target matrix sizes:

| Tier | Cells |
|---|---|
| smoke | 3-5 |
| light | 8-12 |
| medium | 15-25 |
| deep | 30-50 + multi-tenant checklist |

**On smoke-tier cell count:** the rubric's "1 persona, single surface, happy path + 1 negative" is the *baseline*. The 3-5 target comes from boundary-state expansion (typical / empty / error) within that single persona+surface combination. So a smoke run is genuinely lightweight: it rotates one cell through 3-5 data states.

Print the matrix as a table at the top of the report (one row per cell). Each cell will be filled with an outcome during EXECUTE.

### 5. SETUP

Verify the dev environment is reachable:

```bash
agent-browser open https://calm-lemur.test
agent-browser get title
```

If unreachable, exit the skill with `BLOCKED: dev server unreachable at https://calm-lemur.test`. Suggest: check Herd status, run `herd start`, verify `.env` `APP_URL`.

If two teams aren't present (required by the multi-tenant checklist when applicable), seed a second team. See the setup section of `references/multi-tenant-checklist.md`.

Gather login-link URLs for every persona in the matrix. Cache them in `.context/testing/state/login-urls.json`.

### 6. EXECUTE

For each cell in the matrix (in order; the multi-tenant checklist runs **first** when present, so a leak halts execution before further cells run):

1. Load `references/surfaces.md` for the relevant surface playbook.
2. Log in as the persona via login-link.
3. Drive the surface through the tour's steps.
4. Apply the data-nasty payload (if the cell is a boundary-value cell).
5. **Visual sweep (Filament UI / Livewire surfaces only):** if the cell's tour is Supermodel, or if any browser-touching cell has not yet been visually swept, run the probes from `references/visual-probes.md` (P1–P8) at the viewports listed there, plus the state-explosion checklist. Take an annotated screenshot per state and per viewport. Then answer the Image-oracle rubric in `references/oracles.md