exam-audit

# exam-audit — workspace health check (read-only)

## Purpose
Inspect a prep workspace built by `exam-ingest` and report health issues. This is a read-only inspector. Do NOT fix anything by default; only fix after the user explicitly grants permission. Emit a concrete issue report; never silently modify or delete files.

## Activation
Activate when the user suspects the workspace is broken (missing chapters, ungradable quiz items, inconsistent progress), or when the user wants a pre-review health check before studying. Do not activate to build, teach, or grade.

## Inputs
- `references/wiki/` — chapter knowledge files (`chN_*.md`).
- `references/quiz_bank.json` — quiz items.
- `references/teaching_examples.json` — optional parallel teaching inventory; examples may remain here even when they are deliberately excluded from the gradable bank.
- `references/teaching_baseline.json` — append-only retention baseline for new workspaces; it must never shrink on re-ingest and must not be hand-edited.
- `references/figure_page_index.json` / `references/image_question_index.json` — regenerable three-sided visual coverage evidence (wiki / prompt / answer).
- `.ingest/source_manifest.json` — source-root-relative paths, source hashes, media types, and parse status.
- `.ingest/base_content_units.jsonl` / `content_units.jsonl` — deterministic extraction baseline and the compiled view after replaying validated patches.
- `.ingest/base_chapter_phase_mappings.jsonl` / `chapter_phase_mappings.jsonl` — explicit chapter-to-study-phase identity.
- `.ingest/review_queue.jsonl` / `review_patches.jsonl` — typed issue lifecycle and append-only evidence-bound patch ledger.
- `.ingest/build_manifest.json` / `unbound_review.json` — source root, page-quality accounting, input/derived hashes, and issues not yet bound to one source record.
- `ingest_report.json` — import counts, current-snapshot statistics, missing-answer alerts, and the legacy retention fallback.
- `study_plan.md` — phase plan with chapter anchors.
- `study_state.json` — the structured progress state (the SINGLE SOURCE OF TRUTH when present: `phase_checklist`, `mistake_archive`, `confusion_log`).
- `study_progress.md` — a GENERATED VIEW of the state (rendered phase checkpoints and recorded wrong-question IDs); stale/hand-edited when it drifts from `study_state.json`.

## Workflow
Inspect read-only. Open and parse files; never write, rename, or delete. Check each item below and record every failure as a concrete issue (file path + what is wrong).

1. **Structure.** For each phase listed in `study_plan.md`, confirm a matching `references/wiki/chN_*.md` file exists. Flag orphan chapters (wiki files no phase references) and broken links (phases pointing to absent chapters).
2. **Quiz bank.** For each item in `references/quiz_bank.json`: confirm `type` is one of the six allowed types (choice / subjective / diagram / fill_blank / true_false / code); confirm `choice` items carry `options`; treat missing `keywords` on subjective items as a grading-quality warning. An item without an answer must declare `answer_status: unknown`; in a structured workspace it also remains a blocking review issue until an evidence-backed official answer, an explicitly labeled AI answer, or an unrecoverable terminal decision is recorded.
3. **Provenance honesty.** Flag any AI-generated answer presented as the teacher's standard answer (missing the ⚠️ marker). Flag any AI-supplement wiki passage that should carry 🟡 but does not.
4. **Plan/progress consistency.** When `study_state.json` exists, treat it as the source of truth: confirm `study_progress.md` is a faithful render of it (flag drift / stale hand-edits where the md disagrees with the state), and check the state's `phase_checklist` phases map to `study_plan.md`. When no `study_state.json` exists, audit `study_progress.md` directly. Either way, confirm each rendered phase-checkpoint line maps to a phase in `study_plan.md` and every wrong-question ID exists in `references/quiz_bank.json`. Note: the template anchor `<!-- PHASE_CHECKLIST -->` is replaced by `scripts/ingest.py` at generation time and is absent from a correct finished workspace — do NOT report its absence as a problem.
5. **Teaching-example retention.** Prefer `references/teaching_baseline.json`; validate its schema, exact per-chapter mapping, append-only policy, and require every baseline ID to have a same-chapter current snapshot in `references/teaching_examples.json`. Presence of the same ID in `references/quiz_bank.json` is diagnostic overlap only and never substitutes for that teaching snapshot. Only old workspaces without the baseline file may fall back to `ingest_report.json.teaching_example_ids`. It is valid for an ungradable worked example to be absent from the bank if the teaching layer retains it; disappearance from the current teaching layer is a blocking retention gap even when a quiz item survives. Validate the current teaching manifest's IDs, chapter/phase tags, source pages, answer source, and asset paths. Read it per chapter in tutoring; do not treat the whole manifest as a new answer source.
6. **Three-sided visual completeness.** Inspect each denominator separately: `figure_page_index.json.wiki_visual_coverage` for detected material pages embedded in wiki, `image_question_index.json.prompt_suspects` for missing prompt context, and `answer_suspects` for missing answer context. A zero on one side is never evidence that the other two are complete. Require matching `integrity` snapshots and re-hash their declared quiz, teaching, wiki, and asset inputs; stale or missing freshness evidence blocks a new-manifest phase from being complete. Flag NUL/control-byte warnings and missing/capped pages. State that this is deterministic recall coverage, not semantic proof that every meaningful figure was found.
7. **Structured ingestion integrity.** When `.ingest/` exists, strictly load every source, unit, mapping, issue, patch, unbound entry, and build-manifest