prd-spec-generator

<!-- mcp-name: io.github.cdeust/prd-spec-generator -->

<p align="center">
  <img src="assets/banner.svg" alt="prd-spec-generator — a stateless reducer that turns a feature description into a PRD the rest of your pipeline can act on" width="100%"/>
</p>

<p align="center">
  <img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="MIT License">
  <img src="https://img.shields.io/badge/TypeScript-5.9+-3178c6.svg" alt="TypeScript 5.9+">
  <img src="https://img.shields.io/badge/Node-20.x_·_22.x-339933.svg" alt="Node 20/22">
  <img src="https://img.shields.io/badge/Tests-877_passing-brightgreen" alt="877 passing">
  <img src="https://img.shields.io/badge/Packages-10-orange" alt="10 packages">
  <img src="https://img.shields.io/badge/MCP_Tools-17-8A2BE2" alt="17 MCP tools">
  <img src="https://img.shields.io/badge/Validators-Hard_Output_Rules-red" alt="Hard Output Rules">
  <img src="https://img.shields.io/badge/Phase_4-Closed_Loop_Calibration-success" alt="Phase 4 closed-loop calibration shipped">
</p>

<p align="center">
  <a href="#what-an-agent-can-ask-it">What An Agent Asks</a> · <a href="#getting-started">Getting Started</a> · <a href="#the-pipeline">Pipeline</a> · <a href="#the-mcp-tools">Tools</a> · <a href="#multi-judge-verification">Verification</a> · <a href="#calibration--falsification">Calibration</a> · <a href="#architecture">Architecture</a> · <a href="#the-zetetic-standard">Zetetic Standard</a>
</p>

<p align="center">
  <strong>Companion projects:</strong><br>
  <a href="https://github.com/cdeust/Cortex">Cortex</a> — persistent memory that injects past decisions into every PRD<br>
  <a href="https://github.com/cdeust/zetetic-team-subagents">zetetic-team-subagents</a> — 97 genius reasoning patterns that judge each claim<br>
  <a href="https://github.com/cdeust/automatised-pipeline">automatised-pipeline</a> — the codebase intelligence layer this generator consumes upstream
</p>

---

Every AI agent that drafts a PRD eventually invents a function that doesn't exist, claims latency it can't measure, or writes acceptance criteria that don't tie back to the requirements they're supposed to test. The output sounds confident. It is not actionable. The next stage in the pipeline — code generation, ticket import, sprint planning — silently inherits the hallucination, ships it, and pays for it later.

**prd-spec-generator** is a TypeScript MCP server that fixes this at the structural level. The pipeline is a stateless reducer (`step(state, result?) → next_state, action`) driven by a host (Claude Code or any MCP-speaking agent). Sections are produced one at a time, validated by deterministic Hard Output Rules before the host ever sees them, and every load-bearing claim is judged by a panel of genius reasoning agents drawn from `zetetic-team-subagents` against the codebase graph from `automatised-pipeline`. **Phase 4** then closes the loop: per-judge reliability is calibrated from history, retry budgets are derived from survival statistics, KPI gates are tuned against frozen baselines, and held-out partitions are mechanically sealed so no calibration result can be peeked at before evaluation.

**10 packages. 17 MCP tools. 20 pipeline steps (11 PRD generation + 9 opt-in implementation). Multi-judge verification with consensus. Closed-loop calibration with externally-grounded falsifiers. 877 tests. Every numeric constant traces to a citation, a benchmark, or a `// source: provisional heuristic` admission.**

---

## Phase 4 — closed-loop reliability calibration (shipped)

The verification subsystem is no longer a one-shot pass/fail report. Every claim resolution can flush an observation back to a calibration repository, every consensus run can pull calibrated posteriors from history, and every closed loop runs an external control arm so the calibration's effect is *measured*, not assumed.

- **Per-judge Bayesian reliability calibration** — Beta(7,3) prior with sensitivity / specificity split per `claim_type`. Posteriors stored in a SQLite-backed `ReliabilityRepository`; observations flushed on every claim resolution.
- **MAX_ATTEMPTS retry calibration** — Kaplan-Meier survival math (`kmEstimate` / `kmMedianAttempts` / `logRankTest` with Greenwood + Brookmeyer-Crowley CIs); Schoenfeld sample-size derivation event-rate-corrected to ~519 (was 823) against the measured `event_rate=0.4762`, CP CI `[0.4456, 0.5069]`.
- **KPI gate tuning** — Clopper-Pearson exact CIs; per-machine-class wall_time normalization with 5-bucket `detectMachineClass`; frozen-baseline content-hash assertion; `loadCalibratedGates` + `hold_provisional` ratchet protection.
- **Plan-mismatch fire-rate** — measured via XmR control charts (Wheeler 1995, Western Electric 1956) with a synthetic injection round-trip pre-flight that catches drift between the diagnostic prefix and the regex matcher.
- **Externally-grounded held-out subsets** — Ajv schema oracle, mathjs oracle, `tsc` subprocess code oracle, `validateSection` spec oracle. `OracleUnavailableError` typed throw replaces stub-mode fabrication. This is the layer that breaks annotator-circularity — judges and oracles share no inference path.
- **CC-3 forced-exploration control arms** — every closed loop carves out a 20% partition that reverts to the prior. Without it, calibration-on-calibration looks like progress whether or not it actually is.
- **Cross-arm comparison metrics** — `computeAblationComparison` / `computeReliabilityComparison` / `computeKpiGateComparison` produce paired-bootstrap CIs (Efron & Tibshirani 1993 §16.4; deterministic mulberry32 RNG; 12-decimal reproducibility pin). Outcome is a falsifiable recommendation: `calibrated_helps`, `prior_helps`, or `inconclusive_underpowered`.
- **Mechanically-enforced held-out partition seals** — three sealed lock files (`maxattempts-heldout.lock.json`, `kpigates-heldout.lock.json`, `heldout-partition.lock.json`) commit a sha256 of the partition before evaluation. The `SEAL_VERIFIED` typeof sentinel is the only way to compute cross-arm metrics on a sealed partition; passing anything else is a type error at the boundary.
- **Production-mode dispatcher** — `makeProductionDispatcher` + `AgentInvoker` interface. The CLI `--mode production|canned` flag selects whether calibration sees real verdicts or canned ones; the canned arm is preserved for offline reproducibility.

---

## What an agent can ask it

```
start_pipeline(feature_description, codebase_path?)
  → returns the first NextAction; the host executes it and feeds the result
    back via submit_action_result. Eleven steps later: 9 PRD files written.

submit_action_result(run_id, result)
  → drives the reducer one more step. The host sees only SUBSTANTIVE actions
    (ask_user, call_pipeline_tool, call_cortex_tool, spawn_subagents,
     write_file, done, failed). emit_message is coalesced into the
     messages array; the host never has to "advance past" a banner.

validate_prd_section(content, section_type)
  → deterministic Hard Output Rules — zero LLM calls, pure regex/parsing.
  → returns: violations[], hasCriticalViolations, totalScore.

validate_prd_document(sections[])
  → cross-section checks: SP arithmetic, AC numbering, FR-AC coverage,
    test traceability. Catches what per-section validation misses.

coordinate_context_budget(prd_context, completed_sections[])
  → per-section retrieval/generation token budgets so Cortex recall and
    section drafting don't fight over the same context window.

map_failure_to_retrieval(violations[])
  → closes the validation→retrieval feedback loop. When a section fails
    validation, this returns the corrective Cortex query that would
    have prevented the failure.
```

---

## Getting started

### Install (marketplace — recommended)

```bash
claude plugin marketplace add cdeust/prd-spec-generator
claude plugin install prd-spec-generator
```

Restart your Claude Code session. The 17 MCP tools register on first
stdio handshake. Then:

```
/generate-prd build OAuth login for the admin console
```

The plugin's bundled MCP server at `mcp-server/index.js` is self-contained
(only `better-sqlite3` is an optional native dependency for the evidence
repository — gracefully degrades to in-memory mode when absent).

### Companion ecosystem

For full effect, install the three companion plugins so the pipeline can
consume codebase intelligence, persistent memory, and the genius-agent
panel:

```bash
claude plugin marketplace add cdeust/automatised-pipeline    # codebase graph intel
claude plugin marketplace add cdeust/Cortex                  # persistent memory
claude plugin marketplace add cdeust/zetetic-team-subagents  # the genius + team agents

claude plugin install automatised-pipeline
claude plugin install cortex
claude plugin install zetetic-team-subagents
```

Each plugin is independently useful; together they are the ai-architect
ecosystem. See [Companion ecosystem](#companion-ecosystem) above.

### Building from source

For development or to run the audit cycle locally:

```bash
git clone https://github.com/cdeust/prd-spec-generator.git
cd prd-spec-generator
pnpm install --frozen-lockfile
pnpm build      # builds all 9 buildable packages via tsc
pnpm bundle     # produces the standalone mcp-server/index.js
pnpm test       # 877 tests (vitest workspace, all packages; live MCP
                # integration env-gated by AIPRD_PIPELINE_BIN)
```

`pnpm verify` runs all of the above (install + build + bundle + test) —
same as CI.

**Prerequisites for source builds:** Node.js 20.x or 22.x, pnpm v10+
(`corepack enable && corepack prepare pnpm@10`).

### Smoke-test offline

```bash
# Reducer end-to-end without a real host (uses the canned dispatcher):
pnpm test --filter @prd-gen/orchestration smoke

# Benchmark KPI run:
pnpm test --filter @prd-gen/benchmark pipeline-kpis
```

Both run in <2s on an M-series Mac. No LLM calls, no MCP traffic — the
reducer is fully driven by canned ActionResults so you can audit behaviour
offline.

### Use with ot