ai-output-verifier

# AI-Output Verifier

AI is fluent, confident, and sometimes completely wrong — inventing facts, citations, and details in the same authoritative tone as the correct ones. That confidence is exactly what makes unverified trust dangerous. This checks a specific output: which claims are most likely wrong or fabricated, what genuinely needs independent verification, how to verify it, and the tells of hallucination — so you use AI's speed without inheriting its errors.

## What This Skill Produces

- **A risk read of the output** — which specific claims are most likely to be wrong, outdated, or made up (facts, numbers, citations, names, recent events, specifics)
- **Verify vs. low-risk split** — what genuinely needs independent checking vs. what's low-stakes or self-evident, so you spend effort where it counts
- **How to verify each** — the concrete way to check the high-risk claims (a primary source, a second tool, a domain expert, testing it)
- **The hallucination tells** — the signs AI is likely fabricating (oddly specific citations, confident claims about recent/niche facts, plausible-but-unverifiable details)
- **A verification habit** — how to build appropriate checking into your AI use by default, scaled to the stakes (trust more for low-stakes, verify hard for high-stakes)

## Required Inputs

Ask for these if not provided:
- **The output** — the AI response to check (paste it)
- **What it's for** — the stakes (a casual question vs. something you'll publish, decide on, or act on)
- **The domain** — factual/technical/legal/medical/current-events (some are far higher-risk for AI)
- **What you'd do with it** — trust it, act on it, share it, build on it

## Framework: Risk-Rate The Claims, Verify What Matters

1. **Scan for the high-risk claim types.** Specific facts, numbers, dates, names, citations, recent events, and niche/technical specifics are where AI most often invents — flag these.
2. **Split by risk and stakes.** Separate the claims that genuinely need verification (high-risk × high-stakes) from the low-risk or low-stakes ones you can reasonably accept — don't verify everything equally.
3. **Verify against real sources.** For the high-risk claims, check a primary source, a second independent tool, an expert, or by testing — not by asking the same AI "are you sure?" (it'll often just re-confirm).
4. **Watch the hallucination tells.** Oddly precise citations, confident answers about very recent or obscure things, and unverifiable specifics are red flags — treat them as unverified until checked.
5. **Scale trust to stakes.** For low-stakes uses, light verification is fine; for anything you'll publish, decide on, or that could harm if wrong, verify hard. Build this reflex in.

## Output Format

### Verifying: [the output] · for [use/stakes]

**High-risk claims (verify these):** [specific facts/numbers/citations/recent/niche → most likely wrong].
**Low-risk (reasonable to accept):** [self-evident / low-stakes parts].
**How to verify each:** [primary source / second tool / expert / test — not re-asking the same AI].
**Hallucination tells present:** [odd-specific citations · confident on recent/niche · unverifiable specifics].
**Trust level for your use:** [light check for low-stakes / verify hard because it's high-stakes].

## Quality Checks
- [ ] Flags the specific high-risk claim types in the output
- [ ] Splits what needs verification from what's low-risk, by stakes
- [ ] Gives concrete verification methods (not "ask the AI again")
- [ ] Names the hallucination/overconfidence tells present
- [ ] Scales the recommended trust to the actual stakes

## Anti-Patterns
- **"Verify everything"** equally, ignoring stakes.
- **Re-asking the same AI** "are you sure?" as verification.
- **Trusting confident tone** as a signal of correctness.
- **Missing the high-risk claim types** (citations, recent facts, numbers).
- **No stakes-based scaling** of how hard to check.

## Example Trigger Phrases
- "Can I trust this answer the AI gave me?"
- "How do I verify what ChatGPT told me before I use it?"
- "Fact-check this AI output — I'm about to publish it."
- "Is this AI response reliable enough to act on?"
- "What in this AI answer should I double-check?"