dream

# Dream - Memory Consolidation

Deterministic memory maintenance: detect stale entries, merge duplicates, resolve contradictions, rebuild the MEMORY.md index. All pruning decisions are based on verifiable checks (file exists? function exists? duplicate content?), not LLM judgment.

## Argument Resolution

```python
DRY_RUN = "--dry-run" in "$ARGUMENTS"  # Preview changes without writing
```

## Overview

Memory files accumulate across sessions. Over time they develop problems:
- **Stale references** — memories pointing to files, functions, or classes that no longer exist
- **Duplicates** — multiple memories covering the same topic with overlapping content
- **Contradictions** — newer memories superseding older ones without cleanup
- **Index drift** — MEMORY.md index out of sync with actual memory files

This skill fixes all four problems using deterministic checks only.

> **Cadence (CC 2.1.142+):** Reactive compaction now sizes its first summarize attempt to the actual overflow, so long sessions stall mid-turn far less often. The "run nightly" cadence can relax toward "run when memory files accumulate" — consolidation is no longer needed to head off compaction inefficiency.

---

## STEP 1: Discover Memory Files

```python
# Find the memory directory (agent-specific or project-level)
# Agent memory lives in: .claude/agent-memory/<agent-id>/
# Project memory lives in: .claude/projects/<hash>/memory/
# Also check: .claude/memory/

memory_dirs = []
Glob(pattern=".claude/agent-memory/*/MEMORY.md")
Glob(pattern=".claude/projects/*/memory/MEMORY.md")
Glob(pattern=".claude/memory/MEMORY.md")

# For each discovered MEMORY.md, glob all *.md files in that directory
for dir in memory_dirs:
    Glob(pattern=f"{dir}/../*.md")  # All memory files alongside MEMORY.md
```

Read every discovered memory file. Parse frontmatter (`name`, `description`, `type`) and body content. Build an in-memory inventory:

```
inventory = [{
    "path": "/abs/path/to/file.md",
    "name": frontmatter.name,
    "type": frontmatter.type,  # user, feedback, project, reference
    "description": frontmatter.description,
    "body": body_text,
    "file_refs": [],      # extracted file paths
    "symbol_refs": [],    # extracted function/class names
    "topics": [],         # key phrases for duplicate detection
}]
```

---

## STEP 2: Detect Staleness

For each memory file, extract references and verify they still exist.

### 2a: File Path References

Extract paths that look like file references (patterns: paths with `/` and file extensions, backtick-wrapped paths):

```python
# Regex-like extraction from body text:
# - Paths containing / with common extensions: .py, .ts, .tsx, .js, .json, .md, .yaml, .yml, .sh
# - Backtick-wrapped paths: `src/something/file.ts`
# - Quoted paths in frontmatter descriptions

for ref in file_refs:
    Glob(pattern=ref)  # Check if file exists
    # If no match → mark as STALE_FILE_REF
```

### 2b: Symbol References

Extract function/class names (patterns: `function_name()`, `ClassName`, `def function_name`):

```python
for symbol in symbol_refs:
    Grep(pattern=symbol, path=".", output_mode="files_with_matches", head_limit=1)
    # If no match → mark as STALE_SYMBOL_REF
```

### 2c: Staleness Classification

| Finding | Classification | Action |
|---------|---------------|--------|
| All file refs valid, all symbols found | FRESH | Keep |
| Some file refs missing | PARTIALLY_STALE | Flag for review |
| All file refs missing AND all symbols missing | FULLY_STALE | Prune candidate |
| No external refs (pure decision/preference) | EVERGREEN | Keep |

Only memories classified as FULLY_STALE are auto-pruned. PARTIALLY_STALE memories are reported but kept — the user decides.

### STEP 2.5: Consult-gate (#2351) — never prune a memory that's still being used

Closing the VERIFY loop: a deletion must survive the question *"was this actually consulted?"*. A memory whose external refs all vanished (FULLY_STALE) but that the agent keeps looking up is still load-bearing — its **refs** are stale, its **knowledge** is live. So before pruning, read `.claude/logs/memory-consult.jsonl` (written by `memory-validator` on every `mcp__memory__search_nodes`/`open_nodes`/`read_graph`) and **downgrade any recently-consulted FULLY_STALE memory to PARTIALLY_STALE** (kept + flagged, not auto-deleted).

```python
import json, time
from pathlib import Path

def recently_consulted_terms(days=14):
    log = Path(".claude/logs/memory-consult.jsonl")
    if not log.exists():
        return set()
    cutoff = time.time() - days * 86400
    terms = set()
    for line in log.read_text().splitlines():
        try:
            e = json.loads(line)
        except ValueError:
            continue  # best-effort: skip malformed lines
        # open_nodes carries exact entity names; search carries a query string
        terms.update(n.lower() for n in e.get("names", []))
        if e.get("query"):
            terms.update(w.lower() for w in e["query"].split() if len(w) > 2)
    return terms

consulted = recently_consulted_terms()
for m in list(fully_stale_files):
    slug = Path(m["path"]).stem.lower()
    name = (m.get("name") or "").lower()
    if name in consulted or any(c in slug or slug in c for c in consulted):
        m["classification"] = "PARTIALLY_STALE"
        m["kept_reason"] = "consult-gate: looked up in the last 14 days (#2351)"
        fully_stale_files.remove(m)
        partially_stale_files.append(m)
```

This is conservative by design — fuzzy term matching errs toward **keeping** a maybe-consulted memory rather than deleting a live one. The Step 6 report records each consult-gated keep (the "did it matter?" audit the loop was missing). If the log is absent (consult instrumentation not yet exercised), the gate is a no-op and pruning proceeds as before.

---

## STEP 3: Detect Duplicates

Compare memories pairwise within the same directory. Two memories are duplicates when:

1. **Same type** (both `feedback`, both `p