dream

# Dream - Memory Consolidation

Deterministic memory maintenance: detect stale entries, merge duplicates, resolve contradictions, rebuild the MEMORY.md index. All pruning decisions are based on verifiable checks (file exists? function exists? duplicate content?), not LLM judgment.

## Argument Resolution

```python
DRY_RUN = "--dry-run" in "$ARGUMENTS"  # Preview changes without writing
```

## Overview

Memory files accumulate across sessions. Over time they develop problems:
- **Stale references** — memories pointing to files, functions, or classes that no longer exist
- **Duplicates** — multiple memories covering the same topic with overlapping content
- **Contradictions** — newer memories superseding older ones without cleanup
- **Index drift** — MEMORY.md index out of sync with actual memory files

This skill fixes all four problems using deterministic checks only.

> **Cadence (CC 2.1.142+):** Reactive compaction now sizes its first summarize attempt to the actual overflow, so long sessions stall mid-turn far less often. The "run nightly" cadence can relax toward "run when memory files accumulate" — consolidation is no longer needed to head off compaction inefficiency.

---

## STEP 1: Discover Memory Files

```python
# Find the memory directory (agent-specific or project-level)
# Agent memory lives in: .claude/agent-memory/<agent-id>/
# Project memory lives in: .claude/projects/<hash>/memory/
# Also check: .claude/memory/

memory_dirs = []
Glob(pattern=".claude/agent-memory/*/MEMORY.md")
Glob(pattern=".claude/projects/*/memory/MEMORY.md")
Glob(pattern=".claude/memory/MEMORY.md")

# For each discovered MEMORY.md, glob all *.md files in that directory
for dir in memory_dirs:
    Glob(pattern=f"{dir}/../*.md")  # All memory files alongside MEMORY.md
```

Read every discovered memory file. Parse frontmatter (`name`, `description`, `type`) and body content. Build an in-memory inventory:

```
inventory = [{
    "path": "/abs/path/to/file.md",
    "name": frontmatter.name,
    "type": frontmatter.type,  # user, feedback, project, reference
    "description": frontmatter.description,
    "body": body_text,
    "file_refs": [],      # extracted file paths
    "symbol_refs": [],    # extracted function/class names
    "topics": [],         # key phrases for duplicate detection
}]
```

---

## STEP 2: Detect Staleness

For each memory file, extract references and verify they still exist.

### 2a: File Path References

Extract paths that look like file references (patterns: paths with `/` and file extensions, backtick-wrapped paths):

```python
# Regex-like extraction from body text:
# - Paths containing / with common extensions: .py, .ts, .tsx, .js, .json, .md, .yaml, .yml, .sh
# - Backtick-wrapped paths: `src/something/file.ts`
# - Quoted paths in frontmatter descriptions

for ref in file_refs:
    Glob(pattern=ref)  # Check if file exists
    # If no match → mark as STALE_FILE_REF
```

### 2b: Symbol References

Extract function/class names (patterns: `function_name()`, `ClassName`, `def function_name`):

```python
for symbol in symbol_refs:
    Grep(pattern=symbol, path=".", output_mode="files_with_matches", head_limit=1)
    # If no match → mark as STALE_SYMBOL_REF
```

### 2c: Staleness Classification

| Finding | Classification | Action |
|---------|---------------|--------|
| All file refs valid, all symbols found | FRESH | Keep |
| Some file refs missing | PARTIALLY_STALE | Flag for review |
| All file refs missing AND all symbols missing | FULLY_STALE | Prune candidate |
| No external refs (pure decision/preference) | EVERGREEN | Keep |

Only memories classified as FULLY_STALE are auto-pruned. PARTIALLY_STALE memories are reported but kept — the user decides.

---

## STEP 3: Detect Duplicates

Compare memories pairwise within the same directory. Two memories are duplicates when:

1. **Same type** (both `feedback`, both `project`, etc.)
2. **Overlapping topic** — 60%+ of significant words (excluding stopwords) appear in both bodies
3. **Same subject** — `name` or `description` fields reference the same concept

```python
stopwords = {"the", "a", "an", "is", "are", "was", "were", "be", "been",
             "have", "has", "had", "do", "does", "did", "will", "would",
             "could", "should", "may", "might", "can", "shall", "to", "of",
             "in", "for", "on", "with", "at", "by", "from", "as", "into",
             "through", "during", "before", "after", "this", "that", "it",
             "not", "no", "but", "or", "and", "if", "then", "than", "so"}

def significant_words(text):
    words = set(text.lower().split()) - stopwords
    return {w for w in words if len(w) > 2}

def overlap_ratio(words_a, words_b):
    if not words_a or not words_b:
        return 0.0
    intersection = words_a & words_b
    smaller = min(len(words_a), len(words_b))
    return len(intersection) / smaller if smaller > 0 else 0.0

# For each pair with same type:
#   if overlap_ratio >= 0.6 → DUPLICATE pair
#   Keep the NEWER file (by filesystem mtime), prune the older
```

---

## STEP 4: Resolve Contradictions

Contradictions occur when two memories of the same type make opposing claims about the same subject. Detection:

1. **Same type + same topic** (overlap >= 0.4 but < 0.6 — related but not duplicate)
2. **Negation signals** — one body contains negation of the other's assertion:
   - "do X" vs "do not X" / "don't X" / "never X"
   - "use X" vs "avoid X" / "stop using X"
   - "prefer X" vs "prefer Y" (for same decision domain)

```python
negation_pairs = [
    ("do ", "do not "), ("do ", "don't "),
    ("use ", "avoid "), ("use ", "stop using "),
    ("prefer ", "don't prefer "), ("always ", "never "),
]

# For each pair flagged as contradictory:
#   Keep the NEWER file (more recent decision supersedes)
#   Prune the older file
```

---

## STEP 5: Execute Changes (or Dry Run)

### Dry Run Mode (`--dry-run`)

If `--dry-run` flag is present, skip all writes. Output the full report (Step 6) with `