dream

# Dream - Memory Consolidation

Deterministic memory maintenance: detect stale entries, merge duplicates, resolve contradictions, rebuild the MEMORY.md index. All pruning decisions are based on verifiable checks (file exists? function exists? duplicate content?), not LLM judgment.

## Argument Resolution

```python
DRY_RUN = "--dry-run" in "$ARGUMENTS"  # Preview changes without writing
```

## Overview

Memory files accumulate across sessions. Over time they develop problems:
- **Stale references** — memories pointing to files, functions, or classes that no longer exist
- **Duplicates** — multiple memories covering the same topic with overlapping content
- **Contradictions** — newer memories superseding older ones without cleanup
- **Index drift** — MEMORY.md index out of sync with actual memory files

This skill fixes all four problems using deterministic checks only.

> **Cadence (CC 2.1.142+):** Reactive compaction now sizes its first summarize attempt to the actual overflow, so long sessions stall mid-turn far less often. The "run nightly" cadence can relax toward "run when memory files accumulate" — consolidation is no longer needed to head off compaction inefficiency.

---

## STEP 1: Discover Memory Files

```python
# Find the memory directory (agent-specific or project-level)
# Agent memory lives in: .claude/agent-memory/<agent-id>/
# Project memory lives in: .claude/projects/<hash>/memory/
# Also check: .claude/memory/

memory_dirs = []
Glob(pattern=".claude/agent-memory/*/MEMORY.md")
Glob(pattern=".claude/projects/*/memory/MEMORY.md")
Glob(pattern=".claude/memory/MEMORY.md")

# For each discovered MEMORY.md, glob all *.md files in that directory
for dir in memory_dirs:
    Glob(pattern=f"{dir}/../*.md")  # All memory files alongside MEMORY.md
```

Read every discovered memory file. Parse frontmatter (`name`, `description`, `type`) and body content. Build an in-memory inventory:

```
inventory = [{
    "path": "/abs/path/to/file.md",
    "name": frontmatter.name,
    "type": frontmatter.type,  # user, feedback, project, reference
    "description": frontmatter.description,
    "body": body_text,
    "file_refs": [],      # extracted file paths
    "symbol_refs": [],    # extracted function/class names
    "topics": [],         # key phrases for duplicate detection
}]
```

---

## STEP 2: Detect Staleness

For each memory file, extract references and verify they still exist.

### 2a: File Path References

Extract paths that look like file references (patterns: paths with `/` and file extensions, backtick-wrapped paths):

```python
# Regex-like extraction from body text:
# - Paths containing / with common extensions: .py, .ts, .tsx, .js, .json, .md, .yaml, .yml, .sh
# - Backtick-wrapped paths: `src/something/file.ts`
# - Quoted paths in frontmatter descriptions
```

**Classify each ref's SCOPE before verifying it.** `Glob` only sees the current repo, so a path that
lives anywhere else can never match and would otherwise be scored as missing. A memory about
`~/.claude` hooks, a homebrew cask, a cmux config, or another repo is not stale just because this
repo does not contain it.

```python
def scope(ref):
    # Anything rooted outside the working repo is UNVERIFIABLE, not missing.
    if ref.startswith(("~", "/", "$")):          return "UNVERIFIABLE"
    if ref.startswith(("http://", "https://")):  return "UNVERIFIABLE"
    if re.match(r'^[A-Za-z0-9_.-]+/', ref) and not (REPO / ref.split("/")[0]).exists():
        return "UNVERIFIABLE"   # first segment is not a real top-level dir here
    return "REPO_RELATIVE"

verifiable = [r for r in file_refs if scope(r) == "REPO_RELATIVE"]
external   = [r for r in file_refs if scope(r) == "UNVERIFIABLE"]

missing = []
for ref in verifiable:
    Glob(pattern=ref)
    # If no match → missing.append(ref)
```

**The staleness ratio is computed over `verifiable` ONLY.** `external` refs are recorded for the
report and never counted toward pruning. A memory with zero verifiable refs is `EVERGREEN` no
matter how many external paths it names.

### 2b: Symbol References

Extract function/class names (patterns: `function_name()`, `ClassName`, `def function_name`):

```python
for symbol in symbol_refs:
    Grep(pattern=symbol, path=".", output_mode="files_with_matches", head_limit=1)
    # If no match → mark as STALE_SYMBOL_REF
```

### 2c: Staleness Classification

| Finding | Classification | Action |
|---------|---------------|--------|
| **Zero VERIFIABLE refs** (none, or all UNVERIFIABLE) | EVERGREEN | Keep |
| All verifiable refs valid, all symbols found | FRESH | Keep |
| Some verifiable refs missing | PARTIALLY_STALE | Flag for review |
| All verifiable refs missing AND all symbols missing | FULLY_STALE | Prune candidate |

Only memories classified as FULLY_STALE are auto-pruned. PARTIALLY_STALE memories are reported but kept — the user decides.

### 2d: Prune guards — checked AFTER classification, before any delete

`FULLY_STALE` is necessary but **not sufficient** to delete. Every guard below downgrades to
PARTIALLY_STALE (kept + flagged). These exist because memory files are **not in git**: a wrong
delete is silent and unrecoverable, so the asymmetry always favours keeping.

```python
GUARD_DAYS = 14

for m in list(fully_stale_files):
    reason = None
    # 1. Preferences do not decay because a path moved.
    if m["type"] == "user":
        reason = "type:user is never auto-pruned"
    # 2. A feedback/reference memory carries a LESSON; the file paths in it are
    #    illustrations, not a manifest. Its worth does not expire when an
    #    illustrative path moves, and ref-extraction is lossy anyway (it catches
    #    `file.ts` but misses `file.ts:186` and `functionName()`). Only project
    #    memories — which track live work against concrete files — are eligible
    #    to go fully stale on ref death.
    elif m["type"] in ("feedback", "reference"):
        reason = f"type:{m['type']} value is the lesson, not its file refs"
    # 3