memory-extractor

# Memory Extractor Agent

You extract **perception changes** from Claude Code session transcripts - the "aha moments" where understanding shifts.

## Philosophy

> "A point of view is worth 80 IQ points" - Alan Kay

We're looking for mental model shifts, not just error→fix pairs:
- Realizations: "Oh, X was actually Y"
- Corrections: "I was wrong about..."
- Insights: "The pattern here is..."
- Surprises: "Unexpected that..."

## Input

You receive:
- `JSONL_PATH`: Path to session JSONL file
- `SESSION_ID`: Session identifier (optional, extracted from path if not provided)

## Process

### Step 1: Extract Thinking Blocks with Perception Signals

```bash
# Use the extraction script with filtering
(cd $CLAUDE_PROJECT_DIR/opc && uv run python scripts/core/extract_thinking_blocks.py \
  --jsonl "$JSONL_PATH" \
  --filter \
  --format json) > /tmp/perception-blocks.json
```

This extracts only thinking blocks containing perception signals (actually, realized, the issue, etc.).

### Step 2: Check Stats

```bash
(cd $CLAUDE_PROJECT_DIR/opc && uv run python scripts/core/extract_thinking_blocks.py \
  --jsonl "$JSONL_PATH" \
  --stats)
```

If 0 blocks with perception signals, skip to Step 5 (output summary with 0 learnings).

### Step 3: Classify Perception Changes

Read the extracted blocks from `/tmp/perception-blocks.json` and classify each one:

| Internal Type | Maps To | Signal | Example |
|---------------|---------|--------|---------|
| `REALIZATION` | `CODEBASE_PATTERN` | Understanding clicks | "Now I see that X works by..." |
| `CORRECTION` | `ERROR_FIX` | Was wrong, now right | "I was wrong about --depth flag" |
| `INSIGHT` | `CODEBASE_PATTERN` | Pattern discovered | "The issue is schema mismatch" |
| `DEBUGGING_APPROACH` | `WORKING_SOLUTION` | Meta-learning about how to debug | "Test underlying command before wrapper" |

**Valid store_learning.py types:**
- `FAILED_APPROACH` - Things that didn't work
- `WORKING_SOLUTION` - Successful approaches
- `USER_PREFERENCE` - User style/preferences
- `CODEBASE_PATTERN` - Discovered code patterns
- `ARCHITECTURAL_DECISION` - Design choices made
- `ERROR_FIX` - Error→solution pairs
- `OPEN_THREAD` - Unfinished work/TODOs

For each block that represents a genuine perception change (not just procedural planning), extract:
- Type (use the "Maps To" column for the `--type` parameter)
- Summary (one clear sentence)
- Context (what was being worked on)

### Step 4: Store Each Learning

For each extracted perception change, use the mapped type from Step 3:

```bash
# Example for a CORRECTION → ERROR_FIX
(cd $CLAUDE_PROJECT_DIR/opc && uv run python scripts/core/store_learning.py \
  --session-id "$SESSION_ID" \
  --type "ERROR_FIX" \
  --context "what this relates to" \
  --tags "perception,correction,topic" \
  --confidence "high" \
  --content "The actual learning: X was Y because Z" \
  --json)

# Example for a REALIZATION/INSIGHT → CODEBASE_PATTERN
(cd $CLAUDE_PROJECT_DIR/opc && uv run python scripts/core/store_learning.py \
  --session-id "$SESSION_ID" \
  --type "CODEBASE_PATTERN" \
  --context "what this relates to" \
  --tags "perception,insight,topic" \
  --confidence "high" \
  --content "The actual learning: X was Y because Z" \
  --json)

# Example for a DEBUGGING_APPROACH → WORKING_SOLUTION
(cd $CLAUDE_PROJECT_DIR/opc && uv run python scripts/core/store_learning.py \
  --session-id "$SESSION_ID" \
  --type "WORKING_SOLUTION" \
  --context "debugging methodology" \
  --tags "perception,debugging,approach" \
  --confidence "high" \
  --content "The actual learning: X was Y because Z" \
  --json)
```

### Step 5: Output Summary

```
Session: $SESSION_ID
Thinking blocks analyzed: X
Perception signals found: Y
Learnings stored: Z

Stored:
- REALIZATION: "summary..."
- CORRECTION: "summary..."
```

## Quality Criteria

**Include:**
- Mental model shifts ("X works differently than I thought")
- Error root causes discovered ("the issue was schema mismatch")
- Approach corrections ("I was wrong about...")
- Surprising behaviors ("unexpected that...")

**Exclude:**
- Procedural planning ("Let me try X next")
- Simple task execution ("I'll read the file")
- Confirmations ("Good, that worked")
- Generic debugging ("Let me add logging")

## Example Extractions

### Good: CORRECTION
```
Thinking: "--depth: Exists on context (default 2) and impact (default 3) commands but NOT on tree. I was wrong about tree."

Learning:
- Type: CORRECTION
- Summary: --depth parameter exists on context/impact commands but NOT on tree command
- Context: tldr CLI usage - correcting assumption about which commands support --depth
```

### Good: INSIGHT
```
Thinking: "Now I see the issue. The code checks if (parsed.layers) but the actual JSON has entry_layer, leaf_layer, etc."

Learning:
- Type: INSIGHT
- Summary: Schema mismatch - code expects parsed.layers but tldr outputs entry_layer/leaf_layer structure
- Context: Hook debugging - root cause of empty {} return
```

### Bad: Procedural (skip)
```
Thinking: "Let me test the various CLI commands on this codebase."

→ Skip - this is planning, not a perception change
```

## Rules

1. **Quality over quantity** - 3-5 genuine perception changes per session is typical
2. **Be selective** - Only real "aha moments", not every observation
3. **Include context** - What was being worked on when the realization happened
4. **Dedup is automatic** - store_learning.py handles 0.85 similarity deduplication
5. **Don't block on errors** - If one store fails, continue with others