pi-history-ingest

# Pi History Ingest — Session Mining

You are extracting knowledge from the user's Pi coding agent sessions and distilling it into the Obsidian wiki. Pi sessions are stored as structured JSONL with a tree layout — your job is to follow the active branch, extract durable knowledge, and compile it.

**Session knowledge closure:** Pi session files are the only factual source for this skill. Do not add background knowledge from model training, other tools, package docs, local files, or the current conversation unless that fact appears in the selected session entries. If outside context seems useful, mark it as an open question or skip it — never present it as extracted session knowledge.

This skill can be invoked directly or via the `wiki-history-ingest` router (`/wiki-history-ingest pi`).

## Before You Start

**Writing profile:** Before drafting or rewriting natural-language Markdown, read and apply the `Writing Profile Resolution` section in `llm-wiki/SKILL.md`. Framework schema, provenance, safety, and operation-specific requirements take precedence.
`WRITING.md` preferences apply only to newly drafted or rewritten natural-language Markdown; preserve source content and structured records.

1. **Resolve config** — follow the Config Resolution Protocol in `llm-wiki/SKILL.md` (inline `@name` override → walk up CWD for `.env` → global config → prompt setup). This gives `OBSIDIAN_VAULT_PATH` and `PI_HISTORY_PATH` (defaults to `~/.pi/agent/sessions`)
2. Read `.manifest.json` at the vault root to check what has already been ingested
3. Read `index.md` at the vault root to understand what the wiki already contains

## Ingest Modes

### Append Mode (default)

Check `.manifest.json` for each source file. Only process:

- Files not in the manifest (new sessions)
- Files whose modification time is newer than `ingested_at` in the manifest

Use this mode for regular syncs.

### Full Mode

Process everything regardless of manifest. Use after `wiki-rebuild` or if the user explicitly asks for a full re-ingest.

## Pi Data Layout

Pi stores sessions under `~/.pi/agent/sessions/` (or the path set by `PI_CODING_AGENT_SESSION_DIR`).

```
~/.pi/agent/sessions/
├── --<cwd-path>--/                    # Working directory with / replaced by -
│   └── <timestamp>_<uuid>.jsonl       # Session JSONL file
└── ...
```

The session filename contains an ISO timestamp and UUID. The parent directory encodes the working directory where the session was created.

### Session JSONL Format

Each `.jsonl` file is a sequence of JSON objects. The first line is always a `session` header; subsequent lines are tree entries with `id` and `parentId`.

Key entry types:

| `type` | Purpose | Ingest? |
|---|---|---|
| `session` | Header with `cwd`, `version`, `id`, `timestamp` | Metadata only |
| `message` | Conversation turn (`user`, `assistant`, `toolResult`, `bashExecution`, etc.) | **Primary source** |
| `session_info` | Display name set via `/name` | For session title |
| `compaction` | Context compaction summary | **High signal** |
| `branch_summary` | Summary when switching branches via `/tree` | **High signal** |
| `model_change` | Model switch event | Skip |
| `thinking_level_change` | Thinking level change | Skip |
| `custom` | Extension state (not in LLM context) | Skip |
| `custom_message` | Extension-injected message | Context only |
| `label` | User bookmark/label | Skip |

### Message roles inside `message` entries

- `user` — user input; `content` is string or `(TextContent \| ImageContent)[]`
- `assistant` — assistant response; `content` is `(TextContent \| ThinkingContent \| ToolCall)[]`
- `toolResult` — tool execution result; `content` is `(TextContent \| ImageContent)[]`
- `bashExecution` — bash command + output; `command`, `output`, `exitCode`
- `branchSummary` — branch switch summary; `summary` string
- `compactionSummary` — compaction summary; `summary` string

### Key data sources ranked by value

1. **`message` entries (`user` + `assistant`)** — full conversation transcripts; rich but noisy
2. **`compaction` entries** — pre-synthesized summaries of older context; gold
3. **`branch_summary` entries** — summaries of abandoned branches; good signal
4. **`bashExecution` entries** — concrete commands run; useful for workflow patterns
5. **`session_info` entries** — session name for topic inference

Skip `model_change`, `thinking_level_change`, `custom` (extension state), and `label` entries.

## Step 1: Survey and Compute Delta

Scan `PI_HISTORY_PATH` and compare against `.manifest.json`:

```bash
# List all session files
find ~/.pi/agent/sessions -name "*.jsonl" -type f

# Or with custom path
find "$PI_HISTORY_PATH" -name "*.jsonl" -type f
```

Build an inventory. For each session file, record:
- `path` — absolute path
- `cwd` — decoded from parent directory name (`--<path>--` → `/path`)
- `session_name` — from the latest `session_info` entry (if any)
- `modified_at` — file mtime
- `already_ingested` — presence in `.manifest.json`

Classify each file:
- **New** — not in manifest
- **Modified** — in manifest but file is newer than `ingested_at`
- **Unchanged** — already ingested and unchanged

Report a concise delta summary before deep parsing:
> "Found N Pi sessions across K projects. Delta: X new, Y modified."

## Step 2: Parse Session JSONL

For each selected session file, read it line by line. Because sessions use a tree structure, build the active branch first:

1. Parse all entries into a map by `id`
2. Find the current leaf (the entry with no children, or the last `message` entry)
3. Walk `parentId` chain from leaf to root to get the active path
4. Reverse the path so it's chronological

### Extraction rules

From the active path, extract:

- **`session` header** — `cwd`, `timestamp`, `parentSession` (if forked)
- **`session_info`** — `name` field for session title/topic inference
- **`message` entries with `role: "user"`** — extract `content` text (skip images)
- **`message` entries with `role: "assistant"`*