context-keeper

<!-- mcp-name: io.github.jarmstrong158/context-keeper -->

# Context Keeper

_Part of the [xylem](https://github.com/jarmstrong158/xylem) stack._

Project memory for Claude. Records design decisions, pipeline flows, and constraints so Claude maintains context across conversations.

## The Problem

As conversations get long, Claude loses the "why" behind earlier decisions. New conversations start blank. This causes Claude to make changes that break established patterns — like rewriting a pipeline step it doesn't remember exists.

## The Solution

Context Keeper gives Claude 14 tools to record and retrieve structured project context:

| Tool | Purpose |
|------|---------|
| `record_entry` | Unified write tool — record a decision, pipeline, or constraint via `kind`, with per-kind fields validated server-side. Consolidates the former `record_decision`/`record_pipeline`/`record_constraint` (still dispatchable by those names for back-compat) |
| `get_context` | Retrieve relevant entries by query, tags, scope, or ID — relevance-ranked, pulls `related_to` links by default |
| `query_entries` | **Exact structured-field filtering** (status, origin, tags, scope, hardness, supersession, dates) — deterministic, no ranking; distinct from `get_context`'s relevance search |
| `get_project_summary` | Compact overview for conversation start |
| `update_entry` | Update any entry by ID |
| `deprecate_entry` | Retire an entry with reason (optional `merge_into` folds a duplicate into a survivor) |
| `prune_stale` | Find entries not verified recently |
| `get_compaction_report` | Check if last compaction lost any context |
| `verify_quality` | Scan entries for thin rationale, missing tags, isolated arcs (auto-called by PreCompact hook) |
| `export_markdown` | Regenerate `DECISIONS.md` from the decisions store — a derived, read-only projection |
| `reload_constraints` | Re-surface the constraints-only block on demand mid-session (rules refresh, not the full store) |
| `export_snapshot` | Write the whole store to a committable `.context-keeper/memory.json.gz` for sharing project memory via git |
| `import_snapshot` | Import that committed snapshot into the working store — non-destructive, auto-runs on first use when the store is empty |
| `mirror` | Sync with the optional remote store: `op="pull"` merges remote→local (newest wins), `op="backfill"` pushes local→remote. No-op if the remote is unconfigured |

All data stored as human-editable JSON files in `.context/` inside your project directory. Zero dependencies by default, semantic retrieval optional.

### How this relates to Claude Code's built-in memory

Claude Code ships [two memory mechanisms of its own](https://code.claude.com/docs/en/claude-md): **CLAUDE.md** files you write by hand, and **auto memory**, where Claude saves freeform notes to `~/.claude/projects/<project>/memory/`. Context Keeper is not a replacement for either — it sits on different ground, and the differences are the reason to run it:

| | Auto memory | Context Keeper |
|---|---|---|
| **Shape** | Freeform markdown, an index plus topic files | Typed entries (decision / pipeline / constraint) with server-validated fields |
| **Depth** | Whatever Claude writes | Schema-enforced: `problem` ≥40 chars, `why_chosen` ≥60, thin entries rejected at capture |
| **Lifecycle** | Edit or delete the file | `supersedes` (demoted, still recallable), `deprecate`, `merge_into`, drift + staleness scans |
| **Conflicts** | None | Restatement vs contradiction classified at capture, with origin-based trust precedence |
| **Retrieval** | Index loaded whole each session | Relevance-ranked within a token budget, with an abstention signal on no-answer queries |
| **Reach** | Machine-local; explicitly not shared across machines | Cross-device via the mirror, team-shared via a committed snapshot |

They compose rather than compete: auto memory is good at picking up incidental preferences with zero effort, and Context Keeper is for the decisions and rules you want structured, queryable, enforceable, and portable. Running both is fine — and with `rules_export` enabled, Context Keeper writes into the harness's own `.claude/rules/` surface rather than around it.

**Known gap, stated honestly:** subagents do not inherit the main conversation's session-start injection, so a subagent starts without the project summary. If your workflow leans on subagents, have them call `get_project_summary` explicitly — the retrieval-is-unskippable property holds for the main loop only.

## Capabilities at a glance

Context Keeper is a small, offline-first memory layer; several of its capabilities are easy to miss because they live inside existing tools rather than as separate features. The map below names them in memory-system terms:

| Capability | How Context Keeper does it |
|------------|----------------------------|
| **Procedural memory** | `record_entry(kind="pipeline")` stores ordered, dependency-aware workflows (build/deploy/data flows) with `purpose` + `when_to_invoke` — reusable "how we do X", not just facts. |
| **Deduplication** | Every `record_*` runs a word-set Jaccard pass against the store and returns `similar_entries` when a new entry restates an existing one, so duplicates are caught at capture; `deprecate_entry(merge_into=...)` then folds the duplicate's unique content into the survivor and retires it in one non-destructive step. |
| **Contradiction detection** | Those same overlaps are classified `likely_restatement` vs `likely_contradiction` (negation/antonym polarity), and a reversal raises a `contradiction_note` telling the agent to resolve the conflict rather than leave two live rules disagreeing. |
| **Quality refinement** | `verify_quality` scans for thin rationale, missing tags, legacy-schema entries, and isolated (unlinked) arcs; the PreCompact hook runs it automatically so entries get enriched before context is compressed. |
| **Supersede / decay / forget** | `supersedes` demotes-but-keeps prior decisions (recallable history); `prune_stale` surfaces unverified entries for review; `deprecate_entry` removes an entry from retrieval entirely. |
| **Origin + trust / source attribution** | Every entry records `origin` (`user` / `agent` / `import`); retrieval gives user-stated entries a trust boost and it decides the default winner when entries conflict. |
| **Anticipated queries** | `retrieval_hints` stores alternate phrasings a future session might search for, so vocabulary-mismatch queries hit without embeddings. |
| **Hybrid retrieval** | Lexical (tag + word overlap) by default; an opt-in embedding-cosine blend (`semantic.enabled`) adds vector recall, with lexical fallback when the embedder is offline. |
| **Fact-metadata query** | `query_entries` filters entries by exact predicates over structured fields (status, origin, tags-any/all, scope, hardness, supersession, dates), AND-combined and deterministic — a precise lookup path distinct from `get_context`'s fuzzy relevance ranking. |
| **Cache-friendly injection** | The session-start memory block is deterministically ordered with a stable prefix and the only per-session-volatile line (quality-scan IDs) emitted last, so an unchanged store injects byte-identical text across sessions. |
| **Path-triggered rules** | Scoped constraints project into Claude Code's own `.claude/rules/*.md` format with `paths:` frontmatter (`rules_export`), so the harness loads a rule when the agent *reads* a covered file — before an edit, with no hook involved. The `scope_guard` PreToolUse hook covers the write path for clients without rules support. |
| **Narrative + clustering** | `get_project_summary` clusters decisions by topic above a threshold and renders a compact narrative; the `DECISIONS.md` projection mirrors the store as human-readable prose. |
| **Data export / offline / privacy** | Plain JSON in `.context/` you can read, edit, grep, and commit; runs fully offline with zero required dependencies and no data leaving the machine. |

### Evaluation & benchmarks (open methodology)

The retrieval and honesty properties are measured, not asserted — the harness is in [`evals/`](evals/) and reproducible with no network required:

- **Token reduction** — session-start injection vs. dumping the full store: **97.3% / 94.1% / 85.5% / 73.3%** across four real stores ([`evals/token_reduction.py`](evals/token_reduction.py)). The meaningful property is that injected cost stays roughly flat as the store grows.
- **Retrieval quality** — on a held-out 3-store set, the opt-in semantic blend lifts **hit@5 from 80% → 93% and MRR 0.63 → 0.88** ([`evals/retrieval_eval.py`](evals/retrieval_eval.py)).
- **Abstention** — measures whether `get_context` says "nothing relevant" instead of confabulating on no-answer queries; the 0.20 relevance floor is the highest with zero false-abstention on the eval set ([`evals/abstention.py`](evals/abstention.py)).

Every dataset, metric, and caveat is checked into the repo — see [`evals/README.md`](evals/README.md).

## v0.17: Closing the Delivery Gaps

v0.16 got a rule in front of the model *before* the edit. Measuring real
stores afterwards showed two holes that compound, and one that had been open
since the beginning.

- **Truncated entries are no longer undiscoverable.** The SessionStart hook
  prints the summary text and nothing else, so an entry dropped for budget
  wasn't merely unsummarised — the agent had no way to learn it existed in
  order to go ask for it. On the largest real store that was **116 lines of
  memory silently gone**. The summary now ends with the ids it dropped, paid
  for **inside** the same budget (bare ids cost a fraction of the entries they
  stand for) and emitted in stable store order so the prompt-cache prefix stays
  byte-identical. `summary_dropped_ids` carries the same list structurally.

  The constraints floor still wins over the budget when it has to — one real
  store's 19 constraints are 2429 tokens against a 2000 budget — because
  losing the