mgi-mind

# MGI-Mind

**[English](README.md)** | **[Русский](README.ru.md)** | **[中文](README.zh.md)**

**[Latest release: v2.7.1](https://github.com/madgodinc/mgi-mind/releases/tag/v2.7.1)** · **[CHANGELOG](CHANGELOG.md)** · **[Discussions](https://github.com/madgodinc/mgi-mind/discussions)** · **[Issues](https://github.com/madgodinc/mgi-mind/issues)** · **[Contributing](CONTRIBUTING.md)**

Local long-term memory for AI assistants. One Rust binary, a local Qdrant
vector database, local ONNX models. Speaks MCP, so Claude Code and other
assistants read and write memory on their own. Also a normal CLI.

<p align="center">
  <img src="docs/demo.gif" alt="Store a fact, then recall it by meaning: a query with no shared keywords still finds it" width="900">
  <br>
  <em>Store a fact in plain words, recall it with a query that shares none of them. Semantic search matches intent. Your assistant runs the same <code>mind_search</code> on its own over MCP. Regenerate with <code>vhs docs/demo.tape</code>.</em>
</p>

The same exchange inside an assistant:

```
You:  what was the deploy server again?

Assistant (calls mind_search "deploy server"):
  -> "Deploy server is 10.0.0.5:8080, SSH as deploy@, key in vault"  (source: infra.md)

You:  right, thanks
```

Nothing leaves the box. Embeddings, search, reranking and the vault are
all local. No cloud account, no API key, no telemetry.

---

## Contents

- [What it is](#what-it-is)
  - [The validity model](#the-validity-model)
- [Why bother](#why-bother)
- [Quick start](#quick-start)
- [Using it](#using-it)
- [How it works](#how-it-works)
- [Command reference](#command-reference)
- [Configuration](#configuration)
- [Languages and the reranker](#languages-and-the-reranker)
- [Changing the embedding model](#changing-the-embedding-model)
- [Troubleshooting](#troubleshooting)
- [Security](#security)
- [Status and audit](#status-and-audit)
- [Project layout](#project-layout)
- [License](#license)

## What it is

MGI-Mind sits between you and your assistant. The assistant writes short
notes ("memories") and facts as a conversation goes, and pulls the
relevant ones back when they matter.

Retrieval, not just storage:

- **Hybrid search.** Each memory is stored as two vectors: a dense vector
  (multilingual-e5-base, 768 dims) for meaning, and a sparse term-frequency
  vector (TF-IDF, BM25-style) for exact words. A query runs both arms and
  fuses them with Reciprocal Rank Fusion. "The server box" finds "deploy
  host" through the dense arm; `fossilize_replay` finds the one note that
  contains that exact token through the sparse arm.
- **Cross-encoder reranking.** Fused top candidates are re-scored by
  bge-reranker-base, which reads the query and each passage together and
  is more accurate than comparing vectors. On by default and English-tuned;
  see [Languages and the reranker](#languages-and-the-reranker) for the
  trade-off on other languages.
- **One warm process.** `mgimind mcp` is the MCP server itself: it runs
  for the whole session, models load once and stay warm in memory, so a
  lookup costs milliseconds instead of reloading on every call.

Around retrieval there's a knowledge graph for structured facts, per-agent
session logs for cross-session continuity, and an encrypted terminal-only
vault for secrets.

### The validity model

Hybrid search is table stakes; most memory tools have some version of it.
The part that is harder to find is what keeps the store from ossifying as it
fills with old, contradictory, or self-reinforcing beliefs. These run mostly
on their own:

- **Duel rule.** When a new fact contradicts an existing one on the same
  subject and predicate, the second write resolves against the first instead
  of piling up a second "truth". An entrenched fact (many dependants,
  confirmations, age) holds; a strong fresh fact flips it and dampens the
  loser to a hidden `stale` status; a borderline one is marked contested or
  diverted to quarantine. Confirmations accrue live: re-asserting the same
  triple raises its own `confirmations_count`, so a fact confirmed a few
  times resists a single fresh contradiction (contested, not flipped) while
  an unconfirmed young fact still flips as easily as before. `dependants`
  still only comes from the `migrate-v14 dependants --apply` backfill - a
  cheap live producer (it would mean walking the graph on every write) is not
  implemented. Nothing is deleted, so the audit log keeps the loser; a
  re-proposed `stale` loser gets a fair new duel against whatever is current,
  keeping its own original `created_at` and confirmation history rather than
  starting over. Automatic, inside a normal `mind_fact add`.
- **Doubt window.** An entrenched fact has to keep re-justifying itself. Every
  fact query measures how far the current context has drifted from the context
  the fact was written in; after five consecutive drifted retrievals the fact
  stops outranking facts that still hold up, and any single in-context
  retrieval resets the count. Nothing is hidden or deleted. Drift is measured
  on vectors with the corpus mean removed, because raw cosine cannot separate
  topics in an anisotropic embedding space (see `mgimind calibrate` below).
  The current context is the running process's own recent queries, so this
  applies inside a warm `mgimind mcp` or `serve-http` session; a one-shot CLI
  command has no context yet and no fact is penalised for that. This ships in
  shadow: drift is measured and recorded, and no fact changes, until you set
  `doubt_drift_threshold` from your own store's numbers and switch
  `doubt_drift_mode` to `"enforce"`. A background pass separately
  re-tests entrenched facts that have gone quiet, under hard guarantees: never
  during a tool call, a per-tick cap, and a load-aware cadence.
- **Inheritance discount.** `mind_context`'s briefing (what a session sees at
  start) marks every fact it surfaces as inherited for that process. If a
  duel challenger is a re-proposal of one of those inherited facts - the
  agent echoing back what it was told rather than independently reconfirming
  it live - that challenge counts at half weight. The flag clears the moment
  the agent actually re-asserts the fact live and it lands without a
  conflict. Single-source decay (repeated confirmation from the same source
  counting for less each time) has the formula (`diversity_weighted_count`)
  but no live caller yet - not implemented, no "cannot co-confirm" guarantee
  today.
- **Two time axes on facts.** Record-time (`created_at`/`valid_until`: when
  the store learned/retired a value) is bi-temporal with the superseded
  chain: a predicate can be registered (`mind_predicate`) as single-valued,
  temporal-single (one current value, but the previous ones are kept and
  queryable by date via `mind_history`), or multi-valued, and a superseded
  value is hidden from default ranking, not erased. An optional second axis,
  world-time (`valid_from`, settable on add), records when a fact became
  true in the world instead of when the store heard about it; `mind_fact
  action=query` with `as_of` + `world_time=true` resolves point-in-time
  queries against it, falling back to record-time per fact when it has no
  `valid_from`. Existing facts need no migration - they simply have no
  `valid_from` and behave exactly as before.
- **Explicit invalidate is final.** `mind_fact action=invalidate` is a
  one-way door: re-adding the same triple afterward refuses (verdict
  `invalidated`) instead of silently reviving it. `mind_fact
  action=restore` is the only way back.
- **Typed outcome signals.** `mind_outcome` records that a remembered fix
  actually worked (`test_passed`, `code_compiled`, `user_confirmed`,
  `cited_by`). A real success raises a memory's weight and marks a procedure
  verified; a failure pulls the weight down rather than being ignored.

These are research-shaped mechanisms, and their tuning constants are still
being calibrated (the `TODO(phase-4-calibration)` markers in `duel.rs` /
`doubt.rs` are honest). The shape is the point: a memory that argues with
itself and demotes what stops holding up, instead of accreting forever.

`mgimind calibrate` measures whether that shape behaves as intended. It runs a
corpus of realistic conflict situations through the live duel formulas and
reports how many land on the outcome a person would expect: a fresh unsupported
claim cannot overturn an entrenched belief, a CI signal can, repetition alone
coexists rather than overwriting. Today 14 of 15 scenarios match intent; the one
that does not is printed with its reason, not hidden, and the suite runs in CI.
It also reports the doubt window's drift distribution, and that report exists
because of what it found. The drift threshold was 0.4, picked in 384-dim MiniLM
space and left in place when the default model became 768-dim multilingual-e5.
Measured on a live store, 400 memories and all 79800 pairs, raw cosine ran from
0.666 to 0.972: the threshold could not fire once. The corpus mean vector had
norm 0.884, so almost all of that cosine was a component every embedding shares.
Centering fixes the measurement, and `calibrate` proposes a threshold from the
drift your own store produces rather than from a constant that travelled between
embedding spaces.

This measures the *shape* of the validity model, separate from retrieval recall
(R@k, see [BENCHMARKS.md](BENCHMARKS.md)). The two are different numbers and
never belong in the same table.

## Why bother

An assistant without memory asks for the same context every session and
can't build on yesterday's work. The usual workaround is you keeping
notes, tags, and folders, which the assistant still can't read by
meaning.

The thing MGI-Mind does that Obsidian and Notion don't: **the system
decides what to write down, you don't.** The MCP server reads what the
assistant is doing in real time and routes facts, decisions, and fixes
into the store through a relevance gate. You don't file, you don't tag,
you