obsidian-hybrid-search

# Obsidian Hybrid Search

[![npm version](https://img.shields.io/npm/v/obsidian-hybrid-search)](https://www.npmjs.com/package/obsidian-hybrid-search)
[![Tests](https://github.com/flowing-abyss/obsidian-hybrid-search/actions/workflows/ci.yml/badge.svg)](https://github.com/flowing-abyss/obsidian-hybrid-search/actions)
[![Total downloads](https://badgen.net/npm/dt/obsidian-hybrid-search)](https://www.npmjs.com/package/obsidian-hybrid-search)

An [MCP server](https://modelcontextprotocol.io) and CLI tool that makes your Obsidian vault queryable by AI assistants. Indexes notes into SQLite with FTS5 full-text search, trigram fuzzy matching, and `sqlite-vec` vector similarity — results are merged with Reciprocal Rank Fusion (RRF) and scored 0–1.

Once connected, any MCP-compatible AI assistant can answer questions grounded in your actual notes: finding knowledge by meaning, exact phrase, or title; traversing the wikilink graph; filtering by tag or folder; always citing the source note. No guessing from training data, no manual copy-paste.

No external services required. A bundled `@huggingface/transformers` model handles embeddings locally by default. Any OpenAI-compatible API (OpenRouter, Ollama, LM Studio) works as a drop-in replacement.

## Search quality

Evaluated on the [Obsidian Help vault](eval/README.md) (171 notes, 58 queries, local model):

|                | **OHS** (this project) | [qmd](https://github.com/tobi/qmd) |
| -------------- | :--------------------: | :--------------------------------: |
| nDCG@5         |       **0.733**        |               0.659                |
| MRR            |       **0.788**        |               0.665                |
| Hit@1          |       **0.724**        |               0.500                |
| Avg query time |      **571 ms** ¹      |              754 ms ²              |
| Model download |      **~117 MB**       |              ~2.2 GB               |

¹ CPU (Apple Silicon), hybrid mode, no rerank. ² GPU (Apple Silicon Metal), LLM query expansion + reranking.

OHS uses `Xenova/multilingual-e5-small`. [How to reproduce →](eval/COMPARISON.md) · [Full benchmark →](eval/README.md)

### Real knowledge-vault benchmark

OHS is also evaluated on Andy Matuschak’s public evergreen notes, converted into an Obsidian vault with title-based note filenames, source URLs in frontmatter, local attachments, and `5,000+` internal note links across `1,357` notes.

The curated golden set includes `78` hand-judged queries across known-item lookup, paraphrases, quote fragments, ambiguous topics, citation lookup, and multi-note evidence.

Using the default local embedding model, OHS performs strongly on this dense note network.

| Metric    | Value     |
| --------- | --------- |
| nDCG@5    | **0.722** |
| nDCG@10   | 0.753     |
| MRR       | 0.874     |
| Hit@1     | 0.795     |
| Hit@5     | 0.974     |
| Recall@10 | 0.972     |
| AllRel@10 | 0.949     |

The benchmark exercises retrieval over a highly connected real-world knowledge vault, including queries that do not simply repeat note titles.

[Result JSON](eval/results/evergreen-notes-no-rerank.json) · [Reproduce and interpret →](fixtures/evergreen-notes/README.md)

### Large memory benchmark

To test retrieval on a larger public dataset,
[LongMemEval-S](https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned)
was converted into a `22,419`-note Obsidian-style vault with `470` retrieval
queries. Using `baai/bge-m3` embeddings, OHS ranked the answer-bearing notes
strongly:

| Metric    | Value     |
| --------- | --------- |
| nDCG@5    | **0.895** |
| MRR       | 0.920     |
| Hit@1     | 0.889     |
| Hit@5     | 0.968     |
| Recall@10 | 0.950     |
| AllRel@10 | 0.904     |

For this benchmark, each query uses the LongMemEval-provided haystack as its
search scope. That makes the result reproducible and easy to inspect query by
query, while still exercising retrieval over a large generated memory vault.

[Result JSON](eval/results/longmemeval-s-no-rerank.json) · [Reproduce and interpret →](fixtures/longmemeval-s/README.md)

## Features

- **Hybrid search**
  - BM25 + fuzzy title + semantic embeddings, fused with RRF
- **Alias search**
  - notes with `aliases:` in frontmatter are indexed and searchable by any alias; alias matches are boosted in BM25 (weight 5×) and fuzzy title scoring
- **Four search modes**
  - `hybrid`, `semantic`, `fulltext`, `title` (for text queries)
- **Similar note lookup**
  - pass `--path` to find semantically related notes using stored chunk embeddings, with a title + content fallback
- **Graph traversal**
  - `--path --related` shows linked notes at configurable depth; filter by `--direction outgoing|backlinks|both`
- **Links & backlinks**
  - every result includes outgoing links and backlinks
- **Scope filtering**
  - restrict to subfolder(s); supports multiple values and exclusions (`-notes/dev/`)
- **Tag filtering**
  - filter by tag(s); supports multiple values and exclusions (`-category/cs`)
- **Snippet control**
  - `--snippet-length` sets the context window; empty snippets always fall back to note content
- **Extended output**
  - `--extended` adds a TAGS/ALIASES column to the CLI table showing frontmatter tags (`#tag`) and aliases
- **Incremental indexing**
  - only re-indexes changed files; watches for edits in real time
- **Multi-query fan-out**
  - pass multiple queries at once (`ohs "q1" "q2"` or `queries[]` in MCP); results are merged via RRF — a note that ranks well in any one query floats to the top; useful when the note may use different vocabulary than the query
- **Cross-encoder reranking**
  - `--rerank` re-scores results with `bge-reranker-v2-m3` (ONNX int8, ~570 MB download once); improves precision for conceptual and multilingual queries; applied after multi-query merge
- **Local embeddings**
  - works offline via `@huggingface/transformers` (no API key required); default model: Xenova/multilingual-e5-small, 100+ languages
- **Remote embeddings**
  - OpenAI-compatible API (OpenRouter, Ollama, etc.)
- **Note reading**
  - `read` fetches one or more notes by vault-relative path; returns full content with title, aliases, tags, links, and backlinks; on path miss returns top-3 fuzzy suggestions
- **Ignore patterns**
  - exclude folders, extensions, or specific files
- **Obsidian plugin**
  - native search modal inside Obsidian powered by the same CLI — see [obsidian-hybrid-search-plugin](https://github.com/flowing-abyss/obsidian-hybrid-search-plugin)

## Installation

```bash
npm install -g obsidian-hybrid-search
# or run directly without installing:
npx obsidian-hybrid-search
```

## CLI usage

### Quick start

**Option A — recommended: set `OBSIDIAN_VAULT_PATH` once in your shell profile.**

This lets you run the tool from any directory. Add to `~/.zshrc` or `~/.bashrc`:

```bash
export OBSIDIAN_VAULT_PATH="/path/to/your/vault"
```

Then reload (`source ~/.zshrc`) and index your vault once:

```bash
obsidian-hybrid-search reindex
```

After that you can search from any directory:

```bash
obsidian-hybrid-search "zettelkasten"
```

---

**Option B — no env var: run from inside your vault.**

The tool detects the vault root by looking for the `.obsidian/` folder, walking up from the current directory. `cd` into your vault (or any subfolder) and run:

```bash
cd /path/to/your/vault
obsidian-hybrid-search reindex   # detects vault root, creates DB, indexes everything
obsidian-hybrid-search "zettelkasten"
```

Commands work from any directory inside the vault tree. From outside the vault (e.g. via shell aliases called from `~`), use Option A or pass `--db /path/to/vault/.obsidian-hybrid-search.db` explicitly.

---

**Optional: remote embedding API instead of local model.**

By default the local `Xenova/multilingual-e5-small` model is used — works offline, no API key needed. Downloads ~117 MB on first run. Supports 100+ languages including Russian, Chinese, Japanese, and more.

To use a remote API instead, add to your shell profile:

```bash
export OPENAI_API_KEY="sk-..."

# Default API base is https://api.openai.com/v1 — override for other providers:
# export OPENAI_BASE_URL="https://openrouter.ai/api/v1"  # OpenRouter
# export OPENAI_BASE_URL="http://localhost:11434/v1"     # Ollama (no key needed)
# export OPENAI_BASE_URL="http://localhost:1234/v1"      # LM Studio (no key needed)

# Optional: override the embedding model (default: text-embedding-3-small)
# export OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
```

### Search modes

| Scenario        | How                                                                 | Modes                                               |
| --------------- | ------------------------------------------------------------------- | --------------------------------------------------- |
| Text query      | `obsidian-hybrid-search "some topic"`                               | `hybrid` (default), `semantic`, `fulltext`, `title` |
| Similar notes   | `obsidian-hybrid-search --path notes/pkm/zettelkasten.md`           | Semantic similarity from stored chunk embeddings    |
| Graph traversal | `obsidian-hybrid-search --path notes/pkm/zettelkasten.md --related` | Links & backlinks via BFS                           |

`--mode` only affects text queries. When `--path` is given without `--related`, search uses semantic similarity regardless of `--mode`; `--path --related` traverses links/backlinks instead.

```bash
# Hybrid search (default)
obsidian-hybrid-search "zettelkasten atomic notes"

# Fulltext BM25 search
obsidian-hybrid-search "permanent notes" --mode fulltext

# Fuzzy title search (fast, typo-tolerant)
obsidian-hybrid-search "zettleksten" --mode title

# Semantic / vector search
obsidian-hybrid-search "how to build a knowledge graph" --mode semantic

# Limit results and set a score threshold
obsidian-hybrid-search "productivity systems" --limit 5 --threshold 0.3

# Restrict to a subfolder
obsidian-hybrid-search "daily review" --scope notes/perio