obsidian-hybrid-search

# Obsidian Hybrid Search

[![npm version](https://img.shields.io/npm/v/obsidian-hybrid-search)](https://www.npmjs.com/package/obsidian-hybrid-search)
[![Tests](https://github.com/flowing-abyss/obsidian-hybrid-search/actions/workflows/ci.yml/badge.svg)](https://github.com/flowing-abyss/obsidian-hybrid-search/actions)
[![Total downloads](https://badgen.net/npm/dt/obsidian-hybrid-search)](https://www.npmjs.com/package/obsidian-hybrid-search)

<p align="center">
  <img src="assets/banner.png" alt="Obsidian Hybrid Search explains hybrid retrieval from Obsidian notes" />
</p>

Your Obsidian vault already contains your best thinking. Obsidian Hybrid Search makes that thinking easier to find, reuse, and bring into AI-assisted work.

It gives your vault one retrieval engine and three practical ways to use it. The native [Obsidian plugin][obsidian-plugin] gives you fast search, previews, similar notes, link discovery, and graph views while you write. The MCP server lets AI agents search and read your notes as tool calls. The CLI gives power users the same engine for indexing, filtering, reranking, reading, and scripting.

The search understands how real vaults are built. It combines semantic search, BM25 full text, fuzzy title and alias matching, tags, folders, frontmatter, wikilinks, backlinks, and similar-note lookup. You can search by idea, phrase, title, relationship, or metadata without remembering the exact words you wrote.

That turns Obsidian into a stronger personal knowledge system and a better starting point for AI work. Agents can begin from your own notes, pull cited context from source files, follow related material, and work with knowledge you already trust. OHS runs locally by default with SQLite, FTS5, sqlite-vec, RRF ranking, and optional OpenAI-compatible embedding APIs.

## Search quality

Evaluated on the [Obsidian Help vault](eval/README.md) (171 notes, 58 queries, local model):

|                | **OHS** (this project) | [qmd](https://github.com/tobi/qmd) |
| -------------- | :--------------------: | :--------------------------------: |
| nDCG@5         |       **0.733**        |               0.659                |
| MRR            |       **0.788**        |               0.665                |
| Hit@1          |       **0.724**        |               0.500                |
| Avg query time |      **571 ms** ¹      |              754 ms ²              |
| Model download |      **~117 MB**       |              ~2.2 GB               |

¹ CPU (Apple Silicon), hybrid mode, no rerank. ² GPU (Apple Silicon Metal), LLM query expansion + reranking.

OHS uses `Xenova/multilingual-e5-small`. [How to reproduce →](eval/COMPARISON.md) · [Full benchmark →](eval/README.md)

### Real knowledge-vault benchmark

OHS is also evaluated on Andy Matuschak’s public evergreen notes, converted into an Obsidian vault with title-based note filenames, source URLs in frontmatter, local attachments, and `5,000+` internal note links across `1,357` notes.

The curated golden set includes `78` hand-judged queries across known-item lookup, paraphrases, quote fragments, ambiguous topics, citation lookup, and multi-note evidence.

Using the default local embedding model, OHS performs strongly on this dense note network.

| Metric    | Value     |
| --------- | --------- |
| nDCG@5    | **0.722** |
| nDCG@10   | 0.753     |
| MRR       | 0.874     |
| Hit@1     | 0.795     |
| Hit@5     | 0.974     |
| Recall@10 | 0.972     |
| AllRel@10 | 0.949     |

The benchmark exercises retrieval over a highly connected real-world knowledge vault, including queries that do not simply repeat note titles.

[Result JSON](eval/results/evergreen-notes-no-rerank.json) · [Reproduce and interpret →](fixtures/evergreen-notes/README.md)

### Large memory benchmark

To test retrieval on a larger public dataset,
[LongMemEval-S](https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned)
was converted into a `22,419`-note Obsidian-style vault with `470` retrieval
queries. Using `baai/bge-m3` embeddings, OHS ranked the answer-bearing notes
strongly:

| Metric    | Value     |
| --------- | --------- |
| nDCG@5    | **0.895** |
| MRR       | 0.920     |
| Hit@1     | 0.889     |
| Hit@5     | 0.968     |
| Recall@10 | 0.950     |
| AllRel@10 | 0.904     |

For this benchmark, each query uses the LongMemEval-provided haystack as its
search scope. That makes the result reproducible and easy to inspect query by
query, while still exercising retrieval over a large generated memory vault.

[Result JSON](eval/results/longmemeval-s-no-rerank.json) · [Reproduce and interpret →](fixtures/longmemeval-s/README.md)

## Features

- **Hybrid search**
  - BM25 + fuzzy title + semantic embeddings, fused with RRF
- **Alias search**
  - notes with `aliases:` in frontmatter are indexed and searchable by any alias; alias matches are boosted in BM25 (weight 5×) and fuzzy title scoring
- **Four search modes**
  - `hybrid`, `semantic`, `fulltext`, `title` (for text queries)
- **Similar note lookup**
  - pass `--path` to find semantically related notes using stored chunk embeddings, with a title + content fallback
- **Graph traversal**
  - `--path --related` shows linked notes at configurable depth; filter by `--direction outgoing|backlinks|both`
- **Links & backlinks**
  - every result includes outgoing links and backlinks
- **Scope filtering**
  - restrict to subfolder(s); supports multiple values and exclusions (`-notes/dev/`)
- **Tag filtering**
  - filter by tag(s); supports multiple values and exclusions (`-category/cs`)
- **Snippet control**
  - `--snippet-length` sets the context window; empty snippets always fall back to note content
- **Extended output**
  - `--extended` adds a TAGS/ALIASES column to the CLI table showing frontmatter tags (`#tag`) and aliases
- **Incremental indexing**
  - only re-indexes changed files; watches for edits in real time
- **Multi-query fan-out**
  - pass multiple queries at once (`ohs "q1" "q2"` or `queries[]` in MCP); results are merged via RRF, so a note that ranks well in any one query floats to the top; useful when the note may use different vocabulary than the query
- **Cross-encoder reranking**
  - `--rerank` re-scores results with `bge-reranker-v2-m3` (ONNX int8, ~570 MB download once); improves precision for conceptual and multilingual queries; applied after multi-query merge
- **Local embeddings**
  - works offline via `@huggingface/transformers` (no API key required); default model: Xenova/multilingual-e5-small, 100+ languages
- **Remote embeddings**
  - OpenAI-compatible API (OpenRouter, Ollama, etc.)
- **Note reading**
  - `read` fetches one or more notes by vault-relative path; returns full content with title, aliases, tags, links, and backlinks; on path miss returns top-3 fuzzy suggestions
- **Ignore patterns**
  - exclude folders, extensions, or specific files
- **Obsidian plugin**
  - native search modal inside Obsidian powered by the same CLI; see [obsidian-hybrid-search-plugin](https://github.com/flowing-abyss/obsidian-hybrid-search-plugin)

## Installation

```bash
npm install -g obsidian-hybrid-search
```

## CLI usage

### Quick start

The recommended setup is to set `OBSIDIAN_VAULT_PATH` once in `~/.zshrc` or `~/.bashrc`. This lets you run the CLI from any directory.

```bash
export OBSIDIAN_VAULT_PATH="/path/to/your/vault"
```

Open a new terminal and index the vault once.

```bash
ohs reindex
```

You can now search from any directory.

```bash
ohs "zettelkasten"
```

### Run from a vault

Alternatively, run the CLI without an environment variable from any directory inside your vault. It finds the vault root by walking up to the nearest `.obsidian/` folder.

```bash
cd /path/to/your/vault
ohs reindex
ohs "zettelkasten"
```

From outside the vault, set `OBSIDIAN_VAULT_PATH` or pass `--db /path/to/vault/.obsidian-hybrid-search.db` explicitly.

### Optional remote embeddings

By default, the CLI uses the local `Xenova/multilingual-e5-small` model. It works offline without an API key, downloads about 117 MB on first use, and supports more than 100 languages.

To use a remote API, add its settings to your shell profile.

```bash
export OPENAI_API_KEY="sk-..."

# Override the default API base for another provider
# export OPENAI_BASE_URL="https://openrouter.ai/api/v1"  # OpenRouter
# export OPENAI_BASE_URL="http://localhost:11434/v1"     # Ollama (no key needed)
# export OPENAI_BASE_URL="http://localhost:1234/v1"      # LM Studio (no key needed)

# Override the default text-embedding-3-small model
# export OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
```

### Search modes

The CLI supports four search modes called `hybrid`, `fulltext`, `semantic`, and `title`, plus graph traversal for linked notes. The commands below show how to use them, apply filters, rerank results, and control the output.

```bash
# Hybrid search (default)
ohs "zettelkasten atomic notes"

# Fulltext BM25 search
ohs "permanent notes" --mode fulltext

# Fuzzy title search (fast, typo-tolerant)
ohs "zettleksten" --mode title

# Semantic / vector search
ohs "how to build a knowledge graph" --mode semantic

# Limit results and set a score threshold
ohs "productivity systems" --limit 5 --threshold 0.3

# Restrict to a subfolder
ohs "daily review" --scope notes/periodic/
ohs "daily review" --folder notes/periodic/    # alias for --scope

# Restrict to multiple subfolders (OR)
ohs "productivity" --scope notes/pkm/ --scope notes/2024/

# Exclude a subfolder
ohs "programming" --scope notes/ --scope -notes/archive/

# Filter by tag
ohs "productivity" --tag pkm
ohs "machine learning" --tag note/basic/primary

# Filter by multiple tags (AND include, exclude with -)
ohs "learning" --tag pkm --tag work

# Filter by frontmatter / properties (exact match, case-insensitive)
ohs "notes" --frontmatter status:todo
ohs "notes" --prop priority:high          # --prop is alias for --frontmatter

# Filter by multiple frontmatter fields (AND)
ohs "notes" --frontmatter st