obsidian-hybrid-search

# Obsidian Hybrid Search

[![npm version](https://img.shields.io/npm/v/obsidian-hybrid-search)](https://www.npmjs.com/package/obsidian-hybrid-search)
[![Tests](https://github.com/flowing-abyss/obsidian-hybrid-search/actions/workflows/ci.yml/badge.svg)](https://github.com/flowing-abyss/obsidian-hybrid-search/actions)
[![Total downloads](https://badgen.net/npm/dt/obsidian-hybrid-search)](https://www.npmjs.com/package/obsidian-hybrid-search)

<p align="center">
  <img src="assets/banner.png" alt="Obsidian Hybrid Search explains hybrid retrieval from Obsidian notes" />
</p>

Your Obsidian vault already contains your best thinking. Obsidian Hybrid Search makes that thinking easier to find, reuse, and bring into AI-assisted work.

It gives your vault one retrieval engine and three practical ways to use it. The native [Obsidian plugin][obsidian-plugin] gives you fast search, previews, similar notes, link discovery, and graph views while you write. The MCP server lets AI agents search and read your notes as tool calls. The CLI gives power users the same engine for indexing, filtering, reranking, reading, and scripting.

The search understands how real vaults are built. It combines semantic search, BM25 full text, fuzzy title and alias matching, tags, folders, frontmatter, wikilinks, backlinks, and similar-note lookup. You can search by idea, phrase, title, relationship, or metadata without remembering the exact words you wrote.

That turns Obsidian into a stronger personal knowledge system and a better starting point for AI work. Agents can begin from your own notes, pull cited context from source files, follow related material, and work with knowledge you already trust. OHS runs locally by default with SQLite, FTS5, `sqlite-vec`, RRF ranking, and optional OpenAI-compatible embedding APIs.

## Search quality

Evaluated on the [Obsidian Help vault](eval/README.md) (171 notes, 58 queries, local model):

|                | **OHS** (this project) | [qmd](https://github.com/tobi/qmd) |
| -------------- | :--------------------: | :--------------------------------: |
| nDCG@5         |       **0.733**        |               0.659                |
| MRR            |       **0.788**        |               0.665                |
| Hit@1          |       **0.724**        |               0.500                |
| Avg query time |      **571 ms** ¹      |              754 ms ²              |
| Model download |      **~117 MB**       |              ~2.2 GB               |

¹ CPU (Apple Silicon), hybrid mode, no rerank. ² GPU (Apple Silicon Metal), LLM query expansion + reranking.

OHS uses `Xenova/multilingual-e5-small`. [How to reproduce →](eval/COMPARISON.md) · [Full benchmark →](eval/README.md)

### Real knowledge-vault benchmark

OHS is also evaluated on Andy Matuschak’s public evergreen notes, converted into an Obsidian vault with title-based note filenames, source URLs in frontmatter, local attachments, and `5,000+` internal note links across `1,357` notes.

The curated golden set includes `78` hand-judged queries across known-item lookup, paraphrases, quote fragments, ambiguous topics, citation lookup, and multi-note evidence.

Using the default local embedding model, OHS performs strongly on this dense note network.

| Metric    | Value     |
| --------- | --------- |
| nDCG@5    | **0.722** |
| nDCG@10   | 0.753     |
| MRR       | 0.874     |
| Hit@1     | 0.795     |
| Hit@5     | 0.974     |
| Recall@10 | 0.972     |
| AllRel@10 | 0.949     |

The benchmark exercises retrieval over a highly connected real-world knowledge vault, including queries that do not simply repeat note titles.

[Result JSON](eval/results/evergreen-notes-no-rerank.json) · [Reproduce and interpret →](fixtures/evergreen-notes/README.md)

### Large memory benchmark

To test retrieval on a larger public dataset,
[LongMemEval-S](https://huggingface.co/datasets/xiaowu0162/longmemeval-cleaned)
was converted into a `22,419`-note Obsidian-style vault with `470` retrieval
queries. Using `baai/bge-m3` embeddings, OHS ranked the answer-bearing notes
strongly:

| Metric    | Value     |
| --------- | --------- |
| nDCG@5    | **0.895** |
| MRR       | 0.920     |
| Hit@1     | 0.889     |
| Hit@5     | 0.968     |
| Recall@10 | 0.950     |
| AllRel@10 | 0.904     |

For this benchmark, each query uses the LongMemEval-provided haystack as its
search scope. That makes the result reproducible and easy to inspect query by
query, while still exercising retrieval over a large generated memory vault.

[Result JSON](eval/results/longmemeval-s-no-rerank.json) · [Reproduce and interpret →](fixtures/longmemeval-s/README.md)

## Features

- **Hybrid search**
  - BM25 + fuzzy title + semantic embeddings, fused with RRF
- **Alias search**
  - notes with `aliases:` in frontmatter are indexed and searchable by any alias; alias matches are boosted in BM25 (weight 5×) and fuzzy title scoring
- **Four search modes**
  - `hybrid`, `semantic`, `fulltext`, `title` (for text queries)
- **Similar note lookup**
  - pass `--path` to find semantically related notes using stored chunk embeddings, with a title + content fallback
- **Graph traversal**
  - `--path --related` shows linked notes at configurable depth; filter by `--direction outgoing|backlinks|both`
- **Links & backlinks**
  - every result includes outgoing links and backlinks
- **Scope filtering**
  - restrict to subfolder(s); supports multiple values and exclusions (`-notes/dev/`)
- **Tag filtering**
  - filter by tag(s); supports multiple values and exclusions (`-category/cs`)
- **Snippet control**
  - `--snippet-length` sets the context window; empty snippets always fall back to note content
- **Extended output**
  - `--extended` adds a TAGS/ALIASES column to the CLI table showing frontmatter tags (`#tag`) and aliases
- **Incremental indexing**
  - only re-indexes changed files; watches for edits in real time
- **Multi-query fan-out**
  - pass multiple queries at once (`ohs "q1" "q2"` or `queries[]` in MCP); results are merged via RRF — a note that ranks well in any one query floats to the top; useful when the note may use different vocabulary than the query
- **Cross-encoder reranking**
  - `--rerank` re-scores results with `bge-reranker-v2-m3` (ONNX int8, ~570 MB download once); improves precision for conceptual and multilingual queries; applied after multi-query merge
- **Local embeddings**
  - works offline via `@huggingface/transformers` (no API key required); default model: Xenova/multilingual-e5-small, 100+ languages
- **Remote embeddings**
  - OpenAI-compatible API (OpenRouter, Ollama, etc.)
- **Note reading**
  - `read` fetches one or more notes by vault-relative path; returns full content with title, aliases, tags, links, and backlinks; on path miss returns top-3 fuzzy suggestions
- **Ignore patterns**
  - exclude folders, extensions, or specific files
- **Obsidian plugin**
  - native search modal inside Obsidian powered by the same CLI — see [obsidian-hybrid-search-plugin](https://github.com/flowing-abyss/obsidian-hybrid-search-plugin)

## Installation

```bash
npm install -g obsidian-hybrid-search
# or run directly without installing:
npx obsidian-hybrid-search
```

## CLI usage

### Quick start

**Option A — recommended: set `OBSIDIAN_VAULT_PATH` once in your shell profile.**

This lets you run the tool from any directory. Add to `~/.zshrc` or `~/.bashrc`:

```bash
export OBSIDIAN_VAULT_PATH="/path/to/your/vault"
```

Then reload (`source ~/.zshrc`) and index your vault once:

```bash
obsidian-hybrid-search reindex
```

After that you can search from any directory:

```bash
obsidian-hybrid-search "zettelkasten"
```

---

**Option B — no env var: run from inside your vault.**

The tool detects the vault root by looking for the `.obsidian/` folder, walking up from the current directory. `cd` into your vault (or any subfolder) and run:

```bash
cd /path/to/your/vault
obsidian-hybrid-search reindex   # detects vault root, creates DB, indexes everything
obsidian-hybrid-search "zettelkasten"
```

Commands work from any directory inside the vault tree. From outside the vault (e.g. via shell aliases called from `~`), use Option A or pass `--db /path/to/vault/.obsidian-hybrid-search.db` explicitly.

---

**Optional: remote embedding API instead of local model.**

By default the local `Xenova/multilingual-e5-small` model is used — works offline, no API key needed. Downloads ~117 MB on first run. Supports 100+ languages including Russian, Chinese, Japanese, and more.

To use a remote API instead, add to your shell profile:

```bash
export OPENAI_API_KEY="sk-..."

# Default API base is https://api.openai.com/v1 — override for other providers:
# export OPENAI_BASE_URL="https://openrouter.ai/api/v1"  # OpenRouter
# export OPENAI_BASE_URL="http://localhost:11434/v1"     # Ollama (no key needed)
# export OPENAI_BASE_URL="http://localhost:1234/v1"      # LM Studio (no key needed)

# Optional: override the embedding model (default: text-embedding-3-small)
# export OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
```

### Search modes

| Scenario        | How                                                                 | Modes                                               |
| --------------- | ------------------------------------------------------------------- | --------------------------------------------------- |
| Text query      | `obsidian-hybrid-search "some topic"`                               | `hybrid` (default), `semantic`, `fulltext`, `title` |
| Similar notes   | `obsidian-hybrid-search --path notes/pkm/zettelkasten.md`           | Semantic similarity from stored chunk embeddings    |
| Graph traversal | `obsidian-hybrid-search --path notes/pkm/zettelkasten.md --related` | Links & backlinks via BFS                           |

`--mode` only affects text queries. When `--path` is given without `--related`, search uses semantic similarity regardless of `--mode`; `--path --related` traverses links/backlinks instead.