octobrain

# Octobrain

> Persistent memory for AI assistants — store insights, decisions, and knowledge that survives across conversations.

[![Crates.io](https://img.shields.io/crates/v/octobrain.svg)](https://crates.io/crates/octobrain)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-1.95%2B-orange.svg)](https://www.rust-lang.org/)

_MCP Registry: `mcp-name: io.github.Muvon/octobrain`_

**Octobrain** gives your AI assistant a long-term memory. Store code insights, architecture decisions, bug fixes, and knowledge — then retrieve them with semantic search in future sessions. Works as a CLI tool or as an MCP server for integration with Claude Desktop and other AI tools.

## Why Octobrain?

AI assistants start every conversation with zero context. You explain your project, your preferences, your decisions — every single time. Octobrain breaks that cycle:

- **Persistent memory** — Insights survive across sessions, not just within them
- **Semantic search** — Find memories by meaning, not exact keywords
- **Auto-linking** — Related memories connect automatically (Zettelkasten-style)
- **Knowledge indexing** — Ingest docs, articles, and files for retrieval
- **MCP integration** — Works with Claude Desktop and other MCP-compatible tools

## Quick Start

```bash
# Install from crates.io
cargo install octobrain

# Store your first memory
octobrain memory memorize --title "API Design Pattern" \
  --content "Use REST for CRUD, GraphQL for complex queries" \
  --memory-type architecture --tags "api,design"

# Search memories
octobrain memory remember "how should I design APIs"

# Start MCP server for Claude Desktop integration
octobrain mcp
```

## Installation

### From crates.io (Recommended)

```bash
cargo install octobrain
```

### From Source

```bash
# Clone and build
git clone https://github.com/muvon/octobrain.git
cd octobrain
cargo build --release

# Binary location
./target/release/octobrain --help
```

### Feature Flags

Octobrain supports multiple embedding providers:

| Flag | Description | API Key Required |
|------|-------------|------------------|
| `fastembed` | Local embeddings via FastEmbed | No |
| `huggingface` | Local embeddings via HuggingFace | No |
| (default) | Both `fastembed` + `huggingface` | No |
| (no features) | API-based: Voyage, OpenAI, Google, Jina | Yes |

```bash
# Build with local embeddings (default, no API keys needed)
cargo build --release

# Build with API-based embeddings only
cargo build --no-default-features --release
```

For API-based embeddings, set the appropriate environment variable:
- `VOYAGE_API_KEY` for Voyage AI
- `OPENAI_API_KEY` for OpenAI
- `GOOGLE_API_KEY` for Google
- `JINA_API_KEY` for Jina

## Usage

### Memory Management

Store and retrieve insights, decisions, and context:

```bash
# Store a memory
octobrain memory memorize --title "API Design" \
  --content "Use REST for CRUD, GraphQL for complex queries" \
  --memory-type architecture --tags "api,design"

# Search memories (semantic search)
octobrain memory remember "api design patterns"

# Multi-query search for broader coverage
octobrain memory remember "authentication" "security" "jwt"

# Get a memory by ID
octobrain memory get <id>

# Get recent memories
octobrain memory recent --limit 20

# Filter by type
octobrain memory by-type architecture --limit 10

# Filter by tags
octobrain memory by-tags "api,security"

# Find memories related to files
octobrain memory for-files "src/main.rs,src/lib.rs"

# Update a memory
octobrain memory update <id> --title "New Title" --add-tags "new-tag"

# Delete a memory
octobrain memory forget --memory-id <id>
```

### Memory Consolidation

Close a goal and fold all its contributing memories into a consolidated summary:

```bash
# Consolidate a goal (all Achieves-link sources get archived)
octobrain memory consolidate <goal-id> --summary "Final summary"

# Sleep consolidation: auto-cluster recent similar memories
octobrain memory sleep-consolidate --threshold 0.85 --min-size 3
```

### Memory Relationships

Connect related memories for context-rich retrieval:

```bash
# Create a relationship between memories
octobrain memory relate <source-id> <target-id> \
  --relationship-type "depends_on" \
  --description "Source requires target to function"

# View relationships for a memory
octobrain memory relationships <memory-id>

# Find related memories through relationships
octobrain memory related <memory-id>

# Auto-link similar memories (Zettelkasten-style)
octobrain memory auto-link <memory-id>

# Explore memory graph
octobrain memory graph <memory-id> --depth 2
```

### Knowledge Base

Index and search web content, docs, and files:

```bash
# Index a URL
octobrain knowledge index https://docs.rs/tokio/latest/tokio/

# Search knowledge base
octobrain knowledge search "how to handle async tasks"

# Search within a specific source (auto-indexes if outdated)
octobrain knowledge search "spawn blocking" --source https://docs.rs/tokio/

# Read full content of a URL or local file
octobrain knowledge read https://docs.rs/tokio/latest/tokio/

# Search indexed content by regex pattern
octobrain knowledge match "spawn_blocking|block_in_place"

# Store raw text content
octobrain knowledge store "meeting-notes" --content "Discussion points..."

# List indexed sources
octobrain knowledge list --limit 20

# Show statistics
octobrain knowledge stats

# Delete a source
octobrain knowledge delete https://example.com/docs

# Delete stored content by key
octobrain knowledge delete-stored "meeting-notes"
```

### MCP Server

Run as an MCP server for integration with Claude Desktop and other AI tools:

```bash
# Start with stdio transport (for Claude Desktop)
octobrain mcp

# Start with HTTP transport (for web-based tools)
octobrain mcp --bind 0.0.0.0:12345
```

**Available MCP Tools:**

| `memorize` | Store memories with metadata; optional `related_to` for inline relationships |
| `remember` | Semantic search with filters; returns 1-hop graph neighbors |
| `forget` | Delete memories (requires confirmation) |
| `knowledge` | Unified tool: `search`, `store`, `delete`, `read`, `match` via `command` field |
See [MCP Integration](#mcp-integration) for Claude Desktop setup.

## Features

- **Semantic Search** — Find memories by meaning using vector embeddings, not exact keyword matches
- **Hybrid Search** — Combines BM25 full-text search with vector similarity for better results
- **Reranking Support** — Optional cross-encoder reranking for 20-35% accuracy improvement
- **Auto-Linking** — Automatically connects semantically similar memories (Zettelkasten-style)
- **Temporal Decay** — Ebbinghaus forgetting curve for importance management
- **Knowledge Indexing** — Ingest URLs, PDFs, docs for retrieval
- **Project Scoping** — Isolate memories per Git project or share across projects
- **Role Filtering** — Tag memories by role (developer, reviewer, etc.)
- **Query Expansion (HyDE-lite)** — Pseudo-relevance feedback for +10-30% recall on long-tail queries
- **MCP Protocol** — Full MCP 2025-03-26 compliance for AI tool integration

## Benchmarks

Retrieval quality of octobrain's knowledge system on standard [BEIR](https://github.com/beir-cellar/beir) datasets — **nDCG@10**, fully local, no LLM judge, using the default local embedder `bge-small-en-v1.5` (384-dim, 33M params). Each corpus passage is indexed through octobrain's real retrieval path and scored against the official qrels (metrics reproduce `pytrec_eval`).

| Dataset | octobrain vector | octobrain hybrid | BM25¹ | bge-small-en-v1.5² |
|---|---|---|---|---|
| **SciFact** (5.2K docs, 300 q) | 0.722 | **0.742** | 0.665 | 0.713 |
| **NFCorpus** (3.6K docs, 323 q) | 0.341 | **0.363** | 0.325 | 0.343 |

- **vector** = dense-only retrieval; reproduces the embedder's published BEIR numbers (validates the harness).
- **hybrid** = BM25 + vector fused with Reciprocal Rank Fusion (k=60) — octobrain's default. Adds **+2 nDCG@10** over the bare embedding and beats classic BM25 on both datasets.

¹ Canonical BM25 from the [BEIR paper](https://arxiv.org/abs/2104.08663) (Anserini/Lucene, k1=0.9 b=0.4).
² From the [bge-small-en-v1.5 model card](https://huggingface.co/BAAI/bge-small-en-v1.5) (MTEB).

> Scope: this measures the **ranking** layer (embedding + BM25 fusion + reranking). BEIR passages are pre-chunked, so octobrain's chunking strategy is not exercised here.

Reproduce (downloads the datasets, builds a release binary, runs fully offline):

```bash
cd benches && bash scripts/run_retrieval.sh
```

## Configuration

Configuration is stored in `~/.local/share/octobrain/config.toml`. All options have sensible defaults.

### Key Settings

| Section | Option | Default | Description |
|---------|--------|---------|-------------|
| `[embedding]` | `model` | `fastembed:nomic-ai/nomic-embed-text-v1.5` | Embedding model (provider:model format). Default is a local fastembed model — no API key, runs on CPU. |
| `[search]` | `similarity_threshold` | `0.3` | Minimum relevance (0.0-1.0) |
| `[search.hybrid]` | `enabled` | `true` | Enable BM25 + vector fusion |
| `[search.reranker]` | `enabled` | `true` | Enable cross-encoder reranking |
| `[search.hyde]` | `enabled` | `true` | Pseudo-relevance feedback query expansion |
| `[memory]` | `max_memories` | `10000` | Maximum stored memories |
| `[memory]` | `auto_linking_enabled` | `true` | Auto-connect similar memories |
| `[knowledge]` | `chunk_size` | `1200` | Characters per chunk |

### Embedding Providers

```toml
[embedding]
# Local models (no API key, runs on CPU, model auto-downloaded on first use)
model = "fastembed:nomic-ai/nomic-embed-text-v1.5"                      # Default: 768-dim, 8192-token context
model = "fastembed:BAAI/bge-small-en-v1.5"                              # 384-dim, ~62 MTEB, fast + good quality
model = "fastembed:sentence-transformers/all-MiniLM-L6-v2-quantized"  # Smallest (~22MB), fastest
model = "fastembed:BAAI/bge-base-en-v