pulse8-ai-cortex-knowledge-vault

<!-- mcp-name: io.github.synpulse8-opensource/pulse8-ai-cortex-knowledge-vault -->

<p align="center">
  <img src="assets/pulse8-banner.png" alt="PULSE8.ai" width="600" />
</p>

<h1 align="center">PULSE8.ai Cortex</h1>

<p align="center">
  <strong>The open-source knowledge layer for AI agents</strong>
  <br />
  <em>Knowledge that compounds.</em>
</p>

<p align="center">
  <a href="https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault/actions/workflows/pylint.yml"><img src="https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault/actions/workflows/pylint.yml/badge.svg" alt="Build"></a>
  <a href="https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault/releases/latest"><img src="https://img.shields.io/github/v/release/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault" alt="Release"></a>
  <a href="LICENSE.md"><img src="https://img.shields.io/badge/license-Apache%202.0-blue" alt="License"></a>
</p>

<p align="center">
  <img src="https://img.shields.io/badge/python-3.12+-3776AB?logo=python&logoColor=white" alt="Python">
  <img src="https://img.shields.io/badge/FastAPI-009688?logo=fastapi&logoColor=white" alt="FastAPI">
  <img src="https://img.shields.io/badge/MCP-Model%20Context%20Protocol-blueviolet" alt="MCP">
  <img src="https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=white" alt="Docker">
  <img src="https://img.shields.io/badge/NetworkX-graph%20engine-orange" alt="NetworkX">
</p>


PULSE8.ai Cortex is the **open-source knowledge layer for AI agents**: Git-native memory, a typed knowledge graph, and [MCP](https://modelcontextprotocol.io/)-powered retrieval on top of plain Markdown — so agents can build, evolve, and reuse persistent knowledge instead of re-deriving it on every query.

Under the hood it's a unified vault for AI agents and humans, backed by a typed knowledge graph, full-text + hybrid search, and a [MarkItDown](https://github.com/microsoft/markitdown)-powered file compiler. Drop files in (PDF, DOCX, PPTX, XLSX, HTML, images, and more), let agents read, write, search, link, and compile knowledge — no database required.

> Inspired by [Andrej Karpathy](https://github.com/karpathy)'s [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern — a persistent, compounding knowledge base maintained by LLMs instead of re-derived on every query. Search powered by [Tobi Lütke](https://github.com/tobi)'s [QMD](https://github.com/tobi/qmd).

---

## Why PULSE8.ai Cortex?

Most AI agents can access tools, but they cannot accumulate knowledge.

Traditional RAG systems retrieve documents. PULSE8.ai Cortex builds a **persistent, evolving knowledge layer** that grows over time and becomes more valuable the more agents and humans interact with it.

With PULSE8.ai Cortex, agents can:

- **Learn from interactions** — every read, write, ingest, and compile event is logged and graph-linked
- **Build and traverse knowledge graphs** — wikilinks, tags, and typed edges, maintained automatically
- **Store structured insights** — Markdown notes with typed nodes (`note`, `agent_def`, `session`, `daily`, `feedback`)
- **Retrieve context across projects** — `vault_context` builds a ranked subgraph from any seed query
- **Share knowledge through MCP** — one vault, every MCP-compatible client
- **Maintain long-term memory** — files survive sessions, deployments, and model upgrades
- **Version knowledge through Git** — the vault is a plain directory of Markdown, diff-friendly out of the box

| Aspect | Traditional RAG | PULSE8.ai Cortex |
| ------ | --------------- | ---------------- |
| Focus | Documents | Knowledge |
| Memory | Session-based | Persistent |
| Structure | Chunks | Markdown + typed graph |
| Evolution | Static index | Continuous, file-watched |
| Versioning | None | Git-native |
| Agent collaboration | Limited | First-class (MCP) |



## When to use PULSE8.ai Cortex

### Ideal use cases

- ✅ Persistent memory for AI agents
- ✅ Shared knowledge across multiple agents
- ✅ Git-versioned organisational knowledge
- ✅ MCP-compatible knowledge retrieval
- ✅ Knowledge graphs without a dedicated graph database
- ✅ Long-term accumulation of institutional knowledge
- ✅ Human + AI collaborative knowledge management

### Not ideal for

- ❌ Simple full-text document search (use a search engine)
- ❌ Pure vector-only retrieval with no graph (use a vector DB)
- ❌ Short-lived, stateless conversations
- ❌ Workflows that don't need persistent knowledge evolution



## PULSE8.ai Cortex vs alternatives


| Capability                     | PULSE8.ai Cortex | Traditional RAG | GraphRAG |
| ------------------------------ | :--------------: | :-------------: | :------: |
| Persistent knowledge           |        ✅        |       ❌        |    ⚠️    |
| Markdown-native storage        |        ✅        |       ❌        |    ❌    |
| MCP-compatible out of the box  |        ✅        |       ❌        |    ❌    |
| Knowledge graph                |        ✅        |       ❌        |    ✅    |
| Git versioning                 |        ✅        |       ❌        |    ❌    |
| Agent memory layer             |        ✅        |       ❌        |    ⚠️    |
| Human + AI collaboration       |        ✅        |       ❌        |    ⚠️    |
| Continuous knowledge evolution |        ✅        |       ❌        |    ⚠️    |
| Zero database required         |        ✅        |       ❌        |    ❌    |



## Works with

PULSE8.ai Cortex speaks [MCP](https://modelcontextprotocol.io/) — so it plugs into any AI client that does. The same vault is reachable over streamable HTTP or stdio, and mirrored 1:1 by a REST API at `/api/v1/`.


| Category              | Compatible with                                                                |
| --------------------- | ------------------------------------------------------------------------------ |
| **AI agents**         | Claude Desktop, Claude Code, OpenAI Agents, Gemini, custom agent frameworks    |
| **Development tools** | Cursor, VS Code, JetBrains IDEs                                                |
| **Agent frameworks**  | LangGraph, LangChain, CrewAI, AutoGen                                          |
| **MCP ecosystem**     | MCP clients, MCP servers, MCP tool registries                                  |
| **Human tools**       | Obsidian, any Markdown editor, any Git client                                  |


Because the vault is just files, humans and agents collaborate on the same knowledge — no proprietary format, no lock-in.



## Get started

> [!NOTE]
> PULSE8.ai Cortex requires Docker. An [OpenRouter API key](https://openrouter.ai/keys) is optional — needed only for LLM-powered cross-referencing between wiki articles. File conversion works out of the box without any API key.

1. Clone the repository:
  ```bash
    git clone https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault.git
    cd cortex-knowledge-vault
  ```
2. Launch PULSE8.ai Cortex:
  ```bash
    ./scripts/start.sh
  ```
    This builds and starts both **PULSE8.ai Cortex** (API + MCP on `:8420`) and **QMD** (search on `:3100`), waits for health checks, and you're ready to go.
3. Connect your MCP client (e.g. Claude Desktop) to `http://localhost:8420/mcp/`.

To stop: `./scripts/stop.sh`

### Native QMD mode (macOS / Metal GPU)

Docker Desktop on macOS cannot expose the Metal GPU to containers, so containerized QMD embeds on CPU only — over an order of magnitude slower on non-trivial vaults. Run QMD natively instead; the `qmd` binary uses Metal automatically:

```bash
# One-time: install the qmd binary
brew install tobi/tap/qmd   # or: npm install -g @tobilu/qmd

# Start native QMD (background daemon) + Cortex in Docker
./scripts/start.sh --native-qmd
```

The QMD daemon's pid and log are kept in `.qmd-native.pid` / `.qmd-native.log`. To stop both: `./scripts/stop.sh --native-qmd` (a plain `./scripts/stop.sh` also cleans up a native QMD if one is running).

### Cortex-only mode (external QMD)

If you manage QMD yourself (already running elsewhere), start only the Cortex container:

```bash
./scripts/start.sh --cortex-only   # set QMD_URL in .env if not http://host.docker.internal:3100
```

To stop: `./scripts/stop.sh --cortex-only`

### GPU-accelerated QMD (EC2 / Linux with NVIDIA GPU)

For production deployments with NVIDIA GPU acceleration:

```bash
docker compose -f docker-compose.yml -f docker-compose.gpu.yml up --build -d
```

See [docs/ec2-gpu-setup.md](docs/ec2-gpu-setup.md) for a full guide on instance selection, NVIDIA toolkit installation, and cost estimates.



## Features


|                              |                                                                                                                                                                              |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Knowledge Graph**          | Typed graph engine (NetworkX) — wikilinks, tags, and custom edges, auto-maintained on every file change                                                                      |
| **Full-Text Search**         | QMD search with hybrid (BM25 + vector + re-ranking) by default; keyword and semantic modes selectable. Results cached with a configurable TTL.                               |
| **File Compiler**            | Converts raw sources (PDF, DOCX, PPTX, XLSX, HTML, images, etc.) to Markdown via [MarkItDown](https://github.com/microsoft/markitdown). LLM used only for cross-referencing. |
| **MCP Server**               | Streamable HTTP + stdio transport — works with Claude Desktop, Cursor, and any MCP client                                                                                    |
| **Feedback & Notifications** | `vault_feedback` captures quality feedback as notes; optional Microsoft Teams