pulse8-ai-cortex-knowledge-vault

<!-- mcp-name: io.github.synpulse8-opensource/pulse8-ai-cortex-knowledge-vault -->

<p align="center">
  <img src="assets/pulse8-banner.png" alt="PULSE8.ai" width="600" />
</p>

<h1 align="center">PULSE8.ai Cortex</h1>

<p align="center">
  <strong>Agent-native knowledge OS built on Markdown</strong>
</p>

<p align="center">
  <a href="https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault/actions/workflows/pylint.yml"><img src="https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault/actions/workflows/pylint.yml/badge.svg" alt="Build"></a>
  <a href="https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault/releases/latest"><img src="https://img.shields.io/github/v/release/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault" alt="Release"></a>
  <a href="LICENSE.md"><img src="https://img.shields.io/badge/license-Apache%202.0-blue" alt="License"></a>
</p>

<p align="center">
  <img src="https://img.shields.io/badge/python-3.12+-3776AB?logo=python&logoColor=white" alt="Python">
  <img src="https://img.shields.io/badge/FastAPI-009688?logo=fastapi&logoColor=white" alt="FastAPI">
  <img src="https://img.shields.io/badge/MCP-Model%20Context%20Protocol-blueviolet" alt="MCP">
  <img src="https://img.shields.io/badge/Docker-2496ED?logo=docker&logoColor=white" alt="Docker">
  <img src="https://img.shields.io/badge/NetworkX-graph%20engine-orange" alt="NetworkX">
</p>


PULSE8.ai Cortex is an agent-native knowledge OS built on Markdown. It gives AI agents and humans a shared vault backed by a typed knowledge graph, full-text search, and a [MarkItDown](https://github.com/microsoft/markitdown)-powered compiler — all accessible through a unified [MCP](https://modelcontextprotocol.io/) interface.

Drop files in (PDF, DOCX, PPTX, XLSX, HTML, images, and more), let agents read, write, search, link, and compile knowledge — no database required.

> Inspired by [Andrej Karpathy](https://github.com/karpathy)'s [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern — a persistent, compounding knowledge base maintained by LLMs instead of re-derived on every query. Search powered by [Tobi Lütke](https://github.com/tobi)'s [QMD](https://github.com/tobi/qmd).

---

## Get started

> [!NOTE]
> PULSE8.ai Cortex requires Docker. An [OpenRouter API key](https://openrouter.ai/keys) is optional — needed only for LLM-powered cross-referencing between wiki articles. File conversion works out of the box without any API key.

1. Clone the repository:
  ```bash
    git clone https://github.com/synpulse8-opensource/pulse8-ai-cortex-knowledge-vault.git
    cd cortex-knowledge-vault
  ```
2. Launch PULSE8.ai Cortex:
  ```bash
    ./scripts/start.sh
  ```
    This builds and starts both **PULSE8.ai Cortex** (API + MCP on `:8420`) and **QMD** (search on `:3100`), waits for health checks, and you're ready to go.
3. Connect your MCP client (e.g. Claude Desktop) to `http://localhost:8420/mcp/`.

To stop: `./scripts/stop.sh`

### Cortex-only mode (macOS / native QMD)

If you want QMD to run natively (e.g. on macOS with Metal GPU acceleration), start only the Cortex container:

```bash
# Terminal 1: Run QMD natively
npm install -g @tobilu/qmd
VAULT_PATH=./example_vault node docker/qmd/server.mjs

# Terminal 2: Start only Cortex in Docker
./scripts/start.sh --cortex-only
```

To stop: `./scripts/stop.sh --cortex-only`

### GPU-accelerated QMD (EC2 / Linux with NVIDIA GPU)

For production deployments with NVIDIA GPU acceleration:

```bash
docker compose -f docker-compose.yml -f docker-compose.gpu.yml up --build -d
```

See [docs/ec2-gpu-setup.md](docs/ec2-gpu-setup.md) for a full guide on instance selection, NVIDIA toolkit installation, and cost estimates.



## Features


|                              |                                                                                                                                                                              |
| ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Knowledge Graph**          | Typed graph engine (NetworkX) — wikilinks, tags, and custom edges, auto-maintained on every file change                                                                      |
| **Full-Text Search**         | QMD search with hybrid (BM25 + vector + re-ranking) by default; keyword and semantic modes selectable. Results cached with a configurable TTL.                               |
| **File Compiler**            | Converts raw sources (PDF, DOCX, PPTX, XLSX, HTML, images, etc.) to Markdown via [MarkItDown](https://github.com/microsoft/markitdown). LLM used only for cross-referencing. |
| **MCP Server**               | Streamable HTTP + stdio transport — works with Claude Desktop, Cursor, and any MCP client                                                                                    |
| **Feedback & Notifications** | `vault_feedback` captures quality feedback as notes; optional Microsoft Teams webhook posts an adaptive card per submission                                                  |
| **Daily Activity Log**       | Every write/ingest/compile is mirrored into `daily/<date>.md` as a greppable, wikilinked timeline                                                                            |
| **Bulk Ingest**              | Ingest dozens or hundreds of files at once from a local directory with SHA-256 dedup and bounded concurrency                                                                 |
| **REST API**                 | FastAPI endpoints mirroring all MCP tools at `/api/v1/`, including multipart file upload and bulk ingest                                                                     |
| **Vault Watcher**            | Real-time filesystem monitoring — graph stays in sync automatically                                                                                                          |
| **Zero Database**            | Everything persists as Markdown + JSON on your filesystem                                                                                                                    |




## MCP tools


| Tool                   | Description                                                                                |
| ---------------------- | ------------------------------------------------------------------------------------------ |
| `vault_read`           | Read a note by path                                                                        |
| `vault_write`          | Create or update a note                                                                    |
| `vault_search`         | Search the vault (keyword / semantic / hybrid)                                             |
| `vault_link`           | Create, query, or delete graph edges                                                       |
| `vault_context`        | Build a context window: search → graph traversal → ranked subgraph                         |
| `vault_ingest`         | Ingest raw content or binary files (supports `content_base64` for binary)                  |
| `vault_compile`        | Compile unprocessed raw sources into wiki Markdown via MarkItDown                          |
| `vault_feedback`       | Submit feedback on vault quality (`status: OPEN`; optional `related_paths` of `.md` notes) |
| `vault_list_feedbacks` | List feedback note metadata (paths, tags, status; not full body)                           |




## Architecture

```
┌──────────────────────────────────────────────┐
│  MCP Client (Claude Desktop, Cursor, etc.)   │
└──────────┬───────────────────────────────────┘
           │  MCP (HTTP or stdio)
┌──────────▼───────────────────────────────────┐
│  PULSE8.ai Cortex  :8420                     │
│  ┌──────────────────────────────────────┐     │
│  │ Auth (API Key or Microsoft Entra ID) │     │
│  └──────────────┬───────────────────────┘     │
│  ┌─────────┐ ┌──┴───────┐ ┌──────────────┐   │
│  │ MCP     │ │ REST API │ │ Vault Watcher│   │
│  │ /mcp/   │ │ /api/v1/ │ │ (watchfiles) │   │
│  └────┬────┘ └────┬─────┘ └──────┬───────┘   │
│       └───────────┼──────────────┘           │
│            ┌──────▼──────┐                   │
│            │ Graph Engine│                   │
│            │ + Compiler  │                   │
│            └─────────────┘                   │
└──────────┬───────────────────────────────────┘
           │
┌──────────▼───────────────────────────────────┐
│  QMD  :3100                                  │
│  BM25 + vector search, auto-indexes on start │
└──────────┬───────────────────────────────────┘
           │
┌──────────▼───────────────────────────────────┐
│  Vault (bind-mounted volume)                 │
│  wiki/ raw/ agents/ sessions/ daily/ feedback/ │
│  .cortex/ (graph.json, index.md, log.md)     │
└──────────────────────────────────────────────┘
```



## Vault layout

The vault is a plain directory of Markdown files organised by purpose. Cortex classifies each file into a typed node (`NodeType`) used by the graph engine and exposed in REST and MCP responses.


| Folder      | `NodeType`   | Purpose                                                          |
| ----------- | ------------ | ---------------------------------------------------------------- |
| `wiki/`     | `note`       | Compiled, interlinked knowledge articles                         |
| `raw/`      | `raw_source` | Unprocessed sources (PDF, DOCX, TXT, …) the compiler reads from  |
| `agents/`   | `agent_def`  | Agent definitions                                                |
| `sessions/` | `session`    | Per-session notes / conversation transcripts                     |
| `daily/`    | `daily`      | Daily notes (Obsidian Daily Notes convention)                    |
| `feedback/` | `feedback`   | Feedback on vault quality (`status`, `re