datris-platform-oss

# Datris — The First AI Agent-Native Data Platform

[![PyPI](https://img.shields.io/pypi/v/datris-mcp-server)](https://pypi.org/project/datris-mcp-server/)
[![MCP Registry](https://img.shields.io/badge/MCP_Registry-io.github.datris%2Fdatris-blue)](https://registry.modelcontextprotocol.io/servers/io.github.datris/datris)
[![Docker Hub](https://img.shields.io/docker/v/datrisai/datris-server?label=Docker%20Hub)](https://hub.docker.com/u/datrisai)
[![License](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)

[datris.ai](https://datris.ai) · [Documentation](https://docs.datris.ai) · [MCP Registry](https://registry.modelcontextprotocol.io/servers/io.github.datris/datris) · [PyPI](https://pypi.org/project/datris-mcp-server/)

Ingest, validate, transform, store, and retrieve your data — whether you're an AI agent talking through MCP or a developer writing config. One platform for both.

## Why Datris?

- **Agent-native** — Built-in MCP server with 47 tools. Claude, Cursor, and any MCP-compatible agent can operate pipelines through natural conversation
- **Taps** — AI-generated Python scripts that fetch data from external sources (APIs, web scraping, databases) and push it into pipelines. Describe what you want, Datris generates the script. Includes AI diagnosis, CRON scheduling, and credentials via Vault
- **AI at every stage** — AI data quality, AI transformations, AI schema generation, AI profiling, AI error explanation, natural language queries, RAG
- **No vendor lock-in** — 100% open-source infrastructure (MinIO, PostgreSQL, MongoDB, Kafka, Vault). Runs anywhere Docker does
- **Configuration-driven** — Define pipelines through JSON. No code required

## Quick Start

You only need Docker. This pulls pre-built images and runtime files, seeds a
`.env`, and starts the stack into `./datris` — no git checkout required:

```bash
curl -fsSL https://get.datris.ai/install.sh | sh
```

> The `install.sh` installer is a POSIX shell script (macOS/Linux). On **Windows**, run it from **WSL2** or **Git Bash**, or use the single-file Compose option below, which works natively in PowerShell.

<details>
<summary>Single file, no installer (works on Windows)</summary>

A fully self-contained Compose file — the init scripts and config are inlined,
so nothing else is needed (requires Docker Compose ≥ 2.23):

```bash
# macOS / Linux
curl -O https://get.datris.ai/docker-compose.standalone.yml
ANTHROPIC_API_KEY=sk-ant-... docker compose -f docker-compose.standalone.yml up -d
```

```powershell
# Windows (PowerShell) — use curl.exe, and set the key with $env:
curl.exe -O https://get.datris.ai/docker-compose.standalone.yml
$env:ANTHROPIC_API_KEY="sk-ant-..."
docker compose -f docker-compose.standalone.yml up -d
```

</details>

<details>
<summary>From a git clone</summary>

```bash
git clone https://github.com/datris/datris-platform-oss.git
cd datris-platform-oss
cp .env.example .env       # Add your ANTHROPIC_API_KEY and/or OPENAI_API_KEY
docker compose up -d
```

</details>

**UI**: [http://localhost:4200](http://localhost:4200) · **API**: [http://localhost:8080](http://localhost:8080)

### Connect an AI Agent

Add to your MCP client config (Claude Desktop, Claude Code, Cursor, etc.). With the Docker stack running, the `npx mcp-remote` stdio bridge connects to the bundled MCP server on port 3000 — your client appears in the Datris UI **Agent Monitor** tab with live tool-call streaming:

```json
{
  "mcpServers": {
    "datris": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "http://localhost:3000/sse", "--transport", "sse-only"]
    }
  }
}
```

Paste-and-go for the default local setup — no API key required when `USE_API_KEYS=false` (the OSS default). If your instance enables auth (`USE_API_KEYS=true` or hosted/multi-tenant), append `"--header", "x-api-key:<your-key>"` to the `args` array. The Configuration → Connect Your Agent page generates the snippet for you and adds the header automatically when you paste your key.

Requires Node.js on your `PATH` (`brew install node`). For a stdio alternative without Docker, or full Claude Desktop / Claude Code / Cursor walkthroughs, see [Configuring Claude](https://docs.datris.ai/configuring-claude).

### CLI

```bash
brew tap datris/tap
brew install datris
datris ingest data.csv --dest postgres
datris ingest sales.csv --ai-validate "prices > 0" --ai-transform "convert dates to YYYY/MM/DD"
datris query "SELECT * FROM sales"
datris search "quarterly revenue" --store pgvector
datris tap create "Fetch S&P 500 daily prices from yfinance" --pipeline stocks
datris taps
```

## What It Does

```
Source (File Upload / MinIO Event / Database Pull / Kafka)
  → Preprocessor (optional REST endpoint)
  → Data Quality (AI rules, header validation, schema validation)
  → Transformation (AI transformation, destination schema)
  → Destinations (in parallel):
      PostgreSQL, MongoDB, MinIO (Parquet/ORC), Kafka, ActiveMQ,
      REST Endpoint, Qdrant, Weaviate, Milvus, Chroma, pgvector
  → Notifications (ActiveMQ topic)
```

### AI-Powered Features

| Feature | Description |
|---------|-------------|
| **MCP Server** | 47 tools for AI agents — pipeline CRUD, upload, query, search, profiling, taps |
| **AI Data Quality** | Plain English validation rules — AI generates and runs a validation script |
| **AI Transformation** | Plain English transformations — AI generates and runs a transformation script |
| **AI Schema Generation** | Upload a file, get a complete pipeline config |
| **AI Data Profiling** | Upload a file, get statistics + suggested validation rules |
| **AI Error Explanation** | Job failures explained in plain English |
| **Natural Language Query** | Ask questions in English, get SQL results |
| **RAG Pipeline** | Chunk, embed, and search across 5 vector databases |

### Supported Formats

CSV, JSON, XML, Excel, PDF, Word (DOCX), plain text

### AI Providers

Anthropic Claude (Sonnet 4.6 default, Opus 4.8 for CodeGen) · OpenAI (GPT-5.5) · Ollama (local models, optional). Embeddings via TEI sidecar (BAAI/bge-m3) when using Anthropic, or `text-embedding-3-small` when using OpenAI.

## Architecture

| Service | Purpose |
|---------|---------|
| **MinIO** | S3-compatible object store for file staging and data output |
| **PostgreSQL** | Default structured destination, also hosts pgvector for RAG |
| **MongoDB** | Configuration store, job status tracking, metadata |
| **ActiveMQ** | File notification queue, pipeline event notifications |
| **HashiCorp Vault** | Secrets management (database credentials, API keys) |
| **TEI** | Text Embeddings Inference sidecar (BAAI/bge-m3) for vector embeddings without an OpenAI key |
| **Apache Kafka** | Optional streaming source and destination |
| **Apache Spark** | Local Spark for writing Parquet/ORC to MinIO |

## Documentation

Full documentation at [docs.datris.ai](https://docs.datris.ai) or locally at `docs/`.

## License

[AGPL-3.0](LICENSE)