data-aggregator-mcp

# 🔎 data-aggregator-mcp

**One MCP server to find and fetch research data across archives, omics
registries, and literature — behind a single normalized model.**

[![PyPI](https://img.shields.io/pypi/v/data-aggregator-mcp.svg)](https://pypi.org/project/data-aggregator-mcp/)
[![Python](https://img.shields.io/pypi/pyversions/data-aggregator-mcp.svg)](https://pypi.org/project/data-aggregator-mcp/)
[![Downloads](https://img.shields.io/pypi/dm/data-aggregator-mcp.svg)](https://pypi.org/project/data-aggregator-mcp/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/musharna/data-aggregator-mcp/blob/main/LICENSE)
[![CI](https://github.com/musharna/data-aggregator-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/musharna/data-aggregator-mcp/actions/workflows/ci.yml)
[![Glama](https://glama.ai/mcp/servers/musharna/data-aggregator-mcp/badges/score.svg)](https://glama.ai/mcp/servers/musharna/data-aggregator-mcp)

`search` one query across **17 sources** — **Zenodo, DataCite** (Dryad /
Figshare / Dataverse / OSF / OpenNeuro / Mendeley), **NCBI omics**
(GEO / SRA / BioProject), **BioStudies** (EBI, incl. ArrayExpress),
**literature** (PubMed / OpenAIRE), **HuggingFace** datasets, **DataONE**
(eco / environmental), **OmicsDI** (proteomics / metabolomics), **DANDI**
(neurophysiology), **CZ CELLxGENE** (single-cell), **OpenML** (ML datasets),
**RCSB PDB** (structures), **UniProtKB** (proteins), the **GWAS Catalog**,
**GBIF** (biodiversity), **data.gov** (US federal open data), and **NASA CMR**
(Earth science) — deduplicated, normalized, and cross-linked. `resolve` any hit to its file
manifest, citation, trust signals, and the data it points at. `fetch` it to
disk with checksum verification.

mcp-name: io.github.musharna/data-aggregator-mcp

<p align="center">
  <img src="https://raw.githubusercontent.com/musharna/data-aggregator-mcp/main/examples/assets/demo.svg"
       alt="data-aggregator-mcp stdio demo — initialize, tools/list (search, resolve, fetch, operate, relate, list_sources), and a live list_sources call showing the wired sources across archives, omics, and literature"
       width="820">
</p>

## ✨ Why this

Most data MCPs wrap a single source. This one **unifies** them behind six tools
and one `DataResource` model, so an agent searches once and gets back comparable
records:

- **Multi-domain, one model** — generalist archives + raw omics + literature,
  deduplicated by DOI (the fetchable record wins over bare metadata).
- **Taxonomy synonym expansion** — `organism="Orobanche aegyptiaca"` also matches
  `Phelipanche aegyptiaca` (NCBI Taxonomy), so a species rename doesn't cost you
  results.
- **Paper → data bridge** — resolve a paper and get links to the GEO / SRA /
  BioProject / DataCite records it produced.
- **Verified fetch** — streams to disk with md5 verification where the source
  exposes a checksum, optional archive unpacking, and a fail-loud integrity
  sniff that rejects an HTML paywall page served as a "PDF".
- **Citations, access & full text** — render a citation in any CSL style, get
  normalized access/license, and pull open-access full text — all in one
  `resolve`.
- **Trust signals** — usage `metrics` (citations / views / downloads / likes),
  version status (`is_latest` / `superseded_by`), and `last_updated` freshness,
  surfaced wherever the source exposes them.
- **Interop exports** — `resolve(format="croissant")` or `"ro-crate"` hands a
  dataset to an ML or research-packaging pipeline as standard JSON-LD.
- **Operate on data in place** — `operate` reads the schema, previews rows, or
  runs a read-only SQL `SELECT` against a remote Parquet/CSV/TSV **without
  downloading it** (Parquet footer + DuckDB httpfs range reads). Optional
  `[operate]` extra; base install is unchanged.
- **Relate across records** — `relate` takes a handful of resolved ids and
  reports how they connect — shared accession, shared cross-identifier, an
  explicit link, or version lineage — naming the literal shared value as
  evidence. Metadata hints only: it never reads files or executes a join.

→ Full rationale and a comparison vs. single-source servers, breadth gateways, and
ML-dataset tools: **[docs/POSITIONING.md](https://github.com/musharna/data-aggregator-mcp/blob/main/docs/POSITIONING.md)**.

<p align="center">
  <img src="https://raw.githubusercontent.com/musharna/data-aggregator-mcp/main/docs/assets/architecture.svg"
       alt="Architecture: an MCP client speaks stdio to data-aggregator-mcp's six tools, which fan out through one router (DOI dedup, ontology expansion, ranking) to archives (Zenodo, DataCite, HuggingFace, DataONE, OpenML, RCSB PDB), omics (GEO, SRA, BioProject, OmicsDI, DANDI, CELLxGENE, GWAS Catalog), and literature (PubMed, OpenAIRE, EuropePMC, Unpaywall)"
       width="760">
</p>

## ⚡ Quickstart

Run with no install:

```bash
uvx data-aggregator-mcp
```

Register with Claude Code:

```bash
claude mcp add data-aggregator -- uvx data-aggregator-mcp
```

A typical agent flow:

```text
search("drought stress RNA-seq", organism="Sorghum bicolor")
  → [ geo:GSE..., sra:SRX..., zenodo:..., pubmed:... ]   # deduped, taxa-normalized

resolve("sra:SRX079566")
  → DataResource{ files: [ENA FASTQ urls…], access: "open", taxa: [...] }

fetch("sra:SRX079566", dest="./data")
  → ["./data/SRX079566_1.fastq.gz", …]                   # md5-verified
```

<details>
<summary>Other ways to run (pip, python -m, raw client config)</summary>

```bash
pip install data-aggregator-mcp
data-aggregator-mcp        # or: python -m data_aggregator_mcp
```

To use the `operate` tool (query remote tabular files in place), install the
optional extra:

```bash
pip install "data-aggregator-mcp[operate]"
```

Add to a client's MCP config (e.g. Claude Desktop `claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "data-aggregator": {
      "command": "uvx",
      "args": ["data-aggregator-mcp"],
      "env": { "NCBI_API_KEY": "your-optional-key" }
    }
  }
}
```

</details>

## 🗂️ Sources

| Source                       | Discover |       Fetch       |     Checksum     |
| ---------------------------- | :------: | :---------------: | :--------------: |
| Zenodo                       |    ✅    |        ✅         |       md5        |
| DataCite → Figshare          |    ✅    |        ✅         |       md5        |
| DataCite → Dataverse         |    ✅    |        ✅         |       md5        |
| DataCite → OSF               |    ✅    |        ✅         |       md5        |
| DataCite → Dryad             |    ✅    |  manifest only¹   | sha-256 (listed) |
| DataCite → Mendeley & others |    ✅    |         —         |        —         |
| NCBI SRA                     |    ✅    |  ✅ (ENA FASTQ)   |       md5        |
| NCBI GEO                     |    ✅    |   ✅ (`suppl/`)   |      none²       |
| NCBI BioProject              |    ✅    |    → SRA links    |        —         |
| PubMed / OpenAIRE            |    ✅    | ✅ (OA full text) |      none²       |
| HuggingFace datasets         |    ✅    | ✅ (resolve URL)  |       none       |
| DataONE (eco/env)            |    ✅    | ✅ (Member Node)  |  md5 / sha-256   |
| OmicsDI → PRIDE              |    ✅    |  ✅ (HTTPS FTP)   |    size only     |
| OmicsDI → MetaboLights       |    ✅    |  ✅ (HTTPS FTP)   |       none       |
| OmicsDI → other MS repos     |    ✅    |         —         |        —         |
| DataCite → OpenNeuro         |    ✅    |   ✅ (snapshot)   |      none²       |
| DANDI (neurophysiology)      |    ✅    |    ✅ (302→S3)    |      none²       |
| CZ CELLxGENE (single-cell)   |    ✅    |   ✅ (H5AD/RDS)   |      none²       |
| OpenML (ML datasets)         |    ✅    |     ✅ (ARFF)     |       md5        |
| RCSB PDB (structures)        |    ✅    |  ✅ (.cif/.pdb)   |      none²       |
| GWAS Catalog                 |    ✅    |   → PMID bridge   |        —         |

¹ Dryad downloads are token / bot-challenge gated, so `fetch` fails loud;
`resolve` still lists the files.
² No upstream checksum — `fetch` verifies content-type instead (rejects an HTML
page served in place of a binary).

## 🛠️ Tools

### `search(query?, size?, sources?, organism?, disease?, tissue?, chemical?, assay?, kind?, published_after?, published_before?, rank?, cursor?, collapse_mirrors?, understand?, multi_query?, provenance?)`

Fan out across all wired sources in parallel and return compact `DataResource`
records, deduped by DOI. Per-source failures land in `errors{}` — never silently
dropped.

- `organism` — expand the query with NCBI-Taxonomy synonyms; the expansion is
  echoed in `taxon_expansion`, and results carry normalized `taxa[]`
  (`{taxid, name}`) plus a `described_in` link to plant-genomics-mcp for plant
  taxa.
- `sources` — restrict the fan-out, e.g. `["omics"]`.
- `size` — max results (1–50).
- `kind` — keep only `dataset` / `sequencing_run` / `study` / `publication` /
  `software`.
- `published_after` / `published_before` — filter by publication year.
- `rank` — `relevance` (default) or `semantic` (re-rank the fetched page by
  embedding similarity to the query; needs `EMBEDDING_API_BASE`, degrades to
  relevance order otherwise).
- `understand` — opt into LLM query understanding (default false). A free-text
  query is **normalized** into a focused keyword query: conversational fluff
  (`"I'm looking for…"`, `"where can I find…"`) is stripped while the scientific
  and entity terms are kept so they still match by text. The LLM also detects
  structured entities (organism/disease/tissue/chemical/assay, kind) — these are
  **echoed in `query_understanding.extracted` for transparency but not
  auto-applied**, because ANDing LLM-_inferred_ facets across free-text keyword
  upstreams over-constrains and hurts recall. Only the cleaned `keyword_core` and
  explicit `year` scopes are applied; the ontology resolvers still run on the
  facets **you** pass (the LLM proposes, you dispose). Needs an LLM endpoint
  (`LLM_API_BASE`);