tekmeria

<div align="center">

<img src="https://raw.githubusercontent.com/PedroMarkovicz/tekmeria/main/docs/assets/banner.png" alt="Tekmeria: is every citation in this answer real, and does it quote its source verbatim?" width="100%" />

<a id="tekmeria"></a>

### Citations that can be checked: server-issued handles and a deterministic audit for biomedical evidence

<br/>

<a href="https://pedromarkovicz.github.io/tekmeria/"><img src="https://img.shields.io/badge/%E2%96%B6%20%20OPEN%20THE%20AUDIT%20VIEW%20DEMO-pedromarkovicz.github.io%2Ftekmeria-2f5bd3?style=for-the-badge&labelColor=1a2040" alt="Open the audit view demo: pedromarkovicz.github.io/tekmeria" width="700" /></a>

<br/><br/>

[![CI](https://github.com/PedroMarkovicz/tekmeria/actions/workflows/ci.yml/badge.svg)](https://github.com/PedroMarkovicz/tekmeria/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/tekmeria.svg)](https://pypi.org/project/tekmeria/)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.23353865.svg)](https://doi.org/10.5281/zenodo.23353865)
[![Python 3.12](https://img.shields.io/badge/python-3.12-blue.svg)](https://www.python.org/downloads/)
[![MCP Python SDK 2.3](https://img.shields.io/badge/MCP%20Python%20SDK-2.3-6e56cf.svg)](https://github.com/modelcontextprotocol/python-sdk)
[![Pydantic 2](https://img.shields.io/badge/Pydantic-2-e92063.svg)](https://docs.pydantic.dev/)
[![React 19](https://img.shields.io/badge/React-19-61dafb.svg)](https://react.dev/)
[![TypeScript 7](https://img.shields.io/badge/TypeScript-7-3178c6.svg)](https://www.typescriptlang.org/)
[![Contract 0.1.0](https://img.shields.io/badge/citation%20contract-0.1.0-1a2040.svg)](https://github.com/PedroMarkovicz/tekmeria/blob/main/docs/CONTRACT.md)
[![License: Apache 2.0](https://img.shields.io/badge/License-Apache%202.0-brightgreen.svg)](https://github.com/PedroMarkovicz/tekmeria/blob/main/LICENSE)

[Overview](#overview) •
[Quickstart](#quickstart) •
[Results](#results) •
[How a citation works](#how-a-citation-works) •
[Architecture](#architecture) •
[Evaluation](#evaluation) •
[Usage](#usage) •
[Citing](#citing)

</div>

<p align="center">
  <a href="https://pedromarkovicz.github.io/tekmeria/">
    <img src="https://raw.githubusercontent.com/PedroMarkovicz/tekmeria/main/docs/assets/audit-view.png" alt="The audit view showing the offline demo's report" width="100%" />
  </a>
  <br/>
  <em>The audit view on the offline demo: each sentence with its status, each citation with its verdict. The answer fails on a forged and a malformed citation.</em>
</p>

---

<a id="overview"></a>
## Overview

<div align="justify">

Tekmeria is a [Model Context Protocol](https://modelcontextprotocol.io/) server for biomedical evidence from PubMed, Europe PMC, ClinicalTrials.gov and openFDA. Every passage it returns carries a short handle such as `pmid:30831578/abstract.7~ae2tc`, which ends in a check code that only this installation can compute. A model cites those handles, and a deterministic audit then shows which citations are genuine, which point to text that no longer exists, and which quotes are not verbatim. It is meant for people who build or evaluate biomedical question-answering agents, and for developers of MCP hosts who want citations their users can check.

Language models that answer biomedical questions with citations can cite papers that do not exist, misquote the ones that do, and cite retracted work. The problem now reaches the literature itself: an audit of 2.5 million open-access articles found fabricated references in 1 in 2,828 papers in 2023 and in 1 in 277 in early 2026 ([Topaz et al., *The Lancet*, 2026](https://doi.org/10.1016/S0140-6736%2826%2900603-3)). Checks that run after the answer is written must judge whether an imperfect bibliographic match is the right work. Tekmeria moves the guarantee into the tool server. The server that hands out a passage also issues its citation, so a citation is checked against what was issued, not searched for.

> The audit checks provenance and quotation, not support. Whether a passage actually backs its claim is reported as `not_checked`: tekmeria narrows that question to genuine, correctly quoted passages, and leaves it open. A verbatim quote does not settle it: across twelve models, verbatim quotes often failed to support their claims fully ([Zhang et al.](https://arxiv.org/abs/2609.15964)).

</div>

The project stays out of the race for coverage. [BioMCP](https://github.com/genomoncology/biomcp) already reaches about 70 biomedical sources, and retrieval itself helps little: biomedical RAG typically improves on no retrieval by 1 to 2 points ([Nourbakhsh et al.](https://arxiv.org/abs/2606.04127)). Citations are where models fail:
- when asked to cite, the best model in [Ahmed et al.](https://arxiv.org/abs/2604.17114) gave 15.3% relevant PMIDs, most of the rest resolving to real papers in unrelated fields;
- in [Chen et al.](https://arxiv.org/abs/2608.30676), citation fabrication by a medical agent rose from 16.5% to 31.8% when reinforcement learning rewarded only correct answers.

The protocol points the same way: the [2026-07-28 MCP revision](https://modelcontextprotocol.io/specification/2026-07-28/changelog) says that servers needing state across calls use "explicit, server-minted handles passed as ordinary tool arguments".

Others also put trust in the tool rather than the model. An agent runtime can sign a receipt for each tool call and check the claims that cite it ([Basu](https://arxiv.org/abs/2603.10060)); a renderer can mark a number verified only when it matches the record it names ([Solatorio](https://arxiv.org/abs/2509.06902)); and a genome-writing tool layer reachable over MCP requires every quantity to come from a validated tool ([Ali et al.](https://arxiv.org/abs/2608.20412)). Tekmeria applies the idea to citations of biomedical passages: the handle the model writes is itself authenticated, so the server that issued it can check it with its key alone, whichever model or MCP host wrote the answer.

**Status.** Research software, version 0.1.1. The citation contract is versioned separately, at 0.1.0; while it is `0.x`, a breaking change raises its minor version. Changes are listed in [CHANGELOG.md](https://github.com/PedroMarkovicz/tekmeria/blob/main/CHANGELOG.md). A technical note describes the design and the evaluation: [10.5281/zenodo.23374728](https://doi.org/10.5281/zenodo.23374728).

---

<a id="quickstart"></a>
## Quickstart

You need [uv](https://docs.astral.sh/uv/) (it installs a suitable Python if needed). No API keys are required. One command runs the offline demo on recorded responses and opens its audit in your browser:

```bash
uvx tekmeria demo --open
```

To use tekmeria in a chat, add it to an MCP host such as Claude Desktop or VS Code: see [Usage](#usage). The command installs the released package from [PyPI](https://pypi.org/project/tekmeria/). To run the development version instead, use `uvx --from git+https://github.com/PedroMarkovicz/tekmeria tekmeria`, or point `--from` at a local clone.

---

<a id="results"></a>
## Results

<div align="justify">

The evaluation is a conformance test: it plants citations whose correct verdict is known in advance and checks that the audit decides as its specification says. With seed 0 and 25 cases per category:

</div>

| Check | What it measures | Result |
|---|---|---|
| Planted citations | 850 cases in 34 categories, audited as structured claims and as answer text (four categories in one form only) | **1,600 / 1,600** audits correct |
| Whole answers | Every sentence status, every citation, the metrics and the overall verdict of multi-sentence answers | **257 / 257** correct |
| Other seeds | Seeds 1 to 10 at 25 cases per category; seed 0 at 200 (12,800 audits, 1,832 answers) | All correct |
| Injected faults | 38 hand-chosen faults: lenient comparators, lost typographic folds, wrong report rules | **37** caught |
| Mutation analysis | 78 mutants of comparisons, boolean operators and integer constants in the audit's code | **71** killed: 52 by a wrong result, 19 by a crash |
| Live pilot | One question to a frontier model in Claude Desktop; the second of two runs (the first failed on an example handle in the instructions) | 16 / 16 citations issued by the server, 4 / 4 quotes verbatim; `warn`, with 8 of 21 sentences uncited; support not checked |

<div align="justify">

None of this measures how often models fabricate citations, or whether a cited passage supports its sentence. The reference oracle was written by the same author, from the specification and separately from the audit, so it is not independent; whole answers are built from templates; and the faults are chosen by the author. How models cite when they are given handles is left to a later study. The full evaluation, with its output, is [below](#evaluation).

</div>

---

<a id="how-a-citation-works"></a>
## How a citation works

```
pmid:30831578/abstract.7~ae2tc
\__/ \______/ \________/ \___/
scheme  id      path     check code: HMAC of everything before the "~"
```

<div align="justify">

The scheme is one of `pmid`, `pmcid`, `ppr`, `nct` and `setid`; registries that serve only their current version (trials and labels) carry the version that was cited. The model writes `[handle]` after a claim, or `[handle "exact words"]` for numbers and doses. The audit then gives each citation one verdict, from the first check that fails:

</div>

```mermaid
flowchart TD
    C["A citation: a handle, and perhaps a quote"] --> G{"Does it match the handle grammar?"}
    G -- "no" --> M["malformed"]
    G -- "yes" --> K{"Does the check code match this installation's key?"}
    K -- "no" --> F["forged"]
    K -- "yes" --> R{"Could the source be reached?"}
    R -- "no" --> U["unavailable"]
    R -- "yes" --> P{"Do the record and the passage still exist?"}
    P -- "no" --> W["withdrawn"]
    P -- "yes" --> Q{"A quote, and not verbatim in the text as issued or as it is now?"}
    Q -- "yes" --> X["quote