motif

# Motif

<!-- mcp-name: io.github.sleepycobalt/motif -->

An agentic loop that turns a folder of interview transcripts into a research synthesis where every insight carries cited, verified evidence, an honest confidence level, and the counter-evidence against it.

Built for design and research teams who synthesise qualitative interviews and need output they can trust and trace. Motif is the first tool from [ETOT](https://etot.design). Built as an R&D project; the [case study](https://etot.design/tools/motif/case-study/) tells the story.

## What it does

```
transcripts/  →  intake  →  synthesis  →  critic  →  revise  →  report.md
                                            ↑            │
                                            └────────────┘  until the critic passes or 3 rounds
```

- **Intake** (one call per transcript) maps topics and notable positions with turn references.
- **Synthesis** produces 8–14 insights. Each has a claim, cited turns with verbatim receipts, sources, confidence, counter-evidence, and a design opportunity.
- **Critic** checks every insight against the transcripts using rules you can edit — unsupported claims, missing dissent, overconfidence, merged findings, themes present in the corpus but absent from the report. Some rules run in code (citations exist, quotes match, confidence thresholds); the rest are judged by the model.
- **Revise** fixes what the critic flagged. It may not delete an insight to make an objection go away.
- **Report** shows every insight with its evidence expanded, and marks any insight the critic still objected to when the loop stopped. Silence is never treated as agreement.

Sample output: [docs/exhibits/best-report-v2/output.md](docs/exhibits/best-report-v2/output.md).

## Install (about 5 minutes)

You need Python 3.10+ and an Anthropic API key ([console.anthropic.com](https://console.anthropic.com)).

```bash
pip install etot-motif
export ANTHROPIC_API_KEY=your-key-here      # or put it in a .env file in the working directory
```

Or from a checkout, if you want to edit the critic rules or run the evals:

```bash
git clone https://github.com/sleepycobalt/motif.git
cd motif
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
echo "ANTHROPIC_API_KEY=your-key-here" > .env
```

## Run

Put your transcripts in a folder, one speaker turn per paragraph or line, each starting with the speaker's name and a colon. Label the interviewer `Researcher`, `Interviewer`, or `Moderator` so their turns are never cited as evidence.

```
Interviewer: Can you tell me about the last time you used the app?
Priya: Sure. I opened it on the train and it logged me out again, which...
```

Then:

```bash
motif ./transcripts --out report.md --question "What frustrates users about onboarding?"
```

Fifteen transcripts of ~45 minutes each take about 20 minutes and cost about $2.50 in API usage. Every prompt, response, and iteration is saved under `runs/` so you can see exactly what the critic objected to and how the synthesis changed.

Try the sample corpus first:

```bash
motif data/raw/Dataset-2 --out report.md
```

## Use it from Claude Code or Cursor

Motif is also an MCP server: the same engine, callable from any MCP host.

```bash
pip install "etot-motif[mcp]"
claude mcp add motif -e ANTHROPIC_API_KEY=your-key-here -- motif-mcp
```

Or, with [uv](https://docs.astral.sh/uv/) and no install step, `claude mcp add motif -e ANTHROPIC_API_KEY=your-key-here -- uvx --from "etot-motif[mcp]" motif-mcp`. From a checkout: `pip install -e ".[mcp]"` and point the host at `$PWD/.venv/bin/motif-mcp`.

Five tools: `motif_synthesize`, `motif_critique` (check any synthesis, yours or someone else's, against the transcripts), `motif_receipts` (verbatim turn text for a citation), `motif_board` (a run laid out for FigJam, executed by the host through Figma's MCP server), `motif_runs_get`. Install snippets for Claude Code, Cursor, and Claude Desktop, plus a skill that teaches an agent the verify-before-you-quote workflow: [surfaces/mcp/README.md](surfaces/mcp/README.md).

## Use it from Figma

Motif for Figma (FigJam and Figma Design) is in `surfaces/figma/`: paste your Anthropic key once, drop transcripts, get the synthesis in the plugin and as Markdown. Live on Figma Community, approved 2026-09-07: https://www.figma.com/community/plugin/1678295978273812914. Build and import steps: [surfaces/figma/README.md](surfaces/figma/README.md).

## Tune it

Everything a team might want to change lives in [`config/synth.yaml`](config/synth.yaml):

- which model plays which role
- how many revision rounds
- what "high confidence" requires (default: 4+ participants and no counter-evidence)
- the critic's rules, in plain language — add, remove, or reword them

## What the evaluation found

Tested on 15 real research interviews (University of Sheffield, CC-BY-NC) against a human-built ground truth of 16 themes and 12 traps, with blind scoring:

| | Single prompt | Motif v2 | Motif v3 |
|---|---|---|---|
| Insights whose cited evidence doesn't support them | 1.7 of 4 checked | 0.7 | 0.0 |
| Insights with overstated confidence | 1.3 | 0.7 | 0.0 |
| Themes found | 75% | 69% | 88% |
| Time | 4 min | 22 min | 25 min |
| Cost | $0.37 | $2.28 | $2.51 |

The loop makes fewer errors and, since v3, finds more. Its first version found much less (51%) — the critic only checked what was on the page, and the reviser's cheapest fix was deletion. A recall check against the intake topic maps recovered most of that gap; a second check, which asks whether an already-cited turn contains a *second* finding nobody used, recovered the rest (3 of 3 runs, on the two themes that were missed in every report of the previous eval). Full results: [docs/eval1-results.md](docs/eval1-results.md), [docs/eval2-results.md](docs/eval2-results.md), [docs/eval3-results.md](docs/eval3-results.md).

Known gaps: the loop never reaches `critic_pass` — 0 of 10 runs in Eval 3, at three rounds and at five — so it always stops on the iteration cap with objections outstanding; a newly added insight arrives without counter-evidence and the counter-evidence check does not revisit it; and the unsupported-evidence figure above is zero *in a fixed sample of four insights per report*, not zero outright.

## Repo layout

```
synth/      this tool: engine (the shared service), agents, prompts, corpus loader, report renderer, board layout, CLI
surfaces/   mcp/ — the MCP server (Claude Code, Cursor, any MCP host)
tests/      offline tests with a stubbed model; the MCP server is exercised over stdio
config/     synth.yaml — models, thresholds, critic rules (symlink to synth/synth.yaml, which ships in the package)
scripts/    ingest.py (transcripts → citable text), eval_pack.py (blind scoring packs)
data/       sample corpus (CC-BY-NC, see LICENSE) and its processed form
docs/       R&D brief, working log, ground truth, eval results, exhibits, case-study notes
eval/       blind scoring packs and completed sheets
```

The loop controller, run logger, LLM client, and config loader live in [etot-core](https://github.com/sleepycobalt/etot-core), a standalone package Motif depends on. It is written to be reused by other loops; Motif is the first tool built on it.

## Data attribution

Sample transcripts: Hanchard, M. and San Roman Pineda, I. (2023). *Fostering cultures of open qualitative research: Dataset 2 – Interview Transcripts.* University of Sheffield. [doi:10.15131/shef.data.23567223.v2](https://doi.org/10.15131/shef.data.23567223.v2). CC-BY-NC 4.0. Non-commercial use only.

## License

MIT for the code. See [LICENSE](LICENSE).