video-extract-mcp

# video-extract-mcp

**Give an AI agent any video link: download the file, read the transcript, or get just the frames that matter — all on your own machine.**

Two jobs, and you can use either on its own:

**Get the video.** A YouTube link, a TikTok, a WeChat Channels share URL, a raw `.mp4`, or a page from a site nobody has heard of — it resolves and downloads it, whole or just the section you asked for. If that is all you need, stop there; nothing forces you to analyse anything.

**Or read it.** A transcript (real captions when the platform has them, local speech recognition when it does not) and a small set of *important* keyframes — deduplicated, scene-aware, and scored — instead of a thousand near-identical stills.

Built for AI agents. Three MCP tools, no cloud, no API keys, no Python.

[![npm](https://img.shields.io/npm/v/@yanlinglabs/video-extract-mcp)](https://www.npmjs.com/package/@yanlinglabs/video-extract-mcp)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Node](https://img.shields.io/badge/node-%E2%89%A526-brightgreen.svg)](https://nodejs.org)
[![TypeScript](https://img.shields.io/badge/TypeScript-strict-blue.svg)](https://www.typescriptlang.org/)
[![Tests](https://img.shields.io/badge/tests-685%20passing-success.svg)](#testing)
[![MCP](https://img.shields.io/badge/MCP-server-orange.svg)](https://modelcontextprotocol.io)
[![MCP Registry](https://img.shields.io/badge/MCP%20Registry-com.yanlinglabs%2Fvideo--extract--mcp-blue.svg)](https://registry.modelcontextprotocol.io/v0/servers?search=video-extract)

---

## What it can do

**Get the video, from almost anywhere.** Paste a link and it resolves: YouTube, TikTok, Instagram, X, Facebook and Reels, Twitch, Vimeo, Reddit, WeChat Channels, a bare `.mp4` or `.m3u8`, or a page on a site nobody has heard of. Local files work too. Anything unsupported comes back as a clear failure rather than a crash.

From there you choose how much work to pay for:

- **Just look it up.** Title, creator, duration, chapter list, description, comment count — without downloading a byte. On a long video this is how you find the one section worth analysing.
- **Download the file.** The whole thing, or just seconds 300–420 of it. Where the platform supports ranged fetching, only that section is transferred rather than the full video.
- **Get a transcript.** Real captions when the platform has them, in any language it publishes; local speech recognition when it has none. The result says which you got. A captions-only request skips the download entirely — seconds instead of minutes.
- **Get the frames that matter.** Not every Nth frame: scene changes, on-screen text appearing, genuinely new visuals — deduplicated and scored, typically a few dozen for an hour of video. Or uniform sampling, or one exact frame at one exact second, if that is what you need.
- **Do several at once.** Pass a list of videos and get a result per video; one failing does not sink the rest.
- **Reach private or rate-limited content.** Point it at your browser's cookies and it can fetch age-restricted, members-only or followers-only media, and shrug off the bot checks that block anonymous downloads.
- **Run long jobs in the background.** A full analysis of an hour-long video takes minutes; it can run as a background task, report progress over a local HTTP endpoint, and hand back the result even if your client gave up waiting.

Everything happens on your machine. No API key, no upload, no third-party service.

## Why this exists

Two problems, really. Getting the video at all — every platform hides its media behind a different mechanism, and none of them want a script fetching it. And then reading it: an LLM cannot watch a video, and the usual workaround — dump every Nth frame into the context window — burns enormous amounts of context on frames that are 98% identical to the one before, while still missing the slide that changed when nothing else moved.

`video-extract-mcp` handles both:

- **Fetching, from almost anywhere.** One resolver chain covers the big platforms, direct media URLs, and generic sites, with ranged fetches where the platform allows them and cookie support for anything that needs a login. Downloading is a complete use of this tool, not a step on the way to something else.

- **Transcript, honestly sourced.** The platform's own captions are used whenever the video has any — human-written first, otherwise the platform's automatic ones. Audio is transcribed locally (Whisper or SenseVoice) only for videos with no captions at all. The result tells you which you got, via `transcript.source`.
- **Keyframes chosen, not sampled.** Scene-boundary detection, blur/quality filtering, on-screen-text novelty (subtitle-aware, so burned-in captions don't preserve redundant frames), and image-embedding similarity feed an iterative diversity-aware selector.
- **Output goes to disk, not into your context.** The tool reply is a compact summary plus file paths. A 35-frame manifest and a full transcript don't belong in a conversation where the agent needs three numbers from them.
- **Everything runs on your machine.** No third-party API, no upload, no key. Long analyses can run as MCP background tasks — the tool returns a handle immediately and pushes progress; see Background tasks below.

## Quick start

Install the system binaries first — these can't come from npm:

```bash
# macOS; use your package manager elsewhere
brew install ffmpeg yt-dlp tesseract tesseract-lang
```

Then point your MCP client at the package. There are two ways, and they differ in ways worth thirty seconds of your time.

**Option A — `npx`, nothing installed.** Simplest, and it picks up new releases on its own.

Claude Code:

```bash
claude mcp add --scope user video-extract -- npx -y @yanlinglabs/video-extract-mcp@latest
```

Codex:

```bash
codex mcp add video-extract -- npx -y @yanlinglabs/video-extract-mcp@latest
```

**Another agent?** Point it at **[SKILL.md](https://github.com/yanlingLabs/video-extract-mcp/blob/main/SKILL.md)** and it can install itself.

**Keep the `@latest`** — without it npx pins to the first version it cached and never updates.

**Option B — installed globally.** Starts faster and gives you the `video-extract` status CLI as a real command.

```bash
npm install -g @yanlinglabs/video-extract-mcp
```

Then register it — Claude Code:

```bash
claude mcp add --scope user video-extract -- video-extract-mcp
```

Codex:

```bash
codex mcp add video-extract -- video-extract-mcp
```

|  | `npx` (A) | global install (B) |
|---|---|---|
| Updates | automatic **only with `@latest` in the spec** — a bare `npx -y @yanlinglabs/video-extract-mcp` pins to the first version it cached and never updates | **manual: `npm update -g @yanlinglabs/video-extract-mcp`**. You stay on the installed version until you run it |
| Startup | ~0.9s (npm resolution on every launch) | ~0.1s |
| `video-extract status` in your shell | not on `PATH` — needs `npx -y -p @yanlinglabs/video-extract-mcp video-extract status` | works directly |
| Working directory | must not be this package's own checkout (see below) | irrelevant |

Neither affects what agents can do: an agent checks on background work over HTTP using the `statusUrl` handed to it in the reply, never a shell command. The CLI is for humans.

Or in any MCP client's config — `"command": "npx", "args": ["-y", "@yanlinglabs/video-extract-mcp@latest"]` for A, or `"command": "video-extract-mcp"` with no args for B:

```json
{
  "mcpServers": {
    "video-extract": {
      "command": "npx",
      "args": ["-y", "@yanlinglabs/video-extract-mcp@latest"]
    }
  }
}
```

> **One gotcha with `npx`, and it only bites contributors.** Run inside this package's own git checkout, `npx @yanlinglabs/video-extract-mcp` fails with `command not found` — npx sees the local `package.json` claiming that name, looks for the binary in a local `node_modules/.bin` that was never populated, and gives up. Since MCP clients launch servers with the working directory set to your project, option A cannot work *in this repo*. Working on the tool itself? Point that one project at your build — `claude mcp add --scope local video-extract -- node "$PWD/dist/mcp.js"` — which also means a `npm run build` takes effect immediately, with no publish round-trip. Everywhere else, `npx` is fine.

That is enough for any video that has captions — which, thanks to the caption-first transcript policy, is most of them. The vision model downloads itself on first use.

**Speech models are only needed for videos with no captions at all**, and they are fetched automatically the first time one is. Only the engine that video needs is downloaded — 233 MB for the Chinese/Japanese/Korean model, 1.3 GB for Whisper — into `~/.cache/video-extract-mcp/models`. Set `VIDEO_EXTRACT_AUTO_FETCH_MODELS=0` to keep it manual, or pre-fetch them yourself:

```bash
npx -y @yanlinglabs/video-extract-mcp --help   # installs the package
curl -fsSL https://raw.githubusercontent.com/yanlingLabs/video-extract-mcp/main/scripts/fetch-models.sh \
  | bash -s -- ~/.cache/video-extract-mcp/models
```

`~/.cache/video-extract-mcp/models` is where the tool looks by default. Override with `VIDEO_EXTRACT_MODELS_DIR`. If the fetch is disabled or fails, an uncaptioned video still returns frames and records a warning explaining why the transcript is missing — it degrades rather than fails.

### From source (contributors)

```bash
git clone https://github.com/yanlingLabs/video-extract-mcp.git
cd video-extract-mcp
npm install && npm run build
./scripts/fetch-models.sh    # into ./models, which takes precedence when present
npm run preflight            # verifies ffmpeg / ffprobe / yt-dlp / tesseract
```

### Environment variables

| Variable | Purpose |
|---|---|
| `VIDEO_EXTRACT_MODELS_DIR` | Where speech models live. Defaults to `./models` when that exists, else `~/.cache/video-extract-mcp/models`. |
| `VIDEO_EXTRACT_AUTO_FETCH_MODELS` | Set `0` to