watch-skill

<!-- mcp-name: io.github.oxbshw/watch-skill -->
<div align="center">

<img src="docs/assets/watch-skill-hero.webp" alt="Watch Skill: watch a video, remember the evidence with timestamps, and verify an agent's work through THE LOOP" width="88%">

# Watch Skill

**Give every AI agent eyes for video — and a way to check its own work.**

[![CI](https://github.com/oxbshw/watch-skill/actions/workflows/ci.yml/badge.svg)](https://github.com/oxbshw/watch-skill/actions/workflows/ci.yml)
[![Install](https://github.com/oxbshw/watch-skill/actions/workflows/install.yml/badge.svg)](https://github.com/oxbshw/watch-skill/actions/workflows/install.yml)
[![PyPI](https://img.shields.io/pypi/v/watch-skill)](https://pypi.org/project/watch-skill/)
[![Downloads](https://img.shields.io/pypi/dm/watch-skill)](https://pypi.org/project/watch-skill/)
[![Agent Skills](https://www.skills.sh/b/oxbshw/watch-skill)](https://www.skills.sh/oxbshw/watch-skill)
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-3776AB)](pyproject.toml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

[Install](#install) · [Documentation](docs/README.md) · [Examples](examples/README.md) · [Comparison](docs/comparison.md) · [Roadmap](docs/ROADMAP.md)

</div>

Watch Skill turns videos, live streams, meetings, and screen recordings into a searchable,
timestamped index. An agent can ask what happened, get an answer that cites the exact
moment behind it, and ask again tomorrow without processing the video a second time.

When the video is the agent's *own* browser or desktop session, **THE LOOP** closes the
circle: record the work, critique it against plain-language criteria, and show before and
after. That critique is *advisory* — a model describing pictures. To decide whether the
work actually succeeded, attach a **verification contract**: deterministic checks, frozen
before the run, that hold the verdict.

```bash
uvx --from "watch-skill[standard]" watch-skill setup
```

<p align="center">
  <img src="docs/assets/loop_before_after.gif" alt="A checkout flow fails with a NaN total, is fixed, and passes verification" width="720">
  <br>
  <sub>THE LOOP catching a <code>$NaN</code> total that an end-state screenshot misses, then showing the fix.</sub>
</p>

## What it does

| | |
|---|---|
| **Watch** | Scene-aware frames, on-screen text, and local-first transcription from 1,800+ sites, live HLS/DASH streams, local media, meetings, browsers, windows, and desktops. |
| **Watch live** | A session that reports what changed **while the source is still playing** — bounded queues, counted drops, cursor-addressed events, and a rolling buffer that pins the evidence around each one. [Guide](docs/live.md) |
| **Remember** | A persistent, searchable index with timestamp citations, hybrid retrieval, cross-video synthesis, and reusable lessons. |
| **Verify** | A capture → critique → fix → re-capture loop for browser flows, interfaces, generated video, gameplay, and monitored streams — with deterministic contracts deciding pass or fail. |
| **Operate** | Drive a browser and prove the effect of each action — deterministic target resolution, per-step receipts, and verdicts that reject a page reporting success over a failed request. [Guide](docs/browser-runtime.md) |

Available as Claude Code skills, 37 MCP tools, a CLI, a REST API, and native adapters for
LangChain/LangGraph, CrewAI, the OpenAI Agents SDK, LlamaIndex, and AutoGen.

Four things it will not do, because each one is a way of being confidently wrong:

- **Answer from a video that changed.** Identity follows the bytes, not the path. Overwrite
  `demo.mp4` and the next question returns `stale`, not yesterday's frames.
- **Upload a frame you did not agree to send.** A configured API key is not consent.
  `watch-skill plan` prints every network action before a run makes one.
- **Call an absent judgement a pass.** No frames, no OCR, an unreachable model, a timed-out
  check — all `inconclusive`. Only a required deterministic check produces a `pass`.
- **Claim a capability it has not checked.** `watch-skill capture-capabilities` says what this
  machine can actually record, and whether each answer was machine-tested or merely probed.

## Install

Two pieces, and you want both. The **engine** does the work; the **skills** teach your
agent when to reach for it.

```bash
# 1. the engine — installs, wires up every AI agent on the machine, backs up each config
uvx --from "watch-skill[standard]" watch-skill setup

# 2. the skills — into Claude Code, Codex, Cursor, Copilot, Gemini CLI, and 20+ more
npx skills add oxbshw/watch-skill -g
```

Watch Skill ships on PyPI, not npm. The second command runs
[Vercel's `skills` CLI](https://www.skills.sh), which reads the ten `SKILL.md`
files out of this repository and installs them into whichever agents you have —
there is no `watch-skill` npm package to install, and the engine is Python
either way.

Neither needs a clone, and the engine command works the same on macOS, Linux, and
Windows — [CI runs it on all three](https://github.com/oxbshw/watch-skill/actions/workflows/install.yml)
on every push.

Prefer a permanent install to `uvx` fetching on demand?

```bash
pipx install "watch-skill[standard]"     # or: pip install "watch-skill[standard]"
watch-skill setup
```

<details>
<summary>Other ways in — Claude Code plugin, Docker, from source</summary>

**Claude Code plugin** — skills, slash commands, and the MCP server in one:

```text
/plugin marketplace add oxbshw/watch-skill
/plugin install watch-skill@watch-skill
/watch-skill:setup-watch-skill
```

**Docker** — nothing on the host; the volume is where the index lives, so do not skip it:

```bash
docker run --rm -i -v watch-skill-data:/data ghcr.io/oxbshw/watch-skill serve
```

Built for `linux/amd64` and `linux/arm64`, with an SBOM and a signed build attestation.

**From source** (installs uv and Python if either is missing):

```bash
curl -fsSL https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.sh | sh
```

```powershell
powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.ps1 | iex"
```

**Wiring an agent by hand** — the block most MCP clients take:

```json
{ "mcpServers": { "watch-skill": {
    "command": "uvx",
    "args": ["--from", "watch-skill[standard]", "watch-skill", "serve"] } } }
```

Zed, Amp, and a few others name that key differently; each
[agent guide](docs/agents/README.md) shows the exact shape.

</details>

`standard` is frames, retrieval, and MCP — about 200 MB. `watch-skill[all]` adds OCR,
local Whisper, REST, and the browser THE LOOP drives. `watch-skill doctor` names anything
missing and prints the one command that installs it, so starting small is safe.

Coming from [claude-video](https://github.com/bradautomates/claude-video)? Your `/watch`
commands and flags work unchanged — see the [migration guide](docs/migrate-from-claude-video.md).

## First run

```bash
watch-skill watch "https://youtu.be/..." "Summarize the important moments."
```

That prints a report and an id. Everything after it is a lookup against the index, not a
second download:

```bash
watch-skill ask <video_id> "when does the demo first fail?"
watch-skill search "pricing decision"        # across every video you've watched
watch-skill library ask "what did the team decide about auth?"
```

Useful flags on `watch`:

| Flag | Use it when |
|---|---|
| `--detail transcript` | You want the words, not the pictures — much faster |
| `--detail balanced` \| `token-burner` | More frames, more cost |
| `--start 4:10 --end 6:00` | Only a slice of a long video matters |
| `--word-timestamps` | You need the exact word, not the ten-second segment it sat in |
| `--no-cache` | Re-fetch a source that changed |

And the rest of the surface:

```bash
watch-skill serve                            # MCP over stdio — what agents connect to
watch-skill api                              # REST, port 8748
watch-skill doctor                           # check and repair the setup
watch-skill viewer <video_id> --out r.html   # one self-contained page to share
watch-skill loop viewer <loop_id>            # a run's iterations, compared
watch-skill bench providers                  # compare every provider you have a key for
```

Transcription, OCR, and search run locally and need no API key. Visual question
answering uses whichever provider you already pay for — Anthropic, OpenAI, Gemini,
OpenRouter, Groq, Together, Fireworks, DeepSeek, xAI, Mistral, MiniMax, Moonshot,
Z.ai, or Qwen — or nothing at all with a local Ollama model. Anything else that
speaks the OpenAI format (vLLM, LM Studio, llama.cpp, LiteLLM, Azure OpenAI, a
company gateway) works through the `custom` provider:

```bash
watch-skill setup-vision --provider groq            # or any of the above
watch-skill setup-vision --provider custom \
  --base-url http://127.0.0.1:8000/v1               # your own server
```

See [Getting started](docs/getting-started.md) for manual installation and
[Configuration](docs/configuration.md) for provider and privacy settings.

## Why use it

- **Evidence instead of frame dumps.** Scene detection and perceptual deduplication spend
  the frame budget on distinct moments. Answers include timestamps, confidence, and the
  evidence used to support them.
- **Persistent video memory.** Analyze once, ask again without downloading or transcribing
  the same video. Hybrid full-text and vector retrieval works within one video or across
  the entire library.
- **Local-first processing.** Original-language captions are preferred, local Whisper is
  the default fallback, and cloud speech-to-text is opt-in. An Ollama configuration keeps
  the complete pipeline on the machine.
- **Flow verification.** THE LOOP records an agent's browser, screen, or window and checks
  the result against plain-language criteria, producing a before/after comparison. The
  model's read of that rec