clickcast

# 🎞️ clickcast — Playwright browser automation, screenshots, and GIF/MP4/WebP recording for AI agents

> Give AI agents visual + structured feedback about live web UIs — and give humans deterministic demo reels while you're at it.

[![PyPI version](https://img.shields.io/pypi/v/clickcast.svg)](https://pypi.org/project/clickcast/)
[![Python](https://img.shields.io/pypi/pyversions/clickcast.svg)](https://pypi.org/project/clickcast/)
[![CI](https://img.shields.io/github/actions/workflow/status/AlexKay28/clickcast/ci.yml?branch=main)](https://github.com/AlexKay28/clickcast/actions)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)

> **What's new (v0.3.0)** — live agent control via a new [MCP server](docs/mcp.md), pixel-level `clickcast diff`, accessibility semantics fused with the pixel-grid overlay, an official CI GitHub Action, day-one Homebrew/apt packaging, and a self-healing first run. See [`CHANGELOG.md`](CHANGELOG.md) for the full notes.

![clickcast · scripted tour of tailwindcss.com — sticky A→B arrow, actions panel bottom-right, symmetric open/close pairs](docs/demo.gif)

> **See [`docs/ONE_PAGE_NAVIGATION_ORDER_TIPS.md`](docs/ONE_PAGE_NAVIGATION_ORDER_TIPS.md)** for the nine principles behind why this reel reads as legibly as it does — and the scenario template you can copy for your own reels.

`clickcast` drives a real browser through a website and hands back **two things**:

1. A watchable **reel** — GIF / MP4 / WebP / raw frames.
2. A machine-readable **JSON sidecar** — every step's selector, timings, per-step frame paths, discovered elements, and post-action page state (title, URL, console errors, failed requests). Versioned. See [`docs/feedback-schema.md`](docs/feedback-schema.md).

Point it at a URL and it will *auto-discover* the interactive elements and build a tour for you, or hand it a small YAML **scenario** for a scripted, repeatable walkthrough.

---

## Install

One line, ready to run immediately after:

```bash
pip install clickcast && clickcast install --with-deps chromium && clickcast doctor
```

Broken down — `pip install clickcast` (requires Python ≥ 3.10) gets you the
CLI; `clickcast install` downloads Chromium (~one-time, ~180MB — kept out of
the pip package itself since it's versioned independently and every project
doesn't need every engine); `--with-deps` also pulls the system libraries
Chromium needs (Linux only, may prompt for `sudo`); `clickcast doctor`
confirms everything above actually worked.

Forgot the second step? Any command that needs a browser (`auto`, `run`,
`shot`, `elements`, `mcp`) detects a missing engine itself and offers to
install it right then instead of failing — say yes once and it retries
automatically. That prompt only fires in an interactive terminal; CI/scripted
runs fail fast with the exact fix command instead of hanging on stdin.

### Homebrew (macOS/Linux)

```bash
brew install --build-from-source ./Formula/clickcast.rb   # works today, clone this repo first
brew install AlexKay28/clickcast/clickcast                # tap not bootstrapped yet -- see docs/packaging/homebrew.md
```

### apt (Debian/Ubuntu, `.deb`)

```bash
bash scripts/build_deb.sh 0.2.9 && sudo dpkg -i dist-deb/clickcast_0.2.9_amd64.deb   # works today, clone this repo first
sudo apt install clickcast                                                           # hosted repo not bootstrapped yet -- see docs/packaging/apt.md
```

Both native packages skip the Chromium download (~180MB, versioned
independently of clickcast) and never install a second `ffmpeg` (clickcast
already bundles one via `imageio[ffmpeg]`) -- run `clickcast install
--with-deps chromium` once after either install path. Full design rationale,
what's live today vs. what needs one-time bootstrapping, and the exact
bootstrap steps: [`docs/packaging/homebrew.md`](docs/packaging/homebrew.md),
[`docs/packaging/apt.md`](docs/packaging/apt.md).

### npm (`clickcast` + `clickcast-mcp`)

Two npm packages -- thin Node wrappers whose `postinstall` provisions an
isolated Python venv and pip-installs the pinned PyPI `clickcast`, since
there's no way to ship clickcast's Playwright/Pillow/ffmpeg runtime as pure
JS. Built specifically because the MCP ecosystem's install pattern is
`npx <package>`, not `pip install`:

```bash
npx -y clickcast-mcp                 # MCP entry point -- execs `clickcast mcp`
npx clickcast --version              # general CLI wrapper
```

Both are published on the npm registry. To run against your working copy
instead of the published version:

```bash
cd npm/clickcast-mcp && npm pack && npm install -g ./clickcast-mcp-*.tgz && clickcast-mcp --help
cd npm/clickcast      && npm pack && npm install -g ./clickcast-*.tgz      && clickcast --version
```

Full design rationale
(including why the shared provisioning code is a vendored copy rather than
a `file:` dependency), what's live today vs. what needs bootstrapping, and
the exact bootstrap steps: [`docs/packaging/npm.md`](docs/packaging/npm.md).

---

## For AI agents — copy this prompt into your chat

Paste the block below into your coding agent (Claude Code, Cursor, Copilot Chat, Codex, etc.) to teach it clickcast in one message. The agent will install the tool, verify the environment, generate a visual + machine-readable report for your project, and know how to gate CI on the results.

```text
You are going to use `clickcast` (https://pypi.org/project/clickcast/) to visually verify
this project's frontend. It produces a watchable GIF reel + a JSON sidecar with
per-step selectors, timings, page state, error codes, and a graph of the app's
pages. Follow this exact sequence:

1. INSTALL
   pip install clickcast
   clickcast install --with-deps chromium   # first time only, ~180 MB
   clickcast doctor                          # bail out if this fails

2. LOAD FULL CAPABILITIES
   Run `clickcast skill --json` and read the output. It lists every subcommand,
   flag, docs URL, and sidecar schema pointer. Prefer this over guessing.

3. RUN A QUICK TOUR OF THE APP
   If we have a running URL (dev server / staging / production):
     clickcast auto <URL> --for-humans --emit-events --out tour.gif
   If we have a specific flow to verify, write a YAML scenario and run:
     clickcast run tour.yml --emit-events --out tour.gif
   `--for-humans` gives a legible reel for the user to watch;
   `--emit-events` prints a machine-readable JSONL line you can parse.

4. READ THE SIDECAR (JSON at `<gif>.json`, schema v3)
   For each step, gate on the structured fields — NOT regex over prose:
     - `status`      : "ok" | "failed" | "skipped"
     - `error_code`  : "timeout" | "locator_missing" | "cross_origin" |
                       "navigation_error" | "selector_ambiguous" | "other"
     - `skip_reason` : "optional_no_reaction" | "pre_action_failed" |
                       "element_vanished" | "cross_origin_bounce"
     - `page_state`  : title, url_after, console_errors, page_errors,
                       network_failed
   The top-level `graph` block gives page nodes + navigation edges you can
   use to reason about the app's shape, not just the sequence you ran.

5. WATCH STDERR FOR ADVISORIES
   clickcast prints `⚠ <message> — see <docs-url>` lines for known anti-
   patterns (nav-heavy tour, click without DOM reaction, very short reel,
   cross-origin bounce, incoherent cursor styling). Each has a stable
   kebab-case id you can dedupe or gate on.

6. FOR CI REGRESSION GATES
   `clickcast assertions <sidecar>.json --baseline golden.json` diffs the
   run against a committed baseline; nonzero exit on drift. Byte-identical
   across runs (timestamps, frame paths, and URL query strings excluded).

Reference docs (all in-repo, load lazily as needed):
  - Sidecar shape:   https://github.com/AlexKay28/clickcast/blob/main/docs/feedback-schema.md
  - Human-legible reel authoring: https://github.com/AlexKay28/clickcast/blob/main/docs/ONE_PAGE_NAVIGATION_ORDER_TIPS.md
  - Agent integration:  https://github.com/AlexKay28/clickcast/blob/main/docs/ai-integration.md

If something is unclear, run `clickcast <subcommand> --help` before asking me.
```

Once the agent has this, ask it something concrete like *"run clickcast auto against `http://localhost:3000` and tell me which clicks had DOM reactions"* — it now has everything it needs.

---

## Live agent control (MCP)

<!-- mcp-name: io.github.AlexKay28/clickcast -->

Everything above is batch mode: record a whole tour, then read back a GIF + sidecar. `clickcast mcp` is the live counterpart — an MCP server that drives one action at a time (`goto`/`click`/`type`/`scroll`/...) and hands back clickcast's richer per-call payload (annotated frame, `page_state`, grid coordinates, an enumerated `error_code`) instead of a bare screenshot, so an agent can react before deciding the next step.

```bash
pip install 'clickcast[mcp]'
```

```json
{
  "mcpServers": {
    "clickcast": { "command": "clickcast", "args": ["mcp"] }
  }
}
```

Reach for `mcp` when an agent needs to explore and react live; reach for `auto`/`run` for a repeatable, one-shot artifact (CI, docs, release notes). Full tool reference, client config for Claude Code / Claude Desktop, and the schema doc: [`docs/mcp.md`](docs/mcp.md) · [`docs/mcp-tool-schema.md`](docs/mcp-tool-schema.md).

---

## First run — 30 seconds

```bash
clickcast auto https://example.com --out tour.gif
```

Produces two files:

- `tour.gif` — the reel
- `tour.gif.json` — the AI-consumable sidecar (`schema_version: 1`, spec at [`docs/feedback-schema.md`](docs/feedback-schema.md))

For a walkthrough of how an LLM agent consumes both, see [`docs/ai-integration.md`](docs/ai-integration.md).

---

## Three modes

| Mode | Command | When |
|---|---|---|
| **Auto** | `clickcast auto <url>` | Quick tour of a site; you don't care about the exact script. |
| **Scenario** | `clickcast run tour.yml` | Precise, repeatable walkthrough. Docs, release notes, CI. |
| **Shot** | `clickcast shot <url