Pointcast

<p align="center">
  <a href="docs/launch/video/out/pointcast-demo.mp4"><img src="docs/launch/video/out/pointcast-hero.gif" alt="Alt+click a table cell while talking: your agent gets the source line that makes it, not the page's HTML" width="800"></a>
</p>

# Pointcast

**Talk and Alt+click on your web app. Your coding agent gets the exact elements, and the lines of code behind them.**

One recording covers a whole list of UI changes: say what you want while you point, press Stop, and Claude Code, Codex, Gemini CLI or Cursor get a spec that leads with `src/…:line`. In [an evaluation on three real admin dashboards](docs/eval/results-2026-09-27.md), pointing raised the agent's accuracy from 78 % to 89 % over the same words without pointing; in a follow-up, adding the code lines [cut the tokens it spent finding the elements by more than half](docs/eval/stage0-code-pointer-2026-09-27.md). Your voice is transcribed locally, in the browser.

In a [typed-mode evaluation on four apps](docs/eval/results-2026-09-28-batching.md) (React, Vue, Svelte, Django), an agent reading one Pointcast recording with six changes found the right code 96 % of the time, against 85 % for a quick hand-typed request with the same six changes, with 24 % fewer tokens and 75 % fewer searches. One change per request is more accurate but [not cheaper](docs/eval/results-2026-09-28.md): the saving comes from batching.

<img src="docs/launch/video/out/pointcast-batch.gif" alt="Three Alt+clicks in one recording become one spec with three requests" width="640">

[![CI](https://github.com/Hugelidus/pointcast/actions/workflows/ci.yml/badge.svg)](https://github.com/Hugelidus/pointcast/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/Hugelidus/pointcast?include_prereleases&label=beta)](https://github.com/Hugelidus/pointcast/releases)
[![npm](https://img.shields.io/npm/v/pointcast?label=npm)](https://www.npmjs.com/package/pointcast)
[![MIT license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

**[Quick start](#quick-start) · [How it works](#how-it-works) · [Eval](docs/eval/results-2026-09-27.md) · [Discussions](https://github.com/Hugelidus/pointcast/discussions) · [Changelog](CHANGELOG.md)**

> **Public beta (0.6).** Chrome and Edge on desktop. CI-tested on Windows, macOS and Linux (Chrome); Edge developed and tested on Windows. Feedback: [issues](https://github.com/Hugelidus/pointcast/issues).

## Quick start

**One command, in your project** (Node.js ≥ 22.12):

```bash
npx pointcast@latest setup
```

It finds your coding agents (Claude Code, Codex, Gemini CLI, Cursor) and, after asking you, adds Pointcast to each one; says what your stack needs (Django: [pointcast-django](integrations/django/README.md); React, Vue, Svelte: nothing); tells you how to add the browser extension; and ends with `pointcast doctor`. Each change shows the exact command or file first. `--dry-run` only shows the plan, `--yes` skips the questions ([D14](docs/decisions.md#d14-one-setup-command)).

**Or by hand:**

**1. Add the extension.** The Chrome Web Store listing is in review; meanwhile, [load the release zip](#manual-install).

**2. Connect your agent** (optional: it fetches your latest recording itself and resolves each element to its line in your repository):

```bash
# Claude Code, then /pointcast
claude plugin marketplace add Hugelidus/pointcast && claude plugin install pointcast@pointcast
# Codex CLI, then in a new session $pointcast:pointcast (or "apply my latest pointcast recording")
codex plugin marketplace add Hugelidus/pointcast && codex plugin add pointcast@pointcast
# Gemini CLI, then /pointcast in a folder you trust
gemini extensions install https://github.com/Hugelidus/pointcast
# Cursor, Windsurf or any MCP client: run this MCP server (config: packages/cli/README.md#mcp-server)
npx -y pointcast@0.6.0 mcp
```

Any other agent: paste the spec from the clipboard. (In Windows PowerShell 5.1, run each `&&` half on its own line.)

**3. Record** on your app on `localhost`: press **Record**, talk while you **Alt+click** things (**⌥ Option+click** on macOS), press **Stop**.

**Rather type than talk?** Pick **⌨️ Typed** above Record in the popup. Each Alt+click then opens a small box next to the element: type what should change, press Enter (Esc drops that gesture). No microphone, no speech model, and Stop gives you the spec at once.

<img src="docs/launch/video/out/pointcast-typed.gif" alt="Typed mode: Alt+click, type a note, press Enter" width="640">

<details id="manual-install">
<summary><b>Manual install (release zip)</b>, updating from 0.1.x, and where recordings go</summary>

Download the newest `pointcast-<version>-chrome.zip` from [Releases](https://github.com/Hugelidus/pointcast/releases) and unzip it. Open `chrome://extensions` (`edge://extensions` in Edge), turn on *Developer mode*, choose *Load unpacked* and pick the unzipped folder. To build it yourself, see [CONTRIBUTING.md](.github/CONTRIBUTING.md#try-it-locally).

**Updating an unpacked 0.1.x?** Remove it and load 0.2 as a new extension, once. From 0.2 on, every build has the same extension id (the Chrome Web Store item's), so the id changes this one time: allow the microphone again, and the speech model downloads once more.

**Where recordings go.** While your agent's `pointcast` MCP server runs (the plugins above, or `npx -y pointcast@0.6.0 mcp`), recordings go straight to it, with no downloads. Without it, Chrome's downloads save them to `Downloads/pointcast/`; then turn off *Ask where to save each file before downloading* (`chrome://settings/downloads`), or Chrome asks for every file and the sessions miss the folder where the plugins, the MCP server and the CLI look. The popup tells you when that happened.

**Other sites.** Pointcast runs on `localhost`, `127.0.0.1`, `[::1]`, `*.localhost` and `*.test` out of the box. For a staging server or a preview deployment, open the popup there and press **Enable on `<host>`**: Chrome asks for access to that host only.

**The CLI** (optional, Node ≥ 22.12): `npx pointcast process` re-renders the latest session with its code lines resolved in the current repository; `npx pointcast --help` lists every option.
</details>

## How it works

1. **Record.** Press Record in the extension (or Alt+Shift+S, ⌥⇧S on macOS) and talk while you use your app.
2. **Point.** Alt+click (⌥ Option+click on macOS) or select text on whatever you are talking about. The Alt+click never reaches your app.
3. **Stop.** Your voice is transcribed on your machine, and each sentence becomes a request with the elements you pointed at while saying it (in **Typed** mode, each note you typed becomes a request with its element):

```markdown
## Request 2

> And this [a] button should export the order status.

- [a] «Export» → code:
  - used at: `src/pages/Dashboard.tsx` — `<OrdersTable>`
  - text at: `src/components/OrdersTable.tsx:22` — `<button type="button" id="orders-export" className="btn btn-export"> Export </button>`
  - within: `<Dashboard>` in `src/App.tsx` ← `<App>` in `src/main.tsx`
  - on screen: button «Export» in «Orders» on `/`
```

(From a real recording on [dev/examples/react-dashboard](dev/examples/react-dashboard), spoken in Spanish and translated here; selector, DOM path and styles lines trimmed. The app has a second «Export» button, in another component: the spec names this one.)

When something is broken ("this button does nothing"), the element also lists what failed on the page around that moment: uncaught errors, `console.error`/`warn` and failed requests (`POST /api/export → 500`), never a request body ([D13](docs/decisions.md#d13-debug-capture)).

<img src="docs/launch/video/out/pointcast-errors.gif" alt="Alt+click a broken Export button: the spec lists the failed POST /api/export under it" width="640">

Your agent fetches the spec through its plugin, or you paste it. On React 19, Vue 3 and Svelte 5 dev builds, and Django templates with [pointcast-django](integrations/django/README.md), each element leads with its code: where that instance is used, which component defines it (marked when it is shared) and the line where its text or data lives. Other pages get the DOM description: selector, path, HTML and text.

**Next.js App Router** (`next dev`, Turbopack): Client and Server Components both get their chain (`used at:` the file and line, up to the page or layout), mapped through Next's own dev source maps. Through the MCP server or the CLI, which read your repo, text written in a component or in a data module it imports gets its `text at:`/`data at:` line; text computed at runtime gets none. The pasted spec has the chain without those lines, and `next dev --webpack` gets no chain ([what was measured](docs/eval/nextjs-2026-09-28.md)).

<details>
<summary>Where the code lines come from</summary>

The component chain comes from the page. The exact lines need your source, and Pointcast reads it where it is:

- **Clipboard** (what Stop does): at Stop, the extension reads the chain's files from your page's own **Vite** dev server (`/src/…?raw`), within 2 s and in memory only; the spec keeps paths, line numbers and one source line per location. With webpack or Next.js the pasted spec has the chain but no `text at:` lines, and the popup says why.
- **MCP server and agent plugins**: the lines are resolved in your local repository, whatever the dev server. `pointcast process` does the same from the CLI.

Nothing is added when the text is written more than once: no line beats a wrong line.
</details>

## Why it helps

A coding agent can't see what "this" is in "make *this* sortable and move *this* next to *that*", and a screen recording doesn't give it the DOM element or the file behind it. In the [evaluation](docs/eval/results-2026-09-27.md), pointing helped exactly where narration alone is ambiguous: two "Export" buttons, two identical cards, a shared `Button` component. A careful, deliberately written request is still more accurate (9