pinpoint

# Pinpoint

[![tests](https://github.com/gowtham012/pinpoint/actions/workflows/test.yml/badge.svg)](https://github.com/gowtham012/pinpoint/actions/workflows/test.yml)
[![licence: MIT](https://img.shields.io/badge/licence-MIT-blue.svg)](LICENSE)
[![stars](https://img.shields.io/github/stars/gowtham012/pinpoint?style=flat)](https://github.com/gowtham012/pinpoint/stargazers)
[![npm](https://img.shields.io/npm/v/pinpoint-bridge.svg)](https://www.npmjs.com/package/pinpoint-bridge)
[![Glama score](https://glama.ai/mcp/servers/gowtham012/pinpoint/badges/score.svg)](https://glama.ai/mcp/servers/gowtham012/pinpoint)

Click an element on your local dev site, write what should change, and your coding agent gets it — with the selector, DOM path, computed styles, React/Vue component chain, source-file hint and a cropped screenshot. No screenshot files piling up in your Downloads folder, no describing "the third button on the left". Building a native app? The same works on the [iOS Simulator](#native-ios-apps-simulator).

It is not only for styling. Say what the thing should *do* — "disable this until the form is valid",
"this should save the draft before closing", "show the error the API returned here" — and the agent
is told to treat the element as where to start, then follow it into the handlers, state, API calls
and backend behind it. [More below](#looks-or-behaviour).

```
Browser (extension) ──POST──▶ pinpoint bridge (127.0.0.1:7331) ──MCP / hooks──▶ Claude Code, Cursor, Codex…
        pins ◀──live events───┘   ~/.pinpoint/annotations.json   └── <repo>/.pinpoint/pending.md (optional)
iOS Simulator ─▶ 127.0.0.1:7331/ios ─┘  (screenshot + accessibility tree instead of the DOM)
```

![Pinpoint on a local dev page: picking an element, writing what should change, and the agent picking it up](docs/demo.gif)

*Click an element, say what should change, your agent gets it. ([full-speed video](docs/demo.mp4))*

## It is not a browser-driving agent

The distinction matters, because the two get filed together and they are opposites:

- **Playwright MCP, agent-browser, computer-use** put the agent in the driver's seat — it navigates,
  clicks and asserts. Good for testing and for browsing on your behalf.
- **Pinpoint keeps you in it.** You click, you say what should change, and the agent gets structured
  context to edit *code* with: a selector, the styles, the component chain, a source-file hint, a crop
  of the element. It never touches the page itself.
- **A pasted screenshot** carries none of that. The agent gets pixels and has to guess which control
  you meant, what it is called, and where it lives in the tree.

So it composes with browser automation rather than competing: point at the thing, let the agent change
it, let your tests drive the browser.

## Quick start

**You need** Node 18+ and a Chromium browser (Chrome, Edge, Brave, Arc, Opera, Vivaldi).

```bash
git clone https://github.com/gowtham012/pinpoint
node pinpoint/bridge/cli.js setup
```

That is the install. `setup` installs its own dependencies first, then asks — every question has a
default, so Enter all the way through is a working setup:

- **which project's UI you want to annotate** — the repo whose files your agent will be editing
- **whether it is a web app or a native iOS app** (or both) — web gets the browser extension; iOS
  gets the [Simulator picker](#native-ios-apps-simulator), and setup checks for Xcode, Maestro and
  Java and says how to get whichever is missing. It installs none of them for you.
- **which agents to wire up** — it detects Claude Code, Cursor and Codex and writes the MCP entry for
  each: `claude mcp add` for Claude Code, `<project>/.cursor/mcp.json` for Cursor,
  `~/.codex/config.toml` for Codex. Existing entries are merged, never replaced, and the TOML file is
  backed up before it is touched.
- **whether to install the Claude Code hooks**, which carry pending notes in with your next message
- **which browser** to load the extension into, listing the ones you actually have (web only)

Then it registers the launcher behind the popup's **Start bridge** button and starts the bridge.
Non-interactive, for a scripted machine: `node pinpoint/bridge/cli.js setup ~/code/my-app --yes`
(add `--for ios` or `--for both`; the default is `web`).

Building only a native iOS app? You can skip the clone: the iOS picker needs no extension.

```bash
npx -y pinpoint-bridge                                      # the bridge, with the picker at /ios
claude mcp add pinpoint -s user -- npx -y pinpoint-bridge mcp   # once, for Claude Code
```

Pinpoint is also listed in the [official MCP Registry](https://registry.modelcontextprotocol.io)
(`io.github.gowtham012/pinpoint`), on [cursor.directory](https://cursor.directory/plugins/pinpoint)
(one-click **Add to Cursor**) and on [Glama](https://glama.ai/mcp/servers/gowtham012/pinpoint). All
of them install the MCP server only — for web pages you still need the extension and bridge from the
clone above.

**The one step that cannot be a command.** Chrome does not let a terminal load an unpacked extension
into your own profile — only the Web Store or an enterprise policy can. So setup opens your browser's
extensions page and puts the `extension/` folder on your clipboard: **Developer mode** on →
**Load unpacked** → paste. The terminal confirms **✓ Pinpoint is live in Chrome** on its own, because
the bridge can see the extension connect.

**Restart Claude Code once** afterwards — it reads MCP servers and hooks when a session starts.

**Then try it.** Open your dev site (or [the demo page](#a-page-to-try-it-on)), press **⌥⇧A** /
**Alt+Shift+A**, click any element, type what should change, **⌘↩** / **Ctrl+Enter**. Type anything at
all in Claude Code and your note arrives with it. If the bar never appears, see
[Troubleshooting](#troubleshooting).

<details>
<summary>Doing it by hand instead, and the Windows / <code>file://</code> notes</summary>

`setup` writes config into your repo and your agent's files. If you would rather do that yourself,
it is four steps — and the order matters: `$PWD` is baked into the MCP entry, so step 2 has to run
from `pinpoint/bridge`, before the bridge takes over the terminal.

```bash
# 1. get it
git clone https://github.com/gowtham012/pinpoint
cd pinpoint/bridge && npm install

# 2. wire up Claude Code (once, from this folder)
claude mcp add pinpoint -s user -- node "$PWD/cli.js" mcp
node cli.js install-hooks ~/code/my-app                  # notes arrive without being asked
node cli.js install-native-host --project ~/code/my-app   # optional: the popup's "Start bridge" button

# 3. load extension/ unpacked at chrome://extensions (Developer mode on)
# 4. start the bridge, and leave it running
node cli.js --project ~/code/my-app
```

Restart Claude Code once after step 2. `--project` is optional: it keeps
`<repo>/.pinpoint/pending.md` current for agents that read a file instead of MCP.

Annotating a page you opened as a `file://` URL? Chrome keeps that off by default. On
`chrome://extensions`, open Pinpoint's **Details** and turn on **"Allow access to file URLs"**, then
reload the page.

On Windows, `$PWD` works in PowerShell and Git Bash but not `cmd.exe` — run `node cli.js --help` and
copy the ready-made `claude mcp add …` line it prints, which carries the full path already. The
shortcuts are `Alt+Shift+A` and `Ctrl+Enter`, and the UI labels them that way. `install-native-host`
is macOS and Linux only; on Windows, start the bridge in a terminal.
</details>

## Which browsers

**Chromium** — Chrome, Edge, Brave, Arc, Opera, Vivaldi. Load unpacked, as above; the test suite
drives headless Chromium, so that is the one continuously verified.
**Safari** — built rather than loaded: `bash tools/make-safari.sh` (needs Xcode).
**Firefox** — not yet.

<details>
<summary>What differs on Safari and what blocks Firefox</summary>

Safari does not take unpacked extensions, so `tools/make-safari.sh` wraps it in a small macOS app —
it converts, builds, and prints the four Safari settings to flip (the important one is
**Develop ▸ Allow Unsigned Extensions**, which resets every time Safari quits). One capability is
missing there: `"world": "MAIN"` content scripts are unsupported, so `inspector.js` cannot read React
fibers or Vue instances, which costs you the **component chain and source-file hint**. Everything else
— picking, regions, comments, pins, screenshots, the bridge, MCP — is unchanged.

Firefox is closer than it was (every script prefers `browser` where it exists), but two manifest
blockers remain: `background: { service_worker }` where Firefox MV3 wants `background: { scripts }`,
and a missing `browser_specific_settings.gecko.id`. Both are fixable, but keeping them honest needs a
Firefox job in CI rather than a claim in a README. Open an issue if you want it.
</details>

## Native iOS apps (Simulator)

A native app has no DOM, so this path reads the **accessibility tree** instead. It works for SwiftUI,
UIKit, React Native and Flutter apps, and needs nothing added to your app.

**You need** Xcode with a booted Simulator and [Maestro](https://maestro.dev) (it reads the tree,
and needs Java — a Homebrew `openjdk` is found even when it is not on your PATH). The bridge keeps one
`maestro mcp` process running for it, so only the first load waits (~10s, while Maestro starts its
on-device driver); every refresh after that takes about a second. Every Maestro on your Mac shares that
one driver, so if another one (an agent's Maestro MCP server, a test run) restarts it, the bridge's next
read fails, and the bridge then starts a fresh Maestro and tries once more. If the first load takes
minutes, close the Claude Code sessions you are not using that have the Maestro MCP server.

Setup asks whether your app is web or iOS, and for iOS opens this page for you. Otherwise, with the
bridge running — `node cli.js` from the clone, or `npx -y pinpoint-bridge` with no clone at all — open
**http://127.0.0.1:7331/ios**.