vibe-testing

# vibe-testing

[![npm version](https://img.shields.io/npm/v/vibe-testing.svg)](https://www.npmjs.com/package/vibe-testing)
[![Node.js](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](https://nodejs.org)
[![MCP](https://img.shields.io/badge/MCP-compatible-blue.svg)](https://modelcontextprotocol.io)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![CI](https://github.com/AishwaryShrivastav/vibe-testing/actions/workflows/ci.yml/badge.svg)](https://github.com/AishwaryShrivastav/vibe-testing/actions)

**Code-aware browser testing for AI coding agents.**

vibe-testing reads your codebase so tests use your real routes and field names, runs them in a real Playwright browser, remembers what broke, and tells you what your last change fixed or regressed. It works as an MCP server that gives your editor (Claude Code, Cursor, Windsurf, VS Code Copilot, Roo Code) 13 testing tools, or as a standalone CLI.

```bash
cd /path/to/your/project
npx vibe-testing@latest init
```

Then open your editor and say:

> "Scan this codebase and test it against http://localhost:3000"

---

## Why not just a browser tool

Playwright MCP gives your agent hands. vibe-testing gives it a testing workflow: code-derived scenarios, memory across runs, and a report. Two things a stateless browser tool cannot do:

**1. Run it twice and it tells you what you broke.** Every run writes `.vibe/run-snapshot.json` and diffs it against the previous run. Every scan writes `.vibe/route-manifest.json` and diffs your routes. The second run prints regressions and fixes instead of a wall of results:

```
Changes since last run
  Fixed:      /login
  Regression: /checkout
  New:        /admin/users
```

The same diff reaches your editor as `snapshot_diff` on `run_full_test` and `run_converge`, and as `route_changes` on `scan_codebase`, so the agent can flag "checkout broke after that commit" without anyone scrolling a report. Flaky routes, working selectors, and measured timeouts are also remembered between runs.

**2. Zero LLM calls inside the tool.** Pass/fail verification is heuristic: URL changes, toast detection, API errors. Your editor's model decides what to test; vibe-testing does the browsing and checking. No API key, no per-run cost beyond the editor subscription you already pay for.

---

## Contents

- [How it works](#how-it-works)
- [MCP setup (per editor)](#mcp-setup)
- [MCP tools reference](#mcp-tools-reference)
- [Recommended workflow](#recommended-workflow)
- [init command](#init-command)
- [CLI commands](#cli-commands)
- [VIBE.md, project guidance](#vibemd-project-guidance)
- [Configuration (vibe.config.json)](#configuration)
- [Supported frameworks](#supported-frameworks)
- [Memory and regression detection](#memory-and-regression-detection)
- [FAQ](#faq)

---

## How it works

```
npx vibe-testing@latest init
        |
Registers 13 MCP tools in your editor
        |
You ask: "Test the checkout flow"
        |
AI calls: scan_codebase -> get_context("checkout") -> login -> explore_page -> execute_scenario -> generate_report
        |
HTML report opens in browser with screenshots of every step
```

No test cases to write. The AI reads your source code to understand real field names and routes, opens a browser, tests everything, and shows you what's broken.

`init` also:

- Detects which AI editors you have installed
- Registers vibe-test in global editor configs (`~/.claude/settings.json`, `~/.cursor/mcp.json`, and so on) so the tools are available in every project, every session
- Creates project-level MCP configs and AI instruction files
- Auto-detects your app's URL (reads `.env`, `vite.config`, framework defaults)
- Creates `VIBE.md` (edit with your test credentials) and `vibe.config.json`
- Installs the matching Playwright Chromium build

---

## MCP setup

### Option 1: automatic (recommended)

```bash
npx vibe-testing@latest init
```

Detects and configures all installed editors. Done.

### Option 2: manual per editor

#### Claude Code

Add to `~/.claude/settings.json` (global, works in every project):

```json
{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}
```

Or add to `.mcp.json` in your project root (project-level only):

```json
{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}
```

#### Cursor

Add to `~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project):

```json
{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}
```

#### Windsurf

Add to `~/.codeium/windsurf/mcp_config.json`:

```json
{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}
```

#### VS Code (GitHub Copilot)

Add to `.vscode/mcp.json` in your project:

```json
{
  "servers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}
```

#### Roo Code / Cline

Add to `.roo/mcp.json`:

```json
{
  "mcpServers": {
    "vibe-test": {
      "command": "npx",
      "args": ["-y", "vibe-testing@latest", "--mcp"]
    }
  }
}
```

#### From local build (development)

```json
{
  "mcpServers": {
    "vibe-test": {
      "command": "node",
      "args": ["/path/to/vibe-testing/dist/mcp-server.js"]
    }
  }
}
```

---

## MCP tools reference

13 tools available to your AI editor after setup:

| Tool | When to call | Returns |
|------|-------------|---------|
| `scan_codebase` | Always first. Reads source code, finds routes/forms/tests/gaps | Routes, forms, coverage map, generated scenarios, `route_changes` since last scan |
| `get_context` | Before writing test steps. Returns source files for a feature | Actual source code with real field names and selectors |
| `login` | When app requires authentication | Post-login screenshot, token state, API calls observed |
| `scan_page_elements` | To see all interactive elements on a page | Element list with selectors plus page screenshot |
| `explore_page` | Broad "does everything work?" testing | Interaction results, API calls, errors, screenshot |
| `execute_scenario` | Run specific test steps | Step-by-step logs plus screenshots |
| `get_coverage` | View coverage map and untested routes | Coverage entries, gaps, available scenarios |
| `suggest_tests` | Find coverage gaps after exploration | Prioritized, ready-to-run scenarios with steps |
| `take_screenshot` | Quick visual verification | Screenshot of any URL |
| `generate_report` | Build HTML report (auto-opens) | Report path plus summary |
| `run_full_test` | One-shot: scan, execute, explore, report | Full results plus `snapshot_diff` vs last run |
| `run_converge` | Iterative testing until thresholds | Summary across all rounds plus `snapshot_diff` vs last run |
| `cleanup` | Close browsers, free resources | - |

### Tool inputs

**`scan_codebase`**
```json
{
  "codebase_path": "/path/to/project",
  "url": "http://localhost:3000",
  "mode": "deep"
}
```

**`get_context`**
```json
{ "feature": "login" }
{ "feature": "/checkout" }
{ "feature": "user profile form" }
```

**`login`**
```json
{
  "email": "test@example.com",
  "password": "TestPass123!",
  "login_url": "/login"
}
```

**`scan_page_elements`** / **`explore_page`**
```json
{
  "route": "/dashboard",
  "authenticated": true
}
```

**`execute_scenario`**
```json
{
  "scenario": {
    "id": "create-item",
    "name": "Create a new item",
    "route": "/items",
    "steps": [
      { "action": "navigate", "url": "/items", "description": "Open items page" },
      { "action": "click", "selector": "text=Add Item", "description": "Open create form" },
      { "action": "fill", "selector": "[name='title']", "value": "Test Item", "description": "Fill title" },
      { "action": "fill", "selector": "[name='description']", "value": "Test description", "description": "Fill description" },
      { "action": "click", "selector": "button[type='submit']", "description": "Submit form" }
    ],
    "expected_outcome": "New item appears in the list",
    "requires_auth": true
  }
}
```

Step actions: `navigate`, `fill`, `click`, `select`, `wait`, `assert`, `upload`

CLI and MCP use the same scenario runner. An `assert` step evaluates every
supplied condition:

- `selector`: the target must be visible. CSS, `text=`, `label=`, and
  `placeholder=` are supported; the named locators match exactly.
- `value`: visible text must contain this string, on the selected element or on
  the page body when no selector is supplied.
- `url`: the current URL must exactly match this absolute URL or path resolved
  against the configured base URL, including query and fragment.

For example:

```json
{ "action": "assert", "selector": "#confirmation", "value": "Saved", "url": "/settings", "timeout": 5000, "description": "Check saved settings" }
```

These checks wait up to the step timeout (default 15000 ms per check). A failed
assertion stops the scenario, records a failed step and reason, and returns
`status: "fail"`; MCP also sets `isError: true`. Explicit conditions are used as
the verdict after the existing authentication and form API checks. The
`description` and `expected_outcome` prose are not executable assertions.
Without `selector`, `value`, or `url`, `assert` only checks page health: at least
10 characters of visible body text and no nonempty visible error indicators.
This compatibility check does not prove a described business outcome.

`upload` requires a file-input `selector` and a nonempty file path in `value`.
Relative paths resolve against the codebase root. Hidden file inputs are
supported. A missing file, directory, missing target, or target that is not a
file input returns `status: "error"`. The action selects one file using
Playwright; add an assertion of the resulting UI to verify application-side
proc