testivai-oss

# TestivAI Open Source

[![@testivai/witness](https://img.shields.io/npm/v/@testivai/witness.svg?label=%40testivai%2Fwitness)](https://www.npmjs.com/package/@testivai/witness)
[![@testivai/witness-playwright](https://img.shields.io/npm/v/@testivai/witness-playwright.svg?label=%40testivai%2Fwitness-playwright)](https://www.npmjs.com/package/@testivai/witness-playwright)
[![@testivai/witness-webdriverio](https://img.shields.io/npm/v/@testivai/witness-webdriverio.svg?label=%40testivai%2Fwitness-webdriverio)](https://www.npmjs.com/package/@testivai/witness-webdriverio)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)

**Local-first visual regression testing SDKs for modern web applications.**

This is the home of TestivAI. It contains everything you need to capture, diff, and report visual regressions **fully locally** — MIT-licensed, no account, no server.

> **[See a live report →](https://testivai.github.io/testivai-example/)** — a real TestivAI OSS report rendered in your browser, straight from CI. No install, no signup.

[![Real TestivAI report — style-only-change verdict, selector-attributed regions, heatmap diff](./docs/assets/oss-report.png)](https://testivai.github.io/testivai-example/)

## Why TestivAI?

Pixel-only visual testing drowns you in false positives — a font re-hint or an anti-aliasing shift across machines lights up as a "change," and you spend your time re-approving noise.

**TestivAI pairs the screenshot with a snapshot of the page DOM** (every adapter and the crawler capture one by default; the experimental `testivai run` sidecar does not) and, from most test-framework adapters, a digest of every element's computed styles. When pixels differ but the DOM and the computed styles are identical and the diff is small and scattered, the report labels the diff **likely render noise** instead of crying wolf. When the DOM and styles match but the diff sits in one area, it says that too: the style check covers a fixed list of properties, not size, overflow, transform or images, so that is a bad capture or a real change, not noise. It still shows the diff; it tells you what it most likely is. When only the styles changed, it says so: a style-only change is real. When the styles could not be compared, it says *styles not compared* and does not call the diff noise, because a stylesheet-only change is not ruled out. When the DOM actually changed, you see exactly what (`2 added, 1 removed`). That layered signal is the difference between a flaky test wall and a report you trust.

- **Fully local, no account** — captures, diffs, and a self-contained HTML report all stay on your machine.
- **DOM + style-aware noise hint** — tells likely render jitter from real changes, and catches the stylesheet-only case. When both captures carry an element map (the Playwright JS/TS, WebdriverIO, Selenium and Ruby adapters record one), an identical DOM with changed computed styles reads as "Styles changed on `button.cta`", never as noise. Without an element map on both sides (the `testivai witness <url>` crawler, playwright-python, playwright-java), computed styles are not compared, and an identical-DOM diff reads "styles not compared", not "likely render noise".
- **Auditable masks & region-level diffs** — exclude dynamic areas (selectors or coordinates) with the masked region hatched in the diff, and get "3 changed regions" with bounding boxes instead of a raw pixel percentage.
- **Element attribution & exact shift detection** — the report names *which element* changed ("`div.card:nth-of-type(2)` shifted +8px vertically — content unchanged") and spots the injected-banner case ("everything below y=80 moved +24px"), derived from layout, not pixel guesswork. No local-first tool does this.
- **First-class adapters** — Playwright (TS/JS **and Python**, Java experimental) and WebdriverIO, using each framework's native APIs; every language shares one set of baselines and one report.
- **Mobile web, no extra package** — Playwright device emulation works through the same `witness()` call: one project per device profile, baselines keyed per device automatically, and the DOM and computed-style attribution survives emulation, so a change behind a `@media` query is caught on mobile and correctly passes on desktop. See the [mobile web guide](docs/guides/mobile-web.md).
- **PR-native workflow** — a GitHub Action posts the diff and approves baselines from a `/testivai approve` comment.
- **No lock-in** — MIT license, baselines live in your git, and `results.json` is a semver-governed public contract.

> **Evaluating this for a team?** [Maintenance & roadmap](docs/maintenance.md)
> covers who builds it, the release cadence, and what happens to your setup if
> maintenance ever stops.

> **Already using Playwright's `toHaveScreenshot()`?** It's good, and if a pixel
> diff answers your question you should keep using it. Here's an honest look at
> [what TestivAI adds and when it isn't worth the dependency](docs/vs-playwright-builtin.md).

## Eyes for your coding agent

If an AI agent (Claude Code, Cursor, Copilot, …) writes your UI code, someone still has to check what the UI *looks like* — and it shouldn't be you, one screenshot at a time. TestivAI is built to be that check:

- **No account, no API key, no network** — an agent can run it inside any sandbox without you provisioning secrets.
- **Machine-readable output** — every run writes `visual-report/results.json` (a [semver-governed schema](./docs/how-it-works.md)) with per-snapshot diff percentages and DOM change summaries, so an agent can read the result and self-correct.
- **Noise-aware verdicts** — the DOM and computed-style checks tell the agent whether a pixel diff is *likely render noise* (identical DOM, styles compared and identical, a small and scattered diff), a *concentrated or large diff* (identical DOM and styles, but the diff sits in one area or covers over 1% of pixels: a bad capture or a change the style check does not cover), a *style-only change* (identical DOM, restyled elements), a *real structural change* (`2 added, 1 removed`), or *styles not compared* (identical DOM, but no style data to compare, so a stylesheet-only change is not ruled out). It doesn't chase anti-aliasing ghosts, and it doesn't label a diff it couldn't check as noise.
- **Explanations, bring-your-own-model** — the MCP `explain_snapshot` tool hands your agent layered evidence (which selectors shifted vs changed, whole-page shift detection, style-only changes) and your model writes the narrative: *"card #2 shifted +24px — likely the banner injected above it."* No hosted AI service in the loop.
- **Human approval stays in the PR** — the agent iterates locally; you approve baselines with one `/testivai approve` comment.

Paste this into your project's `AGENTS.md` / `CLAUDE.md` to wire it up (full guide with MCP setup, a real agent transcript, and the approval rule: [docs/guides/ai-agents.md](./docs/guides/ai-agents.md)):

```markdown
## Visual verification
After changing any UI code, run `npx playwright test` (TestivAI captures
screenshots automatically), then read `visual-report/results.json`.
- `status: "changed"` with `dom.changed: true` → describe the DOM summary and
  ask whether the change is intended before approving.
- `status: "changed"` with `dom.changed: false` and `dom.styleCheck: "mismatch"`
  → style-only change; real, not noise. Compare `dom.styleChanges` with what
  you intended.
- `status: "changed"` with `dom.noiseHint: true`, `dom.styleCheck: "match"`,
  `diffPercent` at most 1, and no `regions` or regions spread out (their
  combined box covers more than a third of the page) → likely render noise;
  mention it but don't block.
- `status: "changed"` with `dom.noiseHint: true` and `dom.styleCheck: "match"`
  but every region within a third of the page, or `diffPercent` over 1 → not
  noise: a bad capture or a change the style check does not cover (size,
  overflow, transform, images). Look at the diff image before approving.
- `status: "changed"` with `dom.noiseHint: true` and `dom.styleCheck`
  `"unavailable"` or missing → styles not compared: could be render noise or a
  stylesheet-only change. Don't treat it as noise: look at the diff image before approving.
- Never run `testivai approve` yourself — baseline approval is a human decision.
```

## Packages

Live versions are shown by the badges at the top of this README.

| Package | Description |
|---|---|
| [`@testivai/witness`](./packages/witness) | Core SDK: CLI, local diffing, baselines, HTML report generator |
| [`@testivai/witness-playwright`](./packages/playwright) | Playwright reporter/adapter built on top of `@testivai/witness` |
| [`@testivai/witness-webdriverio`](./packages/webdriverio) | WebdriverIO service + capture function (local mode) |
| [`@testivai/witness-selenium`](./packages/selenium) | Selenium WebDriver capture adapter (Python/Java Selenium live in `python/` and `java/`) |
| [`@testivai/mcp`](./packages/mcp) | MCP server — visual results + diff images for AI coding agents |
| [`testivai` (PyPI)](./python) | Python adapter for playwright-python + pytest plugin — same baselines & report |
| [`ai.testiv:testivai`](./java) | Java adapter for playwright-java + JUnit 5 extension (experimental) |
| [`testivai` (RubyGems)](./ruby) | Ruby adapter for Capybara / RSpec / Cucumber — same baselines & report |

Plus:
- [`action/`](./action) — GitHub Action for PR-based visual approvals
- [`examples/`](./examples) — minimal real-world example projects
  - [`examples/site-check/`](./examples/site-check) — an hourly GitHub Actions check that fails when a page is up but broken (stylesheet, script or font missing, JavaScript error, blank page), with one issue per outage. testiv.ai runs it on itself.
- [`docs/`](./docs) — public documentation
- [`e2e/`](./e2e) — OSS smoke E2E test suite

## No test suite? One command.

AI-built and vibe-coded apps (Lovable, Bolt, v0, ...) usually ship with zero
t