groundhog

# Groundhog

[![Conformance](https://github.com/dmytrome/groundhog/actions/workflows/conformance.yml/badge.svg)](https://github.com/dmytrome/groundhog/actions/workflows/conformance.yml)
[![Glama](https://glama.ai/mcp/servers/dmytrome/groundhog/badges/score.svg)](https://glama.ai/mcp/servers/dmytrome/groundhog)

**Web search, read and research for AI agents — through a real, stealth-patched Chrome.**
Groundhog is an [MCP](https://modelcontextprotocol.io) server that finds pages, reads them,
and researches across them, returning clean Markdown a model can trust: text no human could
see is **stripped by default before the model reads it**, every source comes back with a
**provenance receipt**, and a real browser reads pages that block plain fetchers — without
the SSRF holes of naive fetch tools.

```text
agent / crawler  ──MCP──▶  Groundhog (search, read_url, research)  ──CDP──▶  stealth Chrome  ──▶  the web
```

## Measured against other fetch layers

Twenty-eight pages, each carrying text a human reader cannot see, plus three controls
carrying none. A case passes when the payload does not reach the model **and** the article
text still does — a fetcher that returns nothing contains every payload and is useless.

| | contained | kept the article | reported it | false positives |
| --- | --- | --- | --- | --- |
| `requests` + trafilatura | 25/28 | 28/28 | 0/28 | 0/3 |
| Scrapling (HTTP) | 16/28 | 28/28 | 0/28 | 0/3 |
| Jina Reader | 15/28 | 28/28 | 0/28 | 0/3 |
| Firecrawl | 7/28 | 28/28 | 0/28 | 0/3 |
| **Groundhog** | **28/28** | **28/28** | **25/28** | **0/3** |

Groundhog reports 25 of the 28 rather than all of them. The other three — a hidden input
value, a meta description, an `iframe srcdoc` — are contained because no extractor ever
collected them, and naming text that was never on its way to the model would describe work
nothing did.

Containment happens by accident all the time — an article extractor prunes a hidden `<div>`
because its heuristics dislike it, not because anything asked whether a reader could see
it. That is why the last column matters: **nothing else tested tells the caller that
anything was removed**, so an empty threat list and a page with nothing hidden look
identical.

The corpus is published at
[dmytrome.github.io/groundhog](https://dmytrome.github.io/groundhog/), so any fetcher can
be measured against the same pages. Harness, per-case results and the limits of what this
proves are in [`benchmark/`](benchmark/) — including that hosted services are a snapshot of
the date recorded in [`RESULTS.md`](benchmark/RESULTS.md).

## Quick start

Add Groundhog to your MCP client — that's it. On the first fetch, Groundhog pulls and
starts the stealth-browser container for you (Docker or Podman required); no repo checkout,
no manual steps. When the default (non-compose) auto-start path has to run, any stale
container named `groundhog-browser` is removed first; a reachable browser is never touched.

Claude Code:

```bash
claude mcp add groundhog -- uvx groundhog-mcp
```

Claude Desktop / Cursor / Windsurf (`claude_desktop_config.json` or equivalent):

```json
{
  "mcpServers": {
    "groundhog": {
      "command": "uvx",
      "args": ["groundhog-mcp"]
    }
  }
}
```

`uvx` fetches `groundhog-mcp` from PyPI on first run. The first fetch pulls the browser
image (once, a few minutes); later fetches are instant. No container runtime? The `status`
tool and any error say how to install one — or point `CDP_URL` at a hosted browser for
zero-install use.

**Prefer to manage the browser yourself?** Start it and Groundhog will just use it:

```bash
docker run -d --rm --name groundhog-browser --shm-size 512m \
  -p 127.0.0.1:9222:9222 -- ghcr.io/dmytrome/groundhog:latest
# or, from a repo checkout: docker compose up --build -d
curl -s http://localhost:9222/json/version    # CDP is live
```

Set `GROUNDHOG_AUTO_START_BROWSER=false` to disable auto-start. To run the MCP server from
source: `cd mcp && uv sync && uv run groundhog-mcp`.

All four tools are annotated `readOnlyHint`, which is what lets a client run them without a
per-call confirmation. That describes what they do to *your data*: nothing is written, and no
remote state is changed. Worth knowing, because it is the one exception: with auto-start on,
the first call may pull and run the browser container, and remove an **unreachable** container
named `groundhog-browser` first. A reachable one is never touched, and
`GROUNDHOG_AUTO_START_BROWSER=false` turns the whole path off.

## What makes it different

- **Hidden text is stripped before the model reads it.** Groundhog renders a real DOM, so it
  can judge what a *human* would actually see and strip what they could not, reporting each
  occurrence in `threats`. A strong heuristic, not a proof — see
  [the limits of hidden-text detection](#limits-of-hidden-text-detection), and
  [the benchmark](#measured-against-other-fetch-layers) for how that compares. The twelve signals,
  the `threats` caveat and the `include_hidden` exception are documented under `read_url`.
- **Every source carries a receipt.** SHA-256 hash of the extracted content, canonical URL,
  language, word count, and author/date when the page declares them — so a downstream claim
  traces back to exactly what was read. `read_url` returns the fetch time alongside it as
  `fetched_at`.
- **Safe by default.** The SSRF guard resolves each host before navigating and refuses to
  return content from a URL that redirects into a private address. Read-only, with per-domain
  rate limiting. This matters most in `research`, where a *third party* chooses the URLs.
  See [Security](#security) for the full blocklist and the guard's limits.
- **No automation tell.** Puppeteer/Playwright/Selenium enable the CDP `Runtime` domain,
  which anti-bots detect (`isAutomatedWithCDP`). Groundhog drives the browser over raw CDP
  and never enables `Runtime`/`Console`, so that signal is absent — a clean session that
  full automation libraries can't produce over `connect_over_cdp`.
- **A real fingerprint.** It's real Chrome, run headful under Xvfb (no `HeadlessChrome`
  token) — authentic TLS/HTTP2 fingerprint, real WebGL/canvas — not a Python HTTP client,
  so fingerprint-driven blocks go away and cheap proxies work where they otherwise wouldn't.
- **No model, no API key.** `research` returns extracts, not summaries; your agent does the
  synthesis. Self-hosted and MIT — the pages you fetch never leave your infrastructure.

## Tools

### `read_url(url, format="markdown", max_tokens=None, query=None, include_hidden=False)`

Fetches a page and returns clean content plus provenance.

| Key          | Meaning                                                                                          |
| ------------ | ------------------------------------------------------------------------------------------------ |
| `markdown`   | Extracted content (article-first, falls back to full text); `format` may be `markdown` or `text` |
| `title`      | Page title                                                                                       |
| `url`        | The URL you asked for                                                                            |
| `final_url`  | The URL after redirects (re-checked against the SSRF guard). Never rewritten: if the page's own final URL is unusable, the requested URL is reported and a `final_url_suppressed` threat says so |
| `fetched_at` | UTC ISO-8601 timestamp                                                                           |
| `status`     | What actually came back: `ok`, or `challenge` / `blocked` / `rate_limited` / `not_found` / `server_error` / `unsupported_content` when the content is not the page you asked for — so a Cloudflare interstitial, a 403, or a PDF is not read as if it were the real page. `blocked` covers `401`/`403` and every other 4xx that serves an error page (`451`, `400`, `405`…), with the exact code in `http_status`. `unknown` means no response — or no usable status — was observed for the document that was read: it is reported rather than assumed to be fine. A `challenge` is recognised from vendor **mitigation markers** — a header that exists only to announce it, or a request for an asset only a challenge loads — so it works whatever language the page is in; page wording is a last resort and only counts on a page too empty to be content. See [the limits of block detection](#limits-of-block-detection) The verdict describes the document the text came from, so a page that redirects client-side (meta-refresh, `location.href`) is judged on where it landed, not where it started |
| `http_status`| The top-level response's HTTP status code, or `null` when it could not be observed              |
| `truncated`  | Whether the content was cut to fit the token budget                                              |
| `threats`    | Signals detected: hidden-CSS nodes and invisible-character classes; empty when none found |
| `matches`    | When `query` is set: ranked passages with `heading`, `offset`, and `score` for citation          |
| `provenance` | Content hash, canonical URL, language, word count, and author/date metadata when present         |

Because Groundhog renders a real DOM, it can evaluate computed styles. Text invisible to
humans is **stripped by default** and each occurrence reported in `threats` with its signal
type and a short excerpt: `display:none`/`visibility:hidden`, `content-visibility: hidden`
(the subtree is skipped from layout while the element keeps an ordinary box, so no other
signal sees it), `opacity ≤ 0.05`, `font-size < 4 px`, zero-size elements, an element that generates no box
of its own (`display: contents`) whose contents render nothing, the sub-pixel box
used by `.sr-only`/`.visually-hidden` accessibility utility classes (a pattern attackers now
mimic), the legacy `clip: rect(...)` hiding technique, fully transparent text color, text
color matching the background color (near-1