Skip to main content
ClaudeWave
YawLabs avatar
YawLabs

fetch-mcp

View on GitHub

Comprehensive HTTP fetch MCP server with SSRF protection, HTML-to-markdown, and robots.txt awareness

MCP ServersOfficial Registry2 stars1 forks● TypeScriptMITUpdated today
ClaudeWave Trust Score
95/100
✓ Verified
Passed
  • ✓Open-source license (MIT)
  • ✓Actively maintained (<30d)
  • ✓Clear description
  • ✓Topics declared
  • ✓Documented (README)
Last scanned: 9/16/2026
Install in Claude Code / Claude Desktop
Method: NPX · @yawlabs/fetch-mcp
Claude Code CLI
claude mcp add fetch-mcp -- npx -y @yawlabs/fetch-mcp
claude_desktop_config.json (Claude Desktop)
{
  "mcpServers": {
    "fetch-mcp": {
      "command": "npx",
      "args": ["-y", "@yawlabs/fetch-mcp"]
    }
  }
}
1. Run the command above in your terminal (Claude Code), or paste the JSON config into claude_desktop_config.json (Claude Desktop).
2. Replace any <placeholder> values with your API keys or paths.
3. Restart Claude. The MCP server and its tools appear automatically.
Use cases
Content Creative Dev Tools
About

MCP Servers overview

# @yawlabs/fetch-mcp

A comprehensive HTTP fetch MCP server for AI assistants. Bring-your-own client: runs as a stdio MCP server so any MCP-compatible client (Claude Code, Claude Desktop, Cursor, `mcph`, …) can fetch web content safely.

[![Add to Yaw MCP](https://yaw.sh/yaw-mcp-button.svg)](https://yaw.sh/mcp/install?name=Fetch&command=npx&args=-y%2C%40yawlabs%2Ffetch-mcp&description=HTTP%20fetch%20for%20AI%20agents%20-%20HTML-to-markdown%2C%20reader%20mode%2C%20metadata%2C%20RSS%2C%20sitemaps&source=https%3A%2F%2Fgithub.com%2FYawLabs%2Ffetch-mcp)

One click adds this to your local Yaw MCP config so it's available in every Yaw Terminal session. Or install manually below.

## What it gives the model

| Tool | What it does |
|------|--------------|
| `http_get` / `http_head` / `http_options` | Bare HTTP requests with headers, auth, timeout, size cap, retry |
| `http_post` / `http_put` / `http_patch` / `http_delete` | Write-method HTTP with JSON or raw body |
| `fetch_html_to_markdown` | GET a page and convert to clean markdown (3–8× smaller than raw HTML) |
| `fetch_html_to_text` | GET a page and convert to plain text with block structure preserved |
| `fetch_reader` | Reader-mode extraction — isolates the article body and returns title + markdown |
| `fetch_meta` | Extract `<head>` metadata: title, description, OpenGraph, Twitter cards, JSON-LD, feeds, icons |
| `fetch_links` | Extract every outbound link, resolved to absolute URLs, classified internal/external |
| `fetch_sitemap` | Parse `sitemap.xml` (including gzipped and sitemap-index chaining) |
| `fetch_feed` | Parse an RSS 2.0 or Atom 1.0 feed into entries |
| `fetch_robots` | Parse a site's `robots.txt`, return the verdict for a given path & user-agent |

## Safety

SSRF protection is on by default. The server refuses requests to:

- Loopback (`127.0.0.0/8`, `::1`)
- RFC1918 private ranges (`10/8`, `172.16/12`, `192.168/16`)
- Link-local (`169.254/16`, `fe80::/10`) — including the cloud metadata endpoint `169.254.169.254`
- CGNAT (`100.64/10`)
- Unique-local IPv6 (`fc00::/7`)
- Multicast / broadcast
- IPv4-mapped IPv6 (`::ffff:0:0/96`) re-checked against the IPv4 rules
- Non-`http`/`https` schemes (`file://`, `gopher://`, `javascript:`, …)
- Hostname `localhost` and any `*.localhost`

DNS is resolved once per redirect hop, every returned address is checked, and the verified IP is pinned into the HTTP dispatcher so the subsequent TCP connection dials that exact address — closing the DNS-rebinding TOCTOU window. `Authorization` headers are stripped on cross-origin redirects. A 302 to `http://127.0.0.1` through a public host gets caught. Set `allow_private_hosts: true` per-request when you really do need internal access (e.g. development).

## Install & run

```bash
# One-off (auto-updates each spawn via @latest)
npx -y @yawlabs/fetch-mcp@latest

# Or globally
npm i -g @yawlabs/fetch-mcp
fetch-mcp
```

Requires Node ≥20.

## Configure in Claude Code / Claude Desktop

Add to your client's MCP config (usually `claude_desktop_config.json` or `~/.claude.json`):

```json
{
  "mcpServers": {
    "fetch": {
      "command": "npx",
      "args": ["-y", "@yawlabs/fetch-mcp@latest"]
    }
  }
}
```

Or via `mcph`:

```bash
mcph add fetch
```

## Tool reference

### `http_get`, `http_post`, `http_put`, `http_patch`, `http_delete`, `http_head`, `http_options`

Common parameters:

| Field | Type | Default | Meaning |
|-------|------|---------|---------|
| `url` | string | — | Absolute URL |
| `headers` | object | — | Custom request headers |
| `timeout_ms` | int | `10000` | Request timeout |
| `max_bytes` | int | `5242880` (5 MiB) | Truncate body if larger |
| `max_redirects` | int | `5` | Redirect hops allowed |
| `retries` | int | `0` | Retry count on 408/425/429/5xx with backoff (honors `Retry-After`) |
| `user_agent` | string | `@yawlabs/fetch-mcp/<v>` | `User-Agent` override |
| `basic_auth` | `{username,password}` | — | Injects `Authorization: Basic …` |
| `bearer_token` | string | — | Injects `Authorization: Bearer …` |
| `allow_private_hosts` | bool | `false` | Bypass SSRF block |
| `decode_text` | bool | *auto* | When unset, auto-detects by response Content-Type (text for text/*, JSON, XML, JS, form-urlencoded; binary otherwise). Set explicitly `true` to force text decoding, `false` to force base64 in `body_base64`. |

Body-capable tools (POST/PUT/PATCH/DELETE) also take:

| Field | Type | Meaning |
|-------|------|---------|
| `body` | string | Raw request body |
| `body_json` | any | Structured body — encoded as JSON, `Content-Type: application/json` set automatically |
| `content_type` | string | Overrides `Content-Type` |

Response shape:

```ts
{
  ok: boolean;
  status: number;
  statusText: string;
  url: string;              // final URL after redirects
  headers: Record<string,string>;
  body_text?: string;
  body_base64?: string;     // when decode_text=false
  json?: unknown;           // auto-parsed when response is application/json
  truncated?: boolean;      // set when max_bytes hit
  redirects?: string[];     // chain of intermediate URLs
  duration_ms: number;
  error?: string;
}
```

### `fetch_html_to_markdown`

GET the URL, strip scripts/styles/iframes/svg/canvas plus `<nav>`, `<footer>`, `<aside>`, convert to atx-headed markdown with fenced code blocks and dash bullets. Intended for feeding pages into an LLM without blowing the context budget.

### `fetch_html_to_text`

Same fetch, but emits plain text with block-level structure preserved as newlines. Useful when the model doesn't need markdown formatting.

### `fetch_reader`

Isolates the main article body using, in order: `<article>`, `<main>`, `itemprop="articleBody"`, common CMS class names (`post-content`, `entry-content`, etc.), then `<body>` as fallback. Returns:

```ts
{
  url: string;          // final URL after redirects
  title?: string;       // og:title, then <title>, then <h1>
  byline?: string;      // meta[name=author] / article:author
  wordCount: number;
  markdown: string;     // main content converted to markdown
}
```

### `fetch_meta`

GET a URL and return its head metadata without downloading the full body (caps at 2 MiB by default):

```ts
{
  url: string;
  title?: string;
  description?: string;
  canonical?: string;
  language?: string;
  robots?: string;
  og:      Record<string, string>;       // first value per key (og:title, og:image, og:type, ...)
  twitter: Record<string, string>;       // first value per key
  article: Record<string, string>;       // first value per key
  ogAll:      Record<string, string[]>;  // only keys that appear > once (e.g. multiple og:image)
  twitterAll: Record<string, string[]>;
  articleAll: Record<string, string[]>;
  icons:   Array<{ rel: string; href: string; sizes?: string }>;
  feeds:   Array<{ href: string; title?: string; type?: string }>;   // RSS/Atom
  jsonLd:  unknown[];                    // parsed application/ld+json blocks
}
```

### `fetch_links`

GET a page and return every `<a href>` with text, resolved to absolute URLs. Respects `<base href>`. Skips `#`, `javascript:`, `mailto:`, `tel:`, `data:`, `file:`. Each link is classified `internal` or `external` vs. the page host. Optional `filter`/`dedupe`/`limit`.

### `fetch_sitemap`

Fetch a `sitemap.xml` or sitemap-index and return the URL list:

```ts
{
  sitemaps: string[];       // indexes followed, in order
  urlCount: number;
  truncated: boolean;       // hit max_urls
  urls: Array<{
    loc: string;
    lastmod?: string;
    changefreq?: string;
    priority?: number;
  }>;
}
```

Gzipped `.xml.gz` sitemaps are auto-decompressed. `max_depth` controls how many levels of sitemap-index to follow (default 1). Setting `max_depth: 0` on a sitemap-index returns the index's `childSitemaps` list without fetching any child (useful to discover structure cheaply). Partial failures — one child sitemap 500s while others succeed — are returned under `warnings` rather than aborting the whole call.

### `fetch_feed`

Parse an RSS 2.0 or Atom 1.0 feed:

```ts
{
  kind: "rss" | "atom" | "unknown";
  title?: string;
  description?: string;
  link?: string;
  updated?: string;
  entryCount: number;
  truncated: boolean;       // hit limit
  entries: Array<{
    title?: string;
    link?: string;
    id?: string;
    published?: string;
    updated?: string;
    author?: string;
    summary?: string;
    content?: string;
    categories?: string[];
  }>;
}
```

### `fetch_robots`

Fetches `<origin>/robots.txt`, parses it, and returns:

```ts
{
  robotsUrl: string;
  status: number;
  userAgent: string;
  path: string;
  allowed: boolean;
  matchedRule: string | null;  // the longest-match Allow/Disallow that decided it
  crawlDelay: number | null;   // from the matched group
  sitemaps: string[];          // top-level Sitemap: declarations
  rawRobotsText: string;       // first 512KB
  note?: string;               // only when robots.txt 404s
}
```

A `/robots.txt` that returns 404 means there are no rules, so the verdict is `allowed: true`. The response keeps the same shape: `status: 404`, `matchedRule: null`, `crawlDelay: null`, `sitemaps: []`, `rawRobotsText: ""`, plus the `note`. Any other non-2xx status is returned as an error.

The parser follows Google's rules: longest match wins, `*` is a wildcard segment, `$` anchors the end of the path, specific user-agent group beats the `*` wildcard group when the UA matches (comparison uses the length of the actually-matched agent token, not the group's first agent). Allow beats Disallow on equal-length ties.

## Development

```bash
npm install
npm run build
npm test         # vitest
npm run lint     # biome
npm run typecheck
```

Tests spin up a local loopback HTTP server on `127.0.0.1:0` to exercise the real request/response path — no mocking of HTTP. SSRF tests verify that the default-deny still applies to that local server unless the request opts into `allow_private_hosts`.

## License

MIT © Yaw Labs

## Links

- npm: https
Topics
aiai-agentsclaude-codecursorfetchfetch-mcphtml-to-markdownhttphttp-clientmarkdownmcpmcp-servermetadatamodel-context-protocolopengraphreader-modersssitemapssrfweb-scraping
Frequently asked

What people ask about fetch-mcp

What is YawLabs/fetch-mcp?

+

YawLabs/fetch-mcp is mcp servers for the Claude AI ecosystem. Comprehensive HTTP fetch MCP server with SSRF protection, HTML-to-markdown, and robots.txt awareness It has 2 GitHub stars and its last recorded update is dated 2026-09-15.

How do I install fetch-mcp?

+

You can install fetch-mcp by cloning the repository (https://github.com/YawLabs/fetch-mcp) or following the README instructions on GitHub. ClaudeWave also provides quick install blocks on this page.

Is YawLabs/fetch-mcp safe to use?

+

Our security agent has analyzed YawLabs/fetch-mcp and assigned a Trust Score of 95/100 (tier: Verified). See the full breakdown of passed checks and flags on this page.

Who maintains YawLabs/fetch-mcp?

+

YawLabs/fetch-mcp is maintained by YawLabs. The last recorded GitHub activity is dated 2026-09-15, with 1 open issues.

Are there alternatives to fetch-mcp?

+

Yes. On ClaudeWave you can browse similar mcp servers at /categories/mcp, sorted by popularity or recent activity.

1-click deploy

Deploy fetch-mcp to your cloud

Ship this repo to production in minutes. Each platform spins up its own environment with editable env vars.

VercelRailwayRender
Embeddable badge

Maintain this repo? Add a badge to your README

Drop the badge into your GitHub README to show it's tracked on ClaudeWave. Each badge links back to this page and reflects the live Trust Score.

Featured on ClaudeWave: YawLabs/fetch-mcp
[![Featured on ClaudeWave](https://claudewave.com/api/badge/yawlabs-fetch-mcp)](https://claudewave.com/repo/yawlabs-fetch-mcp)
<a href="https://claudewave.com/repo/yawlabs-fetch-mcp"><img src="https://claudewave.com/api/badge/yawlabs-fetch-mcp" alt="Featured on ClaudeWave: YawLabs/fetch-mcp" width="320" height="64" /></a>
Related

More MCP Servers

fetch-mcp alternatives
n8n-io
n8n
today

Fair-code workflow automation platform with native AI capabilities. Combine visual building with custom code, self-host or cloud, 400+ integrations.

204.5k60.7kTypeScript
MCP ServersaiapisInstall
open-webui
open-webui
today

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

152.2k22.3kPython
MCP ServersaillmInstall
google-gemini
gemini-cli
today

An open-source AI agent that brings the power of Gemini directly into your terminal.

107k14.6kTypeScript
MCP Serversaiai-agentsInstall
koala73
worldmonitor
today

Real-time global intelligence dashboard. AI-powered news aggregation, geopolitical monitoring, and infrastructure tracking in a unified situational awareness interface

86.5k13.1kTypeScript
MCP ServersagentaiInstall
D4Vinci
Scrapling
yesterday

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ

81.2k8.2kPython
MCP Serversaiai-scrapingInstall
netdata
netdata
today

The fastest path to AI-powered full stack observability, even for lean teams.

80.5k6.6kGo
MCP ServersaialertingInstall