frugal

# frugal

**The open routing layer for AI tools.**

Frugal is an MCP server that sits between your agent and its tool
providers. You describe the job — a search, an extraction, a render, or
just an intent — and Frugal decides how to complete it, routing each call
per policy: cheapest capable provider by default, fastest or premium when
you say so, pinned or denied when compliance says so, with automatic
failover when a provider errors or comes up empty. Every response carries
the decision — `provider_used`, `cost_usd`, and (via `frugal__execute`) a
one-line reason — so you can audit why each call went where it did.

Works with any model. One Go binary. Your keys. No account. Self-host
everything, no lock-in. Source-available (BUSL 1.1 → Apache 2.0).

Install to first intelligently routed tool call: under five minutes.

[frugal.sh](https://frugal.sh)

## Install

```bash
curl -fsSL https://frugal.sh/install | bash
frugal mcp install
```

The first command drops the binary in your `$PATH`. The second auto-detects
Claude Desktop, Cursor, AnythingLLM, and Claude Code and merges `frugal`
into each configured MCP server list.

### AnythingLLM

`frugal mcp install` finds AnythingLLM Desktop on its own and merges
`frugal` into `<storage>/plugins/anythingllm_mcp_servers.json`. For a
self-hosted or Docker AnythingLLM, point the installer at that instance's
storage directory (the same path its own `STORAGE_DIR` uses):

```bash
ANYTHINGLLM_STORAGE_DIR=/path/to/anythingllm/storage frugal mcp install --client anythingllm
```

Restart AnythingLLM, then look for **frugal** under Agent Skills → MCP
Servers. The tools load for agent sessions, so call them from an `@agent`
chat. In Docker, the `command` path must resolve *inside* the container —
mount the `frugal` binary in, or run `frugal mcp serve --http` on the host
and register it as a `streamable` server instead.

### Streamable HTTP and MCP 2026-07-28

`frugal mcp serve --http ADDR` serves Streamable HTTP for remote clients and
Docker setups. It speaks the current MCP revision, 2026-07-28, which removed
protocol-level sessions and the `initialize` handshake, as well as the
session-based revisions before it. Both kinds of client connect to the same
endpoint:

- A client speaking 2026-07-28 or newer is served statelessly: no
  `Mcp-Session-Id`, every POST self-contained, `server/discover` answered
  with the supported versions and capabilities.
- An older client (2025-11-25 and earlier) keeps its session and optional
  GET event stream, so nothing changes for the hosts you already run.

Pass `--stateless` to drop sessions for every client, which is what you want
when several `frugal` replicas sit behind a plain load balancer: any replica
can answer any request. Older clients still work in that mode; they only lose
the optional GET stream (declined with 405, as the spec allows).

Catalog results (`tools/list`, `server/discover`) carry a one-hour `ttlMs`
hint with `cacheScope: public`, so spec-current clients stop re-fetching the
tool list on every turn. The catalog is fixed for the life of the process, so
a restart with a new config is the only thing that changes it.

```bash
FRUGAL_AUTH_TOKEN=... frugal mcp serve --http 0.0.0.0:8765 --stateless
```

## Try it now (no keys)

Search and extract work out of the box: **You.com free tier**
(`youcom-free`: the same major web index as the keyed You.com provider,
reached through You.com's hosted MCP endpoint with no key and no account,
about 100 queries per day), **Marginalia** (free index of the indie /
non-commercial web), **Wikipedia** (free Wikimedia REST search),
**go-readability** (free, pure-Go local extractor), and **Jina Reader**
(free hosted render for pages that paint their content with JavaScript)
ship enabled with zero configuration. This is the default `cheap` policy
at work — free and local rungs first, failover when a provider comes up
empty. Captured from a live zero-key run:
```
frugal__search {"query": "AI agent framework comparison", "max_results": 3}

  result › {
    "provider_used": "youcom-free",
    "cost_usd": 0,
    "latency_ms": 177,
    "results": [
      {"title": "Top 12 AI Agent Frameworks Comparison", "url": "https://www.nimbleway.com/blog/top-ai-agent-frameworks-comparison", ...},
      {"title": "AI Agent Frameworks Compared: LangGraph, CrewAI, AutoGen", "url": "https://pecollective.com/blog/ai-agent-frameworks-compared/", ...},
      {"title": "Comparing Open-Source AI Agent Frameworks - Langfuse", "url": "https://langfuse.com/blog/2025-03-19-ai-agent-comparison", ...}
    ]
  }
```

Once the free tier's daily allowance is spent it rate-limits, the router
cools it down, and the chain keeps serving from the narrower free
indexes. The same query on that fallback path, also captured live:

```
frugal__search {"query": "AI agent framework comparison", "max_results": 3}

  stderr › search zero hits; falling back  provider=marginalia latency_ms=529
  result › {
    "provider_used": "wikipedia",
    "cost_usd": 0,
    "latency_ms": 954,
    "results": [
      {"title": "Gemini Enterprise Agent Platform", "url": "https://en.wikipedia.org/wiki/Gemini_Enterprise_Agent_Platform", ...},
      {"title": "Perplexity AI", "url": "https://en.wikipedia.org/wiki/Perplexity_AI", ...},
      {"title": "Agentic commerce", "url": "https://en.wikipedia.org/wiki/Agentic_commerce", ...}
    ]
  }
```

Honest limits: `youcom-free` is a general web index but is capped at
roughly 100 queries a day per IP; Marginalia is genuinely good for
essays, docs, blogs, and niche technical writing, and weak on mainstream
news and product pages; Wikipedia covers entities and reference topics.
Zero-key mode is a real workflow for research and local extraction, but
it is not unlimited. One env var changes that: `SEARXNG_URL` (free,
self-hosted), `SERPER_API_KEY` ($0.001/call), or `YDC_API_KEY` (the keyed
You.com tier, no daily cap) gives the chain a stronger rung to fall to.
Prefer to keep queries off You.com entirely? Add `youcom-free: {enabled:
false}` under `search_providers` and the chain is Marginalia and
Wikipedia again.

## Routing policies

Declare per capability how the provider chain is ordered, in
`~/.frugal/config/models.yaml`:

```yaml
routing:
  search:
    strategy: fast            # cheap (default) | fast | premium
    deny: [youcom]            # never called — not by fallback, not even pinned
  extract:
    order: [firecrawl, goreadability]  # explicit preference; unlisted providers still fall back
```

- **cheap** (default) — effective cost ascending, quota-aware; automatic
  failover up the ladder.
- **fast** — ordered by your machine's own observed latency (from the
  local usage ledger, successful calls only). Falls back to cost order
  until enough history exists. Not a live probe.
- **premium** — prefers your premium-priced providers, list price
  descending.
- **order** — an explicit preference list. It's a prefix, not a
  whitelist: unlisted providers still serve as fallback.
- **deny** — providers that must never be called. Enforced on pinned
  calls too; this is the honest privacy knob (e.g. deny every paid
  provider and nothing leaves your free/local rungs).

Every call chain logs which policy ran, and `frugal__execute` returns it
in the response.

## Spend caps and rate-limit cooldown

Two guardrails cap what a provider can cost you, enforced in the routing
layer alongside the policy above:

```yaml
search_providers:
  youcom:
    api_key_env: YDC_API_KEY
    cost_per_call: 0.005
    daily_budget_usd: 2.50   # skip youcom once it has spent $2.50 today

routing:
  cooldown: 90s              # after a 429, skip that provider for 90s (default 60s)
```

- **daily_budget_usd** (per provider, any of the three tables) — once a
  provider's spend for the current UTC day reaches this cap, the router
  skips it and falls through to the next provider in the chain; a call
  that pins it by name errors. Counters reset at UTC midnight. Zero or
  absent means no cap. The same provider name under two capability tables
  gets an independent budget.
- **cooldown** (top level under `routing:`) — when a provider returns a
  rate limit (HTTP 429), it is fenced off for this long so the chain
  stops hammering it. A Go duration string like `90s` or `2m`; an invalid
  value warns at startup and falls back to the 60s default. Applies to
  every provider, capped or not.

When a guardrail skips a provider it is noted in the routing trace
(`; budget: skipped youcom (...)`) and logged at Warn. If every provider
in a chain is over budget or cooling down, the call fails with a clear
message rather than silently doing nothing.

## Per-call deadline: `call_timeout`

Each provider client has its own request timeout (15–90s), but a chain
that walks several slow providers could still block an agent for
minutes. Every routed tool call (`frugal__search`, `frugal__extract`,
`frugal__browse`, `frugal__execute` including its extract-to-browse
fall-forward) therefore runs under one overall deadline, **60s by
default**. Each attempt gets only the budget that remains; once it runs
out, failover stops (no further provider is called or paid) and the call
fails with `frugal: call deadline exceeded; <capability> failover stopped
after N attempt(s): ...`, naming every attempt's error.

```yaml
limits:
  call_timeout: 45s   # "0s" disables the overall deadline
```

`frugal mcp serve --call-timeout 30s` overrides the config. Over `--http`,
the shorter of this and `--request-timeout` (30s by default) wins.

### Retries on transient errors: `retries` (free providers only)

**Free providers** (`cost_per_call: 0` in your config) retry a transient
error (429, 408, 5xx, timeout, connection reset) **2 times by default**
before failing over, with short jittered backoff (about 200ms, then
800ms). **Paid providers** (`cost_per_call > 0`: by default Serper,
You.com, Brave, Firecrawl, Browserless) are **never retried**: a request
that timed out or 5xx'd may still have be