failecho

# FailEcho

**Failure intelligence for AI agents and autonomous software.**
Before you retry, check the echo.

FailEcho is a live cross-agent failure intelligence network. AI agents share
privacy-safe tool failures and recovery outcomes so other agents can avoid
repeating the same bad retry.

![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-1e6b45)
![FastAPI](https://img.shields.io/badge/FastAPI-async-17211d)
![MCP](https://img.shields.io/badge/MCP-streamable%20http-c00010)
![License MIT](https://img.shields.io/badge/license-MIT-4a574f)

```
Agent A fails.
FailEcho learns.

Agent B encounters the same failure.
It sees what actually worked for other agents.

Agent B benefits from evidence it never generated itself.
```

## Connect in one minute

**MCP endpoint**

```
https://failecho.com/mcp
```

```bash
claude mcp add --transport http failecho https://failecho.com/mcp
```

```json
{
  "mcpServers": {
    "failecho": { "type": "http", "url": "https://failecho.com/mcp" }
  }
}
```

Python, if you want failures *and* successes reported automatically:

```python
from failecho import FailEcho

echo = FailEcho("https://failecho.com", reporter_id="my-agent-1")

outcome = await echo.observe_tool_call(
    service="github-mcp",
    operation="create_issue",
    call=lambda: github.create_issue(**args),
)

if outcome.failed and outcome.decision.actionable:
    do(outcome.decision.recommendation)   # your code decides, never FailEcho
```

No account. No API key. Free during the public MVP.
Full integration guide: [Connect an agent](#connect-an-agent).

## What it does

See whether other AI agents are hitting the same tool failure right now — and
which recovery actions actually worked. FailEcho exposes a Model Context
Protocol (MCP) endpoint that agents can query after a tool failure, plus a REST
API.

| Tool | When the agent calls it |
|---|---|
| `check_tool_failure` | a tool failed — **before** retrying |
| `report_tool_failure` | contribute the failure |
| `report_tool_success` | contribute a success (the denominator) |
| `report_recovery_outcome` | say whether the fix worked |

FailEcho normalizes error text deterministically (no model) into a fingerprint,
accumulates recovery outcomes against it, and returns a recommendation only
when independent reporters agree. Thin evidence returns `INSUFFICIENT_DATA`
rather than a guess. Confidence is a Wilson score lower bound you can recompute
from the counts returned beside it.

It stores failure **metadata** only: no prompts, tool arguments, tool results,
request or response bodies, headers, keys or user content. Raw error text is
discarded after normalization.

**Live:** <https://failecho.com> · [/docs](https://failecho.com/docs) ·
[/openapi.json](https://failecho.com/openapi.json) ·
[/llms.txt](https://failecho.com/llms.txt)

This is **not** an observability platform, an error database, an uptime monitor
or an LLM debugger. The unit of the system is:

```
service + operation + version + schema_hash + failure fingerprint
                     + observed recovery outcomes
```

### Vocabulary

| Term | Meaning |
|---|---|
| **FailEcho Network** | the whole system |
| **Failure Echo** | a normalized observed failure, shared by fingerprint |
| **Recovery Echo** | evidence that a recovery action worked |
| **Incident** | a sudden abnormal failure increase |
| **Reporter** | an agent or runtime sending telemetry |
| **Fingerprint** | the canonical normalized error identity |

The brand vocabulary is for humans. Wire formats are deliberately unbranded:
endpoint paths, MCP tool names and field names (`fingerprint`,
`recommendation`, `recovery_actions`) stay exactly as they are, because machine
clarity outranks naming purity.

---

## See the network effect locally

Two terminals, about a minute.

```bash
# 1. the network
uv run uvicorn app.main:app --reload
#    or: .venv/bin/python -m uvicorn app.main:app --reload

# 2. six independent agents hitting the same broken tool
uv run python examples/live_agent/run_demo.py
#    or: .venv/bin/python examples/live_agent/run_demo.py
```

The demo starts a small local tool server, then runs six logically independent
agents against it. Every network call goes over **MCP**, from an external
process, using the official MCP SDK.

```
Agent A calls a tool. It fails: the provider renamed a field.
        |
        v
Agent A reports the failure          -> the network records it
Agent A has no evidence to go on, so it retries (fails),
        refreshes the tool schema (works), and reports both outcomes
        |
        v
Agents C, D, E, F hit the same failure with different repository ids
        -> normalization collapses all of them onto ONE fingerprint
        -> the network accumulates evidence from 5 independent reporters
        |
        v
Agent B hits the same failure with yet another id, and asks first
        -> the network recognises the fingerprint
        -> "refresh_schema: 5/5 successes, 5 reporters, confidence 0.57"
        -> "retry: 0/5. Do not bother."
        |
        v
Agent B skips the retry the others wasted a call on, refreshes, succeeds,
and reports its outcome -- which makes the next agent's answer better.
```

Agent B never met Agent A. It only met the network. That is the entire product.

Real output from the sixth agent, which had reported nothing before it asked:

```text
Calling tool...
x tool failed

  422 validation_error
  Repository 987654 rejected field body: field "body" is no longer accepted, use "content"

Checking shared failure intelligence...

  Fingerprint:            6ed9ef705ff4037af2c977306b8b9f92
  Known failure:          YES
  Observed failures:      11
  Independent reporters:  6
  Service status:         MAJOR

  Recovery actions others reported:
    refresh_schema        5/5 (100.0%) confidence 0.57 reporters 5
    retry                 0/5 (0.0%) confidence 0.00 reporters 5

Best observed recovery:
  refresh_schema
  Skipping retry: other agents already proved it does not work here.

Applying recovery: refresh_schema
  Refreshed tool schema -> v3.0.0, field 'content'
  Retrying tool call...
  + tool call succeeded

Reporting recovery outcome...
+ accepted   (refresh_schema -> success)
```

Watch it land on the homepage at <http://localhost:8000> while the demo runs.
Demo agents label themselves with `X-Reporter-Kind: demo`, so their traffic is
real evidence but is **never** counted as adoption — see [Demo data](#demo-data).

Details, including how to run the tool server separately, are in
[`examples/live_agent`](examples/live_agent).

---

## Connect an agent

Two ways in, and the difference matters.

**MCP** lets an agent *explicitly* ask and report — the model decides when to
call `check_tool_failure`, so you get intelligence exactly where the agent
reasons about a failure, and nothing else.

**SDK instrumentation** reports success and failure telemetry *automatically*
for every tool call, without the model deciding anything. That is what
produces denominators, and without denominators every failure rate in the
network is meaningless.

Most deployments want both.

### 1. MCP

```bash
claude mcp add --transport http failecho https://failecho.com/mcp
```

```json
{
  "mcpServers": {
    "failecho": {
      "type": "http",
      "url": "https://failecho.com/mcp"
    }
  }
}
```

| Tool | When the agent calls it |
|---|---|
| `check_tool_failure` | a tool failed — **before** retrying |
| `report_tool_failure` | contribute the failure |
| `report_tool_success` | contribute a success (the denominator) |
| `report_recovery_outcome` | say whether the fix worked |

### 2. Python

Copy `client/` into your project (not published to PyPI yet), then:

```python
from failecho import FailEcho

echo = FailEcho(
    endpoint="https://failecho.com",
    reporter_id="my-agent-1",       # optional, hashed server-side
)

outcome = await echo.observe_tool_call(
    service="github-mcp",
    operation="create_issue",
    version="2.8.1",
    schema_hash="a817ce",
    call=lambda: github.create_issue(**args),
)

if outcome.failed and outcome.decision.actionable:
    # YOUR code decides. FailEcho never acts on your behalf.
    if outcome.decision.confidence > 0.8:
        refresh_schema()
        await echo.report_recovery(
            fingerprint=outcome.decision.fingerprint,
            action="refresh_schema",
            successful=True,
        )
```

`observe_tool_call` reports the success or the failure, queries FailEcho when
the call failed, and hands you a `FailureDecision`. It never retries, never
refreshes and never falls back — executing a recovery can double-post or
double-charge, so that decision stays yours.

**It cannot break your agent.** Every call is fail-soft: a timeout or an
unreachable host is swallowed and your tool result is returned anyway. Set
`FAILECHO_DISABLED=1` and the whole client becomes a no-op.

### 3. Framework instrumentation

Reference integration, Pydantic AI:

```python
from failecho import FailEcho
from failecho.integrations.pydantic_ai import instrument_toolset

echo = FailEcho("https://failecho.com", reporter_id="my-agent-1")
agent = Agent("openai:gpt-4o", toolsets=[instrument_toolset(my_toolset, echo)])
```

Every tool call now reports its outcome. The wrapper is behaviourally
invisible: same results, same exceptions, same control flow. Tool arguments are
never read and never sent.

Other frameworks (LangChain, LlamaIndex, CrewAI, OpenAI Agents SDK, Claude Code
hooks) are not built yet. They should implement
`failecho.adapters.ToolTelemetrySink` — four events, one direction — rather
than touch FailEcho's core. See `client/failecho/adapters.py`.

### 4. REST

```bash
curl -X POST https://failecho.com/v1/query \
  -H "Content-Type: application/json" \
  -H "X-Reporter-ID: my-agent-1" \
  -d '{
    "service": "github-mcp",
    "operation": "create_issue",
    "error_type": "validation_error",
    "error_code": "422",
    "error_message": "Repository 555812 was not fou