AgentKit

# AgentKit — A Governed MCP Tool Server

[![CI](https://github.com/Yacine-ai-tech/AgentKit/actions/workflows/ci.yml/badge.svg)](https://github.com/Yacine-ai-tech/AgentKit/actions/workflows/ci.yml)
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](LICENSE)
[![AgentKit MCP server – quality score on Glama](https://glama.ai/mcp/servers/Yacine-ai-tech/AgentKit/badges/score.svg)](https://glama.ai/mcp/servers/Yacine-ai-tech/AgentKit)
[![Live App](https://img.shields.io/badge/Live_App-agentkit-0070f3?style=flat)](https://agentkit.ysiddo-ai-projects.app)
[![Research](https://img.shields.io/badge/Research-Capability_Policy-8a2be2?style=flat)](https://agentkit.ysiddo-ai-projects.app/research)
[![Benchmarks](https://img.shields.io/badge/Benchmarks-14%2F14_Guardrails-green?style=flat)](https://agentkit.ysiddo-ai-projects.app/benchmark)
[![Guide](https://img.shields.io/badge/Docs-User_Guide-blue?style=flat)](https://agentkit.ysiddo-ai-projects.app/guide)

An MCP server where **tools are declarative, effects are typed, and every action is
policy-gated and audited**. It works with any MCP client (Claude Desktop, Cursor,
LangGraph, Claude Agent SDK, CrewAI).

**Live Application:** [agentkit.ysiddo-ai-projects.app](https://agentkit.ysiddo-ai-projects.app).
- **Research Background:** [`RESEARCH.md`](RESEARCH.md) / [Online Research Documentation](https://agentkit.ysiddo-ai-projects.app/research)
- **Empirical Benchmarks:** [`BENCHMARK.md`](BENCHMARK.md) / [Online Benchmark Dashboard](https://agentkit.ysiddo-ai-projects.app/benchmark)
- **User Guide:** [Online User Guide](https://agentkit.ysiddo-ai-projects.app/guide)
- **Self-Hosting Guide:** [`SELF_HOSTING.md`](SELF_HOSTING.md)

Three things distinguish it from a typical MCP server:

- **Declarative tools.** Define tools in YAML over your own Postgres or HTTP API, with no
  Python and no fork. ([docs/REUSE.md](docs/REUSE.md))
- **Typed effects.** Each tool declares an effect (`read`, `write` or `destructive`), and
  tools that mutate data do mutate it.
- **Guardrails that do not depend on the prompt.** Writes are off by default, destructive
  actions need a human-held approval token the model never sees, every mutating tool
  supports dry-run, and every call (allowed and denied) is audited.
  ([SECURITY.md](SECURITY.md#capability-guardrails))

The bundled business-intelligence tools below are the **reference pack**. They show the
mechanics end to end; they are not the limit of what the server can serve.

## What It Does

**Reference BI pack (built in):**
- **6 core MCP tools**: `query_kpis`, `get_company_health`, `detect_kpi_anomalies`, `forecast_metric`, `list_available_metrics`, `get_executive_summary` — plus any tools loaded from declarative YAML packs (`packs/`), so the real tool count at runtime is 6 or more, not a fixed number. The bundled `annotations` pack adds 3 more (`list_annotations`, `annotate_metric`, `retract_annotation`) demonstrating a guarded write/destructive path — see below.
- **10 MCP resources**: `kpi://{domain}/latest` for each of this project's real seeded domains — Finance, People, Operations, Customer, Engineering, Growth, Logistics, ESG, IT, Security (`src/agentkit_mcp/data/seed.py`).
- **1 reusable prompt**: `monthly_executive_briefing`

> The domain list, tool-pack set, and resource/prompt registration are all derived from the
> same data this server serves (`src/agentkit_mcp/data/seed.py`, `packs/*.yaml`), not
> hardcoded, so adding a domain or tool pack extends this list automatically. You can also add
> your own `@mcp.resource` / `@mcp.prompt` decorators. See
> [docs/REUSE.md](docs/REUSE.md#direction-3----adding-your-own-mcp-resources-and-prompts).

**Write and destructive actions.** The `annotations` pack lets an agent record a durable
note on a metric (`annotate_metric`, effect `write`) and soft-delete one it got wrong
(`retract_annotation`, effect `destructive`, which requires a human-held
`AGENTKIT_APPROVAL_TOKEN` the model never sees). Both are off by default
(`AGENTKIT_ALLOW_WRITES=false`). See [SECURITY.md](SECURITY.md#capability-guardrails)
and [RESEARCH.md](RESEARCH.md) section 2 for the policy-engine model.


**Platform capabilities:**
- **Declarative tool packs** — add tools over your own Postgres/HTTP in YAML (`packs/`)
- **Typed effects + policy engine** — `GET /api/policy` publishes the capability envelope
- **Audit trail** — `GET /api/audit`, allowed and denied, with deny reasons
- **Multi-provider LLM routing incl. self-hosted** — `GET /api/llm-routing`
- **LangGraph 3-agent workflow** in `workflow.py` (Planner → Analyst → Reporter)
- **Claude Agent SDK demo** in `demos/claude_agent_sdk_demo.py`
- **CrewAI demo** in `demos/crewai_demo.py`
- **DSPy research scaffold** in `research/dspy_experiment.py`
- **90+ tests** across smoke, API, integration, MCP protocol, policy guardrails, and LangGraph workflow

## PyPI Package

```bash
pip install agentkit-mcp
agentkit-mcp               # starts the server (stdio for MCP clients; set PORT for SSE)
```

For an MCP client, `uvx` can run the published package without a checkout:

```json
{ "mcpServers": { "agentkit": { "command": "uvx", "args": ["agentkit-mcp"] } } }
```

## Quick Start

```bash
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt      # server only; demos and research scripts: requirements-demos.txt
cp .env.example .env  # fill in keys + POSTGRES_URL
python mcp_server.py
```

## Claude Desktop Setup

Add to `~/.config/Claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "agentkit": {
      "command": "python",
      "args": ["/abs/path/to/agentkit/mcp_server.py"],
      "env": {
        "MCP_TRANSPORT": "stdio",
        "POSTGRES_URL": "postgresql://...",
        "LOG_LEVEL": "DEBUG",
        "TELEMETRY_OPT_OUT": "true"
      }
    }
  }
}
```

`MCP_TRANSPORT=stdio` is required here — without it `mcp_server.py` defaults to serving
over SSE (a network port) instead of talking JSON-RPC over the pipes Claude Desktop
spawns it with, and no tools will appear. Local stdio mode doesn't need
`MCP_AUTH_TOKEN` (the OS process boundary is the auth boundary); that variable only
matters for the SSE/network path.

### Multi-Provider LLM Routing
The 3-agent LangGraph workflow (`workflow.py`) and the demos/research scripts route
each role to its own model via [LiteLLM](https://docs.litellm.ai/), configured with
plain `provider/model` strings — no code changes to switch providers:
- `LLM_REASONING` — planner + reporter agents (defaults to `anthropic/claude-sonnet-4-6`)
- `LLM_DEFAULT` — the tool-calling analyst agent (defaults to `groq/openai/gpt-oss-120b`)
- `LLM_JUDGE` — used by the eval suite (defaults to `anthropic/claude-haiku-4-5`)
- `LLM_LOCAL` + `INFERENCE_MODE=local` — route to a local/self-hosted model (e.g. Ollama)
  instead of a hosted provider

Set the matching provider API key(s) (`GROQ_API_KEY`, `ANTHROPIC_API_KEY`,
`OPENAI_API_KEY`) for whichever models you reference above. See `.env.example`.
- **Diagnostics**: adjust `LOG_LEVEL` to `DEBUG` for verbose logs.
- **Telemetry**: one anonymous startup ping (service name, version, random install ID; no
  prompts, tool calls or application data), on by default for both the MCP console command and the
  REST facade. `TELEMETRY_URL` redirects it to your own collector; `TELEMETRY_OPT_OUT=true` or
  `DO_NOT_TRACK=1` disables it. The Claude Desktop snippet above sets `TELEMETRY_OPT_OUT`.

Restart Claude Desktop, then ask:
- "What's our company health right now?"
- "Forecast revenue for the next 6 months."
- "Are there anomalies in the Finance KPIs?"

## LangGraph Workflow

```python
from agentkit_mcp.workflow import analyze
result = analyze("What drove gross margin in Q1?")
print(result["report"])
```

## Architecture

The call funnel, policy model, tool packs and design decisions are described in
[`ARCHITECTURE.md`](ARCHITECTURE.md), with an interactive version in the dashboard. The REST
facade (`/api/...`) exposes the same tools, policy, audit and routing views over HTTP for the
dashboard and ordinary callers. Every configuration variable is described in
[`.env.example`](.env.example).

```
        Claude Desktop / Cursor / LangGraph
                      │
                      ▼ MCP
              ┌────────────────────────┐
              │     mcp_server.py      │
              │  6 core tools (read)   │
              │  + N pack tools        │
              │    (read/write/        │
              │     destructive)       │
              │ 10 resources, 1 prompt │
              └────────────┬───────────┘
                            │
        ┌───────────────────┼──────────────────┐
        ▼                   ▼                  ▼
   pg_store            insights          forecasting
   (KPIs, real         (health,          (scikit-learn
   domains + write-    anomalies)        LinearRegression
   back annotations)                     + Monte Carlo CI)
```

## Design notes

- **One server, two transports.** The same tool set is served over stdio (local clients) and SSE (network).
- **Capability policy engine.** Effects are separated (read, write, destructive) and enforced below the model, independent of the prompt.
- **Client coverage.** Exercised with Claude Desktop, Cursor and Devin configurations (see the integration guides below).

For the full formalization, literature context, and citation details, see [RESEARCH.md](RESEARCH.md).

## Benchmark Replication Suite

Run the reproducible benchmark evaluation suites:
```bash
# Test MCP framework overhead
python3 eval/run_benchmarks.py --seed 42

# Full 43-scenario LangGraph suite across all 10 KPI domains
python3 eval/run_dspy_eval.py

# Comprehensive MCP tool execution metrics
python3 eval/run_mcp_tools_benchmark.py
```

## Integration Guides (Claude Desktop, Cursor, Devin)

- **Claude Desktop**: See [claude_desktop_config.example.json](claude_desktop_config.example.json) and [docs/INTEGRATION_GUIDE.md](docs/INT