Tokenectomy

# Tokenectomy Razor 🗡️ — Autonomous Context Surgery & Secret Redaction for AI Agents (M2M MCP Server)

<p align="left">
  <a href="https://tokenectomy-web.vercel.app"><img src="https://img.shields.io/badge/Website-tokenectomy--web.vercel.app-000000?style=flat&logo=vercel" alt="Website" /></a>
  <a href="https://crates.io/crates/tokenectomy"><img src="https://img.shields.io/crates/v/tokenectomy.svg?logo=rust" alt="Crates.io" /></a>
  <a href="https://github.com/daffa2555/Tokenectomy/actions/workflows/ci.yml"><img src="https://github.com/daffa2555/Tokenectomy/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
  <a href="SECURITY.md"><img src="https://img.shields.io/badge/Security-Audited%20(RustSec)-2ea44f?logo=rust" alt="Security" /></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License" /></a>
  <a href="https://modelcontextprotocol.io"><img src="https://img.shields.io/badge/MCP-Compatible-purple" alt="MCP" /></a>
  <a href="https://registry.modelcontextprotocol.io"><img src="https://img.shields.io/badge/Official%20MCP%20Registry-Active-brightgreen" alt="Official MCP Registry" /></a>
  <a href="https://github.com/marketplace/actions/tokenectomy-razor"><img src="https://img.shields.io/badge/GitHub%20Marketplace-Tokenectomy%20Razor-blue?logo=githubactions" alt="GitHub Marketplace" /></a>
  <a href="CONTRIBUTING.md"><img src="https://img.shields.io/badge/PRs-Welcome-brightgreen.svg" alt="PRs Welcome" /></a>
  <a href="https://github.com/daffa2555/tokenectomy-bechmark-history"><img src="https://img.shields.io/badge/Benchmarks-Verifiable%20History-blue?logo=github" alt="Benchmarks" /></a>
  <a href="https://github.com/daffa2555/Tokenectomy"><img src="https://img.shields.io/github/stars/daffa2555/Tokenectomy?style=social" alt="GitHub Stars" /></a>
</p>

> 🌐 **[Interactive Web Playground & Live Architecture →](https://tokenectomy-web.vercel.app)**  
> 🔬 **Need AST Auto-Healing, Test Rollback & Pro Engine? [Tokenectomy Sentinel (Pro Tier) →](https://tokenectomy.gumroad.com/l/kiznsu)**  
> 🔀 **Pair with [Tokenectomy Git (OSS)](https://github.com/daffa2555/tokenectomy-git)** for autonomous Git fix branches & PR creation!  
> 📊 **Telemetry Receipts (25K ➔ 1M Lines):** Check out the transparent **[Benchmark History](https://github.com/daffa2555/tokenectomy-bechmark-history)** repository.  
> 🏷️ **Official MCP Registry:** Officially listed as [`io.github.daffa2555/razor`](https://registry.modelcontextprotocol.io/) (`mcp-name: io.github.daffa2555/razor`)


<p align="center">
  <img src="demo.gif" alt="Tokenectomy Razor Demo" width="100%" />
</p>

**Tokenectomy Razor 🗡️** (Community / OSS Tier) is an agent-native **Machine-to-Machine (M2M) MCP server** built with Rust. Designed specifically as an autonomous background sub-cortex for AI coding agents (Claude Desktop, Cursor, Cline, Roo Code, Windsurf, Google Antigravity), Razor surgically scrubs 90%+ of internal framework noise (`node_modules`, `site-packages`, `.cargo/registry`) from error logs, auto-redacts sensitive credentials before cloud transmission, and provides sub-millisecond AI reverse proxying—**with zero human babysitting**.

```
                     ┌──────────────────┐
   Agent Error Dump  │   TOKENECTOMY    │      Clean Agent Context
   (38K tokens) ───► │     RAZOR 🗡️     │ ───►  (2K tokens)  ───► LLM Brain
                     │  Community / OSS │
   node_modules/     │  🔍 Smart Filter │      Only YOUR code
   site-packages/    │  🛡️ Redact       │      + error message
   .cargo/registry/  │  💾 Cache        │      + StackOverflow refs
                     └──────────────────┘
```

---

## ✨ Key Features

- **🌐 Polyglot Trace Surgery** — Natively parses stack traces and crashes across **Rust, Python, Node.js/TypeScript/JSX, Golang, Java/Kotlin, C/C++ (ASan & GDB), and PHP (Laravel/Symfony)**, auto-filtering thousands of lines of framework dependency noise (`node_modules`, `site-packages`, `go/src`, `pkg/mod`, `.gradle`, `.m2`, `/usr/include`, `vendor`).
- **🛡️ AI Gateway Reverse Proxy (`--proxy`)** — Intercepts OpenAI/Anthropic/Ollama API traffic locally (`127.0.0.1:8080`), surgically scrubbing prompt token waste and auto-redacting secrets with sub-millisecond latency before forwarding to upstream LLMs.
- **🔍 Smart Framework Filter** — Strips thousands of lines of noisy internal stack frames and preserves strictly the lines of code *you* wrote.
- **🌐 Stack Overflow Search** — Silently queries StackExchange APIs and injects top community solutions into the AI's context.
- **⚡ SHA-256 Response Cache** — Identical errors hit local cache (24h TTL). Recurring CI/CD failures cost $0.00 in API calls.
- **🛡️ Secret Redaction** — Regex engine strips API keys, AWS secrets, JWTs, and database connection strings before any data leaves your machine (ReDoS-safe, linear-time).
- **⚡ High-Throughput Stream Surgery** — Zero-allocation linear-time $O(N)$ evaluation handling 250,000+ lines in ~330ms without memory bloat.
- **🔒 Path Traversal Protection** — MCP file operations are strictly canonicalized and locked within your workspace.
- **🤖 MCP Server Mode** — Full JSON-RPC 2.0 over stdio. Works with Claude Desktop, Cursor, VS Code, Google Antigravity, and any MCP-compatible client.
- **🔌 Multi-Provider** — Supports OpenAI, Anthropic, and Ollama (100% offline mode).

---

## 📊 Verifiable Real-World Performance Benchmark

Every developer can verify the core performance claims directly on their physical machine:

| Feature Under Test | Tested Heavy Input | Real Measured Outcome | Status |
|---|---|---|:---:|
| **Quarter-Million Log Redaction** | **250,000 lines (24.44 MB)** enterprise dump with DB URLs, API keys, JWTs | **333.49 ms (73.3 MB/sec, 749,652 lines/sec)**. 100% sanitized. | ✅ Verified |
| **ReDoS Immunity** | 50,000-character malicious backtracking exploit string | **1.44 ms**. Linear $O(N)$ evaluation, 100% ReDoS immune. | ✅ Verified |
| **High Concurrency Torture** | 100 concurrent OS threads hammering redaction & extractor | **100/100 in 27.35 ms (7,312.7 ops/sec)**. Zero race conditions. | ✅ Verified |
| **Kernel Memory Footprint** | Peak Resident Memory during 250,000-line stress test | **76.24 MB VmRSS** via Linux `/proc/self/status`. Zero memory ballooning. | ✅ Verified |

> 💡 **Verify on your own hardware:** Clone this repository and run the standalone benchmark:
> ```bash
> cargo test --release --test stress_benchmark -- --nocapture
> ```

> 🔬 **Need enterprise workloads (1M+ lines, 250 threads, AST Smart Healer & test rollback)?**  
> Check out **[Tokenectomy Sentinel (Pro Tier) on Gumroad ($9) →](https://tokenectomy.gumroad.com/l/kiznsu)**.

---

## 📦 Installation

### ⚡ Install via Cargo (crates.io)

```bash
cargo install tokenectomy
```

### 🦀 Build from Source

```bash
git clone https://github.com/daffa2555/Tokenectomy.git
cd Tokenectomy
cargo build --release
sudo cp target/release/razor /usr/local/bin/razor
sudo cp target/release/tokenectomy /usr/local/bin/tokenectomy
# Optional alias for backward compatibility:
sudo ln -sf /usr/local/bin/razor /usr/local/bin/tkmy
```

### ⚙️ Install via Smithery (for Claude Desktop)

```bash
npx -y @smithery/cli install tokenectomy --client claude
```

### 🐳 Run via GitHub Container Registry (GHCR)

Pull the multi-arch container image:
```bash
docker pull ghcr.io/daffa2555/razor:latest
```

Or configure your MCP client to run the containerized Razor server directly:
```json
{
  "mcpServers": {
    "tokenectomy": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "ghcr.io/daffa2555/razor:latest", "razor", "--mcp"]
    }
  }
}
```

---

## 🔌 M2M Agent Setup (1-Minute Integration)

Tokenectomy Razor is architected to run silently between your AI Coding Agent and your repository over **JSON-RPC 2.0 stdio**. You configure it once, and your agent autonomously invokes Tokenectomy Razor in the background during debugging and refactoring loops—**no manual copy-pasting or piping required**.

```bash
razor --mcp
# (or legacy alias: tkmy --mcp)
```

### Claude Desktop

Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "tokenectomy": {
      "command": "razor",
      "args": ["--mcp"]
    }
  }
}
```

### Cursor

Add to `.cursor/mcp.json` in your project root:

```json
{
  "mcpServers": {
    "tokenectomy": {
      "command": "razor",
      "args": ["--mcp"]
    }
  }
}
```

### Cline / Roo Code / Windsurf / VS Code

Add to your MCP settings (`settings.json` or `cline_mcp_settings.json`):

```json
{
  "mcpServers": {
    "tokenectomy": {
      "command": "razor",
      "args": ["--mcp"]
    }
  }
}
```

### Google Antigravity CLI

```bash
agy mcp add tokenectomy-razor -- razor --mcp
```

### 🤖 Available M2M MCP Tools

| Tool | Autonomous Agent Role |
|------|-------------|
| `get_error_context` | Performs deep log surgery: strips framework noise, redacts secrets prior to parsing, extracts source context and git diff via `WorkspaceBoundary` |
| `search_stack_overflow` | Searches Stack Overflow for a specific error (query is auto-sanitized of secrets) |
| `apply_code_patch` | Safely applies a patch within `WorkspaceBoundary` with language syntax verification (`cargo check`, `py_compile`, `node --check`) and automated rollback on failure (guaranteeing 0 dirty git diffs) |
 
### 🔀 Companion MCP Server: Tokenectomy Git

Close the autonomous loop from error diagnosis all the way to a published GitHub Pull Request! Pair Tokenectomy Razor with our official companion MCP server: **[Tokenectomy Git](https://github.com/daffa2555/tokenectomy-git)**.

```json
{
  "mcpServers": {
    "tokenectomy": {
      "command": "razor",
      "args": ["--mcp"]
    },
    "tokenectomy-git": {
      "command": "tkmy-git",
      "args": ["--mcp"]
    }
  }
}
```

Together, they enable your AI coding assistant to:
1. Scrub noisy error logs & redact credentials (`tokenectomy`)
2. Generate an accurate fix patch
3. Creat