llmtrim

<p align="center">
  <img src="assets/logo.png" alt="llmtrim" width="450">
</p>

<h1 align="center">llmtrim</h1>

<p align="center">
  <strong>Local proxy that compresses LLM API traffic so you pay less. Same answers, smaller bill.</strong>
</p>

<p align="center">
  <sub>
    <b>−31% input · −74% output · −66% round-trip cost</b>
    · 112 live A/B cases · ~5&nbsp;ms/call · no model to load
  </sub>
</p>

<p align="center">
  <sub>
    Using <b>Claude Code</b>? One install also gets you <a href="#claude-code"><code>/sub</code></a>
    to serve it through CLIProxyAPI.
  </sub>
</p>

<p align="center">
  <sub>Proxy · CLI · MCP · library (Python · Ruby · Swift · Kotlin · JS/WASM)</sub>
</p>

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: light)" srcset="assets/status-watch-light.svg">
    <img src="assets/status-watch-dark.svg" alt="llmtrim status: Overview savings dashboard, Sessions, Detail, and Sub routing tab (CLIProxyAPI presets, fallback hops, and model map)" width="760">
  </picture>
</p>

<p align="center">
  <a href="https://github.com/fkiene/llmtrim/actions/workflows/ci.yml"><img src="https://github.com/fkiene/llmtrim/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MPL--2.0-blue" alt="License: MPL 2.0"></a>
  <a href="https://crates.io/crates/llmtrim"><img src="https://img.shields.io/crates/v/llmtrim?logo=rust&cacheSeconds=3600" alt="crates.io"></a>
  <a href="https://www.npmjs.com/package/@llmtrim/cli"><img src="https://img.shields.io/npm/v/@llmtrim/cli?logo=npm&cacheSeconds=3600" alt="npm"></a>
  <a href="https://www.npmjs.com/package/@llmtrim/cli"><img src="https://img.shields.io/npm/dm/@llmtrim/cli?cacheSeconds=3600" alt="npm downloads"></a>
  <img src="https://img.shields.io/badge/rust-1.88%2B-orange" alt="Rust 1.88+">
</p>

<p align="center">
  <a href="#what-it-does">What it does</a> &bull;
  <a href="#get-started">Install</a> &bull;
  <a href="#day-to-day">Day to day</a> &bull;
  <a href="#in-action">In action</a> &bull;
  <a href="#works-with">Works with</a> &bull;
  <a href="#claude-code">Claude Code</a> &bull;
  <a href="#the-numbers">Numbers</a> &bull;
  <a href="#configuration">Config</a> &bull;
  <a href="#use-it-as-a-cli-mcp-or-library">CLI &amp; library</a>
</p>

---

## What it does

You run Claude Code, Codex, Cursor, or your own app. Every turn, the tool sends a large request: system prompt, tools, history, raw command output. You pay for every token of that, including the parts that do not help the model.

A 200-line build log with two errors. Tool schemas resent on every call. JSON with hundreds of near-identical rows. That bulk is still billed.

llmtrim sits on your machine as a local proxy, trims the waste, and forwards a smaller request. The reply is unchanged. You keep the same tools and answers; you spend less.

```
  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …
                    ◀──────── reply ──────────

  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …
                            (on your machine)
                    ◀──────── reply ──────────  (same answer)
```

Compression cannot raise your bill or break a request; worst case is zero savings. Everything runs locally, nothing is sent to us. [In action →](#in-action)

For Claude Code the same install also wires `/sub` to route sessions through another subscription. [Details →](#claude-code)

---

## Get started

```bash
npm install -g @llmtrim/cli@latest && llmtrim setup
# open a new terminal, then keep working
llmtrim status
```

That's it. `setup` starts a local proxy and wires your shell. When Claude Code is present, it also turns on `/sub`. You do not run a separate install for that.

| You want | Run |
|---|---|
| First install | `llmtrim setup` |
| New version | `llmtrim update` (then `llmtrim ensure` after npm/brew/cargo) |
| Something broken | `llmtrim ensure` · `llmtrim doctor --fix` · or **`f`** in `status` |

> Any tool that honors `HTTPS_PROXY` works (Claude Code, Codex, Cursor, Aider, your SDK). GitHub Copilot does not (certificate pinning). [Full list →](#works-with)

<details>
<summary><b>Other installers</b> (Homebrew, curl, Scoop, Cargo, Docker)</summary>

```bash
# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

# Package managers
brew install fkiene/tap/llmtrim
cargo binstall llmtrim
scoop install llmtrim
docker run -d -p 43117:43117 -v llmtrim-state:/data ghcr.io/fkiene/llmtrim
```

Full options: [INSTALL.md](INSTALL.md).

</details>

<details>
<summary><b>Desktop tray</b> (menu bar / system tray)</summary>

Menu-bar / system-tray popover with the same savings numbers. Bundled in Homebrew, Scoop, and npm; `setup` can enable open-at-login. Open with `llmtrim tray`. On Linux desktops, interactive `ensure` can fetch the tray binary from the [latest release](https://github.com/fkiene/llmtrim/releases) (needs `libwebkit2gtk-4.1` and `libayatana-appindicator3`).

<p align="center"><img src="crates/llmtrim-tray/docs/popover.svg" alt="llmtrim tray popover" width="320"></p>

</details>

<details>
<summary><b>Is this safe?</b></summary>

Same technique as [mitmproxy](https://mitmproxy.org), scoped to LLM API hosts only. `setup` changes three things; `llmtrim uninstall` reverses all three:

1. Private CA in `~/.llmtrim/` (name-constrained; cannot intercept your bank or email)
2. Shell env: `HTTPS_PROXY` + CA trust
3. Login service: daemon at login

No API keys stored (your tool's auth is forwarded). Prompts never touch disk; only anonymous token counts. Recoverable tool results stay in bounded daemon RAM for five hours by default and disappear on restart. Full threat model: [SECURITY.md](SECURITY.md).

```bash
llmtrim ca
openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"
```

</details>

---

## Day to day

```bash
llmtrim status     # savings + health  (aliases: monitor, gain)
llmtrim update     # new release, restart daemon, refresh integrations
llmtrim ensure     # match the recommended install state on this machine
```

| Situation | Command |
|---|---|
| Watch savings | `llmtrim status` |
| After `npm` / `brew` / `cargo` upgrade | `llmtrim ensure` (or **`f`** in status) |
| Diagnose | `llmtrim doctor` · repair with `doctor --fix` |
| Pause / resume proxy | `llmtrim stop` · `llmtrim start` |
| Force one session through llmtrim | `llmtrim wrap claude` |
| Remove everything | `llmtrim uninstall` |

After `setup`, `update`, or `ensure`, owned Claude Code `/sub` stays in sync with the binary.

Time series: `llmtrim status --daily` · `--weekly` · `--monthly` · `--json` · `--csv`.

---

## In action

An agent ran a build. The tool returned 58 lines; two were errors. All 58 would have been billed.

4,662 chars → 978 (−79%). Errors stay verbatim. Repeated INFO lines fold into a template plus the values (lossless when the range is regular).

```text
# before (noise + signal)
[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)
… 28 more near-identical INFO lines …
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
… 25 more INFO lines …
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

# after (errors verbatim; INFO folded losslessly)
[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors
```

Try it on a request body of your own:

```bash
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai
```

Log folding is one stage. Others kick in on different waste:

| Waste | What happens |
|---|---|
| Build logs, diffs, grep dumps | Keep errors / changes / matches; fold the rest |
| Long pasted context | Keep chunks relevant to the question |
| Source code | Keep useful bodies; rest → signatures |
| Tool schemas every turn | Trim + keep the cache prefix stable |
| Huge JSON arrays | Compact table (TOON) or sample |
| Verbose model replies | Ask for terser output where safe |

> [!IMPORTANT]
> Compression cannot raise your bill or break a request. Each stage is re-measured with the provider's real tokenizer and undone if it does not save tokens. If the provider rejects the compressed body, the original is resent. Worst case is zero savings.

Existing prompt-cache prefixes (`cache_control`) are left alone. First-arrival recoverable shaping (`llmtrim recall r_…`) is off by default; live-zone tool-output windowing still runs, and a re-run of the same tool ships in full. To skip shaping for a command (verbatim stdout, including a terminal trailer), set `toolout_passthrough = ["*gpt.sh*"]` or prefix the command with `LLMTRIM_TOOL_OUTPUT=passthrough`. Lines starting with `LLMTRIM_KEEP:` survive windowing even when the rest is clipped.

<details>
<summary><b>All 10 compressors</b></summary>

Stages run in savings order. Nothing under a `cache_control` marker is rewritten.

| Stage | What it does | When it runs |
|---|---|---|
| **tool-output** | Lossless template fold first, then window logs · diffs · grep · dumps down to errors / changes / matches; opt-in first-arrival recall can restore omitted cache-boundary results with `llmtrim recall` | tool results |
| **cache discipline** | Mark + stabilize the invariant prefix (sort tools/schema · OpenAI `prompt_cache_key`) so it stays cached | tools |
| **lexical retrieval** | BM25+ ranking wi