llmtrim

<p align="center">
  <img src="assets/logo.png" alt="llmtrim" width="450">
</p>

<h1 align="center">llmtrim</h1>

<p align="center">
  <strong>Local proxy that compresses LLM API traffic so you pay less. Same answers, smaller bill.</strong>
</p>

<p align="center">
  <sub>
    <b>−31% input · −74% output · −66% round-trip cost</b>
    · 112 live A/B cases · ~5&nbsp;ms/call · no model to load
  </sub>
</p>

<p align="center">
  <sub>
    Using <b>Claude Code</b>? One install also gets you a live <a href="#claude-code">status line</a>,
    a <a href="#claude-code">cold-cache guard</a>, cheaper <a href="#claude-code"><code>/compact</code></a>,
    and <a href="#claude-code"><code>/sub</code></a> to serve it from a Codex / Kimi / SuperGrok plan.
  </sub>
</p>

<p align="center">
  <sub>Proxy · CLI · MCP · library (Python · Ruby · Swift · Kotlin · JS/WASM)</sub>
</p>

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: light)" srcset="assets/status-watch-light.svg">
    <img src="assets/status-watch-dark.svg" alt="llmtrim status: a live dashboard showing tokens trimmed, dollars saved off your real bill, input/output savings bars, and a per-model breakdown" width="760">
  </picture>
</p>

<p align="center">
  <a href="https://github.com/fkiene/llmtrim/actions/workflows/ci.yml"><img src="https://github.com/fkiene/llmtrim/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MPL--2.0-blue" alt="License: MPL 2.0"></a>
  <a href="https://crates.io/crates/llmtrim"><img src="https://img.shields.io/crates/v/llmtrim?logo=rust&cacheSeconds=3600" alt="crates.io"></a>
  <a href="https://www.npmjs.com/package/@llmtrim/cli"><img src="https://img.shields.io/npm/v/@llmtrim/cli?logo=npm&cacheSeconds=3600" alt="npm"></a>
  <a href="https://www.npmjs.com/package/@llmtrim/cli"><img src="https://img.shields.io/npm/dm/@llmtrim/cli?cacheSeconds=3600" alt="npm downloads"></a>
  <img src="https://img.shields.io/badge/rust-1.88%2B-orange" alt="Rust 1.88+">
</p>

<p align="center">
  <a href="#what-it-does">What it does</a> &bull;
  <a href="#get-started">Install</a> &bull;
  <a href="#day-to-day">Day to day</a> &bull;
  <a href="#in-action">In action</a> &bull;
  <a href="#works-with">Works with</a> &bull;
  <a href="#claude-code">Claude Code</a> &bull;
  <a href="#the-numbers">Numbers</a> &bull;
  <a href="#configuration">Config</a> &bull;
  <a href="#use-it-as-a-cli-mcp-or-library">CLI &amp; library</a>
</p>

---

## What it does

You run Claude Code, Codex, Cursor, or your own app. Every turn, the tool sends a large request: system prompt, tools, history, raw command output. You pay for every token of that, including the parts that do not help the model.

A 200-line build log with two errors. Tool schemas resent on every call. JSON with hundreds of near-identical rows. That bulk is still billed.

llmtrim sits on your machine as a local proxy, trims the waste, and forwards a smaller request. The reply is unchanged. You keep the same tools and answers; you spend less.

```
  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …
                    ◀──────── reply ──────────

  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …
                            (on your machine)
                    ◀──────── reply ──────────  (same answer)
```

Compression cannot raise your bill or break a request; worst case is zero savings. Everything runs locally, nothing is sent to us. [In action →](#in-action)

For Claude Code the same install goes further: a status line with live trim % and rate limits, a guard that warns before an expired prompt cache re-bills your whole context, `/compact` on a cheaper model, and `/sub` to route sessions through another subscription. [Details →](#claude-code)

---

## Get started

```bash
npm install -g @llmtrim/cli@latest && llmtrim setup
# open a new terminal, then keep working
llmtrim status
```

That's it. `setup` starts a local proxy, wires your shell, and (when Claude Code is present) turns on the status line, cold-cache guard, `/sub`, and cheaper `/compact`. You do not run a separate install for each of those.

| You want | Run |
|---|---|
| First install | `llmtrim setup` |
| New version | `llmtrim update` (then `llmtrim ensure` after npm/brew/cargo) |
| Something broken | `llmtrim ensure` · `llmtrim doctor --fix` · or **`f`** in `status` |

> Any tool that honors `HTTPS_PROXY` works (Claude Code, Codex, Cursor, Aider, your SDK). GitHub Copilot does not (certificate pinning). [Full list →](#works-with)

<details>
<summary><b>Other installers</b> (Homebrew, curl, Scoop, Cargo, Docker)</summary>

```bash
# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

# Package managers
brew install fkiene/tap/llmtrim
cargo binstall llmtrim
scoop install llmtrim
docker run -d -p 43117:43117 -v llmtrim-state:/data ghcr.io/fkiene/llmtrim
```

Full options: [INSTALL.md](INSTALL.md).

</details>

<details>
<summary><b>Desktop tray</b> (menu bar / system tray)</summary>

Menu-bar / system-tray popover with the same savings numbers. Bundled in Homebrew, Scoop, and npm; `setup` can enable open-at-login. Open with `llmtrim tray`. On Linux desktops, interactive `ensure` can fetch the tray binary from the [latest release](https://github.com/fkiene/llmtrim/releases) (needs `libwebkit2gtk-4.1` and `libayatana-appindicator3`).

<p align="center"><img src="crates/llmtrim-tray/docs/popover.svg" alt="llmtrim tray popover" width="320"></p>

</details>

<details>
<summary><b>Is this safe?</b></summary>

Same technique as [mitmproxy](https://mitmproxy.org), scoped to LLM API hosts only. `setup` changes three things; `llmtrim uninstall` reverses all three:

1. Private CA in `~/.llmtrim/` (name-constrained; cannot intercept your bank or email)
2. Shell env: `HTTPS_PROXY` + CA trust
3. Login service: daemon at login

No API keys stored (your tool's auth is forwarded). Prompts never touch disk; only anonymous token counts. Full threat model: [SECURITY.md](SECURITY.md).

```bash
llmtrim ca
openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"
```

</details>

---

## Day to day

```bash
llmtrim status     # savings + health  (aliases: monitor, gain)
llmtrim update     # new release, restart daemon, refresh integrations
llmtrim ensure     # match the recommended install state on this machine
```

| Situation | Command |
|---|---|
| Watch savings | `llmtrim status` |
| After `npm` / `brew` / `cargo` upgrade | `llmtrim ensure` (or **`f`** in status) |
| Diagnose | `llmtrim doctor` · repair with `doctor --fix` |
| Pause / resume proxy | `llmtrim stop` · `llmtrim start` |
| Force one session through llmtrim | `llmtrim wrap claude` |
| Remove everything | `llmtrim uninstall` |

After `setup`, `update`, or `ensure`, owned Claude Code pieces (status line, guard, `/sub`, compact defaults) stay in sync with the binary. You should not need `statusline install` or similar after an upgrade.

Time series: `llmtrim status --daily` · `--weekly` · `--monthly` · `--json` · `--csv`.

---

## In action

An agent ran a build. The tool returned 58 lines; two were errors. All 58 would have been billed.

4,662 chars → 978 (−79%). Errors stay verbatim. Repeated INFO lines fold into a template plus the values (lossless when the range is regular).

```text
# before (noise + signal)
[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)
… 28 more near-identical INFO lines …
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
… 25 more INFO lines …
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

# after (errors verbatim; INFO folded losslessly)
[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors
```

Try it on a request body of your own:

```bash
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai
```

Log folding is one stage. Others kick in on different waste:

| Waste | What happens |
|---|---|
| Build logs, diffs, grep dumps | Keep errors / changes / matches; fold the rest |
| Long pasted context | Keep chunks relevant to the question |
| Source code | Keep useful bodies; rest → signatures |
| Tool schemas every turn | Trim + keep the cache prefix stable |
| Huge JSON arrays | Compact table (TOON) or sample |
| Verbose model replies | Ask for terser output where safe |

> [!IMPORTANT]
> Compression cannot raise your bill or break a request. Each stage is re-measured with the provider's real tokenizer and undone if it does not save tokens. If the provider rejects the compressed body, the original is resent. Worst case is zero savings.

Prompt-cache prefixes (`cache_control`) are left alone.

<details>
<summary><b>All 10 compressors</b></summary>

Stages run in savings order. Nothing under a `cache_control` marker is rewritten.

| Stage | What it does | When it runs |
|---|---|---|
| **tool-output** | Lossless template fold first, then window logs · diffs · grep · dumps down to errors / changes / matches | tool results |
| **cache discipline** | Mark + stabilize the invariant prefix (sort tools/schema · OpenAI `prompt_cache_key`) so it stays cached | tools |
| **lexical retrieval** | BM25+ ranking with RM3 feedback · TextTiling topic cuts · budgeted non-redundant selection; question protected | long context