ratchet

<h1 align="center">
  <img src="docs/assets/banner.svg" alt="Ratchet — an effect gate for AI agents" width="860">
</h1>

[![CI](https://github.com/thearchitect0x-glitch/ratchet/actions/workflows/ci.yml/badge.svg)](https://github.com/thearchitect0x-glitch/ratchet/actions/workflows/ci.yml)
[![CodeQL](https://github.com/thearchitect0x-glitch/ratchet/actions/workflows/codeql.yml/badge.svg)](https://github.com/thearchitect0x-glitch/ratchet/actions/workflows/codeql.yml)
[![npm](https://img.shields.io/npm/v/ratchet-mcp?label=ratchet-mcp)](https://www.npmjs.com/package/ratchet-mcp)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)
[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/thearchitect0x-glitch/ratchet/badge)](https://scorecard.dev/viewer/?uri=github.com/thearchitect0x-glitch/ratchet)
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/14440/badge)](https://www.bestpractices.dev/projects/14440)
[![NIST SSDF](https://img.shields.io/badge/NIST%20SSDF-conformance-informational)](docs/SSDF.md)
[![REUSE](https://api.reuse.software/badge/github.com/thearchitect0x-glitch/ratchet)](https://api.reuse.software/info/github.com/thearchitect0x-glitch/ratchet)

**An effect gate for AI agents.** Your agent asks before it does anything it cannot take back —
charge a card, ship a deploy, publish a package, send the email — and gets a durable decision, so
the same real-world action is attempted at most once across crashes and retries. Agents can also
read back what a run already did, and spend against a limit they cannot raise.

Agents retry. LLM control flow is non-deterministic, network calls fail ambiguously, and processes
die mid-action. The result is duplicate emails, double charges, and repeated writes — and nothing
in the stack knows which. Vendor idempotency keys help for the few vendors that offer them, and
never across separate agent processes or model providers.

Ratchet does not execute your actions. It holds a durable decision record in front of them.

```
POST /v1/effects/begin  →  decision: execute | duplicate | in_flight
                                    | blocked | approval_required | denied
```

Only `execute` authorises the caller to act.

---


## Project documents

| | |
|---|---|
| [Architecture](docs/handoff/ARCHITECTURE.md) | High-level design — what the gate is and what it deliberately is not |
| [Assurance case](ASSURANCE_CASE.md) | Threat model, trust boundaries, and the argument for each security requirement — including what is *not* defended |
| [Roadmap](ROADMAP.md) | What the next year holds, and what will never be built |
| [Governance](GOVERNANCE.md) | Who decides, and what happens if they stop |
| [Contributing](CONTRIBUTING.md) | How to report a bug or propose a change |
| [Security policy](SECURITY.md) | How to report a vulnerability, and how fast you hear back |
| [Code of conduct](CODE_OF_CONDUCT.md) | What is expected, and who to tell |
| [Known limitations](docs/handoff/KNOWN_LIMITATIONS.md) | Everything that is not true yet, stated plainly |

## The part that matters

If your process dies between "go" and "done", most systems quietly let the next caller retry.
Ratchet won't. The lease expires and the effect becomes **`indeterminate`** — a known unknown,
surfaced instead of buried. What happens next is the policy you declared for that effect type:

| `on_indeterminate` | Behaviour | Use for |
|---|---|---|
| `block` (default) | No automatic retry. A human or verifying agent resolves it. | Anything irreversible |
| `retry` | A fresh attempt is granted, up to `max_attempts`. | Vendors that are genuinely idempotent |
| `probe` | Caller must verify at the vendor and record evidence first. | Charges, transfers, payouts |

Exactly-once delivery is not achievable in a distributed system and this project does not claim it.
What Ratchet guarantees is **at-most-once initiation**, a recorded outcome that later callers
replay, and an explicit state for the case nobody else admits exists.

---

## Quick start

Requirements: Node 20.11+, Docker (for local Postgres).

```bash
npm install
cp .env.example .env          # defaults work for local development
npm run dev:db                # Postgres on :5433 via Docker
npm run migrate
npm run dev                   # control plane on :8787
npm run dev:worker            # lease reaper + webhook delivery (separate terminal)
```

Then open <http://localhost:8787>, or drive it from the shell:

```bash
bash examples/curl/walkthrough.sh
```

`npm run seed` populates a workspace with realistic state — a completed effect, a duplicate, an
indeterminate one, and one awaiting approval — so the console has something to show.

### Or with Docker Compose

```bash
AUTH_SECRET=$(openssl rand -base64 32) docker compose up --build
```

---

## The core loop

```bash
# 1. Ask, before you act.
curl -X POST http://localhost:8787/v1/effects/begin \
  -H "Authorization: Bearer $RATCHET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "effect_type": "email.send",
    "idempotency_key": "welcome:user_123",
    "payload": { "to": "sam@example.com" },
    "estimated_cost_micros": 800
  }'
# → { "decision": "execute", "effect_id": "eff_...", "lease_token": "lt_..." }

# 2. Do the real thing, yourself. Ratchet never touches it.

# 3. Say what happened.
curl -X POST http://localhost:8787/v1/effects/eff_.../report \
  -H "Authorization: Bearer $RATCHET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "lease_token": "lt_...", "outcome": "succeeded",
        "result": { "message_id": "msg_9f2" } }'

# Any later caller with the same key now gets:
# → { "decision": "duplicate", "result": { "message_id": "msg_9f2" } }
```

**The one rule:** report `failed` only when you *know* the action did not reach the outside world.
If you are unsure — a timeout, a dropped connection — report nothing. The lease lapses and Ratchet
records an honest `indeterminate`. A false `failed` is worse than silence, because it licenses a
duplicate.

### Idempotency keys

Derive the key from the work, deterministically.

| Good | Broken |
|---|---|
| `welcome-email:user_123` | `uuid4()` |
| `invoice:2026-08:acct_88123` | `"send-" + Date.now()` |
| `pr:acme/api:feature-auth` | `f"job-{attempt_number}"` |

A key that changes on every attempt makes every retry look like new work.

---

## When Ratchet is unreachable

Ratchet sits in your critical path, so decide this before integrating: on an outage your agent
either **acts without the gate** (fail-open) or **refuses to act** (fail-closed). Use fail-closed
for anything you would have to apologise for; fail-open where the vendor deduplicates anyway.

Full contract, client patterns, and the honest availability posture:
[`docs/FAILURE_MODES.md`](docs/FAILURE_MODES.md).

## Architecture

```
                    ┌──────────────────────────────┐
  agents ──────────▶│  control plane (stateless)   │
  REST + MCP        │  Fastify · /v1 · /mcp · web  │
                    └──────────────┬───────────────┘
                                   │
                    ┌──────────────▼───────────────┐
                    │  Postgres                    │
                    │  effects · policies · ledger │
                    │  spend windows · audit       │
                    └──────────────▲───────────────┘
                                   │
                    ┌──────────────┴───────────────┐
  webhooks ◀────────│  worker (long-running)       │
                    │  lease reaper · delivery · GC│
                    └──────────────────────────────┘
```

The control plane is stateless and scales horizontally — it may run on serverless infrastructure.
**The worker may not.** It expires leases on a timer whether or not a request is in flight; a
serverless function cannot do that. Run it as a long-running container. Multiple replicas are safe
(every claim uses `FOR UPDATE SKIP LOCKED`).

At-most-once is enforced by a database unique constraint on
`(workspace_id, effect_type, idempotency_key)` — not by application logic.

Full detail: [`docs/handoff/ARCHITECTURE.md`](docs/handoff/ARCHITECTURE.md).

---

## Deploying

The control plane is stateless and can scale freely. **The worker cannot** — it expires leases on a
timer whether or not a request arrives, so it must be a long-running process. That single
constraint rules out purely serverless hosts (Vercel, Netlify functions) despite their being
easier, and is why `fly.toml` runs both process groups from one image.

```bash
brew install flyctl && fly auth login   # once, needs your browser
npm run deploy:fly
```

The script is idempotent: it creates the app, provisions managed Postgres, generates `AUTH_SECRET`
once (never rotating it, since that would invalidate every API key), deploys both processes, and
verifies readiness. It refuses to proceed unless preflight passes:

```bash
npm run deploy:preflight
```

Preflight runs the full suite and production build, then checks that `AUTH_SECRET` is strong and
not the dev default, `PUBLIC_URL` is set (otherwise the manifest would advertise `localhost`),
`RATE_LIMIT_OVERRIDE` is unset, private-network webhooks are off, CORS carries no wildcard, and —
if Stripe is selected — that both the key and the webhook secret are present. It prints no secret
values.

Any container platform works; only `fly.toml` is Fly-specific. Set `DATABASE_URL`, `AUTH_SECRET`,
`PUBLIC_URL`, `NODE_ENV=production`, then run `node dist/api/server.js` (scale freely) and
`node dist/worker/main.js` (at least one, always on).

To rehearse the exact production containers locally:

```bash
AUTH_SECRET=$(openssl rand -base64 32) docker compose up --build
```

## Commands

| Command | What it does |
|---|---|
| `npm run dev` | Control plane with reload |
| `npm run dev:worker` | Worker with reload |
| `npm run dev:db` / `dev:db:down` | Local Postgres in Docker |
| `npm run migrate` | Apply migrations (advisory-locked; safe to run concurrently)