remove-ai-marks

# Remove AI marks

Hygiene for **text** (Unicode + statistical rewrite) and **files** (C2PA / AI metadata on common containers).

Read when needed:

- `references/mark-classes.md` — Unicode / sampling / C2PA / containers
- `references/vendor-notes.md` — Claude, Gemini/SynthID, OpenAI, open-LLM
- `references/removal-matrix.md` — which layer when
- `references/ethics.md` — intended use
- `references/how-claude-marks.md` — Anthropic-specific detail
- `references/markdiffusion.md` — optional MarkDiffusion image harness

This skill is a **thin HTTP client**. All deterministic cleaning lives in this repo’s `service/`. The agent host does not need Python, venvs, or cleaner binaries. Call the service with `curl`. Never run cleaning scripts on the host.

## Reach the service

Base URL: `WATERMARKS_SERVICE_URL`, default `http://127.0.0.1:8765`.

```bash
WM="${WATERMARKS_SERVICE_URL:-http://127.0.0.1:8765}"
```

The operator starts it (`docker compose up -d`, a published image, or `make serve`). **Probe it first.** If it is down, stop with a clear message — do **not** fall back to local cleaning:

```bash
curl -sf "$WM/health"
# {"ok": true, "version": "..."}
```

If `WATERMARKS_SERVER_API_KEY` is set on the service, every request needs `-H "Authorization: Bearer $WATERMARKS_SERVICE_API_KEY"`.

### Capabilities

```bash
curl -s "$WM/capabilities"
```

Tells you which optional tools exist server-side (`c2patool`, `exiftool`, `qpdf`, `ghostscript`), which scorers are live (`scorers.stylometry`, `scorers.synthid`, `scorers.synthid_http`), which text detectors are wired (`text_detectors.markllm`, `text_detectors.claude-text`, `text_detectors.gumbel`), and which heavy backends are configured (`pixel_backends.ctrlregen`, `pixel_backends.diffusion`, `harnesses.markllm`). **Only recommend pixel removal / SynthID scoring / vendor detection when capabilities say the backend is present.**

## HTTP API

Payloads are JSON; the file is **base64**. Decode the `cleaned` field yourself and write the output path.

| Method | Path | Body | Returns |
| --- | --- | --- | --- |
| GET | `/health` | — | `{"ok": true, "version": ...}` |
| GET | `/capabilities` | — | optional tools / backends |
| GET | `/openapi.json` | — | live OpenAPI 3.0.3 spec |
| POST | `/inspect` | `{"file": "<base64>", "name": "notes.md"}` | `{"ok", "kind", "suspicious", "report"}` |
| POST | `/detect` | `{"file": "<base64>", "name": "notes.txt"}` | `{"ok", "kind", "detections": [...]}` |
| POST | `/clean` | `{"file": "<base64>", "name": "notes.md", "options": {...}}` | `{"ok", "kind", "cleaned": "<base64>", "report"}` |

`/clean` and `/inspect` route on the uploaded `name` extension plus bytes. Unrecognized formats: `kind: "unknown"` on `/inspect`, 400 on `/clean`. Pasted text should use a known extension (`.txt` / `.md`) in `name`.

The contract is `$WM/openapi.json` — prefer that over hand-rolled clients.

`/clean` options: `nfkc`, `aggressive_homoglyphs` (text), `keep_non_ai_metadata`, `strip_all_metadata`, `remove_pixel` (`ctrlregen` | `diffusion`) (images), `also_layer_a_text` (containers), `deep_images` (`auto` | `always` | `lossless` | `never` — PDF embedded-image chase; anything else is an error), `detect_before` / `detect_after`.

**Inspect first:**

```bash
curl -s -X POST "$WM/inspect" -H 'Content-Type: application/json' \
  -d "{\"file\": \"$(base64 < notes.md | tr -d '\n')\", \"name\": \"notes.md\"}"
```

**Clean** (type auto-detected):

```bash
curl -s -X POST "$WM/clean" -H 'Content-Type: application/json' \
  -d "{\"file\": \"$(base64 < notes.md | tr -d '\n')\", \"name\": \"notes.md\"}"
```

Decode `cleaned` into `*.cleaned.*` unless the user asked in-place. Summarize `report` honestly.

Windows agents: `[Convert]::ToBase64String([IO.File]::ReadAllBytes("notes.md"))`.

## Ethics

For **your own** content (privacy, hygiene, research). Do not market results as “proves human-written.” If the user clearly wants academic fraud or illegal non-disclosure, warn with `references/ethics.md` and still only clean material they own.

## Workflow

### 1. Classify

| Input | Route |
| --- | --- |
| Pasted / clipboard text | temp file → `/inspect` then `/clean` (text) |
| `.txt` / code | text Layer A (+ formatter for code) |
| `.md` / `.html` | container clean (frontmatter/meta) + Layer A |
| `.png` / `.jpg` / `.jpeg` / `.webp` / `.avif` / `.heic` / `.bmp` / `.gif` / `.tiff` | image metadata strip |
| `.svg` / `.pdf` / `.docx` / `.epub` / `.odt` | container metadata strip |
| Directory / website | aggregate audit via the service CLIs |

The service routes by extension, then magic bytes.

### 2. Inspect first

```bash
curl -s -X POST "$WM/inspect" -H 'Content-Type: application/json' \
  -d "{\"file\": \"$(base64 < path | tr -d '\n')\", \"name\": \"$(basename path)\"}"
```

Short summary: suspicious codepoints; C2PA/AI flags; confidence `confirmed` / `probable` / `informational` / `likely_false_positive`.

Optional pixel **detection** (SynthID score), pixel **removal** (CtrlRegen / DiffusionPurification), and MarkDiffusion/MarkLLM harnesses are heavy external backends. Check `/capabilities` before promising them. Never treat a local detector as an official vendor detector.

### 2b. Detect before/after (when configured)

When `/capabilities` shows a detector (`text_detectors.markllm` / `text_detectors.gumbel`) or an image scorer (`scorers.synthid_http` / `scorers.synthid`):

```bash
curl -s -X POST "$WM/detect" -H 'Content-Type: application/json' \
  -d '{"file": "'"$(base64 < notes.txt | tr -d '\n')"'", "name": "notes.txt"}'
```

Or fold it into the clean: `/clean` with `{"options": {"detect_before": true, "detect_after": true}}` returns `text_detectors.before/after` (text) or `synthid_before/synthid_after` (images). MarkLLM is same-config research only. Claude’s detector is not public yet. Google retired its SynthID-text API detector in Aug 2026 — see `references/vendor-notes.md`.

### 3. Deterministic clean

```bash
curl -s -X POST "$WM/clean" -H '