unicode-le

<p align="center">
  <img src="src/assets/images/icon.png" alt="Unicode-LE Logo" width="96" height="96"/>
</p>
<h1 align="center">Unicode-LE: The Characters That Are Not What They Look Like</h1>
<p align="center">
  <b>Find the Unicode that hides meaning in the current file or the whole workspace</b><br/>
  <i>Bidi controls, invisibles, homoglyphs, mixed scripts, non-NFC text, spaces that are not the space</i>
</p>

<p align="center">
  <a href="https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.unicode-le">
    <img src="https://img.shields.io/badge/Install%20from-VS%20Code-blue?style=for-the-badge&logo=visualstudiocode" alt="Install from VS Code Marketplace" />
  </a>
  <a href="https://open-vsx.org/extension/OffensiveEdge/unicode-le">
    <img src="https://img.shields.io/open-vsx/dt/OffensiveEdge/unicode-le?style=for-the-badge&label=Open%20VSX&color=blue" alt="Open VSX downloads" />
  </a>
  <a href="https://www.npmjs.com/package/unicode-le-mcp">
    <img src="https://img.shields.io/npm/v/unicode-le-mcp?style=for-the-badge&label=MCP%20server&color=blue&logo=npm" alt="unicode-le-mcp on npm" />
  </a>
  <a href="https://crates.io/crates/unicode-le">
    <img src="https://img.shields.io/crates/v/unicode-le?style=for-the-badge&label=Rust%20CLI&color=blue&logo=rust" alt="unicode-le on crates.io" />
  </a>
  <a href="https://letools.dev/tools/unicode-le">
    <img src="https://img.shields.io/badge/LE%20Tools-letools.dev-blue?style=for-the-badge" alt="LE Tools" />
  </a>
</p>

---

> **Useful?** A star or rating is how other developers find it —
> [★ GitHub](https://github.com/nolindnaidoo/unicode-le) ·
> [★ Open VSX](https://open-vsx.org/extension/OffensiveEdge/unicode-le/reviews) ·
> [★ Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.unicode-le&ssr=false#review-details)

## What it does

Open a file, press `Ctrl+Alt+G` (`Cmd+Alt+G` on Mac), and every character in it that is not what it looks like lands in a report beside the editor: the bidirectional controls behind CVE-2021-42574, zero-width and other invisibles, homoglyphs, words no single script accounts for, lines that are not in Normalization Form C, spaces that are not U+0020, and codepoints with no agreed meaning. **Scan Workspace** does the same for every file on disk. Works in VS Code and in VS Code–based editors like Cursor and VSCodium (installable from Open VSX).

- **Review a pull request for Trojan Source** — a right-to-left override that makes the code a reviewer reads differ from the code that runs
- **Screen for forged names** — a Cyrillic `а` in an otherwise Latin `pаypal` is a finding; a word written wholly in Cyrillic is not
- **Explain the string that never matches** — a zero-width space or a decomposed `é` between two values a hash calls different and a person calls identical

**The report never contains a character it found.** Every finding is written as `U+202E`, never as the character itself, and a file path or key path from the document is escaped to `‮`. A report that pasted one raw would reorder the screen of whoever read it, and the tool would become the delivery mechanism for the thing it detects. **It rewrites nothing**: not a normalization, not a stripped space.

## Install

| Where | What you get | Install |
|---|---|---|
| **VS Code** | The screen, in your editor, on a keystroke | [Marketplace](https://marketplace.visualstudio.com/items?itemName=nolindnaidoo.unicode-le) |
| **Cursor, VSCodium, Windsurf** | The same extension | [Open VSX](https://open-vsx.org/extension/OffensiveEdge/unicode-le) |
| **A terminal or a CI step** | The same screen over a whole tree, with exit codes | `cargo install unicode-le` · [crates.io](https://crates.io/crates/unicode-le) |
| **Any MCP agent, via Node** | `detect_unicode_risks` over stdio | `npx unicode-le-mcp` · [npm](https://www.npmjs.com/package/unicode-le-mcp) |
| **Zed** | The MCP server as a context server | [add it by hand](https://zed.dev/docs/ai/mcp) *(no listing yet)* |

## Use it from an AI agent

The same engine runs as an [MCP](https://modelcontextprotocol.io) server, so an agent can call it directly instead of you running a command. It matters more here than for most tools: a model that pasted a document into its own reasoning has already been handed the bidi controls in it, and what comes back from this server is `U+XXXX` and English, so the answer cannot carry them on into a commit message or a review comment.

| Editor | How |
|---|---|
| **VS Code** 1.101+ | Nothing to install — the extension registers `detect_unicode_risks` with agent mode |
| **Zed** | No listing yet — [add the MCP server by hand](https://zed.dev/docs/ai/mcp) |
| **Claude Code** | `claude mcp add unicode-le -- npx -y unicode-le-mcp` |
| **Cursor, Windsurf, anything else** | point it at `npx unicode-le-mcp` |

```
detect_unicode_risks(content, kinds?, scripts?, format?, filename?, maxResults?)
```

Returns each finding with its kind, severity, 1-based line and column (UTF-16, as an editor counts), byte offset, the codepoints, the script and, where the format allows, the key path it sits under — plus any refusal, both structured and as a warning, so an empty finding list is never mistaken for a clean document. Every non-ASCII character in a reply leaves as `\uXXXX`. Capped at 500 by default with `meta.truncated`.

The server takes content and returns data — it reads no files and makes no network requests of its own. Published as [`unicode-le-mcp`](https://www.npmjs.com/package/unicode-le-mcp) on npm and as `io.github.nolindnaidoo/unicode-le` in the [MCP registry](https://registry.modelcontextprotocol.io). It answers exactly as the Rust CLI's server does: one corpus runs against both, a differential test feeds both thousands of generated documents, and both read the same Unicode tables — written out by the crate — rather than whatever version the host's JavaScript engine carries.

<details>
<summary><b>Configuring it by hand</b> — any host with an MCP config file</summary>

```json
{
  "mcpServers": {
    "unicode-le": {
      "command": "npx",
      "args": ["-y", "unicode-le-mcp"]
    }
  }
}
```

Or install it once with `npm install -g unicode-le-mcp` and point at `unicode-le-mcp`. It needs no environment variables, no API key and no configuration of its own. To check it:

```bash
echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | npx -y unicode-le-mcp
```

</details>

## What it finds

| Kind | Severity | What |
|---|---|---|
| `bidi-control` | high | U+202A–U+202E, U+2066–U+2069, U+061C — the Trojan Source class. They reorder how the rest of the line renders. |
| `confusable` | high | A homoglyph in a word of another script, or a compatibility form of an ASCII character (full-width `Ｆ`, mathematical `𝐚`, the Kelvin sign), with the codepoint it resembles |
| `mixed-script` | high | One word that no single script accounts for |
| `invisible` | medium | Zero-width characters, the soft hyphen, and U+FEFF anywhere but the first byte |
| `unassigned-or-private-use` | medium | A codepoint with no assigned meaning: private use, unassigned, a noncharacter |
| `non-nfc` | low | A line that is not in Normalization Form C. Reported, never rewritten |
| `unusual-whitespace` | low | A space that is not U+0020: no-break, ideographic, the en and em spaces |

## It runs on translated code without drowning you

A naive confusable check flags every letter of every Russian and Chinese string in a workspace — thousands of findings on exactly the codebases that most need the check, so it gets switched off, and the Trojan Source screen goes off with it.

- **A word is judged, never a file.** `Привет` is wholly Cyrillic and is Russian. Japanese mixes Han, Hiragana and Katakana in one word constantly, and UTS #39 knows that is Japanese.
- **A file plainly written in another script is not judged for homoglyphs**, and the report says so — at least 10% of its letters in a script nobody declared. Every other check still runs on it.
- **Declaring a script turns the check on, never off.** Name your workspace's scripts in `unicode-le.detection.scripts` — `Han`, `Cyrillic`, `Hira` — and a Latin product name inside a Chinese string is a translation, while a Cyrillic letter in a Latin word in the same file is still caught.

## It says where in the document, not just where in the file

In JSON, YAML, TOML, INI and `.properties`, `.env`, CSV and TSV a finding also carries the key path it sits under — `metrics.headline.eyebrow` rather than line 412 of a five-thousand-line catalogue. **The format never decides whether a finding exists**, only how it is addressed: a file whose format cannot be parsed is still scanned and still reports everything in it.

## It refuses rather than guessing

The workspace scan reads every file as bytes. A UTF-16 or UTF-32 file, a binary file and a file that is not valid UTF-8 are **refused by name** rather than decoded as something else, because a wrong decode invents findings: read UTF-16 as UTF-8 and every second byte becomes an invisible character that is not in the file.

## The CLI

The same screen runs from a terminal or a CI step: a Rust CLI in [`crate/`](crate/README.md), sharing one corpus with the extension — [`crate/fixtures/`](crate/fixtures/) — so the two can never read a document differently.

<p align="center">
  <img src="assets/demo.gif" alt="unicode-le in a terminal" style="max-width: 100%; height: auto;" />
</p>

```bash
unicode-le .                          # every finding in the tree, as JSON on stdout
unicode-le --fail-on bidi .           # in CI, for the CVE and nothing else
unicode-le --script Han,Cyrillic src/ # a translated tree, judged rather than refused
unicode-le mcp                        # the same screen over MCP on stdio
```

**Exit codes are the API** — 0 clean, 1 a finding `--fail-on` counts, 2 the question was malformed (or `--strict` with any refusal).

## Commands

| Command | Description |
|---|---|
| `Unicode-LE: Detect Unico