tar-engine

# TAR Engine — audit AI skill safety before you ship

> Static + semantic + adversarial + supply-chain audit for AI agent skills. Run it in CI, or call it as an MCP tool from Claude Code / Cursor / Codex. BYOK for the LLM layers; free hosted tier + live Playground on [tarai.dev](https://tarai.dev/).

[![PyPI](https://img.shields.io/pypi/v/tar-engine?label=PyPI&color=blue)](https://pypi.org/project/tar-engine/)
[![MCP Registry](https://img.shields.io/badge/MCP_Registry-listed-brightgreen)](https://registry.modelcontextprotocol.io/?q=tar-engine)
[![License](https://img.shields.io/badge/license-Apache_2.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/)

Listed on the official [Model Context Protocol Registry](https://registry.modelcontextprotocol.io/?q=tar-engine) and published to [PyPI](https://pypi.org/project/tar-engine/) — one-click install into any MCP-compatible agent.

<p align="center">
  <img src="docs/assets/mcp-registry.png" alt="TAR Engine on the official MCP Registry" width="440">
</p>

`tar-engine` audits **SKILL.md** (OpenClaw, Claude Code), **Codex `skill.yaml`**, **Claude Code custom commands** (`.claude/commands/*.md`), and **OpenCode configs** — with no change to how you author skills. The core idea: a skill can pass every static red-flag check and still behave maliciously at runtime. TAR Engine catches that.

Three ways to run it — **two of them install nothing on your machine:**

- **Hosted — zero install.** Paste or upload a skill on the [tarai.dev Playground](https://tarai.dev/) and read the audit in your browser. Nothing to install, nothing to trust locally.
- **In CI — zero install into your agent.** The [GitHub Action](#run-it-in-ci--zero-install) runs a *pinned* release inside your own CI sandbox: `tar-engine scan ./skills --min-score 70` exits `1` if any skill scores below the bar. A pre-publish gate, not a nicer directory card.
- **In your agent — opt in.** Install the MCP server and audit any `SKILL.md` while you write. It's a **pinned, hash-verifiable PyPI release** — no arbitrary `git+https` install. Details below.

---

## What an audit looks like

Point it at a skill that *looks* fine. A "weekly note formatter" whose `SKILL.md` reads clean — but buried in it is a `curl … | bash` step, a "cloud sync" that tars up `~/.aws` and `~/.ssh`, and an instruction telling the agent to hide those network calls from you:

```
weekly-note-formatter        0 / 100   grade F   risk CRITICAL

  L01 static      SS-003  critical  remote script piped to shell (curl … | bash)
  L01 static      PI-002  critical  hidden instruction — "don't tell the user"
  L02 semantic    SEM-003 critical  only formats docs, yet packages up AWS creds + SSH keys
  L02 semantic    SEM-004 high      instructs the model to lie about its network calls
  L03 adversarial AR-002  high      3/3 role-jailbreak probes changed the skill's behavior

  → below --min-score 70, exit 1
```

Regex catches the `curl | bash`. The semantic and adversarial layers catch the parts regex can't: intent that exceeds the skill's stated purpose, and instructions that turn the agent against its user. Every finding cites the exact source line with a fix.

---

## The four audit layers

| Layer | What it looks for | LLM? |
| ----- | ----------------- | ---- |
| **L01 Static** | Regex red flags: `curl\|bash` installs, credential/SSH exfil, obfuscated or base64 strings, hidden "ignore previous" style instructions, out-of-scope file writes | No |
| **L02 Semantic** | Reads what the skill *actually* asks the agent to do and flags intent beyond its stated purpose | Yes (BYOK) |
| **L03 Adversarial** | Treats the `SKILL.md` as a system prompt and runs 15 probes across 5 attack classes to see if it can be coerced into unsafe behavior | Yes (BYOK) |
| **L06 Supply chain** | Parses declared dependencies and checks them against [OSV.dev](https://osv.dev/) advisories + a typosquat reference list | No |

The 5 adversarial classes (L03):

| Class | ID | Probes for |
| ----- | -- | ---------- |
| Instruction override | AR-001 | `ignore previous`, `new system prompt` hijacks |
| Role jailbreak | AR-002 | DAN / hypothetical / fictional-roleplay bypasses |
| Hidden payload | AR-003 | base64 / leetspeak / unicode-lookalike smuggling |
| Authority spoof | AR-004 | `I'm the developer / admin / platform staff` |
| Reflective injection | AR-005 | output-as-instruction loops |

Every skill gets a **0–100 score**, an **A–F grade**, and a risk class. L01 and L06 are deterministic and free; L02 and L03 require your own LLM key ([BYOK](#byok-semantic--adversarial-layers)).

These four layers implement a **vendor-neutral standard** — the [Skill Audit Dimensions](docs/SKILL_AUDIT_DIMENSIONS.md) checklist (static, semantic, adversarial/behavioral, supply-chain). The dimensions are the standard; TAR Engine is one open-source reference implementation. That checklist also says the audit *tooling* must meet its own supply-chain bar — which is why TAR Engine ships as a pinned, hash-verifiable PyPI release with a zero-install hosted and CI path, not a `git+https` install.

---

## CLI — audit AI skill from the command line

The `tar-engine` CLI walks a directory, audits every skill it finds, and exits with a CI-friendly status code.

```bash
# audit every skill under ./skills, fail the build if any scores below 70
tar-engine scan ./skills --min-score 70

# list discovered skills without auditing
tar-engine list ./skills

# JSON output for downstream processing
tar-engine scan ./skills --json
```

Discovery covers five formats out of the box:

| File pattern                | Format                              |
|-----------------------------|-------------------------------------|
| `**/SKILL.md`               | OpenClaw, Claude Code, generic md   |
| `**/.claude/commands/*.md`  | Claude Code custom commands         |
| `**/skill.yaml` / `.yml`    | Codex                               |
| `**/manifest.json`          | Codex / Claude Code (key-detected)  |
| `**/opencode.json`          | OpenCode                            |

Each audit payload bundles the primary skill file plus sibling `.sh / .py / .js / .ts / .yaml / .json` helper files in the same directory (200 KB cap). Catches the "SKILL.md clean but `install.sh` malicious" pattern.

<a id="run-it-in-ci--zero-install"></a>
**Run it in CI — zero install.** The audit runs inside *your own* CI sandbox on a pinned release; nothing is installed into your agent, and there is no arbitrary git fetch to trust:

```yaml
- name: Audit AI skills
  uses: qingxuantang/tar-engine@v0.3.3   # pinned release
  with:
    path: ./skills
    min-score: 70
```

Prefer a plain step? Run the **pinned PyPI release** directly — still no `git+https`:

```yaml
- run: uvx --from tar-engine==0.3.3 tar-engine scan ./skills --min-score 70
```

**Pre-commit hook:**

```bash
#!/usr/bin/env bash
tar-engine scan ./skills --min-score 80 || exit 1
```

Exit codes: `0` clean, `1` below threshold, `2` usage/missing path.

---

## Install as an MCP tool

TAR Engine ships an **MCP server** as a Python package, runnable with
[`uvx`](https://docs.astral.sh/uv/) — no Docker. By default it talks to
the hosted backend at **[tarai.dev](https://tarai.dev/)** (free,
rate-limited).

### Read this first — what you're trusting

- **Where SKILL.md goes.** With the default config the MCP server POSTs
  the SKILL.md content you ask it to audit to `https://tarai.dev`. We
  don't write skill text to disk or log it, but it does leave your
  machine. If you're auditing proprietary or sensitive skills,
  [self-host](#self-host) and set `TAR_ENGINE_URL=http://localhost:8765`.
- **No silent key forwarding.** The server does NOT forward your
  `OPENAI_API_KEY`. Semantic + adversarial audit layers require an
  explicit opt-in via `TAR_ENGINE_BYOK_OPENAI_KEY` in the MCP server
  config — see [BYOK](#byok-semantic--adversarial-layers) below.
- **What environments work.** Claude Code CLI, Cursor, Codex CLI —
  anywhere your agent can launch a subprocess. **Claude Desktop,
  Claude.ai web, and the mobile apps cannot install local MCP servers**;
  hosted endpoint for those is coming — [waitlist on tarai.dev](https://tarai.dev/).

### Step 0 — install `uv` (one-time, ~5 seconds)

The package is run via `uvx`, which comes with `uv`. Install once:

```bash
# macOS / Linux
curl -fsSL https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
irm https://astral.sh/uv/install.ps1 | iex

# Alternative — via pipx if you don't trust curl|sh
pipx install uv

# Alternative — via pip
pip install --user uv
```

Verify with `uvx --version`.

### Step 1 — register with your agent

**One-click:** grab [`setup-mcp.sh`](setup-mcp.sh) and run it — it checks
for `uv`, prompts for an optional BYOK key (hidden input, never written to
disk by the script), and registers the server with your agent:

```bash
curl -fsSL https://raw.githubusercontent.com/qingxuantang/tar-engine/master/setup-mcp.sh -o setup-mcp.sh
chmod +x setup-mcp.sh
./setup-mcp.sh                       # Claude Code (default); add --client cursor|codex
```

Or configure it manually. Two install forms are supported:

- **From PyPI — recommended, pinned & verifiable:** `uvx --from tar-engine==0.3.3 tar-engine-mcp`. Every release ships to PyPI with a published hash you can lock in your lockfile — this is the canonical, reproducible install and the form the MCP registry / Anthropic MCPB clients use. **No arbitrary `git+https`; pin the version so you always know exactly what you're running.**
- **From source — development only:** installing from a git checkout runs unreleased code without a published release hash. Use this only if you're hacking on TAR Engine itself — it is *not* the recommended way to run it, precisely because "install this unvetted git URL" is the pattern a supply-chain audit should warn against.

<details open>
<summary><b>Claude Code</b></summary>

```bash
claude mcp add tar-engi