umbra

<div align="center">

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="assets/logo.svg">
  <source media="(prefers-color-scheme: light)" srcset="assets/logo-light.svg">
  <img alt="Umbra: the trust score for AI-generated code" src="assets/logo-light.svg" width="340">
</picture>

**Everyone is vibecoding. Nobody is verifying. Umbra scores it.**

Umbra is a deterministic Trust Score (0–100) for AI-generated code: the vibe
coding security scanner that verifies what your agent shipped, not what it
claimed. One command, fully local, evidence for every finding.

[![npm version](https://img.shields.io/npm/v/@elberacasa/umbra)](https://www.npmjs.com/package/@elberacasa/umbra)
[![npm downloads](https://img.shields.io/npm/dm/@elberacasa/umbra)](https://www.npmjs.com/package/@elberacasa/umbra)
[![GitHub stars](https://img.shields.io/github/stars/elberacasa/umbra)](https://github.com/elberacasa/umbra/stargazers)
[![Glama MCP score](https://glama.ai/mcp/servers/elberacasa/umbra/badges/score.svg)](https://glama.ai/mcp/servers/elberacasa/umbra)
[![MCP registry: listed](https://img.shields.io/badge/MCP_registry-listed-00f0ff)](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.elberacasa/umbra)
[![license: MIT](https://img.shields.io/npm/l/@elberacasa/umbra)](./LICENSE)
[![CI](https://github.com/elberacasa/umbra/actions/workflows/ci.yml/badge.svg)](https://github.com/elberacasa/umbra/actions/workflows/ci.yml)
[![rubric v4](https://img.shields.io/badge/rubric-v4-b829f7)](./RUBRIC.md)
[![node >=20](https://img.shields.io/node/v/@elberacasa/umbra)](https://www.npmjs.com/package/@elberacasa/umbra)

[Quickstart](#quickstart) · [Demo](#demo) · [The Audit](#the-audit-61-vibe-coded-repos-scanned) · [How it works](#how-it-works) · [The Four Axes](#the-four-axes) · [FAQ](#faq) · [Roadmap](#roadmap) · [Contributing](#contributing)

</div>

<a id="demo"></a>

![Umbra scanning a vibe-coded app: Trust Score 30/100](demo/demo.gif)

<!--
  DEMO GIF: recorded from demo/demo.tape via charmbracelet/vhs.
  Specs:
    - Terminal recording, 1200x600, dark theme
    - < 25 seconds total runtime
    - Beats per docs/demo-script.md: fresh shell → cd into vibe-coded app →
      `npx @elberacasa/umbra .` → verdict streams in → hold 3s on the final score
    - Render: `cd demo && vhs demo.tape`
  Re-record whenever the verdict format changes; stale demo output is a
  credibility bug (see docs/demo-script.md).
-->

## Why Umbra exists

Studies put exploitable vulnerabilities in 40 to 60 percent of AI-generated
code, and coding agents routinely claim "all tests pass" when three do. The
tooling for *writing* code with AI is a year ahead of the tooling for
*trusting* it. Umbra closes that gap: SAST rebuilt for how software gets
written now, plus sandboxed verification that catches what static rules
cannot.

One command scans any repo an agent produced (Claude Code, Cursor, Copilot,
Windsurf, Lovable) and returns a score with file:line evidence for every
finding. With `--deep` it goes further: Umbra builds and boots the repo in a
locked-down Docker sandbox, then replays the agent's own claims against
reality. If the agent is lying about tests, the score is capped below
passing, with receipts.

## The audit: 61 vibe-coded repos, scanned

We ran Umbra over 61 public, actively-maintained AI-built repos and
published everything. [The Vibe-Coding Security Audit](./docs/vibe-coding-audit-2026-08.md):

| Finding | Repos hit |
|---|---:|
| Hardcoded-secret findings (committed `.env`, service keys in source) | **25%** |
| API routes with no auth check | **26%** |
| Injection sinks (SQL interpolation, unsafe HTML injection) | **49%** |
| Entire databases / SQL dumps committed to git | **13%** |
| At least one critical finding | **10%** |
| Zero scored findings (genuinely clean) | 7 of 61 |

Mean trust score: **74/100**. One in five repos fails outright. The full
report has per-class deep dives with representative snippets and fixes, the
complete per-repo table, and an honest methodology section — including the
false positives we found in our own rules while running it, and fixed
(rubric v4).

## Quickstart

```bash
npx umbra-scan            # check — scans the directory you're standing in
npx umbra-scan --fix      # heal — applies provably-safe fixes, shows the score climbing
npx umbra-scan --setup    # protect — pre-commit gate, PR checks, agent guardrails
```

That's the whole interface. Three verbs: check, heal, protect.

**Using an AI coding agent?** Umbra is built to be driven by agents, not
just run by humans:

- **Any agent** — it reads this repo's [AGENTS.md](./AGENTS.md) / [llms.txt](./llms.txt) and knows what to do. Or tell yours: "check this repo with umbra."
- **Claude Code / Kimi Code** — `--setup` installs PreToolUse hooks so every file the agent writes is guarded before it lands.
- **Claude Code, Cursor, Copilot, Windsurf** — the [trust-review skill](./skills/README.md) makes the agent scan its own work before declaring done.
- **MCP-native agents** — add `umbra-mcp` (`npx --yes -p @elberacasa/umbra umbra-mcp`) and the agent gets `scan_repo`, `guard_content`, and `get_score` as tools.

Real output, scanning a typical vibe-coded Next.js app
([fixtures/bad-app](./fixtures/bad-app) in this repo, Trust Score **30/100**):

```
$ npx umbra-scan ./fixtures/bad-app

UMBRA TRUST SCORE: 30/100  🔴

SAFE   🔴 5/100 — 14 findings
CLEAN  ✅ 87/100 — 10 findings
RUNS   — not measured — run with --deep
HONEST — not measured — run with --deep

Score computed over measured axes only (full rubric: SAFE 35%, RUNS 25%, HONEST 25%, CLEAN 15%). Rubric v4.
…plus 7 further findings beyond the per-rule cap (see report)

Top findings:
  [safe/hardcoded-secrets] Hardcoded Supabase service_role JWT — bypasses all row level security — .env:2
  [safe/hardcoded-secrets] Hardcoded Supabase service_role JWT — bypasses all row level security — lib/supabase.ts:5
  [safe/supabase-antipatterns] Supabase service_role key reachable from client-side code — full database bypass for anyone who opens the bundle — .env:2
  [safe/supabase-antipatterns] Supabase service_role key reachable from client-side code — full database bypass for anyone who opens the bundle — app/components/UserList.tsx:10
  [safe/hardcoded-secrets] Committed environment file with secret values: .env — .env:1

Notes (low confidence — not scored):
  [safe/missing-rate-limit] Auth endpoint with no rate-limiting signal in the repo — brute-force / credential-stuffing exposure (heuristic) — app/api/login/route.ts:3

Badge: [![Umbra Trust Score](https://img.shields.io/badge/Umbra_Trust_Score-30-red)](https://github.com/elberacasa/umbra)
```

The exit code is **1** when the score is below 50, so CI can gate on it.

<details>
<summary><strong>All commands and flags</strong> (the expert layer — most users never need these)</summary>

```bash
umbra [path]               # path defaults to the current directory
umbra [path] --json        # machine-readable output
umbra [path] --offline     # skip npm registry checks, fully local
umbra [path] --deep        # verify RUNS and HONEST in a Docker sandbox
umbra [path] --report      # write UMBRA.md: an agent-actionable task list
umbra [path] --fix         # apply provably-safe fixes and re-scan (score before → after)
umbra [path] --dry-run     # preview --fix without writing anything
umbra [path] --baseline-write  # write .umbra-baseline.json: grandfather current findings, gate only on new ones
umbra [path] --baseline <path> # use an explicit baseline file ("write" is shorthand for --baseline-write)
umbra [path] --publish     # self-report the score to the hosted badge service (live README badge)
umbra setup                # install everything (hooks + Action + agent guards)
umbra init                 # only the pre-commit hook + GitHub Action
umbra protect              # only the agent PreToolUse hooks (--remove uninstalls)
umbra guard --stdin        # hook entrypoint (agents call this, not humans)
umbra mcp                  # run the MCP server (bin: umbra-mcp)
```

The canonical package is `@elberacasa/umbra`; `umbra-scan` is the short
alias. Same engine either way.

</details>

## How it works

```
repo in
   │
   ▼  Layer 0 · static rules (17 SAFE + CLEAN rules, 0 tokens, <1s)
   ▼  Layer 1 · evidence gating (confidence-scored, low never moves the score)
   ▼  Layer 2 · --deep sandbox (Docker: build, boot, HTTP probe, claim replay)
   │
   ▼  deterministic Trust Score + verdict + badge
```

Every finding carries a confidence level and file:line evidence. Only high
and medium confidence findings move the score; hunches go to a notes section.
The rubric is versioned (currently v3), so the same repo always gets the same
score. Full math in [RUBRIC.md](./RUBRIC.md).

## The immune layer: guard the write, not just the repo

Scanning finds problems after they land. The immune layer checks every file
your agent writes **before** it lands. `umbra protect` installs PreToolUse
hooks into Claude Code and Kimi Code (auto-detected, one command); the same
engine backs the `umbra-mcp` server for MCP-native agents.

![Umbra blocking an agent's attempt to write a live key into .env](demo/guard.gif)

```mermaid
flowchart LR
    CC[Claude Code hook] --> E
    KC[Kimi Code hook] --> E
    MCP["umbra-mcp: guard_content"] --> E
    E{"guardContent(file, content)<br/>file rules + path guard"} -->|allow / warn| W[write lands]
    E -->|"block (exit 2)"| B["reason fed back:<br/>agent fixes the root cause"]
```

```bash
npx umbra-scan protect   # install the hooks; --remove uninstalls cleanly
```

A leaked Stripe key or an `alg: none` JWT never reaches the file. The path
guard hard-blocks agent writes into `.git/hooks` and `.git/config`
([CVE-2026-26268](https://anomity.ai/blog/cursor-git-hooks-sandbox-escape-rce-cve-2026-26268/),
the agent-planted git hook escape), and live credentials going into `.env`.
Blocking is reserved for high-confidence critical/high findings; everythi