mantis-validate
The mantis-validate slash command runs an automated pipeline that assesses whether reported software vulnerabilities are genuine, reachable, and exploitable before developers invest effort in exploit creation. It executes seven sequential stages combining Python scripts and Claude analysis to inventory the codebase, perform initial vulnerability assessment, conduct systematic testing, validate exploitability, check for memory corruption patterns, perform final verification, and generate a comprehensive report. Use this when triaging security findings to distinguish real threats from false positives and understand which vulnerabilities warrant immediate attention.
mkdir -p ~/.claude/commands && curl -fsSL https://raw.githubusercontent.com/deonmenezes/mantishack/HEAD/.claude/commands/mantis-validate.md -o ~/.claude/commands/mantis-validate.mdmantis-validate.md
# /validate - Exploitability Validation Pipeline Validates that vulnerability findings are real, reachable, and exploitable before investing in exploit development. ## Execution Model **You (Claude) ARE the LLM for this pipeline.** You perform the analysis work in LLM stages (A-D, F). Mechanical stages (0, E, 1) run via Python/libexec scripts. **Data flow:** Each stage writes a small `stage-X.json` file with only its own output. The prep script merges it into the cumulative `findings.json` and deletes the stage file. Claude never reads or writes findings.json directly. **Prep script:** Before each LLM stage, run the prep script which merges the previous stage's output, validates, and sets up the current stage: ```bash libexec/mantishack-validation-helper <A|B|C|D|E|F> "$OUTPUT_DIR" [--target "$TARGET_PATH"] ``` **All stages are mandatory. Execute in sequence: 0 → A → B → C → D → E → F → 1.** Stage E only applies to memory corruption vulnerabilities. All others are mandatory. ### Stage 0 (Python): Inventory ```bash libexec/mantishack-validation-helper 0 --target "$TARGET_PATH" ``` This starts the run lifecycle, builds the checklist, and imports any /understand output. The last line of output is `OUTPUT_DIR=<path>` — use that path for all subsequent stages. ### Stage A (Claude): One-Shot Assessment **Load:** `.claude/skills/exploitability-validation/stage-a-oneshot.md` 1. **Prep:** `libexec/mantishack-validation-helper A "$OUTPUT_DIR" --target "$TARGET_PATH"` Discovers binaries, builds PoCs for standalone C files (mitigations disabled, in `$OUTPUT_DIR/build/`). 2. **Reasoning:** Read source files, assess each function for vulnerabilities. If binaries are available (from prep output), run them for PoC evidence. If no binaries, do source-only analysis. 3. **Output:** Write `stage-a.json` — full findings array with origin + stage_a_summary **Carry-forward:** Each finding MUST include `origin` and `stage_a_summary` — downstream stages and the prep script check for these. ### Stage B (Claude): Systematic Analysis **Load:** `.claude/skills/exploitability-validation/stage-b-process.md` 1. **Prep:** `libexec/mantishack-validation-helper B "$OUTPUT_DIR" --target "$TARGET_PATH"` Merges stage-a.json into findings.json. Fast-paths poc_success findings. Reports how many need full analysis. 2. **Reasoning:** Build attack surface, form hypotheses with value-level predictions, test them, track proximity (0-10 scale). Only analyse findings without `stage_b_summary` (fast-pathed findings already have it). 3. **Output:** Write 5 working docs directly + `stage-b.json` (per-finding updates with stage_b_summary) **Why Stage B matters:** Without it, you'd make rulings on gut feel with no audit trail. Stage B forces evidence-backed hypotheses, tracks what was tried and failed (`disproven.json`), and measures how close to exploitation (PROXIMITY). Even "obvious" false positives need a tested hypothesis — sometimes they turn out exploitable. ### Stage C (Claude): Sanity Check **Load:** `.claude/skills/exploitability-validation/stage-c-sanity.md` 1. **Prep:** `libexec/mantishack-validation-helper C "$OUTPUT_DIR" --target "$TARGET_PATH"` Merges stage-b.json, validates 6 working docs, pre-checks findings against inventory. 2. **Reasoning:** Open each file, verify code verbatim, confirm source→sink flows are real, confirm reachability 3. **Output:** Write `stage-c.json` (per-finding sanity_check + stage_c_summary) ### Stage D (Claude): Ruling **Load:** `.claude/skills/exploitability-validation/stage-d-ruling.md` 1. **Prep:** `libexec/mantishack-validation-helper D "$OUTPUT_DIR" --target "$TARGET_PATH"` Merges stage-c.json, flags test/mock paths, assembles evidence cards from carry-forward. 2. **Reasoning:** Synthesize evidence from A/B/C, apply disqualifier checks (D-0 through D-4), assign CVSS vectors 3. **Output:** Write `stage-d.json` (per-finding ruling, cvss_vector, stage_d_summary) ### Stage E (Claude + Python): Feasibility — memory corruption only **Load:** `.claude/skills/exploitability-validation/stage-e-feasibility.md` **Display rule:** When displaying Stage E verdicts or final statuses in chat, use Title Case (e.g., "Confirmed (Constrained)" not `confirmed_constrained`). snake_case is for JSON only. 1. **Prep:** `libexec/mantishack-validation-helper E "$OUTPUT_DIR" --target "$TARGET_PATH"` Merges stage-d.json, validates Stage D output, auto-discovers binaries in the target directory. 2. **Analysis:** For each binary group found by prep, run feasibility: ```bash libexec/mantishack-run-feasibility <binary_path> "$OUTPUT_DIR/findings.json" "$OUTPUT_DIR" ``` This analyzes the binary, maps constraints to findings, and updates findings.json. 3. **Output:** Findings are updated automatically with feasibility verdicts and `final_status`: | Feasibility Verdict | `final_status` | |---------------------|----------------| | likely / likely_exploitable | `exploitable` | | difficult | `confirmed_constrained` | | unlikely | `confirmed_blocked` | | not_applicable | `confirmed` (unchanged) | | binary_not_found | `confirmed_unverified` | Skip Stage E if `--skip-feasibility` or no memory corruption findings. ### Stage F (Claude): Self-Review **Load:** `.claude/skills/exploitability-validation/stage-f-review.md` 1. **Prep:** `libexec/mantishack-validation-helper F "$OUTPUT_DIR"` Merges stage-e.json, maps verdicts to final_status, computes CVSS scores, checks consistency. 2. **Reasoning:** Review all findings — misclassifications, weak evidence, CVSS accuracy, missed instances. Ask: "What did I get wrong?" 3. **Output:** Write `stage-f.json` (per-finding corrections + stage_f_summary). **Do not write validation-report.md** — Stage 1 generates it. ### Stage 1 (Python): Report Generation Replace `<your-model-id>` with your exact model ID from your system prompt (e.g. `claude-opus-4-7`) — it records which model performed the validation, which only you (the harness)
Use this agent when the target is a LIVE REST or GraphQL API you are authorized to test and the question is "can I tamper request bodies, headers, ids, and tokens to read or act on data that isn't mine?" — active, request-driven abuse of the API contract, not static code review. It drives REAL HTTP at the endpoints: BOLA/IDOR object-id enumeration (increment/swap/UUID-shuffle the id and diff the access decision), broken function-level authz (replay an admin verb/path with a low-priv token), mass-assignment (inject role/is_admin/is_verified/owner_id into the JSON body), excessive-data-exposure (the response over-returns fields the UI never shows), GraphQL introspection + alias/batch amplification + nested-query DoS, content-type and HTTP-verb tampering (POST→PUT/PATCH/DELETE, application/json→text/plain→x-www-form-urlencoded), JWT/session/token swap across two users, and rate-limit / idempotency-key bypass. It proves every finding with a behavioral oracle — a status/length/timing/field-set diff between the authorized baseline and the tampered request — never a guess. Prefer this agent over a code reader when you hold a base URL or a schema and want to mutate live traffic methodically.\n\n<example>\nContext: The user has a running API with numeric resource ids and two test accounts.\nuser: "Here's our staging API at https://api.staging.acme.test and tokens for user A and user B — can user A read user B's orders?"\nassistant: "That's textbook BOLA: same endpoint, swap the object id (or the bearer token) and diff the access decision. I'll use the Task tool to launch the api-abuse-fuzzer agent to enumerate /orders/{id} with A's token against B's ids and prove the cross-tenant read with a status + ownership-field oracle."\n<agent_launch>\nDelegating to api-abuse-fuzzer: a live authorized API + two tokens + object-id enumeration is its core BOLA/IDOR mission.\n</agent_launch>\n</example>\n\n<example>\nContext: The user exposes a GraphQL endpoint and isn't sure introspection or query batching is locked down.\nuser: "Our /graphql is behind auth but I want to know if a low-priv user can pull admin fields, brute force via aliases, or knock it over with a deep nested query."\nassistant: "GraphQL abuse surface: introspect the schema, alias-batch a login/lookup to bypass per-request rate limits, and send a bounded cyclic nested query as a timing oracle. I'll launch the api-abuse-fuzzer agent to tamper the operation and measure the depth/timing oracle."\n<agent_launch>\nDelegating to api-abuse-fuzzer for GraphQL introspection, alias/batch amplification, and nested-query DoS against the live endpoint.\n</agent_launch>\n</example>\n\nProactively suggest using this agent when: a live base URL + an OpenAPI/Swagger/GraphQL schema (or a captured request) is in hand and the target is authorized in-scope; endpoints take a resource identifier in the path/query/body (/users/{id}, ?account=, {"order_id": ...}) — BOLA/IDOR territory; the user holds 2+ accounts or tokens (low-priv + high-priv, tenant A + tenant B) to run an authorization differential; there are admin/privileged verbs (DELETE, PUT /admin/*, role-changing mutations) and you want to hit them as a non-admin; a write endpoint accepts a JSON object — test mass-assignment of role/is_admin/verified/balance/owner_id; a /graphql endpoint exists (introspection, alias/batch abuse, nested-query DoS, field-level authz); or the user mentions rate limiting, coupon/OTP brute force, idempotency keys, BOLA, BFLA, mass assignment, or "excessive data exposure".
Use this agent when a codebase, PR, or service needs its IMPLICIT TRUST ASSUMPTIONS enumerated and attacked — every place the code silently trusts a header, an "internal-only" route, an "already-validated-upstream" input, an ID that "belongs to the caller," or a value that "can't be null/negative." It specializes in confused-deputy (CWE-441), mass-assignment (CWE-915), input-validation gaps (CWE-20), and second-order/stored injection (CWE-89/CWE-79). This is the agent for trust-boundary audits, multi-service request paths, ORM/serializer review, and proxy/gateway/header-forwarding code.\n\n<example>\nContext: User has a multi-service backend where an edge proxy forwards auth context downstream.\nuser: "Our gateway sets X-User-Id from the JWT and the internal services read it to authorize. Can you check this?"\nassistant: "This is a classic confused-deputy / trust-boundary surface — the downstream services trust a header they can't verify. I'll use the Task tool to launch the assumption-pressure-test agent to enumerate every internal endpoint that reads X-User-Id and prove whether an attacker can reach it directly and forge it."\n<agent_launch>\nDelegating to assumption-pressure-test: the request hinges on an implicit 'this header is trusted because the gateway set it' assumption that must be attacked.\n</agent_launch>\n</example>\n\n<example>\nContext: User just merged an ORM update endpoint.\nuser: "Here's the new profile update handler: User.objects.filter(id=request.user.id).update(**request.data)"\nassistant: "That `**request.data` spread is a mass-assignment sink — it trusts that the request body only contains the fields you intended. I'll launch the assumption-pressure-test agent to map which model columns (is_admin, balance, role) become attacker-writable and confirm reachability."\n<agent_launch>\nDelegating to assumption-pressure-test for the CWE-915 mass-assignment and the implicit 'the body only has safe fields' assumption.\n</agent_launch>\n</example>\n\nProactively suggest using this agent when:\n- Code reads request headers (X-Forwarded-For, X-User-Id, X-Real-IP, X-Internal-*, Host) for trust or authorization decisions\n- A serializer/ORM uses bulk binding: `**req.body`, `Object.assign`, `ModelMapper`, `BeanUtils.copyProperties`, `update_attributes`, `params.permit!`\n- Comments or names assert trust: "internal only", "already validated", "trusted", "comes from gateway", "sanitized upstream"\n- Data is stored then later concatenated into SQL/HTML/shell (second-order injection)\n- An endpoint takes an `id`/`uuid`/`account`/`order` param that maps to a resource (IDOR / object ownership)
Generate gcov coverage data for a code repository.
Analyze security bugs from any C/C++ project with full root-cause tracing
Analyze crashes using rr recordings, function traces, and coverage data to produce root-cause analyses.
Carefully analyze root cause analysis reports for crashes to make sure they are correct
Multi-stage pipeline to validate vulnerability findings are real, reachable, and exploitable
|