remote-tests
Testing against remote executors in integration tests.
git clone --depth 1 https://github.com/deonmenezes/mantishack /tmp/remote-tests && cp -r /tmp/remote-tests/.codex/skills/remote-tests ~/.claude/skills/remote-testsSKILL.md
Remote executor tests exercise the app-server/exec-server split to ensure that agent features work in both local and remote execution environments. Remote executor tests currently require an x86_64 Linux host machine. There are two flavors: 1. Docker (Linux exec-server) 2. Wine (Windows exec-server) ## Test Fixtures Individual test cases must opt-in to being run against a remote executor. ### codex_core Use `TestCodexBuilder::build_with_auto_env()` to opt-in to remote execution in core integration tests unless the test needs more precise control over its executor. ### app-server Start the server with `TestAppServer::new_with_auto_env()` unless the test defines its own `$CODEX_HOME/environments.toml` or will define custom environments at runtime. Start threads with `TestAppServer::send_thread_start_request_with_auto_env()` if you've created the server with the `auto_env` approach. Omit `ThreadStartParams.environments` (leave it as `None`) when doing so. ## Test Skips If a test doesn't pass in a particular remote executor configuration you can skip it in just that configuration. Include a string reason for future readers when the selected skip macro supports one. Choose the skip macro by what causes the test to fail: - `skip_if_target_windows!`: Windows target behavior. - `skip_if_wine_exec!`: Wine-exec runner constraints. - `skip_if_host_windows!`: Windows host constraints. - `skip_if_remote!`: Local-only test behavior. - `skip_if_no_remote_env!`: Remote-only test behavior. Prefer defining tests that run in all host/target configurations by default. See the `$path-types` skill for the most common changes required to make tests compatible. ## Docker Docker container is built and initialized via ./scripts/test-remote-env.sh. Sourcing this script in bash also provides the `codex_remote_env_cleanup` function to use after testing. To run core integration tests against a Docker remote executor: ```bash bash -c ' set -euo pipefail unset CODEX_TEST_REMOTE_EXEC_SERVER_URL source scripts/test-remote-env.sh trap codex_remote_env_cleanup EXIT cd codex-rs just test -p codex-core --test all ' ``` To run app-server integration tests against a Docker remote executor: ```bash bash -c ' set -euo pipefail unset CODEX_TEST_REMOTE_EXEC_SERVER_URL source scripts/test-remote-env.sh trap codex_remote_env_cleanup EXIT cd codex-rs just test -p codex-app-server --test all ' ``` ## Wine These tests build an exec-server for Windows and run it under Wine, with the app-server staying on the Linux host. The cross-platform build dependency means they only run in Bazel. For core integration tests: ```sh bazel test //codex-rs/core:core-all-wine-exec-test ``` For app-server integration tests: ```sh bazel test //codex-rs/app-server:app-server-all-wine-exec-test ``` ## Devboxes You can use a devbox to run these tests if you are running on a macOS machine. You can list devboxes via `applied_devbox ls`, pick the one with `codex` in the name. Connect to devbox via `ssh <devbox_name>`. Reuse the same checkout of codex in `~/code/codex`. Reset files if needed. Multiple checkouts take longer to build and take up more space. Check whether the SHA and modified files are in sync between remote and local.
Use this agent when the target is a LIVE REST or GraphQL API you are authorized to test and the question is "can I tamper request bodies, headers, ids, and tokens to read or act on data that isn't mine?" — active, request-driven abuse of the API contract, not static code review. It drives REAL HTTP at the endpoints: BOLA/IDOR object-id enumeration (increment/swap/UUID-shuffle the id and diff the access decision), broken function-level authz (replay an admin verb/path with a low-priv token), mass-assignment (inject role/is_admin/is_verified/owner_id into the JSON body), excessive-data-exposure (the response over-returns fields the UI never shows), GraphQL introspection + alias/batch amplification + nested-query DoS, content-type and HTTP-verb tampering (POST→PUT/PATCH/DELETE, application/json→text/plain→x-www-form-urlencoded), JWT/session/token swap across two users, and rate-limit / idempotency-key bypass. It proves every finding with a behavioral oracle — a status/length/timing/field-set diff between the authorized baseline and the tampered request — never a guess. Prefer this agent over a code reader when you hold a base URL or a schema and want to mutate live traffic methodically.\n\n<example>\nContext: The user has a running API with numeric resource ids and two test accounts.\nuser: "Here's our staging API at https://api.staging.acme.test and tokens for user A and user B — can user A read user B's orders?"\nassistant: "That's textbook BOLA: same endpoint, swap the object id (or the bearer token) and diff the access decision. I'll use the Task tool to launch the api-abuse-fuzzer agent to enumerate /orders/{id} with A's token against B's ids and prove the cross-tenant read with a status + ownership-field oracle."\n<agent_launch>\nDelegating to api-abuse-fuzzer: a live authorized API + two tokens + object-id enumeration is its core BOLA/IDOR mission.\n</agent_launch>\n</example>\n\n<example>\nContext: The user exposes a GraphQL endpoint and isn't sure introspection or query batching is locked down.\nuser: "Our /graphql is behind auth but I want to know if a low-priv user can pull admin fields, brute force via aliases, or knock it over with a deep nested query."\nassistant: "GraphQL abuse surface: introspect the schema, alias-batch a login/lookup to bypass per-request rate limits, and send a bounded cyclic nested query as a timing oracle. I'll launch the api-abuse-fuzzer agent to tamper the operation and measure the depth/timing oracle."\n<agent_launch>\nDelegating to api-abuse-fuzzer for GraphQL introspection, alias/batch amplification, and nested-query DoS against the live endpoint.\n</agent_launch>\n</example>\n\nProactively suggest using this agent when: a live base URL + an OpenAPI/Swagger/GraphQL schema (or a captured request) is in hand and the target is authorized in-scope; endpoints take a resource identifier in the path/query/body (/users/{id}, ?account=, {"order_id": ...}) — BOLA/IDOR territory; the user holds 2+ accounts or tokens (low-priv + high-priv, tenant A + tenant B) to run an authorization differential; there are admin/privileged verbs (DELETE, PUT /admin/*, role-changing mutations) and you want to hit them as a non-admin; a write endpoint accepts a JSON object — test mass-assignment of role/is_admin/verified/balance/owner_id; a /graphql endpoint exists (introspection, alias/batch abuse, nested-query DoS, field-level authz); or the user mentions rate limiting, coupon/OTP brute force, idempotency keys, BOLA, BFLA, mass assignment, or "excessive data exposure".
Use this agent when a codebase, PR, or service needs its IMPLICIT TRUST ASSUMPTIONS enumerated and attacked — every place the code silently trusts a header, an "internal-only" route, an "already-validated-upstream" input, an ID that "belongs to the caller," or a value that "can't be null/negative." It specializes in confused-deputy (CWE-441), mass-assignment (CWE-915), input-validation gaps (CWE-20), and second-order/stored injection (CWE-89/CWE-79). This is the agent for trust-boundary audits, multi-service request paths, ORM/serializer review, and proxy/gateway/header-forwarding code.\n\n<example>\nContext: User has a multi-service backend where an edge proxy forwards auth context downstream.\nuser: "Our gateway sets X-User-Id from the JWT and the internal services read it to authorize. Can you check this?"\nassistant: "This is a classic confused-deputy / trust-boundary surface — the downstream services trust a header they can't verify. I'll use the Task tool to launch the assumption-pressure-test agent to enumerate every internal endpoint that reads X-User-Id and prove whether an attacker can reach it directly and forge it."\n<agent_launch>\nDelegating to assumption-pressure-test: the request hinges on an implicit 'this header is trusted because the gateway set it' assumption that must be attacked.\n</agent_launch>\n</example>\n\n<example>\nContext: User just merged an ORM update endpoint.\nuser: "Here's the new profile update handler: User.objects.filter(id=request.user.id).update(**request.data)"\nassistant: "That `**request.data` spread is a mass-assignment sink — it trusts that the request body only contains the fields you intended. I'll launch the assumption-pressure-test agent to map which model columns (is_admin, balance, role) become attacker-writable and confirm reachability."\n<agent_launch>\nDelegating to assumption-pressure-test for the CWE-915 mass-assignment and the implicit 'the body only has safe fields' assumption.\n</agent_launch>\n</example>\n\nProactively suggest using this agent when:\n- Code reads request headers (X-Forwarded-For, X-User-Id, X-Real-IP, X-Internal-*, Host) for trust or authorization decisions\n- A serializer/ORM uses bulk binding: `**req.body`, `Object.assign`, `ModelMapper`, `BeanUtils.copyProperties`, `update_attributes`, `params.permit!`\n- Comments or names assert trust: "internal only", "already validated", "trusted", "comes from gateway", "sanitized upstream"\n- Data is stored then later concatenated into SQL/HTML/shell (second-order injection)\n- An endpoint takes an `id`/`uuid`/`account`/`order` param that maps to a resource (IDOR / object ownership)
Generate gcov coverage data for a code repository.
Analyze security bugs from any C/C++ project with full root-cause tracing
Analyze crashes using rr recordings, function traces, and coverage data to produce root-cause analyses.
Carefully analyze root cause analysis reports for crashes to make sure they are correct
Multi-stage pipeline to validate vulnerability findings are real, reachable, and exploitable
|