agent-release-gate

# Agent release gate

Product-level sanity QA for the agent runtime, one layer below the playground UI. The question
is not "is every detail right" — it is "**if a user opens the product and does the obvious first
things, do they work?**" This is the gate a release passes before shipping.

Every check asserts on the **wire** (the SSE frame types the browser sees) and on **side effects**
(the file really persisted, the revision really incremented) — never on what the model says. That
makes it deployment-agnostic: point it at any stack and the assertions still hold.

## Run it

Set three environment variables for the deployment under test, then run the gate:

```bash
export AGENTA_BASE=https://your-stack.example.com   # deployment origin
export AGENTA_PROJECT_ID=...                         # target project
export AGENTA_API_KEY=...                            # project API key

uv run resources/qa_product.py --all --custom-slug <vault-slug> --custom-name "<display-name>" --require-store  # everything
uv run resources/qa_product.py --cell P1                         # one cell
uv run resources/qa_product.py --cell C1 --only chat              # one journey
uv run resources/qa_product.py --cell S2 --only warm --only cold1 --require-store  # continuity
```

**EXPORT the three variables, do not just set them.** The driver falls back to an env FILE when
`AGENTA_*` is absent from its environment, which is helpful interactively and dangerous in a
release run: a credentials file of bare `KEY=value` lines sourced with `. file` sets the shell
only, the child `uv run` process inherits nothing, and the driver silently runs the whole gate
against WHATEVER DEPLOYMENT the fallback file names. The failure surfaces as `401 Invalid
credentials` from a stage whose key you just watched answer 200, or worse as a green run
recorded against the wrong stack. Use `set -a` around the source, or `export` each variable,
and confirm the stage in the results before trusting them. (Cost a staging gate run on
2026-08-28; the fallback degrades to "wrong deployment", never to "no credentials".)

Paths are relative to this skill's directory. The deployment's vault must hold the provider keys
the cells use (Anthropic / OpenAI / OpenRouter). If the three env vars are unset the driver stops
immediately and names exactly what is missing; a legacy `--env-file <path>` fallback also exists.
`--all` includes cells P2, P2b, and P3 (a custom OpenAI-compatible provider, with P2 and P2b
running locally and P3 on Daytona), which need a vault slug passed via `--custom-slug`; the driver fails
fast if it's missing. `--custom-name` is also required because `model_keys` is built from the
display name rather than the stable slug. Custom-provider cells skip the credential-rotation
journey because their value is write-only and cannot be safely restored. Cells S1, S2 and C1
additionally need
the subscription sidecar logged in on the target deployment — see `resources/coverage.md` for what
each cell requires. The Daytona cells (C2, C4, P3, X2) additionally run the `secret_opaque`
journey and need the runner's Daytona API key to manage Secrets, because credential hiding is on
by default; without it those cells fail at sandbox creation with an error naming the permission.

**The one flag a release conductor must not skip past.** The continuity journeys
(`warm`, `cold1`, `cold2`) only mean anything on a **store-backed** deployment: with no object
store the runner degrades silently to an ephemeral working directory, so those journeys SKIP by
default and FAIL with `--require-store`. Run the gate against a deployment with `AGENTA_STORE_*`
configured and pass `--require-store`, or the greenest possible run still says nothing about
durability. `cold2` additionally needs an operator hook that SIGKILLs the runner replica
(`--cold2-replace-cmd`) and SKIPs without it.

**The flag that makes the gate fit the release: `--release-base`.** The matrix is fixed, so
without it a release that reworked a subsystem gets exactly the coverage of a release that did
not touch it. Pass the ref the release branches from, and the driver reads the release's own
changed paths, matches them against the rules in `resources/path_triggers.py`, and makes the
cells those rules name MANDATORY for this run:

```bash
uv run resources/qa_product.py --all --release-base origin/main --require-store   # every release run
uv run resources/path_triggers.py --release-base origin/main                      # preview only, runs nothing
```

A mandatory cell that lives in `qa_product.py` is added to the run even when `--cell` did not ask
for it. A mandatory cell that is a standalone `matrix_*.py` script is a separate process the
driver cannot observe, so it is printed, written to `mandatory.json`, and listed in `summary.md`
under "Mandatory for this release" — **the release is not green until each of those has a recorded
result of its own.** If a rule names a cell that does not exist, the driver stops before running
anything and says so: the release changed code a rule protects and the coverage was never
written, which is the one outcome that must never read as green. Add `--changed-path` to state
paths by hand where the checkout is not the release branch. With no `--release-base` and no
`--changed-path` nothing changes, so every existing invocation behaves exactly as before.

Adding a rule is one line in `PATH_TRIGGERS` (a glob, and the cells it makes mandatory) plus the
cell it names. Matching is `fnmatch` over the whole repo-relative path, so `*` crosses directory
separators and `a/b/*` covers the whole subtree; write `**` so a subtree rule reads as one.

**Reading the result.** Each journey prints `PASS`, `FAIL`, or `SKIP` with a one-line reason, and
a per-cell markdown table lands with the full JSON in `./qa-gate-runs/<timestamp>/` (override the
location with `AGENTA_QA_RUNS_DIR`). Runs are written to the current working directory, never into
the skill. `SKIP` is expected where a journey does