cao-workflow

# CAO Workflows

A CAO workflow is a **Python script** you write, validate, and — only after asking the user —
run through `cao workflow run`. Each script drives one or more agent *steps* through CAO's
shared substrate, so you can fan work out across agents, collect their results, and resume a
run that was interrupted.

> Your job as an author ends at a **validated script file on disk**. Authoring does NOT run the
> workflow. Never claim a workflow ran, or will run, when all you did was write it. Running is a
> separate step the user must approve (see Lifecycle step c).

## When to use

Reach for this skill when the user asks to **build or run a multi-step or parameterized
workflow** — for example:

- "Analyze every file in `reports/` and summarize the findings."
- "Run a review pipeline: implement, then review, then verify."
- "Do the same batch job but with a different input directory each time."

If the work is a single one-off agent call, you don't need a workflow. Workflows earn their
keep when there are multiple steps, fan-out, parameterization, or a need to resume.

## The script API

Author scripts import from the `cao_workflow` package. This package runs **only in the script
subprocess** and imports nothing from `cli_agent_orchestrator.*` — it talks to CAO over HTTP.
Its public surface:

- `step(provider, agent, prompt, *, recovery, step_id=None, timeout=None, **opts) -> StepHandle` —
  run one agent step and **declare** what re-running it would mean. `recovery` is keyword-only
  with no default, so omitting it is a `TypeError` at the call. See "Declaring a recovery
  policy" below before you pick a value.
- `run_step(provider, agent, prompt, *, step_id=None, timeout=None, **opts) -> StepHandle` —
  the same call, **declaring no policy**. That is the only difference between the two. A
  `recovery=` passed to `run_step` lands in `**opts`; the server validates it, the shim does
  not — see below.
- `StepHandle` has **five** fields: `.step_id`, `.terminal_id`, `.output`, `.status`, and
  `.replayed`. **`.replayed` qualifies `.terminal_id`.** When it is `True` the server returned
  a stored result and ran nothing, and `.terminal_id` is the ORIGINAL id — it names a terminal
  that **no longer exists**. That flag is the only thing standing between you and reading,
  writing to, or waiting on a dead id, so check it before you touch `.terminal_id`.
- `get_inputs() -> dict` — the run's resolved inputs (see Parameterized workflows). Returns
  `{}` when nothing was declared; never raises on absence.
- `emit_output(value)` — print the run-level `CAO_WORKFLOW_OUTPUT:` sentinel (the run's return).
- `ShimError` (and `ShimIdentityError`, `ShimTransportError`, `ShimHTTPError`) — the failure
  hierarchy `step` and `run_step` raise. Failures surface **unchanged** — the shim never
  retries.

## Declaring a recovery policy

`recovery=` is **the author's claim about the step, and nothing more.** CAO has no mechanism to
prove what a step does to the outside world, so it cannot and does not verify the claim. A
recovery policy **DECLARES what re-running this step would mean; it never grants permission.**

The three values, all of which are statements you are making, not protections you are getting:

| Value | What you are asserting |
| --- | --- |
| `"idempotent"` | re-running this step has the same effect as running it once |
| `"reconcile"` | re-running it needs a reconciliation step first (**deferred** — today CAO treats it exactly like `idempotent`) |
| `"manual"` | do not decide this one without me — halt and ask |

**`"idempotent"` grants nothing and protects nothing.** It does not make a step safe to re-run;
it tells the resume gate that *you* believe it already is — and wherever the gate would otherwise
stop and ask a human, it re-executes the step on your word instead. Declare it on a step that
charges a card, sends mail, or files a ticket and CAO will charge the card again, exactly as
instructed. If you cannot show the step is safe to repeat, `"manual"` is the honest declaration.

Omitting a policy is a **fourth, distinct state** — it is never silently read as `"manual"`. Use
`run_step` for it deliberately: an undeclared step still replays (replay executes nothing), but
where the alternative is re-execution it halts for a human.

**`recovery=` on `run_step` is checked late, not never.** `run_step` has no `recovery`
parameter, so the value rides `**opts` to the server, which stores it, lets the resume gate
honour it, and **rejects an unknown value with a `422`** — the route types that field as the
closed policy enum. What `run_step` lacks is `step()`'s client-side check, which refuses a bad
value *before any HTTP attempt*; on `run_step` a typo instead fails that step mid-run. Neither
surface has its value checked by `validate` (the linter sees the keyword, not its contents),
which is why `validate` reports the `run_step` form as `unenforced-recovery-policy`. Use
`step()` to declare, and `run_step` only to declare nothing.

## Lifecycle

Follow every step in order. **No step may be skipped** — validate is mandatory, and you must
ask before running.

### a. AUTHOR

Write a `.py` file to `~/.aws/cli-agent-orchestrator/workflows/<name>.py`. The workflow is
**run by its stem** (`<name>`), so:

- The name must be a bare stem — **no path separators**, no directory prefix.
- Do **not** create a same-stem `.yaml` sibling — a `<name>.yaml` next to `<name>.py` collides
  on the run surface.

### b. VALIDATE (mandatory gate)

```
cao workflow validate ~/.aws/cli-agent-orchestrator/workflows/<name>.py
```

Fix **every** finding before proceeding — the lint findings are **load-bearing**, not style
nits:

- **`import cli_agent_orchestrator` is banned.** The script runs in a separate subprocess and
  must reach CAO only over HTTP (the `cao_workflow` shim). Importing the server package breaks
  that boundary.
- **`random` / `time` / `datetime` / `uuid` warnings.** Resume **re-executes the script
  top-to-bot