maintain-model-list

# Maintain Model List

The supported model list lives in three places that must stay in sync:

| File                                                       | What it holds                                                                                                   |
| ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
| `packages/website/src/pages/docs/features/models/page.tsx` | `MODEL_GROUPS` (public docs, source of truth for display names) and `BASELINE` (recommended models)             |
| `packages/llms/src/models.live.test.ts`                    | Mirrored `MODEL_GROUPS`, plus `OPENROUTER_ID_OVERRIDES` / `ALIYUN_ID_OVERRIDES` for provider-specific model ids |
| `packages/llms/src/utils.ts`                               | `modelPatch` — per-model-family request parameter fixes (thinking/reasoning flags, `tool_choice` quirks, etc.)  |

## Before Touching Anything

1. Run `git status`. If the working tree has uncommitted changes that look like unrelated in-progress work, do NOT edit files — report the situation and ask the user for permission first.
2. Run the full live test suite as a baseline: `npm run test:live -w @page-agent/llms`. API keys come from the repo-root `.env` (`TESTING_OPENROUTER_KEY`, `TESTING_ALIYUN_KEY`, `TESTING_DEEPSEEK_KEY`); tests skip silently when a key is missing, so check which providers actually ran. Record which models pass/fail before making changes, so new failures are attributable.

## Workflow A: A Specific Model Was Given

1. **Research the model** (web search + provider docs):
    - Is it served on OpenRouter? Fetch `https://openrouter.ai/api/v1/models` and find the exact id (`<vendor-slug>/<model-id>`, watch for `-preview`, dated snapshots, dots vs hyphens).
    - Which other channels serve it (vendor native API, Aliyun DashScope, etc.) and what are their native model ids?
    - API differences: can thinking/reasoning be disabled or minimized? Any `tool_choice`, `parallel_tool_calls`, or parameter-schema quirks? Does the vendor's OpenAI-compatible endpoint differ from OpenRouter's behavior?
    - Agent suitability: tool call support is mandatory; note context window, latency, and cost.
2. **Update `modelPatch`** in `packages/llms/src/utils.ts` if the model needs new parameter handling. Model names are matched after `normalizeModelName` (lowercased, `/`-prefix, `.` and `_` stripped) — check whether an existing family branch already covers it.
3. **Add the model to both `MODEL_GROUPS` lists** (website page and live test), newest first within its brand group. Add id overrides if the OpenRouter/Aliyun id differs from the display name.
4. **Test availability carefully.** Run the live suite and confirm the new model passes on every provider that serves it:

```bash
npm run test:live -w @page-agent/llms
```

A failure means the request/response shape is wrong for that model — fix `modelPatch` or the id override, don't shrug it off. If the model fails consistently on a provider, exclude it from that provider (see the `deepseek-3.2` precedent in the test file) and document why in a comment. 5. **Decide on `BASELINE`** only if the model is a fast, cheap, strong-tool-call option; otherwise leave it out. 6. Run `npm run typecheck` and `npm test`.

## Workflow B: No Model Given (Routine Maintenance)

1. **Verify the existing list**: run the live suite (see above). Investigate any model that newly fails — deprecated? renamed? provider dropped it?
2. **Check for drift**: re-verify `OPENROUTER_ID_OVERRIDES` against the current `https://openrouter.ai/api/v1/models` output; ids change when vendors promote previews to stable.
3. **Scan for new models** worth adding (web search for recent releases from the brands in `MODEL_GROUPS`, plus notable newcomers). A candidate must support tool calls via an OpenAI-compatible API.
4. **Act by significance**:
    - Minor updates (a preview id went stable, a small point-release replaces its predecessor): apply the change yourself following Workflow A steps 2–6, then present it to the user for judgment.
    - New models or removals: report findings and recommendations, let the user decide before editing.

## Hand Off to the User

After your own tests pass, always remind the user to verify manually:

1. Tell them which line to put in the repo-root `.env` — give the exact model id per channel, including the OpenRouter id, e.g.:

```bash
LLM_BASE_URL="https://openrouter.ai/api/v1"
LLM_API_KEY="..."
LLM_MODEL_NAME="<exact-openrouter-id>"
```

2. Tell them to run the demo themselves: `npm run dev:demo` (serves on port 5174) and exercise the agent against a real page.

The live test only proves a single forced tool call round-trips; it is not an agent-quality eval. The user's manual run is the real acceptance test.