add-model

# Add a New AI Model

This skill walks through all the steps needed to register a new AI model in the
Pipelex inference system. The process touches several files across the codebase
and must be done in order to keep everything consistent.

## Overview of what needs to happen

1. **Identify the model** — which provider, what capabilities, what costs
2. **Add to backend TOML(s)** — the model spec in each relevant backend config
3. **Sync kit configs** — propagate `.pipelex/` changes into `pipelex/kit/configs/`
4. **Add to test profile collections** — so the model is available for test selection
5. **Run inference tests** — verify the model works end-to-end with real API calls (fixtures are auto-regenerated when `PROF=` is passed)
6. **Gateway (manual)** — the user handles this separately since it's remote config

## Step 1: Gather model information

Before editing any files, collect the following from the user. If the user already
provided some of this, confirm what you have and ask only for the missing pieces.

| Field | Description | Example |
|-------|-------------|---------|
| **Model name** | The handle used in Pipelex (the TOML section header) | `gpt-5.4` |
| **Model type** | `llm`, `img_gen`, `text_extractor`, or `search` | `llm` |
| **Provider** | Who made it — determines which backends to add it to | OpenAI |
| **Backends** | Which backend TOML files to add it to | `openai`, `azure_openai` |
| **Model ID** | The actual API model identifier (if different from name) | `gpt-5.4-2026-03-01` |
| **Inputs** | Supported input types | `["text", "images", "pdf"]` |
| **Outputs** | Supported output types | `["text", "structured"]` |
| **Costs** | USD per million tokens `{ input = X, output = Y }` | `{ input = 2.0, output = 8.0 }` |
| **Thinking mode** | `none`, `manual`, or `adaptive` | `manual` |
| **Constraints** | Any special constraints (e.g. fixed temperature) | `{ fixed_temperature = 1 }` |

### Looking up costs and capabilities

If the user doesn't know the model's pricing or capabilities, look them up on
OpenRouter. Read `references/openrouter-price-lookup.md` for the full API
reference, but here's the quick version:

1. Fetch `https://openrouter.ai/api/v1/models` (for LLMs) or
   `https://openrouter.ai/api/frontend/models?category=image-generation` (for
   image gen models).
2. Filter the response for the model by `id` or `name` (e.g., `openai/gpt-5.4`).
3. Convert prices from **per token** to **per million tokens** (multiply by
   1,000,000). Example: `"prompt": "0.000002"` becomes `input = 2.0`.
4. Map modalities: OpenRouter `image` -> our `images` (input) or `image` (output),
   OpenRouter `file` -> our `pdf`. If `"tools"` is in `supported_parameters`,
   add `"structured"` to outputs.

### Determining which backends

Each provider typically maps to specific backends:

| Provider | Backends to add to |
|----------|-------------------|
| OpenAI | `openai` + `azure_openai` |
| Anthropic | `anthropic` + `bedrock` |
| Google | `google` + `vertexai` |
| Mistral | `mistral` + `scaleway` |
| Meta (Llama) | `groq`, `bedrock`, `ollama` (varies) |
| xAI (Grok) | `xai` |

The **gateway** (`pipelex_gateway`) is always a separate manual step — its config
is fetched remotely. Remind the user about this at the end.

### Backend-specific differences

When adding a model to multiple backends, be aware of these differences:

- **OpenAI direct**: `model_id` is often omitted (defaults to the section name).
  SDK is `openai_responses`. May support `pdf` in inputs.
- **Azure OpenAI**: `model_id` is always required (includes a date suffix like
  `gpt-5.4-2026-03-01`). SDK is `azure_openai_responses`. Inputs typically use
  `images` but not `pdf`. Image gen models use `azure_rest_img_gen` SDK and need
  a `.rules` sub-table.
- **Anthropic direct**: SDK is `anthropic`. Uses
  `structure_method = "instructor/anthropic_tools"`. Often has `max_tokens` and
  `max_prompt_images`.
- **Bedrock**: SDK is `bedrock_converse`.
- **Google direct**: SDK is `google`. Uses
  `structure_method = "instructor/genai_tools"`.

A model spec never declares how its prompts are formatted: templating style is an
authoring decision on the pipe (`templating_style` on `PipeLLM`) with a single
runtime default. Any key you write that the model-spec blueprint does not know is
sent to the provider as an outbound HTTP header — so do not invent fields.

Each backend TOML has a `[defaults]` section — the model entry only needs to
specify fields that differ from those defaults. Read the defaults before writing
the entry so you include the minimum necessary fields.

## Step 2: Add the model to backend TOML files

The backend config files live in two mirrored locations. Edit the **`.pipelex/`**
copy (the project config), then sync to kit in the next step.

```
.pipelex/inference/backends/<backend_name>.toml     # <-- edit this one
pipelex/kit/configs/inference/backends/<backend_name>.toml  # <-- synced by make
```

### How to write the TOML entry

1. Read the target backend TOML file to understand its `[defaults]` and the
   existing model entries — match the style and ordering conventions.
2. Place the new model in the right section (models are grouped by series with
   comment headers like `# --- GPT-5.4 Series ---`).
3. Quote the section header if the model name contains dots: `["gpt-5.4"]`.
4. Only include fields that differ from `[defaults]`. At minimum you need:
   `inputs`, `outputs`, `costs`. Add `model_id` if it differs from the name.

**Example — adding `gpt-5.4` to `openai.toml`:**

```toml
# --- GPT-5.4 Series -----------------------------------------------------------
["gpt-5.4"]
inputs = ["text", "images", "pdf"]
outputs = ["text", "structured"]
costs = { input = 2.0, output = 8.0 }
thinking_mode = "manual"
```

**Example — adding `gpt-5.4` to `azure_openai.toml`:**

```toml
# --- GPT-5.4 Series -----------------------------------------------------------
["gpt-5.4"]
model_id = "gpt-5.4-2026-03-01"
inputs = ["text", "images"]