add-hosted-key

# Adding Hosted Key Support to a Tool

When a tool has hosted key support, Sim provides its own API key if the user hasn't configured one (via BYOK or env var). Usage is metered and billed to the workspace.

## Overview

| Step | What | Where |
|------|------|-------|
| 1 | Register BYOK provider ID | `tools/types.ts`, `app/api/workspaces/[id]/byok-keys/route.ts` |
| 2 | Research the API's pricing and rate limits | API docs / pricing page (before writing any code) |
| 3 | Add `hosting` config to the tool | `tools/{service}/{action}.ts` |
| 4 | Hide API key field when hosted | `blocks/blocks/{service}.ts` |
| 5 | Add to BYOK settings UI | BYOK settings component (`byok.tsx`) |
| 6 | Summarize pricing and throttling comparison | Output to user (after all code changes) |

## Step 1: Register the BYOK Provider ID

Add the new provider to the `BYOKProviderId` union in `tools/types.ts`:

```typescript
export type BYOKProviderId =
  | 'openai'
  | 'anthropic'
  // ...existing providers
  | 'your_service'
```

Then add it to `VALID_PROVIDERS` in `app/api/workspaces/[id]/byok-keys/route.ts`:

```typescript
const VALID_PROVIDERS = ['openai', 'anthropic', 'google', 'mistral', 'your_service'] as const
```

## Step 2: Research the API's Pricing Model and Rate Limits

**Before writing any `getCost` or `rateLimit` code**, look up the service's official documentation for both pricing and rate limits. You need to understand:

### Pricing

1. **How the API charges** — per request, per credit, per token, per step, per minute, etc.
2. **Whether the API reports cost in its response** — look for fields like `creditsUsed`, `costDollars`, `tokensUsed`, or similar in the response body or headers
3. **Whether cost varies by endpoint/options** — some APIs charge more for certain features (e.g., Firecrawl charges 1 credit/page base but +4 for JSON format, +4 for enhanced mode)
4. **The dollar-per-unit rate** — what each credit/token/unit costs in dollars on our plan

### Rate Limits

1. **What rate limits the API enforces** — requests per minute/second, tokens per minute, concurrent requests, etc.
2. **Whether limits vary by plan tier** — free vs paid vs enterprise often have different ceilings
3. **Whether limits are per-key or per-account** — determines whether adding more hosted keys actually increases total throughput
4. **What the API returns when rate limited** — HTTP 429, `Retry-After` header, error body format, etc.
5. **Whether there are multiple dimensions** — some APIs limit both requests/min AND tokens/min independently

Search the API's docs/pricing page (use WebSearch/WebFetch). Capture the pricing model as a comment in `getCost` so future maintainers know the source of truth.

### Setting Our Rate Limits

Our rate limiter (`lib/core/rate-limiter/hosted-key/`) uses a token-bucket algorithm applied **per billing actor** (workspace). It supports two modes:

- **`per_request`** — simple; just `requestsPerMinute`. Good when the API charges flat per-request or cost doesn't vary much.
- **`custom`** — `requestsPerMinute` plus additional `dimensions` (e.g., `tokens`, `search_units`). Each dimension has its own `limitPerMinute` and an `extractUsage` function that reads actual usage from the response. Use when the API charges on a variable metric (tokens, credits) and you want to cap that metric too.

When choosing values for `requestsPerMinute` and any dimension limits:

- **Stay well below the API's per-key limit** — our keys are shared across all workspaces. If the API allows 60 RPM per key and we have 3 keys, the global ceiling is ~180 RPM. Set the per-workspace limit low enough (e.g., 20-60 RPM) that many workspaces can coexist without collectively hitting the API's ceiling.
- **Account for key pooling** — our round-robin distributes requests across `N` hosted keys, so the effective API-side rate per key is `(total requests) / N`. But per-workspace limits are enforced *before* key selection, so they apply regardless of key count.
- **Prefer conservative defaults** — it's easy to raise limits later but hard to claw back after users depend on high throughput.

## Step 3: Add `hosting` Config to the Tool

Add a `hosting` object to the tool's `ToolConfig`. This tells the execution layer how to acquire hosted keys, calculate cost, and rate-limit.

```typescript
hosting: {
  envKeyPrefix: 'YOUR_SERVICE_API_KEY',
  apiKeyParam: 'apiKey',
  byokProviderId: 'your_service',
  pricing: {
    type: 'custom',
    getCost: (_params, output) => {
      if (output.creditsUsed == null) {
        throw new Error('Response missing creditsUsed field')
      }
      const creditsUsed = output.creditsUsed as number
      const cost = creditsUsed * 0.001 // dollars per credit
      return { cost, metadata: { creditsUsed } }
    },
  },
  rateLimit: {
    mode: 'per_request',
    requestsPerMinute: 100,
  },
},
```

### Hosted Key Env Var Convention

Keys use a numbered naming pattern driven by a count env var:

```
YOUR_SERVICE_API_KEY_COUNT=3
YOUR_SERVICE_API_KEY_1=sk-...
YOUR_SERVICE_API_KEY_2=sk-...
YOUR_SERVICE_API_KEY_3=sk-...
```

The `envKeyPrefix` value (`YOUR_SERVICE_API_KEY`) determines which env vars are read at runtime. Adding more keys only requires bumping the count and adding the new env var.

### Pricing: Prefer API-Reported Cost

Always prefer using cost data returned by the API (e.g., `creditsUsed`, `costDollars`). This is the most accurate because it accounts for variable pricing tiers, feature modifiers, and plan-level discounts.

**When the API reports cost** — use it directly and throw if missing:

```typescript
pricing: {
  type: 'custom',
  getCost: (params, output) => {
    if (output.creditsUsed == null) {
      throw new Error('Response missing creditsUsed field')
    }
    // $0.001 per credit — from https://example.com/pricing
    const cost = (output.creditsUsed as number) * 0.001
    return { cost, metadata: { creditsUsed: output.creditsUsed } }
  },
},
```

**When the API does NOT report cost**