realtime-avatar-sdk

<div align="center">

# Realtime Avatar

**Live voice and video characters you can interrupt.**

Your footage. Your voice. Your persona. Talking back in real time — and stopping
mid-sentence the moment someone cuts in, the way a person stops.

[![npm](https://img.shields.io/npm/v/realtime-avatar?color=1a7f37&label=npm)](https://www.npmjs.com/package/realtime-avatar)
[![CI](https://github.com/theinfluencecompany/realtime-avatar-sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/theinfluencecompany/realtime-avatar-sdk/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
[![Docs](https://img.shields.io/badge/docs-realtimeavatar.ai-6f42c1)](https://realtimeavatar.ai/docs)
[![Agent-ready](https://img.shields.io/badge/agent--ready-AGENTS.md-ff6b35)](./AGENTS.md)

[Quickstart](#quickstart) · [Examples](#examples) · [Packages](#packages) · [API](#api) · [Docs](https://realtimeavatar.ai/docs)

</div>

---

## Quickstart

For Claude Code or Cursor, install the [shared Realtime Avatar plugin](./plugins/realtime-avatar/README.md).
It connects your account's avatars, credits and clips to the agent, with opt-in writes
and a shared workflow for building an avatar app. The [plugin source and packaging](./integrations/agent-plugin/README.md)
also document an optional, separately installed hosted call preview.

```bash
npm install --save-exact realtime-avatar@0.17.0
```

Start the call on your **server**. Your key never touches a browser.

```ts
import { RealtimeAvatar, isQueued } from "realtime-avatar";

const rta = new RealtimeAvatar({ apiKey: process.env.REALTIME_AVATAR_API_KEY! });

const call = await rta.startCall({
  avatarId: "ava_…",
  instructions: "You are Rin. Short, warm, specific sentences.",
  maxSeconds: 600,
});

if (isQueued(call)) return { queued: true, position: call.position };
return call.raw;            // relay to the browser, byte-for-byte
```

Join it from your **client**.

```tsx
import { AvatarCall, createProxyClient } from "realtime-avatar/react";

// Holds a URL, not a credential. Your route holds the key.
const client = createProxyClient({ proxyUrl: "/api/realtime-avatar" });

<AvatarCall client={client} avatarId="ava_…" onEnded={({ reason }) => router.push("/")} />;
```

That's the whole integration. Two halves, one payload between them.

> [!TIP]
> Not on Node? The server half is plain HTTP — see
> [`apps/quickstart/python-fastapi`](./apps/quickstart/python-fastapi) for the same split
> from a Python backend.

---

## Why this one

|  |  |
| --- | --- |
| 🎙️ **Full duplex, always** | They listen the entire time they speak. Interrupt and they stop. A cough doesn't derail them; a pause isn't the end of your turn. Video doesn't cost you this. |
| 🎭 **Your character, not a stock presenter** | Hand it one still image: the resting loop and a starter motion library are generated from it — no footage to shoot, no clips to host. Direct her further in JSON (`setClipLibrary`), or re-shoot her later with footage of your own. ([the one rule](./AGENTS.md#the-one-rule-that-decides-whether-multi-clip-looks-good) if you do: every clip starts and ends on the same frame.) |
| 🛠️ **Tools run on your machine** | Your functions, your data, your secrets — never uploaded. They're called mid-conversation and have 2.5s to answer. |
| 💸 **Priced to leave running** | Under $5 per hour of live conversation, billed by the second — not rounded up to the minute. |
| 🧾 **An itemised bill** | `listSessions` returns every session: when it ran, how long, what it cost. Tag calls with your own user id to attribute them. |
| 🤖 **Written for coding agents** | [`AGENTS.md`](./AGENTS.md) ships in the package. Point your agent at it and it writes the integration correctly the first time. |

---

## How it fits together

```
your client  ──▶  your backend  ──▶  Realtime Avatar
                  (holds the key,      (capacity, listening,
                   decides the call)    thinking, speaking, rendering)
     ◀────────── live audio + video, and they're still listening ──────────
```

Two rules fall out of that picture, and they drive everything else:

1. **The key is server-only.** A browser holding it can start unlimited calls on your
   account. The constructor throws in a browser runtime, so that fails loudly instead of
   quietly costing you money.
2. **The connection payload is opaque.** Relay `call.raw` untouched — the client validates
   it strictly and will reject a reshaped object.

---

## Built with an AI agent?

Most people integrating this in 2026 aren't typing it by hand. So the reference is written
for the reader that shows up: **[`AGENTS.md`](./AGENTS.md)** is the entire surface in one
file, plus the twelve rules that are expensive to learn the hard way.

```
"Add a voice call to my Next.js app using the Realtime Avatar SDK.
 Read AGENTS.md in node_modules/realtime-avatar first."
```

`CLAUDE.md` symlinks to it, so Claude Code picks it up with no prompting. Codex, Cursor and
Copilot read `AGENTS.md` directly. There's also an
**[MCP server](./libs/mcp)** if you'd rather your agent query your avatars, balance and bill
live instead of reading about them.

---

The SDK exports composable validation schemas, so Zod 4.4.3 or a compatible newer
Zod 4 is a required peer. Modern npm/Bun installs peers automatically; applications
that already use Zod should declare one compatible version at their root. This is
a JavaScript dependency and introduces no native module.

## Packages

**One.** `npm install --save-exact realtime-avatar@0.17.0` and you have the server client, the route adapters, the
React bindings and the browser helpers. Everything is a subpath, treeshaken per entry, so a
server-only app still ships 18.8 KB with no React and no LiveKit in it.

The credential boundary is enforced by the build, not by the package name: importing a
key-holding entry into a browser bundle **throws at bundle time**. `realtime-avatar-mcp` is a
separate name only because it is a CLI you run with `npx` and never install.

**A server-only app pays for nothing it does not import.** `.`, `/server`, the route adapters,
`/browser` and `/tools` have **zero** external imports — `npm run externals` prints that per entry
and fails the build if it stops being true.

One runtime dependency comes along: `zod` (4.3 MB), which `/react` and `/react-native` need for
runtime validation. It is deliberately **not** bundled — inlining it costs every React app **+64 KB
gzipped** in shipped bytes (15 KB → 79 KB gzip, measured), and shipped bytes are what your users
feel, while `node_modules` is what a developer feels once. React, `livekit-client`, and the
`@livekit/*` packages are optional peers — install the group for your platform. Native compatibility
and the exact verified dependency tuple are documented in the [package README](./libs/sdk-server/README.md#react-native-compatibility).

| Package | Import | What it does |
| --- | --- | --- |
| [`libs/sdk-server`](./libs/sdk-server) | `realtime-avatar` | The whole SDK, one install. `.` and the adapters hold your key. |
| | `realtime-avatar/server` | The same client, without the route adapters |
| | `realtime-avatar/{nextjs,hono,express,tanstack-start}` | Mount a route that keeps the key server-side |
| | `realtime-avatar/react` | React bindings. **Never holds a key.** |
| | `realtime-avatar/react-native` | The same, for Expo and React Native |
| | `realtime-avatar/browser` | Mic and playback, with the six failure modes turned into values |
| | `realtime-avatar/tools` | The browser tool plane — your functions, called mid-conversation |
| | `realtime-avatar/recording` | Client-safe recording schemas and derived types |
| [`libs/mcp`](./libs/mcp) | `realtime-avatar-mcp` | MCP server for coding agents |

---

## Examples

Every example runs standalone — copy one out of the repo and it still works.

### `apps/quickstart/` — the smallest correct integration

| | |
| --- | --- |
| [**nextjs-app-router**](./apps/quickstart/nextjs-app-router) | A route handler and a call button. Start here. |
| [**python-fastapi**](./apps/quickstart/python-fastapi) | The same split from Python, plus verifying the signed session history |
| [**canvas-tools**](./apps/quickstart/canvas-tools) | The tool plane end to end — four tools drawing on a live canvas, including one that deliberately blows the deadline |

### `apps/demo/` — the showcases

| | |
| --- | --- |
| [**coding-companion**](./apps/demo/coding-companion) | A builder you talk to: the character is the conversation, a cheaper model writes the page, and they ship it to a public URL through the tool plane |
| [**pair-programmer**](./apps/demo/pair-programmer) | The same builder, composited into the room instead of sat beside it — same brief, same tools, same build engine, so the pair shows what a layout changes and what it does not |
| [**livestream**](./apps/demo/livestream) | An audience is not a new primitive — comments, gifts and hearts all ride one chat topic |
| [**live-shopping**](./apps/demo/live-shopping) | The character is not allowed to know the price — every commercial figure comes back from a tool and expires when the fact behind it does |
| [**math-studio**](./apps/demo/math-studio) | A tool plane that stays six tools wide however far the app grows — the things on screen register with the page, not with the character |
| [**terminal-tutor**](./apps/demo/terminal-tutor) | The character draws on the screen to teach vim and tmux, and never sends a coordinate — they name the thing, the page resolves where it is |
| [**dating-rehearsal**](./apps/demo/dating-rehearsal) | A first date you practice on, and four tools that are all the character's own call — he rates his interest into a live meter, pins the beats that turned it, walks out if it dies, and writes the debrief |
| [**persuasion**](./apps/demo/persuasion) | A win condition that is the character's own judgement — she concedes only by calling a `concede`