songbrain

# Songbrain

[![CI](https://github.com/songbrain-ai/songbrain/actions/workflows/ci.yml/badge.svg)](https://github.com/songbrain-ai/songbrain/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/songbrain?label=pypi)](https://pypi.org/project/songbrain/)
[![npm](https://img.shields.io/npm/v/songbrain)](https://www.npmjs.com/package/songbrain)
[![MCP Registry](https://img.shields.io/badge/MCP%20Registry-io.github.songbrain--ai%2Fsongbrain-6e56cf)](https://registry.modelcontextprotocol.io/v0/servers?search=io.github.songbrain-ai/songbrain)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

**Song in, video plan out — the music analysis API for AI video.**

Send a song. Get back JSON that a video model can use directly: what the song is, where the beats and sections are, which part to use, what the words are and when they are sung, what the video should show, and a shot list with a prompt for every scene that cuts on the beat.

Under the hood that is beat detection with downbeats, song structure analysis (sections with repeat letters), best-moment detection and word-timed lyrics in one hosted API, plus an MCP server at `https://api.songbrain.ai/v1/mcp` so Claude, Cursor or ChatGPT can read a song directly ([setup guide](https://www.songbrain.ai/guides/mcp-server-music-analysis)).

This repository holds the official SDKs for Python and JavaScript/TypeScript, examples, and the MCP server entry.

- API docs: https://www.songbrain.ai/docs/api
- OpenAPI: https://api.songbrain.ai/v1/openapi.json
- Get a key (5 free songs a month): https://app.songbrain.ai/developers
- Changelog: [SDKs](CHANGELOG.md) · [API](https://www.songbrain.ai/docs/api/changelog)

## Try it now, no key

```bash
curl https://api.songbrain.ai/v1/examples/old-truck-home/shot-plan
```

This returns the story and shot plan of a real song in exactly the format your songs get. `GET /v1/examples` lists all examples.

## What one call returns

One document per song (schema `songbrain.song/1`). Times are seconds from the start of the song.

| Section | What is in it |
|---|---|
| `song_dna` | Genre and subgenre (37 subgenres), tempo, key and key changes, mood, vocal style, instruments, energy, loudness (LUFS, true peak, per-platform check), a tagline and similar tracks |
| `timeline` | Every beat, every downbeat, time signature, sections with repeat letters (A, B, A) and energy, vocal changes, lyric hooks |
| `best_moments` | Ranked windows with start, peak and end, a score, the reason they work and the measured signals behind it, the beats inside, the words sung |
| `lyrics` | Lines with start and end, and every word with its own start and end |
| `scores` | Virality, quality and lyrics scores with breakdowns, what works, what to fix, audience |
| `story` | What the song means, one world, a three-colour palette, the one element the video is about (before, event, after), setup / turn / payoff |
| `shot_plan` | A beat-synced edit of the best moment (about 15 s, 9:16): every scene with absolute start and end seconds, act, kind, camera motion, transition, framing and a ready image/video prompt. Plus a section-level plan for the full song with cut points on the beat |

One scene from the shot plan:

```json
{
  "index": 1,
  "kind": "tease",
  "act": "setup",
  "start_sec": 94.83,
  "end_sec": 96.246,
  "duration_sec": 1.416,
  "beats_in_scene": 4,
  "transition_in": "cold open, hold",
  "motion": "slow push-in on a still world",
  "prompt": "wide establishing shot of the empty space: A distant, warm yellow light source pulses faintly in the deep blue night ...",
  "sung": null
}
```

A song takes 60–90 seconds to analyse (up to about 2 minutes when busy).

## Prompts for video models, starting from the best moment

Most of a song is not worth animating. Songbrain ranks the strongest windows first (`best_moments`, each with the reason it works and the audio signals behind it) and builds the `shot_plan` on the top one: about 15 seconds, cut on the beat, one scene per cut. Every scene carries a ready prompt for an image or video model, written from the song's story, world and three-colour palette, so the scenes look like one film instead of unrelated clips.

```json
{"index": 8, "act": "turn", "start_sec": 102.965, "end_sec": 103.282, "motion": "impact: frame lands displaced and settles, fast energy",
 "prompt": "tight detail shot: The truck's grille, showing rust, illuminated by the faint glow of the low headlight. Faded brown and warm yellow, in ..."}
```

Pass the prompts to Kling, Runway, Luma, Veo or a hosted model on fal or Replicate, and cut the clips at the given seconds. Recipes: [render-with-your-model.md](examples/render-with-your-model.md), [song_to_clips_fal.py](cookbook/song_to_clips_fal.py), [song_to_clips_replicate.py](cookbook/song_to_clips_replicate.py). The full-song plan (`shot_plan.full_song`) gives the cut points for a complete video.

## Coming from Spotify Audio Features or Audio Analysis?

Since November 27, 2024, new Spotify apps get a 403 on `/audio-features` and `/audio-analysis`. Songbrain returns the same kind of data from any audio file, released or not, and more:

| Spotify | Songbrain | Note |
|---|---|---|
| `tempo` | `song_dna.tempo_bpm` | plus `tempo_confidence` |
| `key` + `mode` | `song_dna.key` | `"C minor"`; `key_changes[]` with the second each one happens |
| `energy` | `song_dna.energy` | 0–1, plus energy per section |
| `loudness` | `song_dna.loudness_lufs` | EBU R128 integrated loudness and true peak |
| `valence`, `danceability` | `song_dna.mood[]` | mood words rather than a number |
| `instrumentalness`, `speechiness` | `has_vocals`, `song_dna.vocals` | vocal style or `"instrumental"` |
| audio-analysis `beats` | `timeline.beats` | every beat in seconds |
| audio-analysis `bars` | `timeline.downbeats` | first beat of each bar |
| audio-analysis `sections` | `timeline.sections` | with repeat letters (A, B, A …) and relative energy |
| — | `best_moments`, `lyrics`, `story`, `shot_plan` | not in Spotify: the strongest windows and why, word timings, a shot list |

Migration guide with a ten-line example: https://www.songbrain.ai/guides/spotify-audio-features-alternative

## Install

```bash
pip install songbrain     # Python 3.9+, one dependency (requests)
npm install songbrain     # Node 18+, no runtime dependencies, ESM + CJS + types
```

Packages: [PyPI](https://pypi.org/project/songbrain/) · [npm](https://www.npmjs.com/package/songbrain).

## Quickstart: Python

```python
from songbrain import Songbrain

sb = Songbrain()                                   # reads SONGBRAIN_API_KEY
song = sb.analyze("song.mp3")                      # or analyze(audio_url="https://…")

dna = song["song_dna"]
print(dna["genre"], dna["tempo_bpm"], dna["key"])
for s in song["shot_plan"]["clip"]["scenes"]:
    print(f'{s["start_sec"]:.2f}-{s["end_sec"]:.2f} [{s["act"]}] {s["prompt"]}')
```

## Quickstart: TypeScript / JavaScript

```ts
import { Songbrain } from "songbrain";

const sb = new Songbrain();                        // reads SONGBRAIN_API_KEY
const song = await sb.analyze({ file: "song.mp3" }); // or { audioUrl: "https://…" }

const dna = song.song_dna!;
console.log(dna.genre, dna.tempo_bpm, dna.key);
for (const s of song.shot_plan?.clip?.scenes ?? []) {
  console.log(`${s.start_sec.toFixed(2)}-${s.end_sec.toFixed(2)} [${s.act}] ${s.prompt}`);
}
```

`analyze()` uploads, waits until the song is done and returns the full document. Pass `wait=False` (Python) or `wait: false` (JS) to get the id right away, then use a webhook or `get_song` / `getSong` later.

## SDK methods

Python uses snake_case and JS uses camelCase method names. Response fields are the API's own names in both.

| Python | JavaScript | API | Key |
|---|---|---|---|
| `analyze(file \| audio_url=…, test=False, idempotency_key=None, wait=True)` | `analyze({ file \| audioUrl, test, idempotencyKey, wait })` | `POST /v1/songs` + polling | yes |
| `get_song(id, view, include)` | `getSong(id, { view, include })` | `GET /v1/songs/{id}` | yes |
| `shot_plan(id)` | `shotPlan(id)` | `GET /v1/songs/{id}/shot-plan` | yes |
| `list_songs(limit, starting_after)` | `listSongs({ limit, startingAfter })` | `GET /v1/songs` | yes |
| `iter_songs(page_size)` | `iterSongs({ pageSize })` | `GET /v1/songs`, every page | yes |
| `delete_song(id)` | `deleteSong(id)` | `DELETE /v1/songs/{id}` | yes |
| `account()` | `account()` | `GET /v1/account` | yes |
| `test_webhook(url)` | `testWebhook(url)` | `POST /v1/webhooks/test` | yes |
| `webhook_deliveries(limit)` | `webhookDeliveries({ limit })` | `GET /v1/webhooks/deliveries` | yes |
| `pricing()` | `pricing()` | `GET /v1/pricing` | no |
| `examples()` | `examples()` | `GET /v1/examples` | no |
| `example(id, view, include)` | `example(id, { view, include })` | `GET /v1/examples/{id}` | no |
| `example_shot_plan(id)` | `exampleShotPlan(id)` | `GET /v1/examples/{id}/shot-plan` | no |
| `webhooks.verify(body, header, secret)` | `verifyWebhook(body, header, secret)` | `Songbrain-Signature` header | – |

Both SDKs:

- send the key as `Authorization: Bearer sb_live_…`,
- raise `SongbrainError` (`status`, `code`, `message`, `request_id` / `requestId`) and the subclasses `AuthenticationError`, `InsufficientCredits`, `NotFound`, `RateLimited` (with `retry_after` / `retryAfter`), `AnalysisFailed` and `WaitTimeout`,
- send an `Idempotency-Key` with every new song and retry network errors, 429 and 5xx up to 3 times with backoff, honouring `Retry-After`,
- keep the last rate-limit headers in `last_rate_limit` / `lastRateLimit`.

Package docs: [python/README.md](python/README.md) · [js/README.md](js/README.md)

## Without an SDK

```bash
# 1. send a file (or JSON {"audio_url": "https://…"})
curl -X POST https://api.songbrain.ai/v1/songs \
  -H "Authorization: Bearer $SONGBRAIN_API_KEY" \
  -F "file=@song.mp3" -F "title=My Song"
# → 202 {"id": "…", "status": "processing", "eta_sec": 75, "billing": {…}}

# 2. poll until stat