audio-sonic-mcp

# 🎵 Audio Sonic MCP

[![Tests](https://github.com/ripunjay-kashyap/audio-sonic-mcp/actions/workflows/test.yml/badge.svg)](https://github.com/ripunjay-kashyap/audio-sonic-mcp/actions/workflows/test.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/)
[![MCP](https://img.shields.io/badge/MCP-server-8A2BE2.svg)](https://modelcontextprotocol.io/)

**Turn any song into a structured "sonic signature" — extracting tempo, musical key, a 512-dimension CLAP vibe embedding, human-readable vibe tags, and a production profile — from a single local call.**

Audio Sonic MCP runs entirely on your local machine (requiring no API keys, external servers, or cloud dependencies) and exposes two premium access points to the same underlying high-fidelity audio analysis engine:

| | Tailored For | Core Interface & Mechanics |
|---|---|---|
| 🤖 **MCP Server** | LLMs, AI agents, & IDEs (Claude, Cursor, Windsurf, Cline) | Asynchronous, fire-and-forget analysis of YouTube URLs. Avoids blocking client LLMs during heavy audio processing. |
| 🎚️ **Local CLI** | Musicians, sound producers, & audio engineers | Deep command-line tool targeting local files for full-song multi-window analysis and high-fidelity output. |

---

## 🎹 Quick Taste: What You Get

### 1. Musician-Friendly CLI Summary (`--summary` mode)
```text
🎵 SONIC SIGNATURE — my_demo.mp3  (3:24)

  TEMPO    153.8 BPM  (steady)
  KEY      G Major  ·  shifts to G Phrygian @0:30   (confidence 74%)
  VIBE     aggressive · dark · driving · hip-hop · gritty

  PRODUCTION
     Vocals     forward
     Punch      0.62  (moderate)
     Stereo     wide
     Low end    ~55 Hz dominant

  Overall confidence: 88%   ·   analyzed in 0:28 (GPU-accelerated)
```

### 2. Comprehensive JSON (Returned by MCP and CLI by default)
```json
{
  "header": {
    "job_id": "sig_a3f9b2c1",
    "status": "success",
    "confidence_score": 0.88,
    "source_metadata": {
      "title": "Acoustic Vibe Demo",
      "duration_sec": 204,
      "source_type": "file"
    }
  },
  "sonic_signature": {
    "bpm": 153.8,
    "bpm_engine": "madmom",
    "bpm_variable": false,
    "key": "G Major",
    "key_variable": true,
    "key_map": [
      { "start_sec": 0.0,  "end_sec": 30.0, "key": "G Major" },
      { "start_sec": 30.0, "end_sec": 90.0, "key": "G Phrygian" }
    ],
    "mode_confidence": 0.74,
    "vibe_vector": [0.012, -0.034, "... 512 float dimensions ..."],
    "vibe_tags": ["aggressive", "dark", "driving", "hip-hop", "gritty"],
    "production_profile": {
      "vocal_presence": "forward",
      "transient_punch": 0.62,
      "stereo_width": "wide",
      "dominant_freq_peaks_hz": {
        "harmonic": [55.0, 110.2],
        "percussive": [125.0, 250.1]
      }
    }
  },
  "telemetry": {
    "inference_time_sec": 28.0
  }
}
```

---

## ⚡ Key Features

* 🥁 **Tempo & Beat Tracking** — Full BPM computation with variable-tempo drift detection and transient windowing.
* 🎹 **Key & Harmonic Mapping** — Computes structural musical key + mode, generating a detailed `key_map` tracking section-by-section modulations.
* 🌈 **Vibe & Style Embeddings** — Compiles a 512-dimensional CLAP embedding and human-readable style tags (covering energy, texture, mood, and genre) using zero-shot music vocab classification.
* 🎚️ **Production Analytics** — Measures vocal spatial presence, transient punch coefficients, stereo width, and dominant frequency peaks.
* 🤖 **MCP-Native System** — Fully exposes 4 standardized Model Context Protocol tools for instant integration into AI tools.
* 🪶 **Robust Graceful Degradation** — Automatically utilizes a CUDA GPU if present and falls back to CPU; gracefully degrades to HPSS and standard librosa feature arrays if heavy deep learning packages (`[clap]`) are omitted.
* 🔒 **100% Offline & Private** — All conversion, separation, and inference occur locally.

---

## 📦 Installation & Setup

### System Prerequisites
Ensure you have **Python 3.10+** and **FFmpeg** installed and accessible on your system `PATH`.

#### Installing FFmpeg:
* **macOS**: `brew install ffmpeg`
* **Linux (Debian/Ubuntu)**: `sudo apt update && sudo apt install -y ffmpeg`
* **Windows**: Run `winget install Gyan.FFmpeg` via PowerShell (Administrator), or download manually from [ffmpeg.org](https://ffmpeg.org/download.html) and add the `bin` directory to your system environment variables.

---

### Step-by-Step Installation

1. **Clone the Repository**
   ```bash
   git clone https://github.com/ripunjay-kashyap/audio-sonic-mcp.git
   cd audio-sonic-mcp
   ```

2. **Initialize Virtual Environment**
   ```bash
   python -m venv .venv
   # Activate on macOS/Linux:
   source .venv/bin/activate
   # Activate on Windows (PowerShell):
   .venv\Scripts\activate
   ```

3. **Install Dependencies**
   Choose between the lightweight core engine or the full high-fidelity ML suite:
   
   * **Option A: Full High-Fidelity ML Suite (Recommended)**
     Includes demixing stems (Demucs) and zero-shot vibe vectors (CLAP). Requires ~4 GB disk space.
     ```bash
     pip install -e ".[clap]"
     ```
   * **Option B: Core Lightweight Pipeline**
     Uses standard digital signal processing (HPSS/librosa). Rapid install and minimal footprint.
     ```bash
     pip install -e .
     ```

> [!NOTE]
> The optional `[clap]` stack installs `torch`, `torchaudio`, `transformers`, and `demucs`. Without these, the server automatically switches to light fallbacks (HPSS instead of Demucs, standard feature matrices instead of CLAP vectors, and leaves out `vibe_tags`).

---

## 🤖 MCP Client Configuration Guide

Audio Sonic MCP registers itself as a standard package script. This enables you to run it using the global executable name (`audio-sonic-mcp`) directly from your virtual environment's bin folder, or run the script file manually.

### 1. Claude Desktop Setup
Open your Claude configuration file:
* **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
* **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`
* **Linux**: `~/.config/Claude/claude_desktop_config.json`

Add the server to your `mcpServers` object:

```json
{
  "mcpServers": {
    "audio-sonic-mcp": {
      "command": "C:\\path\\to\\audio-sonic-mcp\\.venv\\Scripts\\audio-sonic-mcp.exe",
      "args": [],
      "env": {
        "JOBS_ROOT": "C:\\path\\to\\audio-sonic-mcp\\jobs"
      }
    }
  }
}
```

> [!IMPORTANT]
> **Windows Users**: Always use **double backslashes** (`\\`) in JSON configuration paths. Point the executable directly to the `.exe` inside your `.venv\Scripts\` directory.

---

### 2. Cursor IDE Integration
To integrate Audio Sonic MCP into Cursor's AI pane:
1. Navigate to **Settings** ➔ **Features** ➔ **MCP**.
2. Click **+ Add New MCP Server**.
3. Fill in the parameters:
   * **Name**: `audio-sonic-mcp`
   * **Type**: `command`
   * **Command**: `/path/to/audio-sonic-mcp/.venv/bin/audio-sonic-mcp` (use `.exe` extension on Windows)

---

### 3. Windsurf Integration
Open your Windsurf MCP configurations file (typically found at `~/.codeium/windsurf/mcp_config.json`) and append the configuration:

```json
{
  "mcpServers": {
    "audio-sonic-mcp": {
      "command": "/path/to/audio-sonic-mcp/.venv/bin/python",
      "args": ["/path/to/audio-sonic-mcp/server.py"],
      "env": {
        "JOBS_ROOT": "/path/to/audio-sonic-mcp/jobs"
      }
    }
  }
}
```

---

### 4. Cline (VS Code Extension) Setup
Open Cline's MCP setting file (usually located at `%APPDATA%\Code\User\globalStorage\saoudrizwan.claude-dev\settings\cline_mcp_settings.json` or equivalent platform storage) and add:

```json
{
  "mcpServers": {
    "audio-sonic-mcp": {
      "command": "/path/to/audio-sonic-mcp/.venv/bin/audio-sonic-mcp",
      "args": [],
      "env": {
        "JOBS_ROOT": "/path/to/audio-sonic-mcp/jobs"
      }
    }
  }
}
```

---

## 🤖 Interaction Flow for AI Agents & LLMs

LLMs automatically learn how to use this server by reading its exposed tool definitions. Because audio stem separation and CLAP embeddings are computationally demanding, Audio Sonic MCP uses an **Asynchronous Fire-and-Forget Job Pattern**.

### Automated LLM Workflow
```
  [User Prompts LLM]
          │
          ▼
1. Submit URL ──────────────► [Tool: get_sonic_signature]
                                      │ (Returns Job ID instantly)
                                      ▼
2. Notify User ◄───────────── [LLM acknowledges job is queued]
          │
          ├───► 3. Wait 10-15s (Or proceed with other tasks)
          │
          ▼
4. Check Progress ──────────► [Tool: get_job_status]
                                      │ (Checks status: running/success/error)
                                      ▼
5. Present Signature ◄─────── [LLM formats rich output for user]
```

### Natural Prompts to Try
* *"Check the health of my audio-sonic-mcp server to make sure all ML components are ready."*
* *"Submit this YouTube track for sonic analysis: `https://www.youtube.com/watch?v=XXXXXX`."*
* *"Check the progress of my sonic signature job `sig_a1b2c3d4` and summarize the BPM, production width, and vibe once complete."*

---

## 🎚️ CLI Usage (Local Files)

For musicians, engineers, and producers working directly in the terminal, you can analyze a full-length local file directly without running any background servers:

```bash
# Get a visual, musician-friendly sonic signature digest (recommended)
python analyze_file.py "my_demo.wav" --summary

# Print full raw JSON directly to the stdout stream
python analyze_file.py "my_demo.wav"

# Dump JSON payload to a file while keeping the stdout clean
python analyze_file.py "my_demo.wav" > signature.json
```

### CLI Command Options Reference

| Option | Shorthand | Description |
|---|---|---|
| `path` | *None* | Absolute or relative path to the local audio file (Required). |
| `--summary` | `-s` | Print a clean, formatted terminal summary