Rapid-MLX

Rapid-MLX is a local AI inference server for Apple Silicon Macs that exposes an OpenAI-compatible HTTP endpoint, allowing any tool that targets the OpenAI API to run against locally hosted models instead. It installs via pip, Homebrew, or a one-line curl script, then serves models including Qwen3.5, Gemma 4, DeepSeek V4, and GPT-OSS through a FastAPI backend built on Apple's MLX framework. Claude Code users point their base URL to localhost:8000 to get fully local, zero-cost inference, and the server also works with Cursor and Aider the same way. The project ships 17 tool-call parsers covering multiple model families, a prompt cache yielding 0.08-second cached time-to-first-token, and a reasoning-separation feature that strips chain-of-thought tokens before returning responses. A cloud routing option falls back to remote providers when needed. Benchmarks on an M3 Ultra show GPT-OSS 20B running at 119 tokens per second, and the project claims 2.3x throughput over Ollama under four-concurrent-user load using identical model weights. Developers running local coding assistants on macOS hardware with 16 GB or more of unified memory are the primary audience.

What people ask about Rapid-MLX