Rapid-MLX docs · v0.15.7

Docs

Rapid-MLX runs open models on your Mac's GPU and serves them through an OpenAI- and Anthropic-compatible API. Three commands get you from nothing to a running model: 

$ curl -fsSL https://rapidmlx.com/install.sh | bash     # or: brew install rapid-mlx
$ rapid-mlx serve qwen3.5-4b-4bit                       # any alias, or an org/repo from Hugging Face
$ curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" \
    -d '{"model":"qwen3.5-4b-4bit","messages":[{"role":"user","content":"hi"}]}'

Or chat in the terminal with rapid-mlx chat — it starts and stops the server for you. Run rapid-mlx on its own for a one-screen start menu: chat, best model for this Mac, connect a coding agent, start a server, or choose a model. Full walkthrough: Install & first request.

Next, pick what you came for

Pick a model for my Mac
rapid-mlx recipe prints the smart and fast pick for your RAM; the tiers page explains each pick and links measured speeds on Macs like yours.
Bring your own model
Run a Hugging Face model that isn't in the catalog: checked before download, with an import and quantize path.
Connect a coding agent
One setup command per agent — Claude Code, Codex CLI, OpenCode, Pi, Aider and more.
Use a chat app
Open WebUI, SillyTavern and other chat front-ends on your local model.
Use it as an OpenAI-compatible server
Base URL http://localhost:8000/v1, port 8000, the endpoints, and curl, Python and Node examples.
Call the API
Chat completions, Responses, Anthropic Messages, embeddings, audio, images and video — and where they differ from OpenAI.
Run it as a service
rapid-mlx service install starts the server at boot on a headless Mac, with health-gated upgrades.
Something's wrong
Start with rapid-mlx doctor, then the common symptoms and fixes.

Agents & integrations

Rapid-MLX serves an OpenAI-compatible endpoint at localhost:8000/v1, so most tools work by just pointing them there. These guides give the exact per-tool config, tested against the engine.

All agent guides
Setup command, file written and backup behaviour for every supported agent, including Pi.
Claude Code Tier-1
Run Anthropic's coding agent against a local model — the launch claude-code setup and endpoint config.
Codex CLI Tier-1
OpenAI's Codex CLI on a local model, including the apply_patch tool path.
Aider Tier-1
Pair-programming in the terminal with a local model — model naming, editor format, and the config that avoids fallbacks to the cloud.
Cursor
Cursor chat and composer on a model served from your Mac, through an HTTPS tunnel.
Continue.dev
Wire the Continue VS Code / JetBrains extension to a local Rapid-MLX server.
All agents & frameworks matrix
The full support matrix — every agent and framework (LangChain, PydanticAI, smolagents, OpenAI SDK, Kilo Code, OpenCode, OpenHands…) with per-tool status and guides.

Hero models

These are the models where Rapid-MLX does engineering work beyond the upstream MLX-LM stack to serve them cleanly: our own MLX quants, alias configuration, and dedicated tool-call and reasoning parsers.

Measured model pages real numbers
The most-run models — Qwen3.5-4B/9B, Qwen3.6-35B-A3B, Qwen3.8 27B, Gemma 4, GPT-OSS 20B, Bonsai 27B and more — each with decode tok/s, first-token latency and peak memory, labelled with the Mac and engine version they were measured on, plus one command to serve it.
Tmax-9B / 27B agent model
Hybrid Gated-DeltaNet agent model from Hamish Ivison (Ai2). Rapid-MLX publishes its MLX quant family (4 / 6 / 8 bit + bf16) with aliases that work end-to-end with tool calling.
DiffusionGemma 26B-A4B-it diffusion
Discrete-diffusion text generation with parallel block decoding, served through Rapid-MLX's own diffusion lane.
UI-TARS computer-use
ByteDance's vision-grounded UI automation agent. 9 MLX quants from 1.5-7B through 72B-DPO with the custom ui_tars tool parser.
VibeThinker low-RAM reasoner
1.5B / 3B chain-of-thought reasoner small enough for any Apple Silicon Mac, with structured reasoning_content.
DeepSeek V4-Flash sparse attention
CSA + HCA sparse-attention V4 family. 2-bit runs on a single 256 GB M3 Ultra through Rapid-MLX's vendored kernel.
EmbeddingGemma 2 local embeddings
Google's text and code embedding model on a local OpenAI-compatible /v1/embeddings endpoint, next to your chat model — for RAG and semantic or code search. 1.13 GB at 4-bit.

Looking for everything else?

The full set of supported models — 197 text/vision/reasoning aliases (250 total incl. 44 audio and 10 video generation) across 15 vendor families including Qwen (3.8 / 3.6 / 3.5 / Coder / VL / legacy / 2.5 / Qwopus), Gemma (4 / 4-mobile / 3 / EmbeddingGemma), Llama, DeepSeek, GLM, Mistral, Phi, GPT-OSS, MiniMax, Hermes, Hunyuan, Liquid, Granite and a curated small-models bucket — is browseable as a family directory with one page per vendor, or as a flat alias catalog for ctrl-F. For "what fits my Mac" use the live RAM picker at models.rapidmlx.com.

Notable small models

Editorial short-list of small-but-interesting models worth watching outside the headline families — including Holo3.1-35B-A3B, VibeThinker, Tmax-9B, Granite 4 H-Micro, and EmbeddingGemma 2. The full callout lives on the model families page.

What is on by default

rapid-mlx serve <alias> applies the alias's tuned profile with no extra flags: the right tool-call and reasoning parsers, the prefix cache, compressed KV cache on the verified Qwen3.5 / 3.6 aliases, and MTP speculative decoding on the aliases qualified for it. PFlash long-prompt compression is opt-in (--pflash auto). The performance flags reference lists each technique, which aliases get it, and how to turn it off. Release history lives in the changelog.