Rapid-MLX docs · v0.12.18

Docs

Rapid-MLX is a high-performance, OpenAI-compatible inference server for Apple Silicon, built on MLX. These pages cover install + first request, the API surface, every supported model family (one page per family), the six hero deep dives, and what is coming next.

Organized by vendor family. The 175 LLM aliases (228 total incl. 44 audio) are grouped into 15 vendor families (Qwen, Gemma, Llama, Muse, Ling, DeepSeek, GLM, Mistral, Phi, Granite, GPT-OSS, MiniMax, Hermes, Hunyuan, Liquid, plus a curated small-models bucket). Each page nests its version lines (e.g. Qwen3.5 / 3.6 / Coder / VL) under <h2> sub-sections. Six models also get a hero deep dive because the engineering work to make them serve cleanly on MLX is non-trivial. See the model families directory to browse.

Start here

Install & first request
One-curl install, rapid-mlx serve, and an OpenAI-compatible cURL against localhost:8000/v1.
API surface
The endpoints Rapid-MLX exposes — chat, responses, embeddings, audio, vision — and where they match (or diverge from) OpenAI.

Agents & integrations

Rapid-MLX serves an OpenAI-compatible endpoint at localhost:8000/v1, so most tools work by just pointing them there. These guides give the exact per-tool config, tested against the engine.

Claude Code Tier-1
Run Anthropic's coding agent against a local model — the launch claude-code setup and endpoint config.
Codex CLI Tier-1
OpenAI's Codex CLI on a local model, including the apply_patch tool path.
Aider Tier-1
Pair-programming in the terminal with a local model — model naming, editor format, and the config that avoids fallbacks to the cloud.
Cursor
Point Cursor's OpenAI-compatible provider at your Mac and keep the codebase on-device.
Continue.dev
Wire the Continue VS Code / JetBrains extension to a local Rapid-MLX server.
All agents & frameworks matrix
The full support matrix — every agent and framework (LangChain, PydanticAI, smolagents, OpenAI SDK, Kilo Code, OpenCode, OpenHands…) with per-tool status and guides.

Hero models

These are the models where Rapid-MLX did the engineering work to make them serve cleanly on MLX before (or better than) the upstream MLX-LM stack handled them. We shipped first-mover MLX quants, alias configuration, parser support, and benched on M3 Ultra.

Measured model pages real numbers
The 18 most-run models, each with decode tok/s, first-token latency, peak memory and cold boot measured on the same M2 Pro — plus one command to serve it.
Tmax-9B / 27B first-mover
Hybrid Gated-DeltaNet agent model from Hamish Ivison (Ai2). We shipped the first MLX quant family (4 / 6 / 8 bit + bf16) and the alias fixes that make it work end-to-end with tool calling.
DiffusionGemma 26B-A4B-it day-1
Discrete-diffusion text generation with parallel block decoding. Day-1 support on Rapid-MLX while it is still listed as unsupported by upstream mlx-lm.
UI-TARS computer-use
ByteDance's vision-grounded UI automation agent. 9 MLX quants from 1.5-7B through 72B-DPO with the custom ui_tars tool parser.
VibeThinker low-RAM reasoner
1.5B / 3B chain-of-thought reasoner. 4-bit fits a 4 GB Mac with structured reasoning_content on /v1/responses.
DeepSeek V4-Flash sparse attention
CSA + HCA sparse-attention V4 family. 2-bit runs on a single 256 GB M3 Ultra; day-0 MLX support via the vendored kernel.

Looking for everything else?

The full set of supported models — 175 LLM aliases (228 total incl. 44 audio) across 15 vendor families including Qwen (3.5 / 3.6 / Coder / VL / legacy / 2.5 / Qwopus), Gemma (4 / 4-mobile / 3 / EmbeddingGemma), Llama, DeepSeek, GLM, Mistral, Phi, GPT-OSS, MiniMax, Hermes, Hunyuan, Liquid, Granite and a curated small-models bucket — is browseable as a family directory with one page per vendor, or as a flat alias catalog for ctrl-F. For "what fits my Mac" use the live RAM picker at models.rapidmlx.com.

Notable small models

Editorial short-list of small-but-interesting models worth watching outside the headline families — including Holo3.1-35B-A3B, VibeThinker, Tmax-9B, Granite 4 H-Micro, and EmbeddingGemma. The full callout lives on the model families page.

What shipped in 0.9

The 0.9 release focused on multi-user and long-context serving: compressed KV cache default-on for 9 popular MoE aliases, radix-tree prefix cache default-on for concurrent shared-prompt reuse, 128k context on Qwen3.5-9B without any flag flips, speculative decoding (MTP + DFlash) wired up and verified lossless when enabled, and 8+ hour agent-session stability. See the changelog for the full list and roadmap for what is planned next.