Rapid-MLX docs · v0.10.5

Docs

Rapid-MLX is a high-performance, OpenAI-compatible inference server for Apple Silicon, built on MLX. These pages cover install + first request, the API surface, every supported model family (one page per family), the five hero deep dives, and what is coming next.

Organized by vendor family. The 128 supported aliases are grouped into 12 vendor families (Qwen, Gemma, Llama, DeepSeek, GLM, Mistral, Phi, Granite, GPT-OSS, MiniMax, Hermes, and a curated small-models bucket). Each page nests its version lines (e.g. Qwen3.5 / 3.6 / Coder / VL) under <h2> sub-sections. Six models also get a hero deep dive because the engineering work to make them serve cleanly on MLX is non-trivial. See the model families directory to browse.

Start here

Install & first request
One-curl install, rapid-mlx serve, and an OpenAI-compatible cURL against localhost:8000/v1.
API surface
The endpoints Rapid-MLX exposes — chat, responses, embeddings, audio, vision — and where they match (or diverge from) OpenAI.

Hero models

These are the models where Rapid-MLX did the engineering work to make them serve cleanly on MLX before (or better than) the upstream MLX-LM stack handled them. We shipped first-mover MLX quants, alias configuration, parser support, and benched on M3 Ultra.

Tmax-9B / 27B first-mover
Hybrid Gated-DeltaNet agent model from Hamish Ivison (Ai2). We shipped the first MLX quant family (4 / 6 / 8 bit + bf16) and the alias fixes that make it work end-to-end with tool calling.
DiffusionGemma 26B-A4B-it day-1
Discrete-diffusion text generation with parallel block decoding. Day-1 support on Rapid-MLX while it is still listed as unsupported by upstream mlx-lm.
UI-TARS computer-use
ByteDance's vision-grounded UI automation agent. 9 MLX quants from 1.5-7B through 72B-DPO with the custom ui_tars tool parser.
VibeThinker low-RAM reasoner
1.5B / 3B chain-of-thought reasoner. 4-bit fits a 4 GB Mac with structured reasoning_content on /v1/responses.
DeepSeek V4-Flash sparse attention
CSA + HCA sparse-attention V4 family. 2-bit runs on a single 256 GB M3 Ultra; day-0 MLX support via the vendored kernel.

Looking for everything else?

The full set of supported models — 128 aliases across 12 vendor families including Qwen (3.5 / 3.6 / Coder / VL / legacy / 2.5 / Qwopus), Gemma (4 / 4-mobile / 3 / EmbeddingGemma), Llama, DeepSeek, GLM, Mistral, Phi, GPT-OSS, MiniMax, Hermes, Granite and a curated small-models bucket — is browseable as a family directory with one page per vendor, or as a flat alias catalog for ctrl-F. For "what fits my Mac" use the live RAM picker at models.rapidmlx.com.

Notable small models

Editorial short-list of small-but-interesting models worth watching outside the headline families — including Holo3.1-35B-A3B, VibeThinker, Tmax-9B, Granite 4 H-Micro, and EmbeddingGemma. The full callout lives on the model families page.

What shipped in 0.9

The 0.9 release focused on multi-user and long-context serving: compressed KV cache default-on for 9 popular MoE aliases, radix-tree prefix cache default-on for concurrent shared-prompt reuse, 128k context on Qwen3.5-9B without any flag flips, speculative decoding (MTP + DFlash) wired up and verified lossless when enabled, and 8+ hour agent-session stability. See the 0.9.9 changelog for the full list and roadmap for what is planned next.