Rapid-MLX 0.11.0 — 25.6× faster to first token, meet your Mac's new agent

Three weeks. Fifteen releases. One goal — make local models on Apple Silicon reliable enough to run your agent workflows, not just demo them.

Rapid-MLX 0.11.0 is out today. The headline numbers are about speed; the headline story is about what that speed unlocks: a built-in chat agent that chains real tools, five new model families supported on day one, and a reliability pass that makes the whole thing boring in the best way.

Install it in one line:

brew install rapid-mlx        # now in homebrew-core
# or: pip install -U rapid-mlx / uv tool install rapid-mlx

⚡️ Performance: stop paying prefill twice

All numbers below are single-request wall-clock measurements on an M3 Ultra from this week's dogfooding — not synthetic peak-decode figures.

• Prefix-cache reuse: 13.1s → 0.51s TTFT (25.6×). Repeat a 6k-token prefix — a shared system prompt, a long agent context — and the second turn reuses the KV instead of recomputing it. One test session saved 38,032 tokens of prefill. For multi-turn chat and agent loops, this is the single biggest latency win we've shipped.
• Response cache: 656ms → 2ms (284×). An identical greedy request comes back byte-identical in 2 milliseconds with zero GPU decode. Opt-in, and ideal for eval loops and CI.
• Throughput: 152 tok/s on Qwen3.5-4B-4bit, 104 tok/s on gpt-oss-20b, ~100 tok/s on Gemma-4-26B-4bit — wall-clock, TTFT included.
• Cold starts in seconds: ~2s for a 4B model, ~3s for gpt-oss-20b.
• Bounded memory on long prompts: chunked text-lane prefill keeps a 19k-token prompt at a 17.1 GB peak.
• KV cache quantization: int4 KV cache is the M-series default; int8 override verified.

🤖 rapid-mlx chat grew into a real agent

chat started as a REPL. This cycle it became an agent we're genuinely happy with — simple, but neat:

• A real MCP agent loop. Point it at any MCP server and it chains tools on its own. In testing against the standard filesystem server — 14 tools exposed — it autonomously ran list_directory → read_text_file and answered from the file contents, with a live tool-activity UI and per-call timing. 2.8 seconds, clean exit.
• Thinking budgets. reasoning_max_tokens force-closes the <think> block at decode time: set a 24-token budget and a model that would reason for 3,396 characters wraps up in ~90 — and still answers. Verified across all four reasoning families we tested.
• Zero ceremony. It spawns the server for you, gives you /model /save /reset /help, and gets out of the way. --base-url mode attaches to an already-running server as a pure MCP client.

🧠 Five new model families, day one

165 aliases, 16 tool-call wires, ~18 model families — dense, MoE, hybrid-linear, ternary, vision, and audio:

• HY3 (Tencent Hunyuan 3) 295B MoE — our fifth Tier-1 family, with native MTP self-speculative decoding (~1.5–1.7× decode)
• Qwen3-Coder-Next 80B — day-one 4bit alias
• MiniCPM5 — including a brand-new tool-call wire
• LiquidAI LFM2.x
• Ternary Bonsai 1.7B-2bit — yes, 2-bit, and it runs
• Audio round-trip: Kokoro TTS → Parakeet STT, verified word-perfect, both services up in under 4 seconds
• Vision: Qwen3-VL reads controlled test images accurately — all 5 ground-truth features in 2.1s
• Also new: KV cache export/import API, and structured output migrated to llguidance (−200 MB install, native Metal token bitmask)

🛠️ Tool calls, schema-valid by construction

One paragraph, because it just works now: agent loops used to die on almost-JSON tool arguments, so we constrain decoding at the grammar level — the model cannot emit a malformed tool call. It's verified across all four major wire formats (harmony/gpt-oss, qwen3_coder_xml, gemma4, deepseek_v3) plus hermes, for auto, forced, and required modes, on by default with zero flags. The 0.11.0 keystone: the guarantee now holds for reasoning models too — models that <think> first used to break forced calls because the grammar engaged at the wrong offset. Not anymore. (Engineering-minded readers: this was the hard part. Everyone else: you'll never notice it, which is the point.)

🛡️ Reliability and honest gaps

Every release goes through an automated dogfood battery — 5 models × 11 checks covering streaming, reasoning, forced tool calls, error paths, and performance — on real hardware. The 0.10.16 round caught three P1 regressions (a Gemma-4 required loop, a flagship-serve crash, a silent schema fallback); all fixed and re-verified within 48 hours. Error paths are now honest across the board: invalid schemas → 400, unknown models → 404 with the list of what is available. No silent fallbacks, ever.

Honest gaps, because you'd find them anyway: Qwen3.6 native MTP is parked (its SSM-hybrid architecture clamps speculative K to 1 — the engine refuses loudly rather than silently degrading), the MCP config key is servers rather than the ecosystem-standard mcpServers (fix queued), and we haven't run a same-machine head-to-head against llama.cpp/Ollama this cycle. That's next.

Get it

brew install rapid-mlx
rapid-mlx chat qwen3.5-4b-4bit

Rapid-MLX is open source (Apache-2.0), MLX-native, and built for people who want their agents fast, local, and correct. 3,400+ of you have starred it — thank you. The best thing you can do for the project is break it and tell us: issues and PRs are answered fast.

All performance figures are single-request wall-clock measurements on an Apple M3 Ultra (256 GB), macOS 26.5.2, from 2026-07 evaluation runs. tok/s includes TTFT and is not comparable across quantization levels or architectures.