Rapid-MLX vs LM Studio: 1.42× single-stream decode on M5 Max
Rapid-MLX led LM Studio in both decode throughput rows on the October M5 Max run. Its first agent turn after restart took 0.20 s versus 32.7 s.
Disclosure: we build Rapid-MLX.
Rapid-MLX at a glance
- October M5 Max, Qwen3.5-9B 4-bit: 115.5 tok/s versus 81.0 tok/s single-stream decode (1.42×) and 248.2 tok/s versus 230.0 tok/s across four streams.
- Agent replay: 0.20 s versus 32.7 s to the first token after restart; both passed 30/30 tool-call prompts.
pip install rapid-mlx rapid-mlx serve qwen3.5-9b-4bit
Measured on a Mac Studio, M5 Max, 36 GB (October 2026)
Qwen3.5-9B 4-bit; Rapid-MLX 0.16.1 default versus LM Studio llmster 0.0.25 with MLX runtime 1.11.0. Lower time is faster; higher token throughput is faster.
| Scenario | Rapid-MLX 0.16.1 default | LM Studio (MLX) |
|---|---|---|
| Decode, 1 stream | 115.5 tok/s | 81.1 tok/s |
| 4 streams, aggregate | 248.2 tok/s | 230.0 tok/s |
| Tool calls passed (30 prompts) | 30/30 | 30/30 |
| Agent session: turns 2–10, median | 0.233 s | 0.613 s |
| After restart: turn 1 | 0.197 s | 32.678 s |
| After restart: turns 2–10, median | 0.229 s | 0.611 s |
| After restart: 10 turns, total | 2.275 s | 37.842 s |
| Agent session: 10 turns, total | 11.547 s | 37.894 s |
| Agent session: turn 1, cold | 9.442 s | 32.694 s |
| TTFT, ~2k-token prompt | 0.815 s | 2.780 s |
| TTFT, ~16k-token prompt | 6.089 s | 21.743 s |
October measurement notes. Mac Studio, M5 Max, 36 GB; Qwen3.5-9B 4-bit; 2026-10-12. Single run per engine; small differences (under ~5%) are within run-to-run variation. Rapid-MLX 0.16.1 default used MTP speculative decoding on a text-only lane with image input off. With --no-spec-decode it serves images, but measured 80.1 tok/s on one stream and 77.0 tok/s across four streams. oMLX ran with its documented SSD cache directory enabled; LM Studio ran headless with its MLX runtime. oMLX and LM Studio were measured earlier the same day on the same machine as Rapid-MLX. Scenario JSON, harness, versions, and summary are published.
Rapid-MLX's default led single-stream decode, both cold prompt TTFT rows, four-stream throughput, and the coding-agent session in this single run. Both passed all 30 tool-call prompts.
What each tool is
- Rapid-MLX: an MLX inference server for Apple Silicon with OpenAI and Anthropic APIs.
- LM Studio: a desktop model browser and local server for macOS, Windows and Linux; this run used its headless
llmsterservice with the MLX runtime.
Why Rapid-MLX
Rapid-MLX led the October decode throughput and agent-session rows, passed 30/30 tool-call prompts, and answered the first turn after restart in 0.20 s. It is Apache-2.0 licensed, with OpenAI and Anthropic APIs, per-model tool-call parsers, agent setup commands, a Desktop app and Homebrew installation (brew install rapid-mlx). See the full comparison, oMLX, Ollama, and mlx-lm.