Rapid-MLX vs oMLX: 1.50× 27B decode on M5 Max
On M5 Max, Rapid-MLX led oMLX in 27B and 35B decode throughput. On 35B, the first turn after restart took 0.27 s versus 0.34 s.
Disclosure: we build Rapid-MLX.
Qwen3.8-27B 4-bit on a Mac Studio, M5 Max, 36 GB
Rapid-MLX measured 37.9 tok/s on one stream and 82.8 tok/s across four on the same weights as oMLX.
rapid-mlx serve qwen3.8-27b-4bit
| Scenario | Rapid-MLX 0.16.1 default | oMLX 0.7.0 |
|---|---|---|
| Decode, 1 stream | 37.9 tok/s | 25.3 tok/s |
| 4 streams, aggregate | 82.8 tok/s | 75.3 tok/s |
Measurement notes. Mac Studio, M5 Max, 36 GB; 2026-10-12. Single run per engine; small differences (under ~5%) are within run-to-run variation. Rapid-MLX 0.16.1 default used MTP speculative decoding on a text-only lane with image input off. With --no-spec-decode it serves images. oMLX ran with its documented SSD cache directory enabled; LM Studio ran headless with its MLX runtime. All three used the same model weights. Scenario JSON, harness, versions, and summary are published.
Qwen3.6-35B 4-bit on a Mac Studio, M5 Max, 36 GB
Rapid-MLX measured 173.4 tok/s on one stream and 293.9 tok/s across four on the same weights as oMLX.
rapid-mlx serve qwen3.6-35b-4bit
| Scenario | Rapid-MLX 0.16.1 default | oMLX 0.7.0 |
|---|---|---|
| Decode, 1 stream | 173.4 tok/s | 145.8 tok/s |
| 4 streams, aggregate | 293.9 tok/s | 281.9 tok/s |
| Tool calls passed (30 prompts) | 30/30 | 29/30 |
| After restart: turn 1 | 0.27 s | 0.34 s |
| TTFT, ~2k-token prompt | 0.55 s | 1.14 s |
Measurement notes. Mac Studio, M5 Max, 36 GB; 2026-10-12. Single run per engine; small differences (under ~5%) are within run-to-run variation. Rapid-MLX 0.16.1 default used MTP speculative decoding on a text-only lane with image input off. With --no-spec-decode it serves images. oMLX ran with its documented SSD cache directory enabled; LM Studio ran headless with its MLX runtime. All three used the same model weights. Scenario JSON, harness, versions, and summary are published.
Qwen3.5-9B at a glance
- October M5 Max, Qwen3.5-9B 4-bit: 115.5 tok/s versus 77.9 tok/s single-stream decode (1.48×) and 248.2 tok/s versus 224.6 tok/s across four streams.
- Agent replay: 0.20 s versus 0.27 s to the first token after restart; both passed 30/30 tool-call prompts.
pip install rapid-mlx rapid-mlx serve qwen3.5-9b-4bit
Measured on a Mac Studio, M5 Max, 36 GB (October 2026)
Qwen3.5-9B 4-bit; Rapid-MLX 0.16.1 at its default for this model versus oMLX 0.7.0 with its documented SSD cache directory enabled. Lower time is faster; higher token throughput is faster.
| Scenario | Rapid-MLX 0.16.1 default | oMLX 0.7.0 |
|---|---|---|
| Decode, 1 stream | 115.5 tok/s | 78.0 tok/s |
| 4 streams, aggregate | 248.2 tok/s | 224.6 tok/s |
| Tool calls passed (30 prompts) | 30/30 | 30/30 |
| Agent session: turns 2–10, median | 0.233 s | 0.238 s |
| After restart: turn 1 | 0.197 s | 0.265 s |
| After restart: turns 2–10, median | 0.229 s | 0.230 s |
| After restart: 10 turns, total | 2.275 s | 2.340 s |
| Agent session: 10 turns, total | 11.547 s | 10.772 s |
| Agent session: turn 1, cold | 9.442 s | 8.576 s |
| TTFT, ~2k-token prompt | 0.815 s | 0.747 s |
| TTFT, ~16k-token prompt | 6.089 s | 5.721 s |
October measurement notes. Mac Studio, M5 Max, 36 GB; Qwen3.5-9B 4-bit; 2026-10-12. Single run per engine; small differences (under ~5%) are within run-to-run variation. Rapid-MLX 0.16.1 default used MTP speculative decoding on a text-only lane with image input off. With --no-spec-decode it serves images, but measured 80.1 tok/s on one stream and 77.0 tok/s across four streams. oMLX ran with its documented SSD cache directory enabled; LM Studio ran headless with its MLX runtime. oMLX and LM Studio were measured earlier the same day on the same machine as Rapid-MLX. Scenario JSON, harness, versions, and summary are published.
Cold prompt prefill was close (within ~10%) in this single run. oMLX had the shorter cold first agent turn. Rapid-MLX's default text-only lane matters to its decode result; its image-capable --no-spec-decode measurements are in the note above.
What each tool is
- Rapid-MLX: an MLX inference server for Apple Silicon with OpenAI and Anthropic APIs.
- oMLX: an MLX inference server for Apple Silicon with an SSD-backed KV cache, menu-bar app and web admin panel.
Why Rapid-MLX
Rapid-MLX led both October decode throughput rows, passed 30/30 tool-call prompts and answered the first turn after restart in 0.20 s. It is Apache-2.0 licensed, with OpenAI and Anthropic APIs, per-model tool-call parsers, agent setup commands, a Desktop app and Homebrew installation (brew install rapid-mlx). See LM Studio, Ollama, and the full comparison.
September 2026 results: M4 Pro 48 GB and M3 Pro 18 GB
Qwen3.5-9B at 4-bit, each server at its defaults except: oMLX with its documented --paged-ssd-cache-dir, Rapid-MLX with PFlash off (see /compare). Medians of 3 runs (2 for the 4-stream row). oMLX was measured in the earlier run on each machine with the same harness; its version did not change between those September runs. On the 18 GB Mac, oMLX refused one of its three ~16k-token requests with its prefill memory guard, so that cell is the median of two.
| Rapid-MLX 0.15.3 (@1394f16), 48 GB | oMLX 0.7.0rc1, 48 GB | Rapid-MLX 0.15.3 (@1394f16), 18 GB | oMLX 0.7.0rc1, 18 GB | |
|---|---|---|---|---|
| Decode, 1 stream | 66.1 tok/s | 51.2 tok/s | 43.6 tok/s | 27.9 tok/s |
| TTFT, ~2k-token prompt (cold) | 5.30 s | 5.40 s | 5.50 s | 5.73 s |
| TTFT, ~16k-token prompt (cold) | 42.8 s | 41.7 s | 44.7 s | 43.9 s |
| 4 streams, aggregate | 68.3 tok/s | 97.1 tok/s | 62.3 tok/s | 85.3 tok/s |
| Tool calls passed (30 prompts) | 30/30 | 30/30 | 30/30 | 30/30 |
| Agent session: turn 1, cold | 65.1 s | 63.1 s | 68.5 s | 67.0 s |
| Agent session: turns 2–10, median | 0.49 s | 0.49 s | 0.69 s | 0.66 s |
| After a restart: turn 1 | 12.3 s | 0.27 s | 13.0 s | 0.56 s |
| After a restart: turns 2–10, median | 0.49 s | 0.27 s | 0.67 s | 0.44 s |
The agent session is a Claude-Code-shaped replay: a 22,819-token first request (a long system prompt plus 30 tool schemas), then 10 turns from a fixed transcript, then the same 10 turns after a server restart. Full method on /compare.