Rapid-MLX
|
Issue #4 |
|
|
Which model should
you run on your Mac?
Will it fit, and will it be fast enough to use? We took the nine popular models from our homepage and ran each one on four Macs: a 32 GB M2 Pro, a 48 GB M4 Pro, a 36 GB M5 Max and a 256 GB M3 Ultra. Same software, same test, every number public.
Tokens per second while answering · higher is faster · tap to open every run
|
|
01 / WHICH MODEL FOR YOUR MAC
Start from your memory, not your chip
Memory decides what fits. The chip decides how fast it goes. Here is what we would run on each, from the table above:
|
16 GB
or less |
Qwen3.5 4B or LFM3 1.7B. Both are small: Qwen3.5 4B uses about 4 GB while running. We did not test a 16 GB Mac this round, so no speeds here yet. |
|
32 GB
M2 Pro |
Gemma 4 26B for quality, 52 tok/s, about 15 GB. Qwen3.5 4B when you want speed, 63 tok/s. |
|
36–48 GB
M5 Max, M4 Pro |
Qwen3.6 35B: 150 tok/s on the M5 Max, 97 on the M4 Pro, about 21 GB. For the strongest dense model, Qwen3.8 27B: 65 tok/s with MTP on the M5 Max. |
|
256 GB
M3 Ultra |
DeepSeek V4.1 Flash, 47 tok/s with DSpark, about 148 GB. Nemotron 3.5 Bolt, 52 tok/s, about 47 GB. Qwen3.6 35B reaches 199 with MTP. |
Try one now
rapid-mlx serve qwen3.6-35b
Leave room for macOS and your apps on top of the memory figures above.
Making images? The GPU matters more than memory. One 1024×1024 Qwen-Image 2.1 picture took 38 s on the M5 Max, 83 s on the M3 Ultra, 264 s on the M4 Pro and 451 s on the M2 Pro.
How we measured: one benchmark run per cell on an idle Mac, the median of five rounds, Rapid-MLX 1.0 built from source (commit 56e36a4). MTP and DSpark are speed-ups that turn on automatically for models that support them; the 1.0.0 release benchmarks with them off, so its numbers read lower for those models until the next release.
|
|
02 / RAPID-MLX 1.0 IS HERE
Same model. Same Mac. Up to 4× faster.
Rapid-MLX 1.0 shipped on October 29. Point your coding agent or your own app at a model on your Mac with one command. It speaks the OpenAI and Anthropic APIs, so the tools you already use just work.
 Qwen3.5-9B 4-bit · whole-file code edit · Mac mini M4 Pro 48 GB · mlx-lm 46.4 vs Rapid-MLX 198.4 tok/s
An agent, built in
Never set up a coding agent? Run rapid-mlx chat --harness in a folder. It searches the web, reads and edits files and runs your tests with a local model, all on your Mac, and asks before it touches the network.
Terminal commands are not sandboxed. They run with your user account's access. Start with --approvals always to review every action, or --no-action-tools to turn off edits and execution. Read the harness guide first →
Already have an agent you like? rapid-mlx launch opencode wires it to your local model. Claude Code, Codex, Hermes, Pi and Goose work the same way.
New models, ready on day one
MiniMax-H3: text or a first image in, video with synchronized audio out, entirely on your Mac. Needs 160 GB+.
LFM3-VL 2B: ask about a photo on any Apple silicon Mac. The photo never leaves it.
DeepSeek V4.1 Flash: 47 tok/s with DSpark on an M3 Ultra.
Flint 40B-A2.5B: 131 tok/s on an M5 Max.
Also new: Qwen-Image 2.1 Turbo, Nemotron 3.5, Ornith 122B (needs a 96 GiB Mac), Bonsai 2 and LFM3 1.7B.
 LFM3-VL 2B on a Mac mini M2 Pro · unedited
Built to take a beating
Out of memory? The server recovers without a restart, and bursts of long prompts queue instead of crashing.
Four chats at once: twice the total throughput, 37.6 to 73.8 tok/s on Qwen3.8 27B on an M3 Ultra.
Safer by default: drive-by browser requests and DNS rebinding are blocked.
Desktop: eight images in one message, and an idle model is released after a timeout you choose.
|
|
03 / THE COMMUNITY
A community that builds on Rapid-MLX
47 people have landed code in Rapid-MLX. Many more build their own tools on top of it. A few of them, all open source:
 |
SiliconScope by kennss
A 1k-star Apple silicon monitor that spots a running Rapid-MLX server and shows its model and speed. GitHub → |
 |
claude-local by jorden-parker
Claude Code on a local Qwen3.8 27B, through Rapid-MLX's Anthropic-compatible API. GitHub → |
 |
RapidManager by leoric
A menu-bar app to start the server, switch and pull models, and tail logs. GitHub → |
The leaderboard you built
Every speed on the leaderboard comes from someone running the same benchmark on their own Mac. It shows the fastest run for each Mac, with the exact settings, so you can copy them.
Add your Mac in two commands.
rapid-mlx benchmark run qwen3.6-35b
rapid-mlx benchmark share <run-id>
Nothing leaves your Mac until you share it. A faster run on your chip helps everyone who owns one.
Find your Mac on the leaderboard →
|
|
04 / THANK YOU
Built by all of you
Rapid-MLX is 47 people who landed code, 94 who benchmarked their Macs, and everyone who took the time to file a report. A few we want to name:
Pierre Lamy @pierre427Performance. The continuous MTP engine behind the Qwen3.8 27B speed-ups in the table above, a fused single-token decode kernel, and an opt-in kernel that serves many requests at once faster: 1.77× total throughput at eight requests on an M4 Pro.
Wayner Barrios @waybarriosRapid-MLX began as his vLLM-MLX, where the paged KV cache, prefix cache and continuous batching were first built.
@janhilgardEarly speculative decoding and MTP, and a Metal leak fix under heavy load.
@guo, @masonjames, @angusgastleThe API contract tests, thinking-token routing and locked-down admin routes, and the first Claude Code connection.
@reports-svg, @Fe2-O3Reports we promised to credit two issues ago: a Metal handle limit hit about 55 times a day (#2834, #2836), and port 8000 collisions, which is why Desktop now starts on 7659, RMLX on a phone keypad (#2879). Both fixed.
|
|
All of it runs on the Mac you already own.
Download for Mac →
Free · open source · macOS 14+ · Apple Silicon · no account
already installed? rapid-mlx upgrade
|
|
|
Ran something on your Mac that beat our numbers?
Share it. We'd love to see it on the board.
|
Rapid-MLX · Run great AI locally on Apple Silicon · rapidmlx.com
|
