Rapid-MLX vs oMLX vs LM Studio: 1.50× 27B decode on M5 Max

Rapid-MLX delivered 173.4 tok/s on one stream with Qwen3.6-35B and 37.9 tok/s with Qwen3.8-27B on the M5 Max runs, ahead of oMLX and LM Studio.

Disclosure: we build Rapid-MLX. All measurements below come from the published runs.

For the direct MLX vs Ollama speed question, read the measured Rapid-MLX vs Ollama benchmark. For a feature-based choice among Rapid-MLX, Ollama and LM Studio, use the three-tool guide.

Qwen3.8-27B 4-bit on a Mac Studio, M5 Max, 36 GB

Rapid-MLX measured 37.9 tok/s on one stream and 82.8 tok/s across four. On the same weights, oMLX measured 25.3 tok/s and 75.3 tok/s; LM Studio measured 26.0 tok/s and 69.7 tok/s.

rapid-mlx serve qwen3.8-27b-4bit

Measurement notes. Mac Studio, M5 Max, 36 GB; 2026-10-12. Single run per engine; small differences (under ~5%) are within run-to-run variation. Rapid-MLX 0.16.1 default used MTP speculative decoding on a text-only lane with image input off. With --no-spec-decode it serves images. oMLX ran with its documented SSD cache directory enabled; LM Studio ran headless with its MLX runtime. All three used the same model weights. Scenario JSON, harness, versions, and summary are published.

Qwen3.6-35B 4-bit on a Mac Studio, M5 Max, 36 GB

Rapid-MLX measured 173.4 tok/s on one stream and 293.9 tok/s across four. On the same weights, oMLX measured 145.8 tok/s and 281.9 tok/s; LM Studio measured 124.5 tok/s and 265.6 tok/s.

rapid-mlx serve qwen3.6-35b-4bit

Measurement notes. Mac Studio, M5 Max, 36 GB; 2026-10-12. Single run per engine; small differences (under ~5%) are within run-to-run variation. Rapid-MLX 0.16.1 default used MTP speculative decoding on a text-only lane with image input off. With --no-spec-decode it serves images. oMLX ran with its documented SSD cache directory enabled; LM Studio ran headless with its MLX runtime. All three used the same model weights. Scenario JSON, harness, versions, and summary are published.

Qwen3.5-9B at a glance

• October M5 Max, Qwen3.5-9B 4-bit: 115.5 tok/s single-stream decode (1.48× oMLX; 1.42× LM Studio) and 248.2 tok/s across four streams (oMLX 224.6 tok/s; LM Studio 230.0 tok/s).
• Agent replay: 0.20 s to the first token after restart, versus 0.27 s for oMLX and 32.7 s for LM Studio. All three passed 30/30 tool-call prompts.

pip install rapid-mlx
rapid-mlx serve qwen3.5-9b-4bit

Measured on a Mac Studio, M5 Max, 36 GB (October 2026)

Rapid-MLX 0.16.1 default, oMLX 0.7.0 and LM Studio llmster 0.0.25 with MLX runtime 1.11.0, all on Qwen3.5-9B 4-bit. Lower TTFT is faster; higher token throughput is faster.

October measurement notes. Mac Studio, M5 Max, 36 GB; Qwen3.5-9B 4-bit; 2026-10-12. Single run per engine; small differences (under ~5%) are within run-to-run variation. Rapid-MLX 0.16.1 default used MTP speculative decoding on a text-only lane with image input off. With --no-spec-decode it serves images, but measured 80.1 tok/s on one stream and 77.0 tok/s across four streams. oMLX ran with its documented SSD cache directory enabled; LM Studio ran headless with its MLX runtime. oMLX and LM Studio were measured earlier the same day on the same machine as Rapid-MLX. Scenario JSON, harness, versions, and summary are published.

The September runs use the script, driver, raw per-request JSON and derived summary.

oMLX vs LM Studio

On this run LM Studio decoded faster than oMLX on one stream and across four streams; oMLX had shorter cold prompt TTFT and first agent turn. Both passed all 30 tool-call prompts. Rapid-MLX led both decode rows against both: 115.5 tok/s on one stream and 248.2 tok/s across four. Its 0.20 s first turn after restart was close to oMLX's 0.27 s and much shorter than LM Studio's 32.7 s. See the oMLX and LM Studio pages for focused comparisons.

September 2026 results on other Macs

The following M4 Pro Mac mini (48 GB) and M3 Pro Mac (18 GB) results are separate September runs. Ollama was not measured on the M5 Max.

The September short version

On an M4 Pro Mac mini with 48 GB (Qwen3.5-9B at 4-bit), with each engine at its defaults except the settings listed under the table. The Rapid-MLX rows are Rapid-MLX 0.15.3 (measured at commit 1394f16, which 0.15.3 includes); the previous release, 0.15.2, is compared in What changed in Rapid-MLX.

• Single-stream decode: Rapid-MLX was fastest at 66.1 tok/s (oMLX 51.2 tok/s, mlx-lm 47.7 tok/s, Ollama 38.4 tok/s). Rapid-MLX runs this model with MTP speculative decoding on by default.
• Four concurrent streams: Rapid-MLX measured 68.3 tok/s, below oMLX (97.1 tok/s) and mlx-lm (96.6 tok/s). Ollama (37.4 tok/s) serves this model one request at a time. These four-stream numbers are from 0.15.3. In 0.15.6, overlapping requests on this model decode together through ordinary batching instead of MTP (how MTP handles concurrency). Re-measured on 2026-10-04 with the same harness against mlx-lm only: Rapid-MLX 0.15.6 104.2 tok/s, mlx-lm 0.32.0 100.2 tok/s, parity (Rapid-MLX vs mlx-lm); oMLX and Ollama were not re-run.
• Long coding-agent session, follow-up turns: every engine reused the ~23k-token prefix and answered in under a second; Ollama was fastest: Ollama 0.41 s, mlx-lm 0.47 s, oMLX 0.49 s, Rapid-MLX 0.49 s (medians of turns 2–10).
• Same session after a server restart: Rapid-MLX restored most of the prefix from its disk save and took 12.3 s; oMLX’s SSD tier took 0.27 s. mlx-lm (65.7 s) and Ollama (73.1 s) prefilled from scratch.
• Cold prefill was close for everyone: 41.7 s (oMLX, fastest) to 49.4 s at ~16k tokens.
• Tool calls: every engine passed all 30 prompts, so this set does not separate them.
• On an 18 GB Mac the order is the same, except that mlx-lm's server ran out of GPU memory in the agent session; the 18 GB table has all numbers.

Rapid-MLX vs mlx-lm

A separate, task-by-task benchmark against mlx-lm, Apple's MLX library and server (2026-10-04, Rapid-MLX 0.15.6 against mlx-lm 0.32.0, Mac mini M4 Pro, 48 GB, greedy decoding, same weights): up to 4× faster decode on whole-file code edits (Qwen3.5-9B 4-bit), 1.50× at the median of the 18 everyday tasks that stream text, close to a tie on long-prompt agent turns. Per-task results, method and raw data: Rapid-MLX vs mlx-lm benchmark.

Feature comparison

What each project ships today, from its own docs and our installs. "Yes" means documented and working in the version we ran, not a roadmap.

Sources: each project's README and docs, checked 2026-09-26 (oMLX, Ollama Anthropic compatibility, Ollama MLX preview, LM Studio Anthropic endpoint, mlx-lm server). Rapid-MLX flags are from rapid-mlx serve --help at 0.15.2.

Measured on an M4 Pro Mac mini (48 GB), September 2026

Conditions. Mac mini, Apple M4 Pro, 48 GB unified memory, macOS 26.5.1, on AC power, idle before each engine (GPU 0% after a 60 s rest + 10 idle seconds). Model: Qwen3.5-9B at 4-bit (MLX engines: mlx-community/Qwen3.5-9B-4bit @ 8b2b98c, MTP sidecar mlx-community/Qwen3.5-9B-MTP-4bit @ 222dfd2; Ollama: qwen3.5:9b, GGUF Q4_K_M, id 6488c96fa5fa), same as run 2. One engine at a time, loopback only, temperature 0, streaming. Before each engine the driver rested 60 s and waited for the GPU to read idle, and gave the engine a fresh home directory (no cache or settings carried over). The competitors were measured in run 3; Rapid-MLX 0.15.3 (measured at commit 1394f16, which 0.15.3 includes) was measured a few hours later the same night with the same harness and protocol in run 4 (the competitors' versions had not changed, so they were not re-run). Decode and TTFT cells are the median of 3 runs, the 4-stream cell the median of 2. Decode tok/s is (completion tokens − 1) / (last token − first token), using each engine's own token count. TTFT is request sent to the first streamed token of any kind, thinking included; each prompt starts with a random nonce so that no prefix cache applies. Engines group tokens into stream chunks differently (oMLX sends several tokens per chunk), which can add a fraction of a second to a TTFT. Every request asked for thinking explicitly (chat_template_kwargs.enable_thinking=true), because Rapid-MLX turns thinking off by default for casual requests while the others leave it on; the raw files store the generated text and reasoning, so you can check. Engine settings: Rapid-MLX with --pflash off (see below), everything else default, which for this model includes MTP speculative decoding; oMLX with its documented --paged-ssd-cache-dir; Ollama with a 32k context so no prompt was truncated and OLLAMA_NUM_PARALLEL=4 (Ollama 0.34.3 ignores it for this model and logs "model architecture does not currently support parallel requests"; its four streams' first tokens arrived about 7 s apart, one after another); mlx-lm's server at its defaults. Ollama loads its own GGUF Q4_K_M build of the model, the others load the same MLX 4-bit files.

Why the main Rapid-MLX row has PFlash off. PFlash is long-prompt prefill compression that keeps a fraction of the prompt's tokens (20% by default) before prefilling. It is off unless you pass --pflash auto or --pflash always; the build we measured turned it on by default for Qwen3.5 and Qwen3.6 models, so the main row passed --pflash off, which matches the default. With PFlash on, long prompts were much faster (8.23 s instead of 42.8 s at ~16k tokens; it did not engage on the agent session, whose requests carry tools), but the model then does not see every prompt token, so it is not a like-for-like comparison with engines that prefill all of them. We did not measure answer quality with PFlash on, and ran only the prefill and agent-session tests in that configuration.

On an 18 GB Mac, September 2026

MacBook Pro, Apple M3 Pro, 18 GB unified memory, macOS 15.6.1, on power. Competitors from run 2, Rapid-MLX 0.15.3 (measured at commit 1394f16, which 0.15.3 includes) from run 4, same protocol. On this machine oMLX refused one of its three ~16k-token requests with its prefill memory guard ("Prefill would require ~9.08 GB peak … dynamic ceiling is 9.00 GB"), so its ~16k cell is the median of the two that ran, and Ollama's four streams started about 11 s apart.

The coding-agent session

oMLX's pitch is that Claude Code "responds in 5 seconds, not 90" (omlx.ai): coding agents resend a long system prompt and tool list on every turn, and oMLX keeps that prefix's KV cache on SSD so it survives a restart. We replayed a Claude-Code-shaped session to measure exactly that: a 22,819-token first request (a long system prompt plus 30 tool schemas), then 10 user turns from a fixed transcript (the assistant replies are canned, so every engine sees identical prompts). Then we restarted the server, warmed it with an unrelated prompt, and replayed the same 10 turns. For Ollama, "restart" means unloading the model (ollama stop), which drops its KV cache the same way. Numbers are time to first token per turn; when a server sends nothing until its 16-token reply is complete, the number is time to the end of the stream instead, and the raw records flag those turns (ttft_fallback_end_of_stream).

M4 Pro, 48 GB:

M3 Pro, 18 GB:

After one cold prefill, oMLX answered every turn in under a second; its first turn after a restart took 0.27 s on 48 GB and 0.56 s on 18 GB. Ollama and mlx-lm reused their caches within the session just as well, and lost them on restart. On 18 GB, mlx-lm's server died with a Metal out-of-memory error on turn 4, and after the restart on turn 3 (server log).

Rapid-MLX 0.15.3 (measured at commit 1394f16, which 0.15.3 includes) kept the session prefix resident on both machines: turns 2–10 prefilled only the 42–66 new tokens each, bringing it under a second like the others (0.49 s on 48 GB, 0.69 s on 18 GB), though still behind Ollama on both machines and slightly behind oMLX on 18 GB. At shutdown it saved the session to disk and reloaded it at start, so the first turn after the restart restored 18,944 of the 22,819 prompt tokens and prefilled the remaining 3,875: 12.3 s on 48 GB. That is far better than a full prefill but still well behind oMLX's 0.27 s. The save is not guaranteed: on 18 GB, one of the four shutdowns (the last one, after the restart replay, so no number above depends on it) skipped the entry because its predicted write time (3.1 s) did not fit the 3.5 s shutdown budget minus its 0.4 s commit headroom. Cache budgets and save results for every start are in each run's versions.json and server logs. The prompt token counts differ slightly between engines (about 150 tokens) because each renders the tool schemas through its own chat template.

What changed in Rapid-MLX

The first published version of this page measured the previous release, Rapid-MLX 0.15.2, and it lost the agent session badly on the 18 GB Mac: its prefix cache sized itself to 0.86 GB there, each ~0.96 GB session entry was "too large" to store, and every turn paid the full prefill; what it saved at shutdown did not include the session, so the first turn after a restart was a full prefill too. Commit 1394f16 (#3794) gives agent sessions a minimum cache budget on small-RAM Macs and makes the shutdown save fit in most cases (see above). Same harness, same protocol, PFlash off:

Install 0.16.2 with pip install rapid-mlx==0.16.2 or Rapid-MLX Desktop 0.16.2. The benchmark was measured at commit 1394f16, which 0.16.0 includes; it was not re-run on the release tag. The 0.15.2 numbers are in summary-run2.json and summary-run3-m4pro-48gb.json.

Tool calling

Thirty fixed prompts, each with the same eight tools (weather, calculator, web search, email, calendar, read file, shell, currency). A prompt passes when the first tool call names the right function and its JSON arguments contain the expected values. Non-streaming, temperature 0. The prompts, expected calls and every raw response are in the raw data.

Rapid-MLX 0.15.3 (@1394f16) (PFlash off): 30/30 · oMLX 0.7.0rc1 (SSD cache on): 30/30 · Ollama 0.34.3 (qwen3.5:9b, Q4_K_M): 30/30 · mlx-lm 0.31.3 (mlx_lm.server): 30/30. Every engine passed every prompt. These are single-call, clearly worded requests, so the result says the plumbing works on all four servers for this model; it does not rank them. Harder multi-step tool use is where parsers differ, and this test does not cover it.

Gaps in the September runs

• M3 Ultra numbers, for the same reason.
• Ollama's MLX engine (qwen3.5:9b-mlx), which needs 32 GB or more and was not in our run-3 plan; it is a gap in the 48 GB table.
• Other models. One model, one quantization per machine. Run the script on your own Mac and models.

What each tool is

• Rapid-MLX: an MLX inference server for Apple Silicon with OpenAI and Anthropic APIs.
• oMLX: an MLX inference server for Apple Silicon with an SSD-backed KV cache and menu-bar app.
• Ollama: a cross-platform model runner and local server.
• LM Studio: a cross-platform desktop model browser and local server.
• mlx-lm: Apple's MLX library with a reference server.

Why Rapid-MLX

On the October M5 Max run, Rapid-MLX led both decode throughput rows against oMLX and LM Studio, passed 30/30 tool-call prompts, and answered the first turn after restart in 0.20 s. It is Apache-2.0 licensed and provides OpenAI and Anthropic APIs, per-model tool-call parsers, agent setup commands, a Desktop app and Homebrew installation (brew install rapid-mlx). Its measured default for this model used a text-only lane, as noted above.

Focused pages: Rapid-MLX vs mlx-lm · Rapid-MLX vs oMLX · Rapid-MLX vs LM Studio · Ollama alternative for Mac · our August Ollama/LM Studio/llama.cpp benchmark.

How these runs came about

We publish every run. Run 1 (18 GB) was superseded after a review found three flaws that favored Rapid-MLX or muddied the result: Rapid-MLX had thinking off by default while the other engines were thinking; the engines ran back to back in a fixed order with Rapid-MLX first, on the coolest machine; and one Rapid-MLX start overlapped the shutdown of the previous server. The same review caught two errors on the first version of this page: it said Ollama's four streams ran in parallel (they did not), and the server logs it cited were missing from the published data. Run 2 (18 GB) fixed all of that; run 3 repeated it on a 48 GB M4 Pro; run 4 re-measured only Rapid-MLX at commit 1394f16, which 0.15.3 includes, on both machines. The data README maps each table to its run.

Reproduce it

curl -O https://rapidmlx.com/blog/assets/compare-2026-09/bench_compare.py
# start any OpenAI-compatible server on loopback, then, per scenario a b c d e:
python3 bench_compare.py --engine NAME --base-url http://127.0.0.1:PORT/v1 --model MODEL --thinking on --scenario a --out a.json

The client is standard-library Python. run_all.sh is the driver we used: it starts and stops each server, rests between engines, and restarts the server for the agent session. It expects the engines installed in a scratch directory (W, see its header) and your normal Hugging Face cache in HF_HOME.

Frequently asked questions

Is Rapid-MLX faster than Ollama?

For decode, yes in our runs: on an M4 Pro Mac mini with 48 GB (Qwen3.5-9B at 4-bit), Rapid-MLX 0.15.3 (measured at commit 1394f16, which 0.15.3 includes) decoded 66.1 tok/s on one stream against Ollama's 38.4 tok/s, and 68.3 tok/s against 37.4 tok/s across four streams (Ollama serves this model one request at a time). In a long coding-agent session Ollama answered follow-up turns faster: a median of 0.41 s against 0.49 s for Rapid-MLX, and 0.44 s against 0.69 s on an 18 GB Mac. The previous release, 0.15.2, was far slower there (70.0 s); that is fixed in 0.15.3.

Rapid-MLX vs oMLX: which should I use?

On the October M5 Max run, Rapid-MLX 0.16.1 default led single-stream and four-stream decode and passed all 30 tool-call prompts. oMLX had shorter cold prompt TTFT and the first agent turn. The first turn after restart was close between them in this single run. Details and the September tables are on the Rapid-MLX vs oMLX page.

Does Rapid-MLX work with Claude Code?

Yes. Rapid-MLX serves the Anthropic Messages API at /v1/messages, so Claude Code connects with ANTHROPIC_BASE_URL (see Claude Code on a local model). oMLX, Ollama (v0.14+) and LM Studio (0.4.1+) also serve that API. In our replay of a 23k-token Claude Code session, Rapid-MLX 0.15.3 (measured at commit 1394f16, which 0.15.3 includes) answered follow-up turns in a median of 0.49 s on a 48 GB Mac and 0.69 s on an 18 GB Mac. The previous release, 0.15.2, re-prefilled the whole prompt every turn on the 18 GB Mac (70.0 s); that is fixed in 0.15.3.

Where is the raw data?

Every request's timings, the engine-reported token usage and every tool-call response are listed in raw/manifest.json, with the script (bench_compare.py), the driver (run_all.sh), the engine versions and flags (each run's versions.json) and the numbers used on this page (summary.json).