Rapid-MLX vs Ollama vs LM Studio — a Measured Benchmark on Apple Silicon

Three models, four engines, one idle Mac mini, every number measured and the raw JSON published — including the ones where we lose. The short version — MLX-native serving wins decode, llama.cpp wins prefill, and Ollama leaves ~25% on the table against its own upstream.

First, the disclosure: we build rapid-mlx. You should expect this benchmark to flatter us, which is exactly why every number below was measured on one idle machine with a written protocol, medians of three runs, token counts taken from each engine's own accounting — and the complete raw results JSON is published, including the tests we lose. Check our work.

Our comparison page covers features and philosophy. This post is only about speed.

Setup

• Machine: Mac mini, Apple M2 Pro, 32 GB unified memory, plugged in,

verified idle before every timed run. (Deliberately not a maxed-out Studio — this is the hardware most people actually have.)

• Engines: Ollama 0.32.7 · llama.cpp build 10330 (Metal, Homebrew) ·

LM Studio (llama.cpp engine 2.28.2 / MLX engine 1.11.0) · rapid-mlx 0.12.11 — all at default settings, context set to 4096 where settable.

• Protocol: one discarded warmup, then 3 timed runs each of a short

prompt (128-token cap) and a fixed 805-word summarization prompt (256-token cap), temperature 0, streaming. TTFT is measured client-side on localhost; decode rate uses the engine's own reported token counts, never a client-side guess. A concurrency lane (4 and 8 simultaneous salted requests, added 2026-08-12) is described in its own section below.

• Models — same or closest-available weights per engine:

The gpt-oss pair is the cleanest comparison in the set: every engine runs the model's native MXFP4 weights. For the Q4_K_M pairs, note the GGUF side carries slightly more bits per weight than MLX 4-bit — reported, not hidden.

Updated 2026-08-12: the rapid-mlx column was re-measured after upgrading 0.12.9 → 0.12.11 (same machine, same Python venv, same prompts and protocol; mlx 0.31.2 / mlx-lm 0.31.3 unchanged, so the delta is the server itself). The other engines' numbers are the original 2026-08-11 measurements at the versions above. Both sets, per-run, are in the raw JSON.

Decode speed — the number you feel

Median tokens/second while generating, long-form task (reasoning enabled everywhere; rapid-mlx, llama.cpp and Ollama all generate the full 256-token budget — LM Studio generated 183–255 tokens at its defaults, which changes totals but not the per-token rate shown):

Short bursts (128-token cap): rapid-mlx 49.9 / 60.6 / 22.7 against llama.cpp's 43.5 / 38.7 / 19.4 and Ollama's 33.4 / 39.3 / 18.9. LM Studio's MLX engine posted the single fastest burst we saw all day — 71.5 t/s on gpt-oss — more on that below.

Three honest readings:

1. The MoE gap is real and big. On Qwen3.6-35B-A3B, MLX-native serving

decodes ~1.5× faster than both llama.cpp and Ollama (58.5 vs 38–39 t/s). The raw framework numbers agree (mlx-lm 61.7 vs llama-bench 39.1), so this is MLX's MoE path, not our server tricks.

1. On a dense 12B the gap is small but no longer a tie. Gemma4 lands

at 18–19 t/s on every llama.cpp-family engine against 21.3 on rapid-mlx — a ~15% edge that appeared with 0.12.11 (0.12.9 measured 18.7, a genuine tie). If your model is small and dense, engine choice still moves decode speed the least.

1. **Ollama is ~25% slower than plain llama.cpp on the same gpt-oss

weights** (32.1 vs 42.6 t/s — llama.cpp is 1.33× faster). That gap is Ollama's own conversion and serving path, not GGUF — which surprised us enough to re-check it.

Concurrency — your Mac as a small API server

(Added 2026-08-12.) One stream is chat; agents and multi-user setups fire several. So we added a concurrency lane: 4 and 8 simultaneous streaming requests, each with the ~1,000-token prompt and a unique salt prefix (so no engine can serve a prompt-cache hit), 256-token budget, medians of 3 batches after a discarded warmup batch. Parallelism was pinned explicitly per lane (OLLAMA_NUM_PARALLEL, llama-server --parallel N -c N×4096) with a server restart between lanes. Unlike the single-stream tables above, every engine in this section was measured fresh on the same day — rapid-mlx 0.12.11, Ollama 0.32.7, llama.cpp build 10330.

Aggregate decode throughput across all streams (t/s), shown as 4-way → 8-way so you can see who scales and who degrades:

Three readings, same honesty policy:

1. On the MoE flagship, batching is a rout. At 8 concurrent streams

rapid-mlx aggregates 82.9 t/s against llama.cpp's 48.8 (1.7×) and Ollama's 27.2 (3.0×) — and it's the only engine on this row that gets faster going from 4 to 8 streams while both others degrade. Whole-batch wall-clock throughput (which includes prefilling all eight 1K prompts) agrees: 41.8 vs 36.2 vs 26.3 t/s.

1. On gpt-oss the decode race is a tie, and the wall clock isn't.

rapid-mlx and llama-server aggregate about 1% apart at 8-way (62.9 vs 63.6), Ollama trails at 49.3. But this workload prefills eight ~1K prompts per batch, and llama.cpp's 2× prefill advantage puts it ahead on whole-batch throughput: 43.2 vs 34.7 t/s. If your concurrent traffic is long-in/short-out, that's the number you'll feel; long generations tilt it back toward decode.

1. On the dense 12B we lose concurrency outright. llama-server

scales to 30.9 t/s aggregate at 8-way while rapid-mlx sits at 22.6 — dense-model batching is llama.cpp's win on this hardware, full stop. (Ollama lands at rapid's level, 22.4, degrading like us.)

The pattern matches the single-stream story: the sparser the model, the bigger the MLX advantage — and the models people actually run on 32 GB Macs (Qwen3.6-35B-A3B, gpt-oss) are exactly the sparse ones.

Where we lose: prefill

llama.cpp chews through prompts faster than MLX, consistently. llama-bench pp512 vs raw MLX prompt throughput: 558 vs 250 t/s (gpt-oss), 460 vs 277 (Qwen), 172 vs 96 (Gemma). On the server path, a cold ~1,000-token prompt confirms it — time to first token:

All three engines cache the prompt prefix, so within a conversation follow-ups drop to warm TTFTs (llama.cpp ~60–390 ms, Ollama ~275–680 ms, rapid-mlx ~130–390 ms). But the first long prompt of a session is llama.cpp's win, and if your workload is mostly cold long-document ingestion, that matters more than decode.

The LM Studio wrinkle

LM Studio's MLX engine hit the highest burst decode of the day (71.5 t/s on gpt-oss short prompts) — but in our runs it re-paid the full prefill on every long request to the same conversation (~4 s TTFT on runs that were warm cache hits everywhere else). Great engine, but as benchmarked, no cross-request prompt reuse — which dominates real chat latency. (We ran LM Studio at defaults; details and per-run numbers in the raw JSON. The Qwen3.6 pair wasn't measured there — its separate model store meant another ~40 GB of downloads and we hit our timebox; recorded as not-measured rather than guessed.)

A finding we didn't go looking for

Ollama's model blobs are no longer portable llama.cpp GGUFs. We tried to reuse Ollama's downloaded weights for the llama.cpp runs — upstream llama.cpp rejected two of the three (unknown model architecture: gptoss; a malformed rope metadata key on Qwen3.6). Only the Gemma blob loaded. "Ollama is just llama.cpp underneath" is operationally no longer true: its conversions are its own, and as the gpt-oss numbers show, they can cost real speed.

What Ollama and LM Studio still do better

This is a speed benchmark, not a verdict. Ollama runs on Linux and Windows, has a huge model library and a giant ecosystem of integrations. LM Studio is the best GUI in local AI and its MLX engine is seriously fast at raw decode. If those are your constraints, they're good tools. Our case is narrow and specific: on Apple Silicon, MLX-native serving is faster where it counts most for interactive and agent use — sometimes by 1.5×, never meaningfully slower at decode — and rapid-mlx speaks the same OpenAI wire. The Ollama migration guide shows the switch.

Reproduce it

Everything above is three timed runs per cell on one machine — the raw results JSON carries every individual run, engine-reported cross-checks, versions, flags, prompts, and an anomalies section. The rapid-mlx side is one install away:

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve qwen3.6-35b-4bit

If you run this on other hardware (we'd love M4 numbers — this mini is an M2 Pro), the protocol is in the JSON's meta.protocol block.

Frequently asked questions

Is rapid-mlx faster than Ollama on a Mac?

At decode — the speed you feel while an answer streams — yes in our measurements: about 1.45× on gpt-oss-20b (46.7 vs 32.1 t/s) and about 1.5× on Qwen3.6-35B MoE (58.5 vs 39 t/s), with a ~15% edge on dense Gemma4-12B (21.3 vs 18.0 t/s). Ollama was also ~25% slower than plain llama.cpp on identical gpt-oss weights. Cold prompt ingestion is the reverse: llama.cpp-family engines prefill faster.

Which is faster on Apple Silicon, MLX or llama.cpp?

Neither, universally. In our runs MLX decodes faster — dramatically so on mixture-of-experts models (1.5×) — while llama.cpp prefills long prompts roughly 1.7–2× faster. For chat and agent loops, decode plus prompt caching dominates, which favors MLX; for one-shot long-document ingestion, prefill dominates, which favors llama.cpp.

Is LM Studio faster than Ollama?

On the same GGUF weights in our runs, yes — LM Studio's llama.cpp engine tracked plain llama.cpp (42 t/s on gpt-oss) while Ollama trailed at 32 t/s. Its MLX engine was faster still at raw decode, but re-processed the full prompt on every long request in our tests, which cost seconds of latency per turn.

Which engine handles concurrent requests best on a Mac?

Depends on the model. On the MoE models most people serve (Qwen3.6-35B-A3B, gpt-oss), rapid-mlx aggregated the most decode throughput in our 8-stream test — 82.9 t/s on Qwen3.6-35B against llama.cpp's 48.8 and Ollama's 27.2 — and was the only engine that got faster moving from 4 to 8 streams on that model. On a dense 12B the ranking flips: llama.cpp's batching won (30.9 vs our 22.6). Ollama's aggregate throughput degraded from 4 to 8 streams on every model we tested.

What hardware were these benchmarks run on?

A Mac mini with an M2 Pro and 32 GB of unified memory — deliberately mid-range, since that's closer to what most people run than a maxed-out Mac Studio. Every engine ran on the same idle machine, plugged in, with medians of three timed runs per test.