MLX vs llama.cpp — an Apple Silicon Performance Comparison, Measured

The eternal r/LocalLLaMA question, answered with a benchmark instead of a take — same Mac, same models, both frameworks' own tools, raw JSON published. Neither wins outright. What decides it is the shape of your workload.

"Should I run MLX or llama.cpp on my Mac?" gets asked weekly, and the answers are usually vibes. So we measured it: one idle M2 Pro Mac mini (32 GB), three models, each framework benchmarked with its own canonical tool — llama-bench for llama.cpp, mlx_lm generate for MLX — plus full server-path runs to confirm the raw numbers survive contact with a real OpenAI-compatible server. Medians of repeated runs, and the complete raw JSON is published.

Disclosure: we build rapid-mlx, an MLX-native server, so we have a horse in this race — which is why the numbers where llama.cpp wins are printed just as large.

The raw numbers

llama.cpp: llama-bench -p 512 -n 128 (Metal, build 10330, 5 reps). MLX: mlx_lm generate, ~490-token prompt, 128 tokens out, 3 reps, mlx-lm 0.31.3. Same quantization class per pair (native MXFP4 for gpt-oss; Q4_K_M ≈ 4.8 bpw vs MLX 4-bit ≈ 4.5 bpw for the others).

Generation (decode) — tokens/second:

Prompt processing (prefill) — tokens/second:

That's the whole story in two tables: MLX decodes faster — hugely so on sparse MoE models — and llama.cpp prefills 1.7–2.2× faster, every time.

(A wrinkle inside the decode table: raw llama-bench actually edges raw MLX on gpt-oss. That lead didn't survive the server path — see below — but it's real at the kernel level and we're not hiding it.)

Do the raw numbers survive a real server?

Raw benchmarks flatter frameworks; users talk to servers. So we re-ran everything through OpenAI-compatible endpoints — llama-server for llama.cpp, rapid-mlx for MLX — streaming, temperature 0, medians of 3 (rapid-mlx column re-measured 2026-08-12 on 0.12.11, same machine and protocol; llama-server numbers are the 2026-08-11 measurements at build 10330):

The MoE gap (1.5×) carries straight through. The dense model, a tie at the raw-framework level and a tie on the server path in our original 0.12.9 run (18.7 vs 18.3), moved to a ~15% rapid-mlx edge on 0.12.11. And cold time-to-first-token flips the sign, exactly as the prefill table predicts: on a ~1,000-token cold prompt, llama-server answered in 2.3 s (Qwen) / 6.7 s (Gemma) against rapid-mlx's 3.4 s / 9.3 s. Both cache prompt prefixes, so follow-up turns in a conversation are sub-second on either — the cold-start penalty is paid once per fresh context.

Concurrent streams

(Added 2026-08-12.) We also ran 4 and 8 simultaneous salted requests (~1K-token prompt, 256-token budget each, medians of 3 batches; llama-server --parallel N -c N×4096, restart between lanes). Aggregate decode across all streams, 4-way → 8-way:

Same shape as everything else in this post, sharpened: on the sparse MoE 35B, MLX serving is the only stack that speeds up with more streams and ends 1.7× ahead; on gpt-oss it's a dead heat; on the dense 12B llama.cpp's batching wins outright. llama.cpp's prefill edge also means it wins whole-batch wall-clock on gpt-oss (43.2 vs 34.7 t/s incl. prefilling all eight prompts) while MLX wins it on the MoE (41.8 vs 36.2). Full per-batch numbers are in the raw JSON.

So which one should you run?

It depends on the shape of your workload, and now you can look it up:

• Chat, coding assistants, agent loops — decode speed and warm-cache

TTFT dominate; you prefill once and then generate for minutes. MLX wins this shape, and wins it big if your model is MoE (which the current best-on-Mac models — Qwen3.6-35B-A3B, gpt-oss — are).

• One-shot long-document ingestion (summarize this 50-page PDF, cold

RAG over fresh chunks) — prefill dominates. llama.cpp wins this shape by roughly 2×.

• Dense small models — the smallest gap in the set (MLX ahead ~10–15%

in our runs); close enough to choose on ecosystem instead.

One more Apple-specific note: MLX is Apple's own framework, built around unified memory, and MoE-heavy model design is where open-weights is heading — the two models with the widest MLX margins here are the two most recent. llama.cpp's portability (Linux, Windows, CUDA, Vulkan) remains its unanswerable advantage; on a Mac, that advantage is moot.

Caveats, so you don't have to find them yourself

• One machine (M2 Pro, 32 GB). Ratios should hold across Apple Silicon;

absolute numbers won't. We'd genuinely like to see M4 replications.

• Q4_K_M carries ~0.3 more bits per weight than MLX 4-bit — quality isn't

identical at "4-bit," in GGUF's favor, and this is a speed test only.

• Both projects move fast (llama.cpp build 10330, mlx-lm 0.31.3, MLX

0.31.2 here; rapid-mlx server numbers re-measured on 0.12.11). Numbers have a shelf life; the protocol doesn't.

• Gemma4's raw MLX number came via mlx-vlm (mlx-lm can't load that

unified checkpoint yet) — noted in the JSON.

Frequently asked questions

Is MLX faster than llama.cpp on Apple Silicon?

At generation, mostly — in our measurements MLX ranged from a near-tie (llama.cpp edged gpt-oss 52.0 vs 49.5 t/s at the raw kernel level, though the served comparison flipped back to MLX) up to 1.58× on a mixture-of-experts 35B. At prompt processing, no: llama.cpp prefilled 1.7–2.2× faster on every model we tested. Which framework is "faster" for you depends on whether your workload is generation-heavy (chat, agents) or ingestion-heavy (long-document one-shots).

Why is MLX so much faster on MoE models?

Sparse mixture-of-experts models like Qwen3.6-35B-A3B store 35B parameters but activate ~3B per token, so decode is dominated by memory-bandwidth- efficient gather-and-compute over a changing subset of weights — a pattern MLX's unified-memory design handles very well. The gap we measured (61.7 vs 39.1 t/s raw, 58.5 vs 38.3 t/s served) reproduced across both raw and server paths.

Does llama.cpp's prefill advantage matter in practice?

For the first long prompt of a session, yes — a ~1,000-token cold prompt reached first token about 1.4× sooner via llama-server in our runs. For everything after that, both stacks cache the prompt prefix and follow-up turns start in well under a second, so conversations feel similar; batch document processing feels different.

Can I use both?

Easily — they read different weight formats (GGUF vs MLX/safetensors), so they coexist on one machine. A practical split: an MLX server like rapid-mlx for your daily chat and coding agents, llama.cpp for bulk document ingestion jobs.