Best local LLMs for an M2 Pro Mac with 32 GB

34 models have been measured on an Apple M2 Pro with 32 GB of unified memory. Every number below was measured on that chip and memory size, by the Rapid-MLX community or by us. None is scaled from another Mac.

What Rapid-MLX picks for 32 GB

These two come from the engine's recommendation policy for its 32 GB tier (rapid/default/text-generation/ram-v1): the same picks rapid-mlx recipe, the installer and the desktop app show. They are the policy's choice, not a ranking of the measurements below. How the tiers are chosen.

Measured on this Mac

Decode speed is how fast the model writes once it has read your prompt; first-token time is how long it reads before it starts. Each section has its own scale and method, so compare rows within a section.

Community runs

Current community benchmark: a 512-token prompt, then 128 generated tokens. Median across runs; each run on the contributor's own Mac.

• qwen3-0.6b258tok/s
  first token 176 ms · peak memory 2.1 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• lfm2.5-1b-4bit210tok/s
  first token 316 ms · peak memory 1.7 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• bonsai-1.7b-2bit190tok/s
  first token 324 ms · peak memory 2.7 GB · rapid-mlx 0.14.2 · 2 runs · 2026-09-16
• vibethinker-1.5b-4bit131tok/s
  first token 499 ms · peak memory 1.8 GB · rapid-mlx 0.13.4 · 1 run · 2026-09-06
• lfm2.5-8b-a1b-4bit124tok/s
  first token 537 ms · peak memory 5.5 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• lfm2.5-2.6b-4bit93.2tok/s
  first token 717 ms · peak memory 2.7 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• llama3-3b-4bit85.6tok/s
  first token 602 ms · peak memory 4.1 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• nemotron-3.5-lightning-30b-4bit64.7tok/s
  first token 1.3 s · peak memory 19.8 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• gemma3-4b-qat-4bit64.3tok/s
  first token 970 ms · peak memory 4.2 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• qwen3-4b-thinking-2507-4bit63.2tok/s
  first token 1.0 s · peak memory 5.4 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• qwen3.6-35b-mxfp463.0tok/s
  first token 1.1 s · peak memory 19.3 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• qwen3.5-4b-4bit62.4tok/s
  first token 1.0 s · peak memory 4.4 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• qwen3.6-35b-nvfp461.9tok/s
  first token 1.1 s · peak memory 20.3 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• qwen3.6-35b61.3tok/s
  first token 1.1 s · peak memory 20.3 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• qwen3-4b-instruct-2507-4bit59.9tok/s
  first token 1.1 s · peak memory 6.2 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• bonsai-8b-2bit59.8tok/s
  first token 1.4 s · peak memory 5.3 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• gemma-4-e4b-4bit52.2tok/s
  first token 709 ms · peak memory 5.2 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• gpt-oss-20b51.9tok/s
  first token 1.2 s · peak memory 12.4 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• qwen3.6-35b-dwq46.6tok/s
  first token 1.1 s · peak memory 21.4 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• qwen3-8b-4bit37.1tok/s
  first token 1.9 s · peak memory 7.4 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• deepseek-r1-8b-4bit37.0tok/s
  first token 2.0 s · peak memory 7.4 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• qwen3.5-9b-4bit36.0tok/s
  first token 2.0 s · peak memory 6.8 GB · rapid-mlx 0.14.2 · 2 runs · 2026-09-15
• k2-horizon-7b-4bit35.8tok/s
  first token 2.3 s · peak memory 7.9 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• qwen3.5-4b-8bit35.5tok/s
  first token 1.3 s · peak memory 6.4 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• qwen3.5-9b-6bit24.8tok/s
  first token 2.1 s · peak memory 8.8 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• gemma-4-12b-4bit23.0tok/s
  first token 3.1 s · peak memory 7.7 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• qwen3.5-9b-8bit20.1tok/s
  first token 2.0 s · peak memory 10.8 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• bonsai-27b-2bit19.3tok/s
  first token 5.2 s · peak memory 11.0 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• devstral-v2-24b-4bit12.6tok/s
  first token 6.2 s · peak memory 17.5 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• qwen3.6-27b11.4tok/s
  first token 7.7 s · peak memory 18.4 GB · rapid-mlx 0.14.2 · 2 runs · 2026-09-16
• qwen3.5-27b-4bit11.2tok/s
  first token 7.2 s · peak memory 18.1 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15
• ornith-1.5-9b-bf1610.9tok/s
  first token 1.8 s · peak memory 18.8 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-16
• qwen3.8-27b-4bit9.9tok/s
  first token 9.6 s · peak memory 18.6 GB · rapid-mlx 0.14.3 · 1 run · 2026-09-19
• qwen3.5-27b-6bit7.7tok/s
  first token 7.5 s · peak memory 24.3 GB · rapid-mlx 0.14.2 · 1 run · 2026-09-15

Older community benchmark

The previous community suite (short prompt). Its protocol differs from the current one, so compare these rows with each other, not with the sections above.

• qwen3.5-4b-4bit61.3tok/s
  first token 1.7 s · rapid-mlx 0.11.9 · 3 runs · 2026-08-02 · details
• gpt-oss-20b48.2tok/s
  first token 1.9 s · rapid-mlx 0.11.9 · 1 run · 2026-08-02
• qwen3.5-9b-4bit36.0tok/s
  first token 3.2 s · rapid-mlx 0.11.9 · 2 runs · 2026-08-02 · details

Run one

rapid-mlx recipe detects your Mac's memory and prints the pick with its command. Any model above runs with rapid-mlx serve <alias> and serves an OpenAI- and Anthropic-compatible API on http://localhost:8000.

$ curl -fsSL https://rapidmlx.com/install.sh | bash
$ rapid-mlx recipe
$ rapid-mlx serve qwen3.8-27b-4bit

Add your Mac's numbers

Have an M2 Pro Mac with 32 GB? Run the benchmark and share it. It lands on the live leaderboard view for this Mac and on this page at its next build.

$ rapid-mlx benchmark run qwen3.8-27b-4bit
$ rapid-mlx benchmark share <run-id>

More

• Every Mac with enough measurements
• The best local LLM for your Mac, by memory
• Hardware tiers: the engine's picks for every memory size
• Rapid-MLX vs mlx-lm, measured task by task
• Use these models with a coding agent