Run Gemma 4 26B (A4B MoE) on a Mac

Mixture-of-experts done right for local use: 26 B of knowledge, but only ~4 B active per token — so it decodes at 50 tok/s on an M2 Pro, more than 4× the dense 27B class, and stays around 6 GB resident.

MoE · 26 B total / 4 B active · 4-bit  32 GB Macs · fast high-quality chat · Tier-1 family

One command

Weights (14.3 GB) download on first run; you get an OpenAI-compatible endpoint at http://localhost:8000/v1. No rapid-mlx yet? It's one line — curl -fsSL https://rapidmlx.com/install.sh | bash — or take the desktop app.

Measured on real hardware

Measured on rapid-mlx 0.12.10, 2026-08-11. Decode: median of 3 runs, temperature 0, 256-token saturating generation, engine-reported token counts, unique salt per request (no prefix-cache hits). TTFT: median of 3, short prompt. Boot: process spawn to first completed token. Peak RSS: 0.5 s sampling across the run.

Will it fit your Mac?

Peak resident memory measured 6.0 GB during a 256-token generation. The KV cache grows with context length, so treat that as a floor, not a ceiling. For the conservative install-default placement see the hardware tiers table; to compare against every model your RAM can hold, use the live picker.

In this 256-token run, peak resident memory came in below the on-disk weight size — with mixture-of-experts weights the runtime doesn't have to touch every expert right away. Don't budget by that number: for sustained use, follow our memory guide and plan for the full weight size plus context headroom. The measured RSS here is a floor, not a plan.

Variants & alternatives

• gemma-4-26b-qat-4bit — The QAT 4-bit variant — quantisation-aware weights
• gemma-4-12b-4bit — Gemma 4 12B dense — smaller, takes images too

FAQ

Where next