Run Gemma 4 12B on a Mac
The dense mid-size of Google's current Gemma 4 generation, with vision input. 23 tok/s on an M2 Pro is comfortable reading speed; 7.8 GB resident fits 16 GB Macs with room to spare.
12 B dense · 4-bit · vision vision + text · 16–32 GB Macs · Tier-1 family
One command
$ rapid-mlx serve gemma-4-12b-4bit
Weights (6.3 GB) download on first run; you get an OpenAI-compatible
endpoint at http://localhost:8000/v1. No rapid-mlx yet? It's one line —
curl -fsSL https://rapidmlx.com/install.sh | bash — or take the
desktop app.
Measured on real hardware
| Machine | Decode | First token | Peak memory | Cold boot | Weights |
|---|---|---|---|---|---|
| Mac mini M2 Pro · 32 GB | 22.6 tok/s | 0.95 s | 7.8 GB | 8.6 s | 6.3 GB |
Measured on rapid-mlx 0.12.10, 2026-08-11. Decode: median of 3 runs, temperature 0, 256-token saturating generation, engine-reported token counts, unique salt per request (no prefix-cache hits). TTFT: median of 3, short prompt. Boot: process spawn to first completed token. Peak RSS: 0.5 s sampling across the run.
Will it fit your Mac?
Peak resident memory measured 7.8 GB during a 256-token generation. The KV cache grows with context length, so treat that as a floor, not a ceiling. For the conservative install-default placement see the hardware tiers table; to compare against every model your RAM can hold, use the live picker.
Variants & alternatives
gemma-4-26b-4bit— Gemma 4 26B — the MoE big sibling, faster decode
FAQ
How much memory does Gemma 4 12B need on a Mac?
Measured peak resident memory was 7.8 GB on rapid-mlx 0.12.10 during a 256-token generation (M2 Pro, 32 GB). The weights are 6.3 GB on disk. Longer contexts grow the KV cache beyond this, so leave headroom.
How fast is Gemma 4 12B on Apple Silicon?
We measured 22.6 tokens/sec sustained decode and 0.95 s time-to-first-token on a Mac mini M2 Pro (32 GB), median of 3 runs at temperature 0.
How do I run Gemma 4 12B locally?
Install rapid-mlx (curl -fsSL https://rapidmlx.com/install.sh | bash, or brew install rapid-mlx), then: rapid-mlx serve gemma-4-12b-4bit — the weights download on first run and you get an OpenAI-compatible endpoint at localhost:8000/v1 that works with Cursor, Claude Code, Aider, and any OpenAI client.