Run Gemma 4 Locally on Your MacBook Pro

Gemma is Google's open-weights family, and every size runs on Apple Silicon — from a 1 GB nano that fits a base MacBook Air to a 31B dense model on a Studio. Here's the size that fits your Mac, and the one command that serves it.

Gemma is Google's open-weights family — the same research lineage as Gemini, released with permissive licensing and small enough to run on your own hardware. It spans a wider size range than almost anything else in open weights: a 1 GB nano built for phones on one end, a 31B dense model on the other, and a mixture-of-experts in between. All of it runs on Apple Silicon.

This guide is the short path: pick the right Gemma for your Mac's RAM, serve it with one command, and point your tools at it.

The 60-second version

curl -fsSL https://rapidmlx.com/install.sh | bash   # or: brew install rapid-mlx
rapid-mlx serve gemma-4-12b-4bit                    # pick your size below

Then talk to it — it's a standard OpenAI-compatible server on http://localhost:8000:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "default", "messages": [{"role": "user", "content": "Explain the difference between a process and a thread."}]}'

First run downloads the weights; after that, startup is seconds. No account, no API key, nothing leaves your machine.

Gemma 4 vs Gemma 3 — which one?

Gemma 4 is the current generation, and it's the one to start with. It covers the whole range on its own: the e2b / e4b nano models (1–2 GB, the on-device sizes designed to run on phones), a dense 12B, a mixture-of-experts 26B-A4B that activates only ~4B parameters per token, and a dense 31B flagship. QAT (quantization-aware trained) variants exist across the range — prefer them at 4-bit and 8-bit when both are available, since they hold quality better at low bit-widths.

Gemma 3 is the previous generation and still fully supported, but unless you have a specific reason to pin to it, Gemma 4 is faster and higher quality at every size.

Which size fits your Mac

Sizes below are what the weights occupy on disk (and roughly in unified memory once loaded). Leave headroom for your context window and the rest of macOS.

Want to go even smaller? gemma-4-e2b-4bit is just 1.0 GB — the smallest real Gemma, useful when you need to leave almost all your RAM for other work. And for retrieval and semantic search, embedding-gemma is a dedicated 300M embeddings model that fits any Mac.

Not sure where you land? The hardware tier guide maps every Mac from an 8 GB Air to a 512 GB Studio to a recommendation, and models.rapidmlx.com lets you filter the whole catalog by your RAM.

Tool calling and vision work out of the box

serve auto-configures the right chat template and tool-call parser for every Gemma alias — Gemma 4 uses the gemma4 parser — and grammar-constrained tool calling is on by default, so function calls come back as valid JSON, not as a paragraph that almost parses. Gemma is natively multimodal, so the 12B and up can take images alongside text on the standard OpenAI wire.

That's the boring-but-load-bearing part: it means Gemma-on-your-Mac slots into tools that expect OpenAI semantics without any glue code.

Wire it into your editor or agent

The server speaks the OpenAI protocol, so anything with a "custom base URL" box can use it:

• Claude Code — run the CLI

agent fully local against your Gemma.

• Cursor — chat, Ctrl-K and

composer against http://localhost:8000/v1.

• Codex CLI and

Aider — both speak the same OpenAI-compatible endpoint.

For agent work specifically, prefer the gemma-4-26b MoE or the gemma-4-31b dense — multi-step tool use is where the extra capability pays for itself. Gemma's strengths are broad general knowledge, strong multilingual coverage, and multimodal input; for the heaviest autonomous coding loops, a Qwen 3.6 is still the family we gate our own releases on.

How fast is it?

We publish real numbers rather than vibes: our five-model MacBook Pro benchmark puts local models through coding, reasoning, tool-calling and memory tests, and the M3 Ultra benchmark covers the desktop end. On a MacBook Pro, the 12B is fast enough for interactive chat and the 26B MoE keeps up surprisingly well for its quality.

Frequently asked questions

Can I run Gemma locally on a Mac with 8 GB of RAM?

Yes — gemma-4-e4b-4bit is 2.0 GB on disk and runs comfortably on an 8 GB MacBook Air with room left for the OS, and gemma-4-e2b-4bit is smaller still at 1.0 GB. Both handle chat, Q&A and light editing well. For heavier work you'll want 16 GB and the 12B, or 24 GB+ for the 26B MoE and 31B dense.

What's the difference between Gemma 3 and Gemma 4?

Gemma 4 is the current generation and covers the whole size range on its own: 1–2 GB nano models (e2b/e4b), a dense 12B, a mixture-of-experts 26B-A4B (4B active per token), and a dense 31B. It's faster and higher quality than Gemma 3 at every comparable size, so start with Gemma 4 unless you have a reason to pin to the older generation.

Does Gemma work with Claude Code and Cursor locally?

Yes. rapid-mlx serves Gemma behind an OpenAI-compatible endpoint, so any tool with a custom base URL box — Claude Code, Cursor, Codex, Aider — can point at http://localhost:8000/v1 and use it with no bridge and no plugin.

Is running Gemma locally actually free and private?

The weights are open, rapid-mlx is Apache 2.0, and inference happens entirely on your Mac — no account, no per-token billing, and your prompts and code never leave the machine. The only costs are disk space for the weights and the electricity your Mac was already using.