Gemma
22 MLX aliases · Google's open-weights family — Gemma 4 / 4-mobile (3n) / 3 / EmbeddingGemma.
The Gemma family on rapid-mlx covers Google's open-weights line end-to-end: the frontier Gemma 4 (12B / 26B / 31B, plus QAT variants for sharper low-bit quants), the mobile-tier Gemma 4 E-series and Gemma 3n, the older but still useful Gemma 3 predecessor, and EmbeddingGemma for local /v1/embeddings. All chat variants use the gemma4 tool-call parser (Gemma 3 uses hermes); EmbeddingGemma routes to the embeddings endpoint, not chat.
- family
- Gemma
- aliases
- 22
- lines
- 4
- install
- rapid-mlx serve <alias>
- OpenAI base URL
- http://localhost:8000/v1
Lines in this family
- Gemma 4 (10 aliases)
- Gemma 4 (mobile / 3n) (4 aliases)
- Gemma 3 (6 aliases)
- EmbeddingGemma (2 aliases)
Gemma 4 · 10 aliases
Google's frontier open-weights family — text + vision. rapid-mlx ships 10 quants including Google's QAT (quantization-aware training) variants for sharper 4/8-bit quality. All variants use the gemma4 tool-call parser and the gemma4 reasoning parser.
parser: gemma4
| alias | hf repo | tool parser | reasoning | flags |
|---|---|---|---|---|
| gemma-4-12b-4bit | mlx-community/gemma-4-12B-it-4bit | gemma4 | gemma4 | spec |
| gemma-4-12b-8bit | mlx-community/gemma-4-12B-it-8bit | gemma4 | gemma4 | spec |
| gemma-4-12b-qat-4bit | mlx-community/gemma-4-12B-it-qat-4bit | gemma4 | gemma4 | spec |
| gemma-4-12b-qat-8bit | mlx-community/gemma-4-12B-it-qat-8bit | gemma4 | gemma4 | spec |
| gemma-4-26b-4bit | mlx-community/gemma-4-26b-a4b-it-4bit | gemma4 | gemma4 | spec |
| gemma-4-26b-qat-4bit | mlx-community/gemma-4-26B-A4B-it-qat-4bit | gemma4 | gemma4 | spec |
| gemma-4-31b-4bit | mlx-community/gemma-4-31b-it-4bit | gemma4 | gemma4 | spec |
| gemma-4-31b-8bit | mlx-community/gemma-4-31b-it-8bit | gemma4 | gemma4 | spec |
| gemma-4-31b-qat-4bit | mlx-community/gemma-4-31B-it-qat-4bit | gemma4 | gemma4 | spec |
| gemma-4-31b-qat-8bit | mlx-community/gemma-4-31B-it-qat-8bit | gemma4 | gemma4 | spec |
Notes & caveats
- QAT variants are noticeably sharper at low bit-widths — prefer
-qat-4bitover plain-4bitwhen both exist. - 31B is the recommended ceiling for a 64 GB Mac; 12B fits comfortably in 16 GB.
Gemma 4 (mobile / 3n) · 4 aliases
The mobile-tier Gemma line — Gemma 4 E-series (effective 2B / 4B with weight-sharing) and the older Gemma 3n family. These are the smallest production-grade Gemmas, designed to fit phones and base-RAM Macs. Useful as draft models for speculative decoding.
parser: gemma4
| alias | hf repo | tool parser | reasoning | flags |
|---|---|---|---|---|
| gemma-3n-e2b-4bit | mlx-community/gemma-3n-E2B-it-4bit | — | — | spec |
| gemma-3n-e4b-4bit | lmstudio-community/gemma-3n-E4B-it-MLX-4bit | — | — | spec |
| gemma-4-e2b-4bit | mlx-community/gemma-4-e2b-it-4bit | gemma4 | gemma4 | spec |
| gemma-4-e4b-4bit | mlx-community/gemma-4-e4b-it-4bit | gemma4 | gemma4 | spec |
Notes & caveats
- Gemma 4 E2B is a strong draft-model candidate for speculative decoding against Gemma 4 31B.
- Gemma 3n entries have no tool-call parser configured — they are text-only.
Gemma 3 · 6 aliases
Kept in the registry for users who want the older, more battle-tested generation. All variants share the hermes tool envelope; reasoning is plain (no <think> stream).
parser: hermes
| alias | hf repo | tool parser | reasoning | flags |
|---|---|---|---|---|
| gemma3-12b-4bit | mlx-community/gemma-3-12b-it-qat-4bit | hermes | — | spec |
| gemma3-1b-4bit | mlx-community/gemma-3-1b-it-4bit | hermes | — | spec |
| gemma3-1b-qat-4bit | mlx-community/gemma-3-1b-it-qat-4bit | hermes | — | spec |
| gemma3-27b-4bit | mlx-community/gemma-3-27b-it-4bit | hermes | — | spec |
| gemma3-27b-qat-4bit | mlx-community/gemma-3-27b-it-qat-4bit | hermes | — | spec |
| gemma3-4b-qat-4bit | mlx-community/gemma-3-4b-it-qat-4bit | hermes | — | spec |
Notes & caveats
- QAT variants ship for the 1B / 4B / 12B / 27B tiers — same quant-quality benefit as Gemma 4.
- If you can pick Gemma 4 instead, you should — it's strictly better on the standard benchmarks.
EmbeddingGemma · 2 aliases
Google's 300M embedding model. We ship it as a first-class alias under rapid-mlx so you can run a local /v1/embeddings endpoint — RAG, semantic search, and offline vector-DB ingest without an API call.
parser: —
| alias | hf repo | tool parser | reasoning | flags |
|---|---|---|---|---|
| embeddinggemma-300m-6bit | mlx-community/embeddinggemma-300m-6bit | — | — | — |
| embeddinggemma-300m-8bit | mlx-community/embeddinggemma-300m-8bit | — | — | — |
Notes & caveats
- Routes to
/v1/embeddingsrather than/v1/chat/completions. - 300M parameters → fits any Apple Silicon Mac, including base 8 GB models.
Notes & caveats
- Gemma 4 + 4-mobile use the
gemma4tool-call parser; Gemma 3 useshermes; EmbeddingGemma has no tool parser (it's an embeddings model). - QAT (quantization-aware trained) variants exist for Gemma 4 and Gemma 3 — prefer them at 4/8-bit when both exist.