Run Qwen3.5 9B on a Mac
The sweet spot of the Qwen3.5 dense line for 16 GB Macs: measurably smarter than the 4B on tool calling and code, still under 6 GB resident.
9 B dense · 4-bit 16 GB Macs · tool calling · general assistant work
One command
$ rapid-mlx serve qwen3.5-9b-4bit
Weights (5.6 GB) download on first run; you get an OpenAI-compatible
endpoint at http://localhost:8000/v1. No rapid-mlx yet? It's one line —
curl -fsSL https://rapidmlx.com/install.sh | bash — or take the
desktop app.
Measured on real hardware
| Machine | Decode | First token | Peak memory | Cold boot | Weights |
|---|---|---|---|---|---|
| Mac mini M2 Pro · 32 GB | 36.4 tok/s | 0.84 s | 5.8 GB | 7.8 s | 5.6 GB |
| Mac Studio M3 Ultra · 256 GB † | 106 tok/s | — | 5.1 GB | — | — |
Measured on rapid-mlx 0.12.10, 2026-08-11. Decode: median of 3 runs, temperature 0, 256-token saturating generation, engine-reported token counts, unique salt per request (no prefix-cache hits). TTFT: median of 3, short prompt. Boot: process spawn to first completed token. Peak RSS: 0.5 s sampling across the run. M3 Ultra row: from our 17-model benchmark post (same alias and quant).
On the M3 Ultra benchmark it scored 83% on tool calling and 70% on code. Full benchmark →
Will it fit your Mac?
Peak resident memory measured 5.8 GB during a 256-token generation. The KV cache grows with context length, so treat that as a floor, not a ceiling. For the conservative install-default placement see the hardware tiers table; to compare against every model your RAM can hold, use the live picker.
Variants & alternatives
qwen3.5-4b-4bit— Qwen3.5 4B — smaller and fasterqwen3.6-35b— Qwen3.6 35B MoE — the flagship, if you have the memory
FAQ
How much memory does Qwen3.5 9B need on a Mac?
Measured peak resident memory was 5.8 GB on rapid-mlx 0.12.10 during a 256-token generation (M2 Pro, 32 GB). The weights are 5.6 GB on disk. Longer contexts grow the KV cache beyond this, so leave headroom.
How fast is Qwen3.5 9B on Apple Silicon?
We measured 36.4 tokens/sec sustained decode and 0.84 s time-to-first-token on a Mac mini M2 Pro (32 GB), median of 3 runs at temperature 0, and 106 tokens/sec on an M3 Ultra in our published 17-model benchmark.
How do I run Qwen3.5 9B locally?
Install rapid-mlx (curl -fsSL https://rapidmlx.com/install.sh | bash, or brew install rapid-mlx), then: rapid-mlx serve qwen3.5-9b-4bit — the weights download on first run and you get an OpenAI-compatible endpoint at localhost:8000/v1 that works with Cursor, Claude Code, Aider, and any OpenAI client.