flint-40b-a2.5b-4bit
Run Flint 40B on a Mac
Run it
Install: pip install rapid-mlx or Rapid-MLX Desktop.
rapid-mlx serve flint-40b-a2.5b-4bit
Minimum RAM: 48 GB or more recommended. Run a coding agent on Mac with rapid-mlx launch <agent> (Codex, Claude Code, OpenCode).
A 40B-total, ~2.5B-active mixture-of-experts model that runs on a 36 GB Mac. Tool-call and reasoning parsing are configured automatically for the alias, and downloads use the validated checkpoint revision pinned in the registry.
MoE · 40 B total / ~2.5 B active · INT436 GB+ Macs · sparse MoE · tool calls and reasoning
First request
rapid-mlx serve flint-40b-a2.5b-4bit
Weights (22.6 GB) download on first run; you get an OpenAI-compatible
endpoint at http://localhost:8000/v1. No rapid-mlx yet? It's one line —
curl -fsSL https://rapidmlx.com/install.sh | bash — or take the
desktop app.
Point a client at it. OpenAI SDK: base_url="http://localhost:8000/v1",
model="flint-40b-a2.5b-4bit". Claude Code:
ANTHROPIC_BASE_URL=http://localhost:8000 (no /v1 suffix) —
see the Claude Code guide.
Current-version run status
Hardware listed in release notes · MacBook Pro M5 Max · 36 GB
Source and status
| Machine | Decode | First token | Peak memory | Cold boot | Weights |
|---|---|---|---|---|---|
| MacBook Pro M5 Max · 36 GB rapid-mlx 0.16.8 · 2026-10-22 · site benchmark pending engine release notes | — tok/s | — | — | — | 22.6 GB |
No site benchmark has been run for this build. The weight size comes from the tagged engine model registry.
Not yet measured on rapid-mlx 0.16.8. The site has not run this build. The engine measured it on an M5 Max 36 GB with greedy decoding and 256 output tokens: ~1K context reached 131 tok/s decode, 0.35–0.38 s time to first token and about 24 GB peak memory; ~8K context reached 118.5 tok/s and 2.2 s time to first token. These are workload-specific measurements, and 48 GB or more unified memory is recommended. Reproduction and measurements →
Will it fit your Mac?
48 GB or more recommended — the catalog alias asks for 48 GB; the engine's measurements used a 36 GB M5 Max.
For the conservative install-default placement see the hardware tiers table; to compare against every model your RAM can hold, use the live picker.
Variants & profiles
deepseek-v41-flash-reap-2bit— DeepSeek V4.1 Flash REAP — the other large-Mac sparse option
FAQ
How much memory does Flint 40B need on a Mac?
48 GB or more recommended — the catalog alias asks for 48 GB; the engine's measurements used a 36 GB M5 Max.
How fast is Flint 40B on Apple Silicon?
The site has not run this build. The engine measured it on an M5 Max 36 GB with greedy decoding and 256 output tokens: ~1K context reached 131 tok/s decode, 0.35–0.38 s time to first token and about 24 GB peak memory; ~8K context reached 118.5 tok/s and 2.2 s time to first token. These are workload-specific measurements, and 48 GB or more unified memory is recommended.
How do I run Flint 40B locally?
rapid-mlx serve flint-40b-a2.5b-4bit — the weights download on first run and you get an OpenAI-compatible endpoint at localhost:8000/v1 for Aider, Codex and any OpenAI client, plus the Anthropic API at localhost:8000 for Claude Code.