# rapid-mlx

OpenAI-compatible LLM inference server for Apple Silicon. MLX-native
quants, curated model support across 247 models / 15 families
(179 text/vision/reasoning + 44 audio TTS/STT + 9 video generation; Tier-1: Qwen 3.6, Gemma 4, DeepSeek, GPT-OSS), and benchmarks
focused on the M-series unified-memory envelope.

## Project
- Homepage: https://rapidmlx.com/
- GitHub: https://github.com/raullenchai/Rapid-MLX
- License: Apache-2.0 (engine)
- Current release: 0.14.3
- Install: `curl -fsSL https://rapidmlx.com/install.sh | bash` — or `brew install rapid-mlx` (now in homebrew/core; no tap, no trust)
- 中文指南 (Chinese guide, incl. mainland-China install + HF mirror notes): https://rapidmlx.com/zh
- 中文本地听写指南 (system-wide Mac dictation with local Whisper / Qwen3-ASR / SenseVoice; audio is not uploaded): https://rapidmlx.com/zh/local-dictation-mac

## Docs
- Docs index: https://rapidmlx.com/docs/
- Install & first request: https://rapidmlx.com/docs/getting-started.html
- Headless macOS service (boot without GUI login, KeepAlive, safe updates): https://rapidmlx.com/docs/headless-macos-service.html
- HTTP API reference: https://rapidmlx.com/docs/api.html
- Roadmap: https://rapidmlx.com/docs/roadmap.html
- CLI reference (serve / chat / pull / bench / doctor / agents): https://rapidmlx.com/docs/cli.html
- Hardware tiers (which model for your Mac's RAM, 8-15 GB through 96 GB+, measured boot + peak RSS): https://rapidmlx.com/docs/hardware-tiers.html
- Model aliases (naming scheme + quant suffixes): https://rapidmlx.com/docs/aliases.html
- Performance flags (prefix cache, speculative decoding, PFlash, TurboQuant): https://rapidmlx.com/docs/perf-flags.html
- Agent + framework compatibility matrix: https://rapidmlx.com/docs/matrix.html
- Troubleshooting (OOM, slow decode, tool calls as plain text): https://rapidmlx.com/docs/troubleshooting.html
- Extras (embeddings, rerank, structured output): https://rapidmlx.com/docs/extras.html
- Telemetry (what is collected, opt-in, how to disable): https://rapidmlx.com/docs/telemetry.html
- Changelog: https://rapidmlx.com/docs/changelog.html

## Integration guides (point your agent / IDE / SDK at a local server)
- OpenAI SDK three-line switch (Python + JS base_url swap, env-var zero-code option): https://rapidmlx.com/docs/frameworks/openai-sdk.html
- Claude Code on a local model: https://rapidmlx.com/docs/agents/claude-code.html
- Codex CLI: https://rapidmlx.com/docs/agents/codex-cli.html
- Aider: https://rapidmlx.com/docs/agents/aider.html
- Hermes Agent: https://rapidmlx.com/docs/agents/hermes.html
- DeepSeek Harness (dsh, DeepSeek's own agent, fully local): https://rapidmlx.com/docs/agents/deepseek-harness.html
- Cursor (IDE chat via custom base URL): https://rapidmlx.com/docs/agents/cursor.html
- Continue.dev (VS Code / JetBrains, one-shot `rapid-mlx launch continue-dev`): https://rapidmlx.com/docs/agents/continue-dev.html
- GitHub Copilot: https://rapidmlx.com/docs/agents/copilot.html
- OpenCode: https://rapidmlx.com/docs/agents/opencode.html
- OpenHands: https://rapidmlx.com/docs/agents/openhands.html
- Qwen Code: https://rapidmlx.com/docs/agents/qwen-code.html
- Kilo Code: https://rapidmlx.com/docs/agents/kilo-code.html
- gpt-oss via the OpenAI SDK (Harmony parsers): https://rapidmlx.com/docs/agents/gpt-oss.html
- LangChain: https://rapidmlx.com/docs/frameworks/langchain.html
- PydanticAI: https://rapidmlx.com/docs/frameworks/pydantic-ai.html
- smolagents: https://rapidmlx.com/docs/frameworks/smolagents.html

## Model pages (per-model, measured on an M2 Pro 32 GB — decode tok/s, TTFT, peak RSS, cold boot)
- Index (18 most-run models, one measured table): https://rapidmlx.com/models/
- Qwen3.6 35B MoE (flagship local agent model, 59.6 tok/s): https://rapidmlx.com/models/qwen3-6-35b
- Qwen3 Coder 30B MoE (coding agents, 53 tok/s): https://rapidmlx.com/models/qwen3-coder-30b-4bit
- GPT-OSS 20B (OpenAI open-weight reasoner, 47.9 tok/s): https://rapidmlx.com/models/gpt-oss-20b
- Gemma 4 26B MoE (50.4 tok/s): https://rapidmlx.com/models/gemma-4-26b-4bit
- Gemma 4 12B (vision, 22.6 tok/s): https://rapidmlx.com/models/gemma-4-12b-4bit
- Qwen3.6 27B dense (11.1 tok/s): https://rapidmlx.com/models/qwen3-6-27b
- Ternary Bonsai 27B 2-bit (17.4 tok/s in 8.4 GB): https://rapidmlx.com/models/bonsai-27b-2bit
- Qwen3.5 9B (36.4 tok/s): https://rapidmlx.com/models/qwen3-5-9b-4bit
- Qwen3 8B (37.1 tok/s): https://rapidmlx.com/models/qwen3-8b-4bit
- Qwen3.5 4B (most-pulled model, 60.7 tok/s): https://rapidmlx.com/models/qwen3-5-4b-4bit
- Qwen3 4B Instruct 2507 (61.2 tok/s): https://rapidmlx.com/models/qwen3-4b-instruct-2507-4bit
- Qwen3 4B Thinking 2507 (61.3 tok/s): https://rapidmlx.com/models/qwen3-4b-thinking-2507-4bit
- Gemma 3 4B QAT (vision, 66.7 tok/s): https://rapidmlx.com/models/gemma3-4b-qat-4bit
- Llama 3.2 3B (82.2 tok/s): https://rapidmlx.com/models/llama3-3b-4bit
- LFM2.5 2.6B (8 GB-Mac pick, 93.5 tok/s): https://rapidmlx.com/models/lfm2-5-2-6b-4bit
- LFM2.5 1.2B (209.8 tok/s): https://rapidmlx.com/models/lfm2-5-1b-4bit
- Ternary Bonsai 1.7B 2-bit (169 tok/s): https://rapidmlx.com/models/bonsai-1-7b-2bit
- Qwen3 0.6B (225.9 tok/s): https://rapidmlx.com/models/qwen3-0-6b

## Blog
- Blog index (benchmarks, MLX internals, Apple Silicon guides): https://rapidmlx.com/blog/
- Blog Atom feed: https://rapidmlx.com/blog/feed.xml
- Fully local dictation in any Mac app (Desktop hotkey + Whisper / Qwen3-ASR / SenseVoice / Parakeet model guide): https://rapidmlx.com/blog/local-dictation-mac
- Rapid-MLX 0.11.0 release (prefix-cache 25.6× TTFT, MCP chat agent, 5 new model families, grammar-constrained tool calls): https://rapidmlx.com/blog/rapid-mlx-0-11-0
- Rapid-MLX vs Ollama vs LM Studio on Apple Silicon (honest comparison, measured rapid-mlx tok/s, reproducible bench): https://rapidmlx.com/blog/rapid-mlx-vs-ollama-vs-lm-studio
- Moving from Ollama to rapid-mlx on Apple Silicon (migration guide, command cheat sheet): https://rapidmlx.com/blog/moving-from-ollama-to-rapid-mlx
- Best local AI models for MacBook Pro (2026 picks by use case): https://rapidmlx.com/blog/best-local-ai-models-macbook-pro
- MLX model benchmark on M3 Ultra (measured throughput / TTFT): https://rapidmlx.com/blog/mlx-model-benchmark-m3-ultra
- How much memory do you need to run a local LLM on a Mac (82 models, real measured memory): https://rapidmlx.com/blog/how-much-memory-local-llm-mac
- The best local LLM for your Mac, by RAM tier (one pick per tier 8 GB-96 GB+, peak RSS + boot measured in one run on one M3 Ultra, capability scored on the AA Intelligence Index): https://rapidmlx.com/blog/best-local-llm-for-your-mac-by-ram
- Local coding agents that actually work on Apple Silicon (verified end-to-end: Claude Code / Codex / Aider / Hermes, July 2026): https://rapidmlx.com/blog/local-coding-agents-apple-silicon
- Run Claude Code on a local model: https://rapidmlx.com/blog/run-claude-code-on-a-local-model
- Run Cursor on a local model: https://rapidmlx.com/blog/run-cursor-on-a-local-model
- Run Codex CLI on a local model: https://rapidmlx.com/blog/run-codex-cli-on-a-local-model
- Run Aider on a local model: https://rapidmlx.com/blog/run-aider-on-a-local-model
- Rapid-MLX vs Ollama vs LM Studio benchmark (measured on M2 Pro: decode/prefill/TTFT, 3 models, 4 engines, raw JSON published): https://rapidmlx.com/blog/rapid-mlx-vs-ollama-benchmark
- MLX vs llama.cpp on Apple Silicon (measured: MLX wins MoE decode 1.58×, llama.cpp wins prefill 1.7-2.2×): https://rapidmlx.com/blog/mlx-vs-llama-cpp-benchmark
- Best local AI models for Apple Silicon 2026 (picks by use case + RAM): https://rapidmlx.com/blog/best-local-ai-models-apple-silicon
- Run Qwen locally on a Mac (Qwen 3.5 + 3.6 RAM table, MoE explainer): https://rapidmlx.com/blog/run-qwen-locally-on-a-mac

## Models — directory
- Family directory + notable-small-models callout: https://rapidmlx.com/docs/models/
- Full alias catalog: https://rapidmlx.com/docs/models/alias-catalog.html
- Live model picker (with RAM filter): https://models.rapidmlx.com/

## Models — vendor family pages
Each page nests its version lines (e.g. Qwen3.5 / 3.6 / Coder / VL) under
h2 sections — one consolidated page per vendor.
- Qwen (56 aliases · 3.6 / 3.5 / Coder / VL / legacy / 2.5 / Qwopus): https://rapidmlx.com/docs/models/families/qwen.html
- Gemma (35 aliases · 4 / 4-mobile (3n) / 3 / EmbeddingGemma): https://rapidmlx.com/docs/models/families/gemma.html
- Llama (4 aliases · Llama 3 / 3.1): https://rapidmlx.com/docs/models/families/llama.html
- Muse Glimmer (2 aliases · Meta's 30B dense reasoner, 131K context, ATEM tool calls): https://rapidmlx.com/docs/models/families/muse.html
- Ling (1 alias · inclusionAI's 7.9B MoE reasoner, 1.3B active, KDA+MLA hybrid, 131K context on an 8 GB Mac): https://rapidmlx.com/docs/models/families/ling.html
- DeepSeek (7 aliases · R1 + Coder; V4-Flash on its own hero page): https://rapidmlx.com/docs/models/families/deepseek.html
- GLM (3 aliases · GLM-5.2 (REAP) + GLM-4.7 + GLM-4.5-Air, custom parser): https://rapidmlx.com/docs/models/families/glm.html
- Mistral (6 aliases · Ministral + Mistral + Mistral-Small-4 + Devstral): https://rapidmlx.com/docs/models/families/mistral.html
- Phi (4 aliases · Phi-3.5 / Phi-4): https://rapidmlx.com/docs/models/families/phi.html
- Granite (2 aliases · Granite 4 hybrid SSM): https://rapidmlx.com/docs/models/families/granite.html
- GPT-OSS (10 aliases · 20B + 120B, mxfp4-q4/q8, Harmony): https://rapidmlx.com/docs/models/families/gpt-oss.html
- MiniMax (2 aliases · M2.5 / M2.7, custom envelope): https://rapidmlx.com/docs/models/families/minimax.html
- Hermes (2 aliases · Hermes 3 / Hermes 4): https://rapidmlx.com/docs/models/families/hermes.html
- Liquid (4 aliases · LFM2.5 2.6B / 8B-A1B / 1.2B + LFM2 24B-A2B — hybrid conv+attention, tiny KV cache): https://rapidmlx.com/docs/models/families/liquid.html
- Hunyuan (1 alias · HY3 preview): https://rapidmlx.com/docs/models/families/hunyuan.html
- Audio (44 aliases · TTS: Kokoro / Qwen3-TTS / IndexTTS / Chatterbox / VibeVoice / VoxCPM / F5-TTS / Dia · STT: Whisper / Parakeet / SenseVoice / Qwen3-ASR / Qwen3-ForcedAligner): https://rapidmlx.com/docs/models/families/audio.html
- Video generation (8 aliases · Wan 2.1 / 2.2, CogVideoX-Fun, LTX-2.3 — async /v1/videos): https://rapidmlx.com/docs/models/families/video.html
- Small / curated (Bonsai / SmolLM3 / Nanbeige / Nemotron): https://rapidmlx.com/docs/models/families/small-models.html

## Models — hero deep dives
- Tmax (hybrid Gated-DeltaNet agent model): https://rapidmlx.com/docs/models/tmax.html
- DiffusionGemma (discrete-diffusion text gen): https://rapidmlx.com/docs/models/diffusiongemma.html
- UI-TARS (vision-grounded computer-use agent): https://rapidmlx.com/docs/models/ui-tars.html
- VibeThinker (1.5B / 3B small reasoner): https://rapidmlx.com/docs/models/vibethinker.html
- DeepSeek V4-Flash (CSA + HCA sparse attention): https://rapidmlx.com/docs/models/deepseek-v4-flash.html
- Holo3 (small-MoE agent model, shipped in 0.9.9 as holo3.1-35b-a3b): https://rapidmlx.com/docs/models/holo3.html

## Surfaces
- Desktop app (local chat + system-wide local dictation): https://rapidmlx.com/desktop
- Performance benchmarks: https://rapidmlx.com/#performance
- Community performance tab: https://rapidmlx.com/performance/

## Discovery
- llms-full.txt (concatenated docs corpus): https://rapidmlx.com/llms-full.txt
- Sitemap: https://rapidmlx.com/sitemap.xml
- robots.txt: https://rapidmlx.com/robots.txt