Frequently asked questions

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. It runs open models such as Qwen, Gemma, DeepSeek and GPT-OSS on your Mac, serves them at localhost:8000/v1, and connects them to coding agents such as OpenCode, Claude Code and Codex.

On the models we measured, yes. Rapid-MLX is built on MLX, so the comparison is against mlx-lm, Apple's MLX library and server (mlx_lm.server 0.32.0), with the same weights. On a Mac mini M4 Pro (48 GB) with Qwen3.5-9B 4-bit and greedy decoding, Rapid-MLX 0.15.6 decoded whole-file code edits up to 4.27× faster (46.4 → 198.4 tok/s) and 1.50× faster at the median of the 18 everyday tasks that stream text; Qwen3.6-35B-A3B 4-bit (MoE) was up to 3.16× faster, 1.57× at the median. The gain comes from speculative decoding (MTP and prompt lookup), on by default for these models. Long-prompt agent turns and the first token of a fresh prompt are close to a tie. Per-task results and method →

It depends on the model and the Mac. On the community benchmarks, Qwen3.6 35B-A3B reaches 151 tok/s on an M5 Max, Qwen3.8 27B 54 tok/s on an M5 Ultra, and the small Qwen3.5 4B 227 tok/s on an M5 Ultra. When several apps or agents call the same Mac, continuous batching keeps each call from waiting behind the others. See the numbers for your chip →

OpenCode, Claude Code, Codex, Hermes, Goose, Qwen Code, Crush, Aider and Pi start with one command: rapid-mlx launch <agent>. It picks a model that fits your Mac, starts it, and opens the agent already connected, without changing your existing agent settings. Cursor and any other OpenAI- or Anthropic-compatible client connect through the API. Agent setup guides →

Yes. It serves an OpenAI-compatible API at localhost:8000/v1, so any OpenAI client works by changing the base URL, and an Anthropic-compatible Messages API for tools such as Claude Code. Cursor calls models from its own servers, so it needs an HTTPS tunnel to your Mac. API reference →

280+ models across text, vision, speech and video, including Qwen, Gemma, DeepSeek, GPT-OSS, GLM, Nemotron and Liquid AI's LFM. Audio covers 24 text-to-speech and 20 speech-to-text models through /v1/audio/speech and /v1/audio/transcriptions. If a model you want is not in the catalog, you can bring your own from Hugging Face. Browse all models →

Yes. It runs Wan 2.2, CogVideoX-Fun and LTX-2.5 / 2.3 for local video generation behind an asynchronous /v1/videos API (install rapid-mlx[video], which needs ffmpeg). It also ships zero-shot voice cloning with IndexTTS, Qwen3-TTS, F5-TTS and Chatterbox. Everything runs on the Mac; nothing is sent to a cloud service.

Yes. Start it with rapid-mlx serve --host 0.0.0.0 and set --trusted-hosts to the Mac's IP or host name; other machines then call the OpenAI-compatible API on port 8000, with no window open. Continuous batching lets one Mac serve several apps, agents or scripts at once. Add --api-key once it is reachable beyond your own machine.

Run curl -fsSL https://rapidmlx.com/install.sh | bash, or brew install rapid-mlx. No sudo is needed. Prefer a window to a terminal? The Rapid-MLX Desktop app runs the same engine behind a one-click Mac interface. Install guide →

No. Rapid-MLX is built on Apple's MLX framework and needs an Apple Silicon Mac (M1 or newer) running macOS 14 or later.

Yes. The engine is free and open source under the Apache 2.0 license, and the source is on GitHub. GitHub →