Install & first request

Install Rapid-MLX, start a server, and make your first OpenAI-compatible chat completion. Roughly 90 seconds end-to-end on a clean machine.

Requirements

• Apple Silicon Mac — M1 or newer.
• macOS 14 or later.
• Python 3.10 or newer (the installer auto-fetches one if you do not have it).

Intel Macs are not supported — Rapid-MLX is built on Apple's MLX framework which targets the unified-memory Apple Silicon GPU.

Install

One curl. No sudo, no Homebrew required.

$ curl -fsSL https://rapidmlx.com/install.sh | bash

The installer probes for Python 3.10+ (auto-installing python-build-standalone if needed), creates a venv at ~/.rapid-mlx, and symlinks the rapid-mlx command into ~/.local/bin. The same flow works from pip directly if you would rather manage your own venv:

$ python3 -m venv .venv && source .venv/bin/activate
$ pip install rapid-mlx

Start a server

Pick any alias from the catalog and pass it to rapid-mlx serve. Aliases are short memorable names that resolve to the canonical Hugging Face repo and apply the right tool-call parser, reasoning parser, hybrid-cache flag, and so on:

$ rapid-mlx serve qwen3.5-4b-4bit
# downloads the MLX weights on first run, then:
INFO  uvicorn running on http://127.0.0.1:8000

The default port is 8000. Override with --port 9000, bind to a different host with --host 0.0.0.0, set the served name with --served-model-name my-model, or pick a quant explicitly with the full alias suffix (e.g. qwen3.5-9b-8bit).

Make your first request

The endpoints are drop-in OpenAI-compatible — point any OpenAI client at http://localhost:8000/v1 with any non-empty API key.

cURL

$ curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "qwen3.5-4b-4bit",
      "messages": [{"role": "user", "content": "ping"}]
    }'

Python (openai SDK)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-used",
)
resp = client.chat.completions.create(
    model="qwen3.5-4b-4bit",
    messages=[{"role": "user", "content": "ping"}],
)
print(resp.choices[0].message.content)

What is on by default

Rapid-MLX 0.9.9 turns two multi-user features on automatically for the popular Qwen3.5 / 3.6 aliases — you do not need to pass any extra flag to get them.

• Compressed KV cache — on 9 hero aliases (Qwen3.5-9B / 27B, Qwen3.6-35B-A3B in 4 / 6 / 8-bit + DWQ). Roughly doubles the concurrent-user count at the same RAM budget with byte-identical output. Turn off with --kv-cache-turboquant none.
• Prefix cache — when several requests share a system prompt or a tool schema, the server reuses the already-computed KV. Measured 13× aggregate throughput at 10 concurrent clients.

Speculative decoding (MTP, DFlash) is opt-in — see the API defaults section for how to enable it.

Wire it into a tool

Anything that speaks the OpenAI API works without code changes — just point the base URL at localhost:8000/v1. There is also a one-shot bootstrap for common IDE clients:

$ rapid-mlx launch cursor      # Cursor
$ rapid-mlx launch claude-code # Claude Code
$ rapid-mlx launch aider       # Aider

launch writes the required client config (base URL, model name, API key placeholder) and starts the server.

Next steps