API surface

Point any OpenAI client at http://localhost:8000/v1 and use the alias you served as the model name. Any non-empty API key works unless you start the server with --api-key. Starting the server, the port and the base URL are covered in OpenAI-compatible server.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-used")
stream = client.chat.completions.create(
    model="qwen3.5-4b-4bit",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

The three things most clients need:

• Chat: POST /v1/chat/completions — tools, vision, streaming and reasoning_content.
• Agents: POST /v1/responses (Codex CLI) and POST /v1/messages (Anthropic Messages, used by Claude Code).
• Discovery: GET /v1/models lists what the server serves, with its parsers and context window.

Endpoints

MCP tool routes (/v1/mcp/*) appear when the server starts with --mcp-config; model management, cache and agent-run routes are listed in /openapi.json.

What's the same as OpenAI

• Request and response shapes for /chat/completions, /completions, /embeddings, /responses.
• SSE streaming via stream: true with a data: [DONE] terminator.
• Function / tool calling via tools and tool_choice, returned as an OpenAI tool_calls array.
• Vision inputs as image_url content parts on multimodal models (Gemma 4, Qwen3-VL and others).
• Authorization: without --api-key (or RAPID_MLX_API_KEY) the header is accepted but not checked; with it, a missing or wrong key returns 401.

Where it differs

• Reasoning is off unless you ask for it on chat completions. For thinking models, a /v1/chat/completions request that sets no thinking preference — plain chat, or a request with tools — gets a direct answer. Ask for reasoning with chat_template_kwargs: {"enable_thinking": true}, reasoning_effort or reasoning_max_tokens.
• reasoning_content is a sibling of content (the DeepSeek-R1 field name), not OpenAI's reasoning blocks.
• reasoning_max_tokens caps the thinking part of a response; the model then answers. It does not bound the answer — pair it with max_tokens for total length.
• Each alias uses its own tool-call format internally (Hermes, Qwen3 XML, GLM, Harmony, Gemma 4, UI-TARS, Llama, DeepSeek, MiniMax…); the client always receives OpenAI's tool_calls array.
• /v1/models adds Rapid-MLX fields — tool_call_parser, reasoning_parser, is_hybrid, is_moe, modality, capabilities, context_window, recommended_sampling — that OpenAI-only clients ignore.
• A finished response can carry a Rapid-MLX metrics object: speculative_decoding when the request ran MTP, and prompt_compression (original_tokens, kept_tokens) when an opted-in PFlash shortened the prompt. Chat Completions, single-prompt Completions and Responses also report experimental request timing: time_to_first_token_ms (scheduler arrival to first output token, so queueing and prefill are included) and, from two tokens on, mean_itl_ms (mean interval between output tokens; 1000 / mean_itl_ms is that request's decode tokens/second). Streaming sends them once on the final event, whether or not stream_options.include_usage is set. Failed, cancelled and response-cache replays omit them, and /v1/messages does not carry them. The metrics object is omitted when none of these apply. Non-streaming responses on every text API, including /v1/messages, also announce compression with the X-Rapid-MLX-Prompt-Compressed: <kept>/<original> header.
• When the server can't take a request it returns 503 with Retry-After. The message says why: Server is busy (max concurrent requests reached) above --max-concurrent-requests, or Server is at its Metal memory limit with how much memory the request needs (see Troubleshooting).
• Streaming SSE for /v1/responses emits response.output_item.added, response.output_item.done, reasoning items and tool-call items in OpenAI's order; a diffusion model emits a block of tokens per delta instead of one.

Examples

List models

$ curl -s http://localhost:8000/v1/models | jq '.data[] | select(.id=="qwen3.5-4b-4bit")'
{
  "id": "qwen3.5-4b-4bit",
  "object": "model",
  "owned_by": "rapid-mlx",
  "is_hybrid": false,
  "is_moe": false,
  "tool_call_parser": "hermes",
  "reasoning_parser": "qwen3",
  "modality": "text",
  "context_window": 262144,
  "capabilities": ["text", "tools"],
  …
}

The list also carries the underlying Hugging Face repo id as its own entry, so either name works as model.

Chat completion with cURL

$ curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model": "qwen3.5-4b-4bit", "messages": [{"role": "user", "content": "What is 17*23?"}],
         "chat_template_kwargs": {"enable_thinking": true}}'

With thinking on, the reply carries both reasoning_content and the final content.

Embeddings

# first install: pip install 'rapid-mlx[vision]'
$ rapid-mlx serve qwen3.5-4b-4bit \
    --embedding-model embeddinggemma-2-4bit --port 8000 &
$ curl http://localhost:8000/v1/embeddings \
    -H "Content-Type: application/json" \
    -d '{"model": "embeddinggemma-2-4bit", "input": "ping"}'

An embedding model is attached to a chat server — on its own, rapid-mlx serve embeddinggemma-2-4bit exits with a hint to use --embedding-model. Setup, task prefixes, dimensions and a worked search example are on the EmbeddingGemma 2 page.

Long inputs. The input limit comes from the model (config.max_position_embeddings, else tokenizer.model_max_length), so a 32K-context embedder uses its full window. Inputs above the limit are never cut silently: by default the tail is truncated with a logged warning and a rapid_mlx_embedding_truncations_total increment on /metrics; with --embedding-overflow-policy error the request is rejected with a 400:

$ rapid-mlx serve qwen3.5-4b-4bit --port 8000 \
    --embedding-model embeddinggemma-2-4bit \
    --embedding-max-length 2048 --embedding-overflow-policy error

# a 3,000-token input then returns:
{"error": {"code": "input_too_long", "message": "…observed vs allowed tokens…"}}

Audio (TTS)

# first install: pip install 'rapid-mlx[audio]'
$ rapid-mlx serve kokoro --port 8002 &
$ curl http://localhost:8002/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{"model": "kokoro", "voice": "af_heart", "input": "hello"}' \
    -o speech.wav

Where to next

For every request and response field, open the Swagger UI at http://localhost:8000/docs on your own server — it always matches the running build. The CLI reference covers server flags; the performance flags reference covers what is on by default and how to change it.