Reference · HTTP API

API surface

Rapid-MLX exposes an OpenAI-shaped HTTP API at http://localhost:8000/v1 by default. If a tool speaks OpenAI, it speaks Rapid-MLX. Point the base URL at your local server and pass any non-empty API key.

Live OpenAPI schema. Every running server exposes its own machine-readable schema at GET /openapi.json and an interactive Swagger UI at GET /docs. The list below is the human-readable summary; the JSON schema is the source of truth.

Endpoints

method	path	description
GET	/v1/models	List available models — alias, parser bindings, capability flags.
POST	/v1/chat/completions	Chat completions — tools, vision, streaming, reasoning_content all supported.
POST	/v1/responses	OpenAI Responses API — streaming with item-by-item events, reasoning items, tool calls.
POST	/v1/completions	Legacy text completion (no chat template).
POST	/v1/embeddings	Embeddings — requires the `[embeddings]` install extra and an embedding alias (e.g. `embeddinggemma-300m-6bit`).
POST	/v1/audio/speech	Text-to-speech (Kokoro, Chatterbox, VibeVoice, VoxCPM, Dia) — requires the `[audio]` extra.
POST	/v1/audio/transcriptions	Speech-to-text (Whisper, Parakeet) — requires the `[audio]` extra.
GET	/v1/health	Health probe.
GET	/metrics	Prometheus-format counters and histograms.
GET	/openapi.json	OpenAPI 3 schema for everything above.
GET	/docs	Swagger UI (interactive).

What's the same as OpenAI

Request and response shapes for /chat/completions, /completions, /embeddings, /responses.
SSE streaming via stream: true with data: [DONE] terminator.
Function / tool calling via tools and tool_choice.
Vision inputs as image_url content parts on multimodal models (Gemma 4, Qwen3-VL).
Authorization header is parsed but not enforced — pass any string.

What is on by default in 0.9

Two features flipped from opt-in to on-by-default in 0.9 because they help every multi-user workload and cost nothing on the single-user path.

Compressed KV cache (K8V4) on 9 hero MoE aliases. Qwen3.5-9B / 27B and the Qwen3.6-35B-A3B family (4 / 6 / 8-bit + DWQ) automatically use the K8V4 compressed KV cache. This roughly doubles the concurrent-user count at the same RAM budget, and the output is byte-identical to the uncompressed baseline. If you'd rather run without it — for a bench or an A/B — start the server with rapid-mlx serve <alias> --kv-cache-turboquant none.
Radix-tree prefix cache. When several requests share a common prefix (system prompt, tool schema, RAG boilerplate) the server reuses the already-computed KV. Turns on automatically — no flag to flip, no per-request opt-in. Measured 13× aggregate throughput at 10 concurrent clients on the K8V4 hero aliases.
Speculative decoding is opt-in. MTP and DFlash are both verified lossless (byte-identical output) but you have to enable them explicitly: MTP via the qwen3.5-9b-mtp-4bit sidecar alias, DFlash via pip install 'rapid-mlx[dflash]' + --enable-dflash.

Where it diverges

reasoning_content is surfaced as a sibling to content for reasoning models — same field name as DeepSeek-R1's API, not OpenAI's reasoning blocks.
/v1/models includes Rapid-MLX-specific fields: tool_call_parser, reasoning_parser, is_hybrid, is_moe, supports_spec_decode.
Tool-call output respects the effective parser for the served model rather than always emitting OpenAI-flavoured JSON — most aliases use one of Hermes / Qwen3-XML / GLM4.7 / Harmony / UI-TARS / Llama / DeepSeek / Minimax. The wire shape the client sees is normalised back to OpenAI's tool_calls array.
Streaming SSE for /v1/responses emits response.output_item.added, response.output_item.done, reasoning items, and tool-call items in the order OpenAI specifies — but the per-token deltas are batched per model: an AR model emits one token per delta, a diffusion model emits a block.

Examples

List models

$ curl http://localhost:8000/v1/models | jq .data[0]
{
  "id": "qwen3.5-4b-4bit",
  "object": "model",
  "tool_call_parser": "hermes",
  "reasoning_parser": "qwen3",
  "is_hybrid": false,
  "is_moe": false,
  "supports_spec_decode": false
}

Streaming chat completion

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
stream = client.chat.completions.create(
    model="qwen3.5-4b-4bit",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Embeddings

# first install: pip install 'rapid-mlx[embeddings]'
$ rapid-mlx serve embeddinggemma-300m-6bit --port 8001 &
$ curl http://localhost:8001/v1/embeddings \
    -H "Content-Type: application/json" \
    -d '{"model": "embeddinggemma-300m-6bit", "input": "ping"}'

Audio (TTS)

# first install: pip install 'rapid-mlx[audio]'
$ rapid-mlx serve kokoro-tts --audio-mode --port 8002 &
$ curl http://localhost:8002/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{"model": "kokoro-tts", "voice": "af_heart", "input": "hello"}' \
    -o speech.wav

Where to next

For the full request/response field reference, hit the live Swagger UI at http://localhost:8000/docs on your own server — it stays in sync with the running build, including any flags wired up by recent releases. The engine README covers CLI flags, environment variables, and the rapid-mlx launch <client> bootstrap targets.