Reference · Local server

Run a Local OpenAI-Compatible MLX Server on Mac

Run rapid-mlx serve qwen3.5-4b-4bit on your Mac. Set your OpenAI client base URL to http://localhost:8000/v1 and send the served alias as the model.

$ brew install rapid-mlx                # or: curl -fsSL https://rapidmlx.com/install.sh | bash
$ rapid-mlx serve qwen3.5-4b-4bit
$ curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" \
    -d '{"model": "qwen3.5-4b-4bit", "messages": [{"role": "user", "content": "Hello"}]}'
SettingValue
OpenAI base URLhttp://localhost:8000/v1
Anthropic base URLhttp://localhost:8000 (no /v1; the SDK adds /v1/messages)
Port8000. If it's taken, the next free port up to 8009, and the startup banner says which. --port N pins it.
Host127.0.0.1, this Mac only. --host 0.0.0.0 listens on your network; allow the names clients use with --trusted-hosts.
API keyNot checked by default; SDKs need any non-empty string. Set --api-key or RAPID_MLX_API_KEY to require one.
Model nameThe alias you served (qwen3.5-4b-4bit) or its Hugging Face repo id. GET /v1/models lists both.

Start the server

Install with Homebrew (brew install rapid-mlx) or the one-line installer, then serve a model by alias or by Hugging Face repo id. The first run downloads the weights. When the server is up it prints the URLs to use:

$ rapid-mlx serve qwen3.5-4b-4bit
…
  Ready: http://127.0.0.1:8000

  OpenAI:    http://127.0.0.1:8000/v1
  Anthropic: http://127.0.0.1:8000
  Model:     qwen3.5-4b-4bit

localhost and 127.0.0.1 are the same address here. Not sure which model fits your Mac? rapid-mlx recipe prints the pick for your memory, and Pick a model for my Mac explains each tier. In another terminal, rapid-mlx connect prints the same URLs for a server that is already running.

Port and base URL

Browser and Host access

A default server accepts native clients using localhost, 127.0.0.1, [::1], or *.localhost. It rejects foreign browser Origins, including simple POST requests, and unexpected Host headers. A same-origin browser client can connect.

For a frontend on another origin, pass its exact origin with --cors-origins http://localhost:3000 (or set RAPID_MLX_CORS_ALLOW_ORIGINS). For LAN or reverse-proxy clients, use --host 0.0.0.0 when needed and pass the Host names or IPs clients send with --trusted-hosts (or RAPID_MLX_TRUSTED_HOSTS). An explicit trusted-host list replaces the loopback defaults, so include them if local clients also connect. Use an API key on shared servers. Wildcard origins or hosts explicitly disable their respective checks.

In 1.0.5, an unexpected Host returns HTTP 400 with a sanitized hostname and guidance to configure --trusted-hosts <name>. Add only the intended hostname. Browser extension clients can opt in with --cors-origins 'chrome-extension://*', 'moz-extension://*' or 'safari-web-extension://*'. Each pattern allows all valid extension origins for that scheme; cross-origin access remains closed by default.

Endpoints

Paths are relative to http://localhost:8000.

EndpointAPINotes
POST /v1/chat/completionsOpenAI Chat CompletionsStreaming, tool calling, vision inputs, reasoning_content.
POST /v1/responsesOpenAI ResponsesWhat Codex CLI speaks; streaming events, reasoning and tool-call items.
POST /v1/messagesAnthropic MessagesWhat Claude Code speaks; tools, streaming, thinking blocks. Plus /v1/messages/count_tokens.
POST /v1/completionsOpenAI CompletionsRaw text completion, no chat template.
GET /v1/modelsOpenAI ModelsThe served model, with its context window and capabilities.
POST /v1/embeddingsOpenAI EmbeddingsSet --embedding-model with a chat model or alone for dedicated embedding serving (EmbeddingGemma 2 uses the bundled vision runtime); without one it returns an error that says so. See EmbeddingGemma 2.
POST /v1/audio/speech, /v1/audio/transcriptionsOpenAI AudioText-to-speech and speech-to-text with an audio model; needs the [audio] extra.
POST /v1/images/generationsOpenAI ImagesText-to-image with an image model; uses the bundled image runtime.
GET /healthProbeHealth check; also /healthz, /readyz, /livez.
GET /docs, /openapi.jsonSchemaSwagger UI and the OpenAPI schema of the running server.

Video, audio translation and music, image edits, metrics and model management are in the full API reference, with every place the server differs from OpenAI.

Call it from curl, Python and Node

curl

$ curl http://localhost:8000/v1/models
$ curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model": "qwen3.5-4b-4bit", "messages": [{"role": "user", "content": "Hello"}]}'

Python (openai)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

r = client.chat.completions.create(
    model="qwen3.5-4b-4bit",
    messages=[{"role": "user", "content": "Hello"}],
)
print(r.choices[0].message.content)

# the Responses API works on the same client
print(client.responses.create(model="qwen3.5-4b-4bit", input="Hello").output_text)

Node / TypeScript (openai)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "http://localhost:8000/v1", apiKey: "not-needed" });
const r = await client.chat.completions.create({
  model: "qwen3.5-4b-4bit",
  messages: [{ role: "user", content: "Hello" }],
});
console.log(r.choices[0].message.content);

Without changing code

The OpenAI SDKs read the base URL and key from the environment:

$ export OPENAI_BASE_URL=http://localhost:8000/v1
$ export OPENAI_API_KEY=not-needed
$ python your_script.py
Tested 2026-10-07 with rapid-mlx 0.15.7 and qwen3.5-4b-4bit on a Mac mini (M2 Pro, 32 GB): the startup banner and port fallback above, the curl calls, the Python snippet (openai 3.26.0) and the Node snippet (openai 7.30.0) all returned replies.

API key and network access

A plain rapid-mlx serve doesn't check keys, and it only listens on 127.0.0.1. To require a key, set it when you start the server:

$ export RAPID_MLX_API_KEY="$(openssl rand -hex 24)"
$ rapid-mlx serve qwen3.5-4b-4bit            # or: --api-key <key>

Clients then send Authorization: Bearer <key> (what the OpenAI SDKs do with api_key); on /v1/messages, Anthropic's x-api-key header works too. A missing or wrong key gets 401; the health probes stay open. Set a key before you use --host 0.0.0.0 or put the server behind a tunnel.

New in 1.0.3: share Desktop's API on your network

In Desktop, open Settings → Tools to enable LAN access. The server restarts with a persistent bearer key saved in Keychain and shows addresses and copyable connection details for other devices on your network. Network clients can use inference; model management, administrative routes and Local Search, including grounded chat, stay on this Mac.

Security. Network access is off by default and uses plain HTTP.

Supervised serve in 1.1.0

Supervised rapid-mlx serve runs a parent process that restarts a crashed runner on the same port. Ctrl-C stops the runner and frees the port. Use rapid-mlx status to inspect it, rapid-mlx log for recent API calls, and rapid-mlx jobs to list agent runs. rapid-mlx jobs cancel <id> cancels one. See CLI monitoring and the loopback control API.

The restarting parent and status, log and jobs controls apply only to catalog text and vision models on the supervised plain path; models that default to MTP, DFlash or Tensorfold (including qwen3.5-9b-4bit, qwen3.8-27b-4bit and qwen3.6-35b-4bit), audio/video and non-catalog repositories use the single-process path, with --no-spec-decode enabling supervision only for models whose exclusion is a default MTP sidecar, while native MTP and Tensorfold remain single-process.

For batch inference, X-Rapid-Priority: background lets waiting interactive requests go first, with 30-second aging and no preemption. Chat completions can continue a partial assistant message with continue_final_message.

Build on Rapid-MLX

Build a menu bar app, editor plugin, script or model picker around one local server. Run rapid-mlx serve as a child process without sudo, or connect to an existing listener. Probe the known port before starting another process; a server returning HTTP 503 may still be starting. Only stop a child your controller owns. For a Mac that serves without login, use the system daemon guide.

rapid-mlx serve qwen3.5-4b-4bit --host 127.0.0.1 --port 8000
rapid-mlx ps --json
rapid-mlx top --port 8000 --once

GET /health/ready returns HTTP 503 while blocked and HTTP 200 when ready; lazy standby can be ready without resident weights. GET /healthz is liveness, not model readiness. GET /v1/status describes the primary engine, while GET /v1/models/residency lists model records, states and the memory ceiling. A registered record does not always have weights loaded.

POST /v1/models/load accepts a model alias or repo ID. To switch the assistant group while rejecting active work, send {"model":"qwen3.5-9b-4bit","replace_group":"assistant","replace_mode":"reject"}. A successful load returns HTTP 200; conflicts return 409 and insufficient capacity returns 507. Re-read residency after failures because eviction may have happened before loading.

DELETE /v1/models/{id} evicts an idle, unpinned model without deleting cached weights. Use the returned residency ID; pinned or busy models return 409, unknown IDs return 404, and success returns 204 with an empty body. Model controls and status require your bearer key when configured. Health probes and /metrics are public; keep the listener on loopback or behind access controls. The documented rapid_mlx_model_loaded gauge tracks the primary, not the number of resident models.

Accept additional response fields. The engine’s Build on Rapid-MLX guide distinguishes stable integration contracts from experimental metrics and shows controller lifecycle examples. See CLI monitoring and JSON discovery and prefix-cache retention.

Frequently asked questions

What port does the MLX OpenAI-compatible server use?

Rapid-MLX listens on port 8000 by default, so the OpenAI base URL is http://localhost:8000/v1. If 8000 is taken it uses the next free port up to 8009 and prints it; --port sets it explicitly.

What is the base URL for a local OpenAI-compatible server on Mac?

http://localhost:8000/v1 for OpenAI clients (Python, Node, LangChain, Codex CLI) and http://localhost:8000 for Anthropic clients such as Claude Code. Use the alias you served as the model name.

Does it need an OpenAI API key?

No. The server doesn't check keys unless you start it with --api-key or RAPID_MLX_API_KEY. The OpenAI SDKs refuse an empty key, so pass any placeholder string such as not-needed.

Which OpenAI endpoints are supported?

Chat Completions, Completions, Models, Responses, Embeddings (with an embedding model attached), Audio speech and transcription, and Images, plus Anthropic Messages. The API reference lists every route and where it differs from OpenAI.

Can I use it from another computer on my network?

Yes. Start it with --host 0.0.0.0, an API key, and --trusted-hosts naming the address clients use, then use http://<your-mac's-ip>:8000/v1 from the other machine. By default it only accepts connections from the Mac itself.

Next