OpenAI SDK — the three-line switch

rapid-mlx serves a drop-in OpenAI-compatible API on localhost. If your script already uses the OpenAI Python or JS SDK, moving it onto a local model is a constructor change — base_url and api_key — and nothing else. This is the single most common way rapid-mlx is called in the wild.

Serve a model

$ rapid-mlx serve qwen3.5-9b-4bit
⚡ serving on http://localhost:8000/v1

Python

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",  # ← the switch
    api_key="not-needed",                  # any non-empty string
)

r = client.chat.completions.create(
    model="default",  # resolves to whatever the server booted
    messages=[{"role": "user", "content": "Explain MLX in one line."}],
)
print(r.choices[0].message.content)

Zero-code: env vars

The SDK reads OPENAI_BASE_URL and OPENAI_API_KEY from the environment, so an existing tool can be pointed local without touching its source at all:

$ export OPENAI_BASE_URL=http://localhost:8000/v1
$ export OPENAI_API_KEY=not-needed
$ python your_existing_script.py

JavaScript / TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8000/v1",
  apiKey: "not-needed",
});

const r = await client.chat.completions.create({
  model: "default",
  messages: [{ role: "user", content: "Explain MLX in one line." }],
});
console.log(r.choices[0].message.content);

What carries over

• Streaming — stream=True (and stream_options={"include_usage": True} for engine-reported token counts) works as on the hosted API.
• Tool calling — pass tools=[…] as usual; rapid-mlx runs grammar-constrained tool calling by default on tool-trained families (Qwen 3.5/3.6, Gemma 4, gpt-oss), so arguments parse as valid JSON.
• Model discovery — client.models.list() returns the served aliases; model="default" always resolves without knowing the alias.

Gotchas

• The API key can't be empty. The SDK raises before sending if api_key is blank — pass any placeholder string. rapid-mlx doesn't check it unless you started the server with auth.
• One server, one loaded model. Requests for an alias the server didn't boot 404 — use model="default" or serve the alias you name.

See also