Anthropic SDK with local models on Mac
Use the Anthropic Python SDK with an open-weight model on your Apple Silicon Mac. Rapid-MLX serves the Messages API locally; it does not run Claude models.
Local setup in 3 steps
# 1. Install
brew install rapid-mlx
# 2. Terminal 1: start the model; leave running
rapid-mlx serve qwen3.5-9b-4bit
# 3. Terminal 2: configure and send a request
python3 -m venv .venv
source .venv/bin/activate
python -m pip install anthropic
python - <<'PYTHON'
from anthropic import Anthropic
client = Anthropic(base_url="http://localhost:8000", api_key="not-needed")
r = client.messages.create(model="default", max_tokens=256, messages=[{"role": "user", "content": "Say hello"}])
print("".join(b.text for b in r.content if b.type == "text"))
PYTHON
Step 3 creates a Python environment in your project. Use Python 3.12 or newer and wait for the server to be ready. A printed text reply confirms the request. Increase max_tokens if a reasoning model produces no visible text.
Use the server root, without /v1
The SDK appends /v1/messages, so pass http://localhost:8000. OpenAI clients use http://localhost:8000/v1 instead. default selects the served model; use the exact model ID from /v1/models when targeting it explicitly.
not-needed is a placeholder for an unprotected loopback server. If you enable server authentication, pass the real server key as api_key. See server ports and authentication.
Stream text
After creating client above, run this in the same Python script:
with client.messages.stream(
model="default", max_tokens=768,
messages=[{"role": "user", "content": "Say hello"}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
print()
Compatibility and tool use
Reasoning models may return thinking blocks before text. Filter content blocks by type == "text" rather than assuming the first block has a text field. Tool use also depends on the served model; your application must execute requested tools and send their results back.
This is a Messages API adapter, not a promise of complete Anthropic platform parity. The engine’s Anthropic SDK integration test covers plain messages, system prompts, multi-turn history, streaming and tool use. Consult the API surface for implemented routes and the integration matrix for model-specific limits.
Code with the same local model
Use Claude Code with a local model for a coding agent, or switch client protocols with the OpenAI SDK guide. Compare local inference servers on Mac using published measurements.
Sources
Anthropic Python SDK documentation · Rapid-MLX Messages route · SDK smoke-test examples and token caps