Framework guide · Anthropic Messages API

Anthropic SDK with local models on Mac

Use the Anthropic Python SDK with an open-weight model on your Apple Silicon Mac. Rapid-MLX serves the Messages API locally; it does not run Claude models.

Local setup in 3 steps

# 1. Install
brew install rapid-mlx
# 2. Terminal 1: start the model; leave running
rapid-mlx serve qwen3.5-9b-4bit
# 3. Terminal 2: configure and send a request
python3 -m venv .venv
source .venv/bin/activate
python -m pip install anthropic
python - <<'PYTHON'
from anthropic import Anthropic
client = Anthropic(base_url="http://localhost:8000", api_key="not-needed")
r = client.messages.create(model="default", max_tokens=256, messages=[{"role": "user", "content": "Say hello"}])
print("".join(b.text for b in r.content if b.type == "text"))
PYTHON

Step 3 creates a Python environment in your project. Use Python 3.12 or newer and wait for the server to be ready. A printed text reply confirms the request. Increase max_tokens if a reasoning model produces no visible text.

Use the server root, without /v1

The SDK appends /v1/messages, so pass http://localhost:8000. OpenAI clients use http://localhost:8000/v1 instead. default selects the served model; use the exact model ID from /v1/models when targeting it explicitly.

not-needed is a placeholder for an unprotected loopback server. If you enable server authentication, pass the real server key as api_key. See server ports and authentication.

Stream text

After creating client above, run this in the same Python script:

with client.messages.stream(
    model="default", max_tokens=768,
    messages=[{"role": "user", "content": "Say hello"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()

Compatibility and tool use

Reasoning models may return thinking blocks before text. Filter content blocks by type == "text" rather than assuming the first block has a text field. Tool use also depends on the served model; your application must execute requested tools and send their results back.

This is a Messages API adapter, not a promise of complete Anthropic platform parity. The engine’s Anthropic SDK integration test covers plain messages, system prompts, multi-turn history, streaming and tool use. Consult the API surface for implemented routes and the integration matrix for model-specific limits.

Code with the same local model

Use Claude Code with a local model for a coding agent, or switch client protocols with the OpenAI SDK guide. Compare local inference servers on Mac using published measurements.

Sources

Anthropic Python SDK documentation · Rapid-MLX Messages route · SDK smoke-test examples and token caps