PydanticAI

PydanticAI is the Pydantic-native agent framework. It uses OpenAIChatModel with an OpenAIProvider — pass rapid-mlx's URL to the provider, hand the model to an Agent, and run_sync.

Install

$ pip install pydantic-ai

Config

Exact snippet from tests/integrations/test_pydantic_ai_full.py:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    model_name="default",
    provider=OpenAIProvider(
        base_url="http://localhost:8000/v1",
        api_key="not-needed",
    ),
)

agent = Agent(model)
print(agent.run_sync("What is 2+2?").output)

Tool calling

Decorate a plain Python callable with @agent.tool_plain; PydanticAI derives the JSON schema from the signature and docstring:

agent = Agent(model)

@agent.tool_plain
def get_weather(city: str) -> str:
    """Get the weather for a city."""
    return f"sunny, 22C in {city}"

r = agent.run_sync("What's the weather in Paris?")
# PydanticAI drives the tool loop end-to-end — r.output is the final
# natural-language answer; r.all_messages() contains the get_weather call.

Structured output

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

agent = Agent(model, output_type=Person)
r = agent.run_sync("Extract: 'Alice is 30 years old'")
# r.output == Person(name="Alice", age=30)

Streaming

import asyncio

async def stream_test():
    agent = Agent(model)
    chunks = []
    async with agent.run_stream("Count from 1 to 5, separated by commas.") as result:
        async for delta in result.stream_text(delta=True):
            chunks.append(delta)
    return "".join(chunks)

print(asyncio.run(stream_test()))

Gotchas

• PydanticAI defaults max_tokens to ~1024. On verbose 4B-class models the second turn of a multi-turn or the answer after a two-tool chain can spill past that. Pass model_settings={"max_tokens": 2048} to the Agent constructor for those cases (the deep test does this for tests 5 and 6).
• Structured output with thinking models: token budget consumed by reasoning before JSON output. Same fix — bump max_tokens.
• Multi-tool chain (3+4)*5 flakes on small models. Some 4B-class models fail sequential tool calls; 9B+ works reliably (see matrix row).

Empirical

The PydanticAI row of the integration matrix is ✅ on Qwen 3.6, Gemma 4, and gpt-oss; DeepSeek R1-Distill XFAIL(arch). Deep flow assertions in test_pydantic_ai_full.py cover plain, streaming, structured output, single tool call, multi-turn, and multi-tool sequential.

See also