Tmax-9B & Tmax-27B

Hamish Ivison's hybrid Gated-DeltaNet agent model — and the first family Rapid-MLX took from "broken on import" to "first-class end-to-end tool calling" on Apple Silicon. We publish every quant and own the alias config.

Why Tmax matters

Tmax is a hybrid Gated-DeltaNet architecture — a 3:1 ratio of RNN-style Gated-DeltaNet blocks to full self-attention. The RNN blocks give it constant per-token memory cost (no quadratic KV explosion at long context) while the attention blocks keep recall strong on agentic workloads. It comes out of Hamish Ivison's agent-tuning lineage at Ai2 and inherits that family's bias toward tool use and code.

For a Mac, this is interesting because the RNN portion means the memory footprint at 64k or 128k context is dramatically smaller than a pure-attention model of the same parameter count. You can hold Tmax-27B 4-bit plus a comfortable working context inside 24 GB of RAM.

Quants we publish

All quants live under mlx-community. The Rapid-MLX aliases below resolve to these repos automatically:

Tmax-9B family

Tmax-27B family

We deliberately do not publish a tmax-27b-bf16 — the ~54 GB footprint is prohibitive for the small set of machines that could load it, and the 8-bit quant is within KLD noise of bf16.

Engineering notes

The interesting part of "first-mover support" is the bugs we had to flush out of the pipeline before rapid-mlx serve tmax-9b would just work. These are documented here so anyone porting other hybrid-Gated-DeltaNet models has a starting point.

Phantom vision tower in the config

The Tmax checkpoints declare Qwen3_5ForConditionalGeneration in config.json — a class that includes a vision tower stub — but ship zero vision-tower tensors in the safetensors shards. Standard mlx-lm sanitization fails on the missing keys at load time. We strip the vision-tower entries during MLX conversion before publishing, which is why our quants live at mlx-community/Tmax-*-MLX-* rather than the broken community uploads that landed earlier.

is_hybrid alias correction

Tmax-27B requires the hybrid (RNN + attention) KV path; without it, the cache is treated as fully rewindable and prefix-cache hits corrupt the RNN state. The 27B aliases now ship with is_hybrid: true (PR #902 landed in 0.8.19). The 9B variants run fine on the standard transformer path and stay at is_hybrid: false.

Tool-call parser

Both 9B and 27B use the qwen3_xml tool-call parser — they emit Qwen3-style <tool_call>…</tool_call> XML blocks rather than JSON-fenced or Hermes-style envelopes. The alias config wires this in by default, so OpenAI-shape tools=[…] requests round-trip cleanly without any client-side massaging.

Run it

$ rapid-mlx serve tmax-9b
# or, for the larger model:
$ rapid-mlx serve tmax-27b

Chat completion (cURL)

$ curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
      "model": "tmax-9b",
      "messages": [
        {"role": "user", "content": "Plan a weekend in Kyoto."}
      ]
    }'

Tool call (Python)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-used")

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
    },
}]

resp = client.chat.completions.create(
    model="tmax-27b",
    messages=[{"role": "user", "content": "Weather in Osaka?"}],
    tools=tools,
)
print(resp.choices[0].message.tool_calls)

Benchmarks

Numbers below were measured on a Mac Studio M3 Ultra with Rapid-MLX 0.8.19, single-stream, median of 3 rounds. They match what we shipped on the model cards at mlx-community/Tmax-27B-MLX-*.

Tmax-27B

Prefill is "tokens per second of input ingestion at a 16k-token context"; tool-call latency is end-to-end time from request to parsed tool_calls array on a small toolchain. Full single-stream / multi-stream tables live on the HF model card and the engine README.

References

• Upstream Tmax repo: github.com/hamishivi/tmax
• MLX quant family: huggingface.co/mlx-community (search "Tmax")
• Alias config: vllm_mlx/aliases.json
• is_hybrid: true correction (PR #902): github.com/raullenchai/Rapid-MLX/pull/902