Changelog / release

0.10.18 — Bounded prefill memory on long prompts

Released 2026-07-24 · full changelog · GitHub releases

Upgrade: pip install -U rapid-mlx  ·  brew upgrade rapid-mlx  ·  or grab the desktop app.
  • Long text-only prompts no longer spike memory. The multimodal batch generator was pushing the entire prompt through the language model in a single forward and projecting logits over every position, peaking around 35 GB on a ~20k-token prompt — enough to max out a 48 GB Mac. Text-only prefill is now chunked to bound peak memory (PR #1193, fixes #1187).
← All releases