Changelog / release

0.9.9 — Multi-user & long-context release

Released 2026-06-30 · full changelog · GitHub releases

Upgrade: pip install -U rapid-mlx · brew upgrade rapid-mlx · or grab the desktop app.

Compressed KV cache by default on 9 hero MoE aliases — Qwen3.5-9B / 27B and the Qwen3.6-35B-A3B family (4 / 6 / 8-bit + DWQ). Roughly 2× more concurrent users at the same RAM budget, with byte-identical output vs the uncompressed baseline.
Radix-tree prefix cache is default-on. Concurrent requests that share a system prompt (IDE assistants, Cursor / Claude Code / Aider fleets, multi-user chat) reuse the KV automatically. Verified 13× aggregate throughput at 10 concurrent clients.
128k context on Qwen3.5-9B out of the box. No flag flips — the alias resolves to the model's native 256k position-embedding cap.
Speculative decoding (MTP + DFlash) is lossless when enabled. Byte-identical output verified vs the non-spec baseline. MTP works with the qwen3.5-9b-mtp-4bit sidecar; DFlash works with qwen3.5-27b-8bit + the z-lab drafter.
DFlash is now flag-gated as experimental. Install with pip install 'rapid-mlx[dflash]', enable with --enable-dflash. Measured 1.4× pooled speedup on Qwen3.5-27B-8bit — the previously advertised 3.5× number was from a broken bench and has been corrected here.
Long-running session stability verified. 8+ hour agent loops show flat memory (±1% RSS drift over 8 h, 2,878 turns).
GLM-5.2 removed from the roadmap. Both offline GGUF → MLX conversion attempts (Q2_K and Q3_K_M sources) landed a working load path but the model produced incoherent output on this REAP-50-pruned checkpoint.
Config. New CLI flag --kv-cache-turboquant none opts out of the compressed KV cache on a per-server basis. Default is per-alias auto.