Changelog / release
0.9.9 — Multi-user & long-context release
Released 2026-06-30 · full changelog · GitHub releases
- Compressed KV cache by default on 9 hero MoE aliases — Qwen3.5-9B / 27B and the Qwen3.6-35B-A3B family (4 / 6 / 8-bit + DWQ). Roughly 2× more concurrent users at the same RAM budget, with byte-identical output vs the uncompressed baseline.
- Radix-tree prefix cache is default-on. Concurrent requests that share a system prompt (IDE assistants, Cursor / Claude Code / Aider fleets, multi-user chat) reuse the KV automatically. Verified 13× aggregate throughput at 10 concurrent clients.
- 128k context on Qwen3.5-9B out of the box. No flag flips — the alias resolves to the model's native 256k position-embedding cap.
- Speculative decoding (MTP + DFlash) is lossless when enabled. Byte-identical output verified vs the non-spec baseline. MTP works with the
qwen3.5-9b-mtp-4bitsidecar; DFlash works withqwen3.5-27b-8bit+ the z-lab drafter. - DFlash is now flag-gated as experimental. Install with
pip install 'rapid-mlx[dflash]', enable with--enable-dflash. Measured 1.4× pooled speedup on Qwen3.5-27B-8bit — the previously advertised 3.5× number was from a broken bench and has been corrected here. - Long-running session stability verified. 8+ hour agent loops show flat memory (±1% RSS drift over 8 h, 2,878 turns).
- GLM-5.2 removed from the roadmap. Both offline GGUF → MLX conversion attempts (Q2_K and Q3_K_M sources) landed a working load path but the model produced incoherent output on this REAP-50-pruned checkpoint.
- Config. New CLI flag
--kv-cache-turboquant noneopts out of the compressed KV cache on a per-server basis. Default is per-alias auto.