0.10.12 — Response cache + trim-free prefix reuse

Released 2026-07-17 · full changelog · GitHub releases

Upgrade: pip install -U rapid-mlx · brew upgrade rapid-mlx · or grab the desktop app.

Community contributors: @kumosan2 — hybrid prefix reuse (#1111); @romanbsd — nested Gemma 4 tool arguments (#1102); @pierre427 — pflash compression visibility (#1106).

Prompt-deterministic response cache — opt-in exact-match short-circuit. New --response-cache-entries N retains up to N fully-computed greedy (temperature 0 / top_k 1) chat completions; a completely repeated request returns the stored completion verbatim with zero GPU decode — the second and later identical calls are effectively free. The cache key spans every output-affecting field (model id, the canonical messages + chat kwargs, resolved sampling kwargs, response_format / logprobs), so any change is a clean miss and recompute; sampled requests are never short-circuited. It is distinct from and complementary to the prefix / KV cache — the prefix cache reuses prefill state and still decodes, this returns the entire stored completion and decodes nothing. New Prometheus counters rapid_mlx_response_cache_hits_total / _misses_total. Ideal for agent retry loops, eval harnesses, and dashboard / health probes that re-issue an identical request. Default 0 = fully disabled, request path byte-for-byte unchanged (PR #1123).
Trim-free prefix reuse for hybrid & sliding-window models — opt-in. New --hybrid-cache-entries N retains up to N non-trimmable prefix-cache entries so a stable prefix + a new suffix each turn reuses prior prefill instead of recomputing it. Covers both hybrid recurrent-state (GatedDeltaNet / Mamba) and sliding-window (Gemma 4, GPT-OSS) models — best for stable-system-prompt / long-context agent workloads. An exact re-request of a rotated sliding-window prompt safely falls back to a full prefill (byte-equal to cold). Default 0 = disabled (PR #1111, #1124).
Gemma 4 nested tool arguments. Correctly parse Gemma 4 tool calls whose arguments contain nested objects (PR #1102).
pflash compression visibility. Surface the previously-silent endpoints-only compression collapse so a degraded pflash configuration is no longer hidden (PR #1106).
Dependency hygiene. Exclude the broken mlx-vlm 0.6.4 from all extras (PR #1119).