Changelog

The 0.8 through 0.10 lines, newest first. For one-shot release notes per tag, check the GitHub releases page — this list is a curated summary of the most load-bearing changes.

0.10.5 — 2026-07-08 · Fix batch

• Spec-decode unified interface migration. Legacy _install_mtp installer removed (−475 LOC dead code) — every MTP path now flows through the vendored installer that was already the live runtime. mtp_optimistic is hard-rejected at all three entry points (SchedulerConfig.__post_init__, server.load_model, CLI normalizer) so misuse fails loud instead of silently ignoring the flag. spec_decode='suffix' is preserved as a canonical value; mtp_max_k conflict detection is now idempotent for the disabled default. Matched-vendored bench on Qwen3.6-27B-4bit + MTP sidecar (90 requests × 3 runs) confirmed behavior parity: tok/s median 43.83 → 46.03 (+5.0% noise), MTP rounds 17704 → 17619 (0.5% delta), finish reasons 59/31 → 60/30 (PR #1050).
• README trim. Slimmed from 1142 → ~280 lines. Each trimmed section carries a → link to the specific docs page anchor instead of losing content silently (PR #1054).
• install.sh Tier-1 refresh. RECOMMENDED_MODEL now maps to the 0.10 Tier-1 families (gpt-oss-120b · qwen3.6-35b-a3b · gpt-oss-20b · qwen3.5-4b fallback). Canonical URLs migrated to rapidmlx.com across the script (PR #1053).
• HF → R2 mirror tool. New scripts/mirror_to_r2.py handles streaming multipart mirrors of gated / large model weights (used to seed gpt-oss-120b MXFP4-Q8 and Qwen3.6-{27,35}B-MTP-4bit into R2). Bench / maintenance tooling only, not a runtime dependency (PR #1056).
• OpenHands integration harness. Corrected the gpt-oss XFAIL reason (CodeActAgent parses text-action XML tags; gpt-oss emits harmony analysis + final channels — the format mismatch, not a stop-scoping bug). Docker digest handling in the harness now uses a two-ref pattern that avoids the OpenHands 0.9.0 base_image.split(':') bug on 3-way digest tags (PR #1055, tracked upstream at All-Hands-AI/OpenHands#15167).

0.10.3 — 2026-07-07 · Integrations + parser fixes

• Harmony parser final-channel stop scoping. User-supplied stop= sequences now only match against the harmony final channel, so gpt-oss on CodeActAgent no longer prematurely truncates when a stop token appears inside the analysis channel. Unblocks gpt-oss under OpenHands / Aider (PR #1051, closes #1049).
• Real Aider bash-CLI integration harness. Four Tier-1 family × Aider cells were un-xfailed after wiring a live aider --model openai/<alias> harness that actually issues a code-edit round-trip (PR #1047).
• Real OpenHands Docker E2E harness. CodeActAgent runs against ghcr.io/all-hands-ai/openhands:0.9.0 with a digest-pinned runtime image; three of four Tier-1 cells un-xfailed (gpt-oss cell still XFAIL — see the 0.10.5 note above) (PR #1048).
• R2 mirror bypass fixes. jlens and bench/bench --submit now honor the R2 mirror before falling back to Hugging Face — matching what serve and pull already did (PRs #1045, #1046).
• Spec-decode unified interface (staging). MTP moved onto the vLLM-style speculative_config plumbing that #1050 then completes in 0.10.5 (PR #1044, operator lane).

0.10.2 — 2026-07-07 · Interpretability

• rapid-mlx jlens — read a model's internal draft. A read-only Jacobian-lens command that decodes what a model is disposed to say at every layer, reports where the answer crystallizes (early-exit headroom) and how far ahead of the logit lens it reads. Runs locally on quantized MLX models via a Jacobian–vector product — the full d×d Jacobian is never materialized and it differentiates through 4/8-bit weights. Follows Anthropic's Global Workspace work. Supports dense decoders (Qwen3, Llama, Phi); linear-attention / hybrid and VLM vision towers are flagged as unsupported rather than crashing. --verbose adds per-layer ranked readouts + the answer's rank trajectory; --json for machine output.

0.10.1 — 2026-07-05

• Version hygiene. Reverted an accidental early 0.10.1 bump from #1010 and re-tagged cleanly (PRs #1020, #1021). No functional change.

0.10.0 — 2026-07-04

• Codex tool groups on /v1/responses. The Responses endpoint now accepts Codex namespace tool groups, fixing tool-call handling for Codex-style clients (PR #993).

0.9.14 — 2026-07-04

• OpenAI reasoning_effort translation. The reasoning_effort request field is now translated at the route layer, so OpenAI-shaped clients that set it map cleanly onto the engine's reasoning controls (PR #1009, closes #448).

0.9.13 — 2026-07-04

• Gemma-4 MTP sidecar CLI. New --mtp-sidecar flag plus an eligibility gate for Gemma-4 speculative decoding — the CLI-facing half of the 0.9.13 MTP work (PR #1000).

0.9.12 — 2026-07-02

• Gemma-4 MTP drafter, stabilized. The 0.9.11 assistant-drafter inject was reverted (PR #997) and re-landed in a polished form (PR #998), so Gemma-4 speculative decoding runs without the first-cut rough edges.

0.9.11 — 2026-07-02

• Gemma-4 speculative decoding (MTP). Google's official assistant-drafter inject plus a Gemma-4 entry in the MTP allowlist (PRs #990 + #988) — faster Gemma-4 generation with byte-identical output.
• Scheduler prefix-cache fix. On an exact prompt-cache hit the cache is trimmed by one token to prevent a duplicate KV entry (PR #994).
• CLI audio aliases. rapid-mlx pull / rm now resolve audio aliases (PR #992).

0.9.10 — 2026-07-01

• Whisper silence-hallucination guard. A Silero VAD pre-trim strips the silent lead-in so STT stops inventing text over quiet audio (PR #980).
• Tool-call argument normalization. Assistant tool_call.arguments is coerced to a dict at the chat-template boundary, fixing downstream shape mismatches (PR #981).
• Qwen3-Coder XML streaming. The streaming parser now anchors on <function=…> instead of the <tool_call> wrapper (PR #979).
• Anthropic adapter. Non-leading system messages are merged before the chat template is applied (PR #976).
• Server flags. python -m vllm_mlx.server now threads TurboQuant flags into SchedulerConfig (PR #983).
• README audited and rewritten for clarity and value-prop (PR #601).

0.9.9 — 2026-06-30 · Multi-user & long-context release

• Compressed KV cache by default on 9 hero MoE aliases — Qwen3.5-9B / 27B and the Qwen3.6-35B-A3B family (4 / 6 / 8-bit + DWQ). Roughly 2× more concurrent users at the same RAM budget, with byte-identical output vs the uncompressed baseline.
• Radix-tree prefix cache is default-on. Concurrent requests that share a system prompt (IDE assistants, Cursor / Claude Code / Aider fleets, multi-user chat) reuse the KV automatically. Verified 13× aggregate throughput at 10 concurrent clients.
• 128k context on Qwen3.5-9B out of the box. No flag flips — the alias resolves to the model's native 256k position-embedding cap.
• Speculative decoding (MTP + DFlash) is lossless when enabled. Byte-identical output verified vs the non-spec baseline. MTP works with the qwen3.5-9b-mtp-4bit sidecar; DFlash works with qwen3.5-27b-8bit + the z-lab drafter.
• DFlash is now flag-gated as experimental. Install with pip install 'rapid-mlx[dflash]', enable with --enable-dflash. Measured 1.4× pooled speedup on Qwen3.5-27B-8bit — the previously advertised 3.5× number was from a broken bench and has been corrected here.
• Long-running session stability verified. 8+ hour agent loops show flat memory (±1% RSS drift over 8 h, 2,878 turns).
• GLM-5.2 removed from the roadmap. Both offline GGUF → MLX conversion attempts (Q2_K and Q3_K_M sources) landed a working load path but the model produced incoherent output on this REAP-50-pruned checkpoint.
• Config. New CLI flag --kv-cache-turboquant none opts out of the compressed KV cache on a per-server basis. Default is per-alias auto.

0.8.19 — 2026-06-25

• Tmax-27B hybrid-cache fix. is_hybrid: true on all Tmax-27B variants — was incorrectly false in 0.8.18, which corrupted the RNN state on prefix-cache hits (PR #902).
• Responses item shape defaults to type: "message" when role+content are present (PR #904).
• thinking accounting. enable_thinking now threads into prompt accounting on the chat + responses paths so reasoning models report token usage correctly (PR #906, follow-up to #891).
• Auto-config routing. V4 / V5 model-type detection aligned with the alias classifier (PR #903).
• CLI: rapid-mlx serve now exits non-zero on a port-already-in-use error (PR #905).
• CI: pr_validate full_unit flakes on macOS uv-managed Python fixed (PR #907).

0.8.18 — 2026-06-24

• Tmax-9B + Tmax-27B aliases added. First-mover MLX support — see the Tmax docs page (PR #899).
• tool_call promotion in reasoning ported from upstream into the refactored think_parser (closes #344, PRs #896 + #898).
• Auto-disable thinking on casual chat completions (matches the M-2 / #891 pattern, PR #895).
• Auto-disable thinking when tools are provided — strict-JSON pattern carried over to reasoning models (PR #891).
• Audio: _serve_audio_mode now honours --embedding-model and --served-model-name (closes R11-K #258, PR #894).
• Tool parser: prevent deepseek_v3 parser from binding to non-V3 Qwen2 distills + warn on misbind (PR #887).
• Tempfile leak fix: managed_tempfile helper applied across leak sites (closes #719, PR #888).
• Strict JSON schema retry on context-length re-check before the repair retry (closes #267b, PR #886).

0.8.16 — 2026-06-22

• /v1/models surfaces effective parsers. The tool_call_parser and reasoning_parser the alias resolved to are now visible per model (V-1, S-2, PR #878).
• reasoning_content sanitization on the tool_choice=required path so the reasoning trace is not leaked into the tool-call output (V-2, PR #881).

0.8.15 — 2026-06-21

• Strict JSON schema mode enforced via post-generate validation + repair retry (closes #423, PR #873).
• Reasoning tail rescue: tail re-routed to content when finish_reason=length hits mid-reasoning (closes 8-round D-carry #259, PR #875).
• CLI: rapid-mlx launch <client> — one-shot IDE bootstrap for Cursor, Claude Code, Aider, etc. (closes #566, PR #870).
• deepseek_v3 variant added for the R1-0528 family (PR #874).
• Chat output always emits the content key on the assistant message — fixes clients that throw on its absence (PR #872).

0.8.14 — 2026-06-20

• Responses streaming fix: exclude the reasoning-cutoff sentinel from downstream_output_seen — regression from PR #860 (PR #869).
• README documents the 0.8.13 audio support (TTS + STT, 26 aliases) (PR #868).
• Length-stop rescue stub for reasoning models restored after refactor (closes #858, PR #860).

0.8.13 — 2026-06-19 · Audio launch

• Audio support shipped. 26 aliases — 13 TTS (Kokoro, Chatterbox, VibeVoice, VoxCPM, Dia) and 13 STT (Whisper, Parakeet) — via /v1/audio/speech and /v1/audio/transcriptions. Install with pip install 'rapid-mlx[audio]'.
• Audio CLI: --audio-mode + comprehensive alias registry (R10-A, PR #854).
• Responses: emit reasoning item on max_output_tokens cutoff (R11-M-F1, PR #866).
• Embeddings: [embeddings] extra + 503 on missing model + /v1/models visibility (H-08 + H-09 + H-13, PR #861).
• VLM penalties: frequency / presence / repetition penalties now pass through to the VLM sampler (closes #512, PR #864).
• Tool-choice=required: finalize invariant pinned in streaming (R11-V1 + R11-V2, PR #859).
• UI-TARS: accumulator-anchor + honour enable_thinking=false (R10-F, PR #850).

Older

Full pre-0.8.13 release notes are in the GitHub releases history. Notable highlights from the 0.7 → 0.8 era:

• 0.8.x: Reasoning-parser refactor, multi-turn prompt cache with KV trimming for sub-100 ms TTFT, RNN state snapshots for hybrid architectures.
• 0.7.41: Cross-thread Stream(gpu, 4) crash fix in logprobs path (hotfix).
• 0.7.40: 5 systematic integration fixes — tool_choice / stop / template-escape / logprobs / think-detection (PR #716).
• 0.7.x: VibeThinker integration, alias bundle for ≤5B models, diffusion-lane fix for mlx-vlm eos_token_ids alias.