Changelog / release
0.10.10 — Ternary Bonsai 27B + Qwen3-Coder-Next 80B
Released 2026-07-15 · full changelog · GitHub releases
- Ternary Bonsai 27B — new flagship small-footprint model.
bonsai-27b-2bitis a 2-bit ternary Qwen 3.5-class 27B that packs into 7.9 GB and runs on a 16 GB Mac (~46 tok/s on M3 Ultra). The quant is stock MLX 2-bit affine — no custom kernel — and it loads through the mlx-lm text path (served text-only via theis_text_onlyprofile flag to sidestep an mlx-vlm SSM bug, even though the config declares a vision tower). Strong on the mainstream: code, math, reasoning, EN/ZH general writing, and agent/framework tool-calling (verified end-to-end with the OpenAI SDK, LangChain, pydantic-ai, smolagents CodeAgent, and Aider). Known limitation: strict Chinese classical regulated verse (五言绝句 with fixed rhyme) can trip a repetition loop — a niche edge case. Weights are on the R2 mirror. - Qwen3-Coder-Next 80B — new coder family SKUs. Added
qwen3-coder-next-80b-4bit(44.8 GB) andqwen3-coder-next-80b-8bit(84.7 GB), the 80B-A3B Coder-Next MoE at 4-bit and 8-bit MLX with theqwen3_coder_xmltool-call parser. HF-pull (not yet on the R2 mirror). See the alias reference. - KV-cache export / import. Serialize a warmed prompt-prefix KV cache to disk and re-hydrate it on a later serve, so a long shared system prompt / document context is paid for once rather than re-prefilled every session.
- Gemma 4 hardening. Correctness and stability fixes to the vendored Gemma 4 text path.
- Release-infra hardening. Toughened the auto-release / version-bump pipeline so version markers and the PyPI + Homebrew publish steps stay in lockstep.