0.11.8 — Embeddings stop truncating silently at 512 tokens

Released 2026-08-02 · full changelog · GitHub releases

/v1/embeddings hardcoded the tokenizer at max_length=512. Anything longer came back HTTP 200, correctly shaped, and quietly missing its tail — which degrades a vector index with no signal anywhere. Reported by a user indexing code chunks with Qwen3-Embedding-4B, whose architecture supports 32K (issue #1381).

• The limit is now derived from the model. auto reads config.max_position_embeddings, then tokenizer.model_max_length, guarding Hugging Face's large "unset" sentinel. It falls back to 512 only when the model declares nothing at all — and logs that it did.
• --embedding-max-length auto|<int> — an operator ceiling for a lower memory or service limit. An explicit value is validated against and clamped to the model maximum.
• --embedding-overflow-policy truncate|error — truncate (the default) still discards the tail, but logs a warning and increments rapid_mlx_embedding_truncations_total on /metrics, so it is no longer silent. error returns a structured 400 with code: "input_too_long" carrying the observed and allowed token counts.
• Both apply to string and pre-tokenized inputs alike, and the usage block no longer over-reports the pre-truncation token count (PR #1386).