Troubleshooting

Common symptoms on rapid-mlx 0.10.3, mapped to root causes and fixes. Every entry on this page was verified against the shipped CLI — if we couldn't trace a symptom to a real error message or source path, we left it off.

Start here — run the doctor

rapid-mlx doctor is the one-shot self-diagnostic — Apple Silicon detection, macOS + Darwin versions, free disk, HF cache size, Python version, mlx / mlx-lm / transformers / fastapi / uvicorn / rapid-mlx versions, optional-extras status, and network probes. Run it before filing an issue.

$ rapid-mlx doctor
$ rapid-mlx doctor -v          # verbose per-probe detail

Performance

Getting much lower tok/s than the community benchmarks?

By far the most common cause: the model is reasoning before answering. Qwen 3.5 and 3.6 default to thinking-on, which roughly doubles the wall-clock work per response.

• In rapid-mlx chat: the REPL defaults to --no-think — so this shows up mainly on serve traffic.
• On rapid-mlx serve: pass --no-thinking, or send chat_template_kwargs.enable_thinking=false per-request.

Tracked as issue #567 — recognised as the most common bug report.

Slow first response, fast follow-ups

Two causes stack:

• Reasoning prelude — the same as above.
• Cold prefill on a long prompt. Add --prefill-step-size 8192 (or 16384) to the serve command. Subsequent turns hit the prefix cache and get 10-30× faster.

OOM and boot failures

Out of memory, or very slow (< 5 tok/s)

Model too big for your RAM. Pick a smaller quant or a smaller model — see the Hardware tiers map for canonical picks per RAM budget. The four safe choices are:

• 96+ GB → gpt-oss-120b-mxfp4-q8
• 48-95 GB → qwen3.6-35b-8bit
• 24-47 GB → gpt-oss-20b-mxfp4-q8
• 8-23 GB → qwen3.5-4b-4bit

Pre-flight disk-space check aborts serve

rapid-mlx serve aborts before download if the model won't fit on the filesystem backing the HuggingFace cache. If your cache actually lives on an external drive (via HF_HOME) that has room, add --force-disk-check to skip the guard.

Free disk dropped after upgrade — check stale cache

Older rapid-mlx installs used ~/.cache/vllm-mlx. After the rebrand this directory is no longer read, but existing bytes hang around. If rapid-mlx doctor reports a high HF cache size and low free disk, verify:

$ ls -lah ~/.cache/vllm-mlx     # stale — safe to remove
$ rm -rf ~/.cache/vllm-mlx

The live cache lives at ~/.cache/huggingface/hub/ (the standard HF path); rapid-mlx rm <alias> cleans individual repos from there.

Output behaviour

Empty responses

Usually a parser mismatch. If you passed --reasoning-parser to a non-thinking model, the parser will strip everything as reasoning_content. Fixes:

• Remove --reasoning-parser for the model in question.
• Or pass --no-reasoning-parser to force-disable auto-detection from the alias profile.
• Or verify rapid-mlx info <alias> — the reasoning_parser field there is what auto-detection uses.

Tool calls arriving as plain text

Set the right --tool-call-parser for your model — the canonical values are listed under rapid-mlx serve --help (parsers section). Or rely on auto-detection: aliases in aliases.json ship with the right parser pre-wired, so passing just --enable-auto-tool-choice is often enough. Even without either, rapid-mlx auto-recovers most cases from the served checkpoint's tokenizer + config.

"parameters not found in model" warnings at startup

Normal for vision-language models (VLMs) — vision weights are auto-skipped when the boot path is text-only. If you actually want vision, install the extra (pip install "rapid-mlx[vision]==0.10.3") and remove --no-mllm / --text-only if you set it.

Reasoning models leak raw <think> tokens

Two knobs — pick whichever fits:

• --no-thinking on serve — suppresses the chain-of-thought prompt template AND the parser, so thinking tokens appear as regular content.
• --no-reasoning-parser — only skips the auto-config step; the model still emits <think> segments but they arrive unwrapped instead of routed to reasoning_content.

Shell integration

Tab completion doesn't fire

Homebrew installs activate argcomplete for rapid-mlx automatically. For pip / install.sh paths, activate once:

# zsh / bash — add to ~/.zshrc or ~/.bashrc
$ eval "$(register-python-argcomplete rapid-mlx)"

# Or system-wide:
$ activate-global-python-argcomplete

argcomplete>=3.6 is a hard dependency (not optional) so the CLI is Tab-completable out of the box once the shell hook is wired.

Platform

"Rapid-MLX requires Apple Silicon"

Intel Macs are not supported — MLX is Apple Silicon (M1 / M2 / M3 / M4) only. Same story for Linux and Windows.

"Rapid-MLX requires macOS 13 (Ventura) or later"

The install.sh platform check enforces macOS 13+. Upgrade to Ventura or later; there is no rapid-mlx build path for earlier macOS.

Service management

"Is there a server already running?"

rapid-mlx ps lists every running rapid-mlx serve process — PID, port, loaded model, uptime. Kill by PID if you need to reclaim a port.

Port already in use

Pick another port with --port <n>, or find the culprit with lsof -i :<port> and stop it. On macOS, note that a wildcard-bound listener (--host 0.0.0.0) can coexist with a more-specific loopback listener on the same port; the serve pre-flight bind check probes 127.0.0.1 explicitly when --host is a wildcard to keep that bypass closed.

Not on this page

This page ships only entries we could verify against 0.10.3 code or a reproducible CLI error message. Other README-level tips that required more digging than the wall budget allowed — cookie rotation for 429s (irrelevant here), specific webhook integrations, per-model quirks — will be added in a follow-up pass with source-of-truth citations. If you hit a symptom missing here, file an issue at github.com/raullenchai/Rapid-MLX/issues.

Next steps