Optional install extras

A bare pip install rapid-mlx==0.12.18 is enough to serve any text model. The optional extras below unlock specific capabilities — vision, MTP speculative decoding, embeddings, audio, dev tooling — without paying their disk cost by default.

How to install an extra

Bracket syntax; quote for zsh:

$ pip install "rapid-mlx[vision]==0.12.18"
$ pip install "rapid-mlx[vision,mtp]==0.12.18"    # combine
$ pip install "rapid-mlx[all]==0.12.18"           # vision + chat + embeddings + audio

User-facing extras

[vision] — multimodal models (Gemma 4, Qwen VL, etc.)

Adds roughly 322 MB of dependencies. Required for any model with vision input; text-only models work without this.

• mlx-vlm>=0.6.3 — DiffusionGemma + Gemma 4 unified path; also unlocks DFlash spec-decode hooks.
• opencv-python>=4.8.0
• torch>=2.3.0
• torchvision>=0.18.0
• pillow>=10.0.0

$ pip install "rapid-mlx[vision]==0.12.18"

[mtp] — MTP speculative decoding (Gemma 4 external assistant path)

Adds the sidecar-path dependency for --speculative-config '{"method":"mtp","num_speculative_tokens":3}'. The assistant-drafter loader transitively imports mlx_lm.utils → transformers, which reaches for PIL when the sidecar checkpoint has an image-processor preprocessor_config.json. Without PIL on PYTHONPATH, boot fails with a misleading ModuleNotFoundError: No module named 'PIL'.

• pillow>=10.0.0

$ pip install "rapid-mlx[mtp]==0.12.18"

For Gemma 4 targets specifically (as opposed to just MTP wiring), also install [vision] — the Gemma 4 text-target constructor surfaces its own actionable ImportError with a size-tiered install hint. Combine as rapid-mlx[mtp,vision]==0.12.18.

Homebrew note. The Homebrew formula does not auto-install [mtp]. On Homebrew installs, add pillow>=10.0.0 manually into /opt/homebrew/Cellar/rapid-mlx/<v>/libexec or run rapid-mlx from a pip venv where you can pass the extra directly.

[dflash] — DFlash speculative decoding for Qwen 3.5/3.6 dense 8bit

Adds ~1-4 GB at runtime (drafter weights) and depends on mlx-vlm's spec-decode runtime. Text-only — does not pull torch / cv2 like [vision]. Only enable if you serve a DFlash-eligible alias (e.g. qwen3.5-27b-8bit) with --speculative-config '{"method":"dflash"}'.

• mlx-vlm>=0.6.3

$ pip install "rapid-mlx[dflash]==0.12.18"

[embeddings] — /v1/embeddings endpoint

Unlocks --embedding-model at serve time. Required to preload an embedding model like mlx-community/embeddinggemma-300m-6bit.

• mlx-embeddings>=0.1.0

$ pip install "rapid-mlx[embeddings]==0.12.18"

[chat] — Gradio chat UI

Adds the rapid-mlx-chat Gradio front-end (separate from the REPL you get from rapid-mlx chat).

• gradio>=4.0.0
• pytz>=2024.1

$ pip install "rapid-mlx[chat]==0.12.18"

[audio] — TTS and STT (kokoro, whisper, parakeet, vibevoice, voxcpm, chatterbox, dia)

Every audio alias (see the audio registry) needs this extra. Ships upper-cap pins the audio surface has actually been dogfooded against — notably mlx-audio is pinned <0.4.4 because 0.4.4 regressed Kokoro's istftnet so every Kokoro request 500s.

• mlx-audio>=0.2.9,<0.4.4
• f5-tts-mlx==0.2.6 — pure-MLX EN+ZH TTS with zero-shot voice cloning.
• sounddevice>=0.4.0, soundfile>=0.12.0
• scipy>=1.10.0, numba>=0.57.0
• tiktoken>=0.5.0
• misaki[zh,ja]>=0.5.0, spacy>=3.7.0, num2words>=0.5.0, loguru>=0.7.0
• espeakng-loader>=0.2.0, phonemizer-fork>=3.3.0 (fork, not vanilla — see below)
• cn2an>=0.5.0 — Chinese number conversion

Kokoro imports misaki.espeak which calls EspeakWrapper.set_data_path(...) — the legacy 3.2-era API. Vanilla phonemizer>=3.3.0 removed it in favour of a property-only data_path accessor, so phonemizer-fork is required (not just preferred).

$ pip install "rapid-mlx[audio]==0.12.18"

[audio-desktop] — desktop sidecar audio subset

The trimmed audio stack shipped inside the macOS desktop sidecar. It exposes transcription plus Qwen3 preset-voice speech, but not F5 voice cloning or Kokoro's language-specific G2P stack — so it skips llvmlite, spaCy, espeak, and the language dictionaries the signed app can't drive from its UI. On the pip engine, install the full [audio] extra instead.

• mlx-audio>=0.2.9,<0.4.4
• soundfile>=0.12.0 — libsndfile encoding of the in-memory result.

$ pip install "rapid-mlx[audio-desktop]==0.12.17"

[image] — local image generation (FLUX.1-schnell, Qwen-Image)

MLX-native text-to-image and image-edit (FLUX.1-schnell, Qwen-Image, Qwen-Image-Edit) via mflux. Apple-Silicon only and requires Python ≥ 3.11 (core still supports 3.10, so the image-lane preflight gives 3.10 users an explicit upgrade diagnostic). Only Apache-2.0 model families are wired.

• mflux>=0.18.0 (Darwin, Python ≥ 3.11)

$ pip install "rapid-mlx[image]==0.12.18"

[video] — local video generation (LTX-2.3 T2V/I2V)

MLX-native text-to-video / image-to-video with synchronized audio (LTX-2.3) via mlx-video-with-audio. Apple-Silicon only and requires Python ≥ 3.11; the video startup preflight gives 3.10 users an explicit upgrade diagnostic.

• mlx-video-with-audio==0.1.36
• mlx-arsenal>=0.10.1
• imageio[ffmpeg]>=2.34.0
• pillow>=10.0.0, sentencepiece>=0.2.0

$ pip install "rapid-mlx[video]==0.12.18"

[guided] — llguidance-based structured JSON output

Guided decoding via llguidance's native MLX Metal mask kernel, which interprets JSON Schema natively. Replaced outlines[mlxlm] in 0.10. As of 0.10.15 llguidance was promoted to core dependencies so default-on grammar-constrained tool-calling works out of the box; this extra is retained purely for backward compat so pip install 'rapid-mlx[guided]' keeps resolving.

• llguidance>=1.7.6

$ pip install "rapid-mlx[guided]==0.12.18"

[all] — vision + chat + embeddings + audio union

One-shot install of the main user-facing runtime extras. Expanded inline (not rapid-mlx[vision,chat,embeddings,audio]) to avoid the self-dependency pitfall that breaks editable installs.

$ pip install "rapid-mlx[all]==0.12.18"

Note: [all] does not include [mtp], [dflash], [guided], [image], [video], [audio-desktop], [mirror], [test], or [dev] — add those explicitly if you need them.

Contributor / test extras

[test] — minimum pytest runtime

What pytest tests/ needs to import + collect every test cleanly. Kept narrow on purpose so pr_validate can rebuild a venv without the linter toolchain.

• pytest>=7.0.0
• pytest-asyncio>=0.21.0 — required because pytest.ini sets asyncio_mode = auto.
• prometheus_client>=0.16.0 — parses /metrics output through the official parser.
• tomli>=2.0.1 (Python < 3.11 only) — for the L-07 vision-extra lock-in tests.
• aiohttp>=3.9.0, pillow>=10.0.0
• Darwin-only: mlx-vlm>=0.6.3, mlx-audio>=0.2.9,<0.4.4
• hypothesis>=6.100.0 — property-based invariant suite.
• pytest-cov>=4.0.0, diff-cover>=8.0.0,<9.0.0 — advisory diff-coverage step in pr_validate.

$ pip install "rapid-mlx[test]==0.12.18"

[dev] — test runtime + linters + type-checker

Mirrors [test] and adds Ruff, mypy. A unit test in tests/test_pr_validate_test_env.py asserts every [test] dep is also in [dev] so a future drift can't silently break the "install dev, run pytest" workflow.

• Everything in [test]
• ruff>=0.1.0, mypy>=1.0.0

$ pip install "rapid-mlx[dev]==0.12.18"

[mirror] — R2 mirror seeding (maintainers only)

HF → Cloudflare R2 mirror tooling (scripts/mirror_to_r2.py). Off the hot path — only maintainers who seed models.rapidmlx.com need it. Users never install [mirror]; the client-side download path uses stdlib urllib and does not depend on boto3.

• boto3>=1.34.0

$ pip install "rapid-mlx[mirror]==0.12.18"

Verify what's installed

rapid-mlx doctor surfaces each optional extra's status:

◆ Optional Packages
  ⚠ mlx-vlm (vision extras) not installed (`pip install 'rapid-mlx[vision]'`)
  ⚠ mlx-audio (audio extras) not installed (`pip install 'rapid-mlx[audio]'`)
  ⚠ mlx-embeddings (embeddings extras) not installed (`pip install 'rapid-mlx[embeddings]'`)
  ⚠ mlx-vlm 0.5.0+ (dflash extras) not installed or too old

Next steps