Video generation

8 video aliases · Wan 2.1 / 2.2 (4) · CogVideoX-Fun (3) · LTX-2.3 (1). Text-to-video and image-to-video through the asynchronous OpenAI-compatible Videos API.

rapid-mlx serves POST /v1/videos — the same job-based shape as OpenAI's Videos API. You submit a prompt, get a job id back immediately, poll it, then download the finished MP4. Generation is serialized: one clip renders at a time, because a diffusion pipeline plus its VAE will not share unified memory politely with a second one.

family

Video generation

aliases

8

lines

3

install

pip install 'rapid-mlx[video]'

also needs

ffmpeg · Python 3.11+

OpenAI base URL

http://localhost:8000/v1

Requirements

Two constraints that do not apply to the rest of rapid-mlx:

• Python 3.11 or newer. The upstream mlx-video-with-audio runtime does not support 3.10. Core text and audio features still run on 3.10 — this is a video-lane-only floor.
• ffmpeg on PATH, for muxing frames into an MP4: brew install ffmpeg.

Usage

Submit, poll, download — three calls. The job id comes back on the first one.

pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve wan2.2-ti2v-5b-q8

# Submit a one-second text-to-video job
curl http://127.0.0.1:8000/v1/videos \
  -F model=wan2.2-ti2v-5b-q8 \
  -F 'prompt=a fox running through fresh snow, cinematic tracking shot' \
  -F seconds=1 \
  -F size=832x512 \
  -F seed=42

# Poll it, then download the MP4
curl http://127.0.0.1:8000/v1/videos/video_ID
curl http://127.0.0.1:8000/v1/videos/video_ID/content --output result.mp4

For image-to-video, add -F input_reference=@start.png — supported when the served checkpoint is TI2V or I2V.

Lines in this family

• Wan 2.1 / 2.2 · CogVideoX-Fun · LTX-2.3

Wan · 4 aliases

Alibaba's Wan, through the mlx-video-with-audio runtime. Four converted Wan 2.2 checkpoints are registered. wan2.2-ti2v-5b-q8 is the recommended starting point — it is the smallest, and TI2V means it does both text-to-video and image-to-video from one checkpoint.

The backend reads each checkpoint's native frame rate rather than accepting an arbitrary one (16 fps for Wan 2.1, 24 fps for Wan 2.2), enforces Wan's 4n+1 temporal shape, and honours the checkpoint's pixel-area ceiling. A locally converted Wan 2.1 or 2.2 checkpoint can override the selected alias with RAPID_MLX_WAN_MODEL_DIR.

Process-level tuning is available through environment variables — steps, scheduler and VAE tiling:

RAPID_MLX_WAN_STEPS=8 \
RAPID_MLX_WAN_SCHEDULER=unipc \
RAPID_MLX_WAN_TILING=auto \
rapid-mlx serve wan2.2-ti2v-5b-q8

LoRAs load through RAPID_MLX_WAN_LORA as path[:strength] entries; dual-model checkpoints additionally accept RAPID_MLX_WAN_LORA_HIGH and RAPID_MLX_WAN_LORA_LOW.

CogVideoX-Fun · 3 aliases

Experimental single-worker backend. The MVP covers one-second, 672×384 text-to-video. The pinned MLX runtime ships inside rapid-mlx — no source checkout or PYTHONPATH surgery. Weights load lazily on the first job, not at boot.

LTX-2.3 · 1 alias

MLX-native LTX-2.3 with an audio-video checkpoint. Same /v1/videos job flow as the other two backends.

Notes & caveats

• Jobs are serialized — a second request queues rather than running concurrently. This is deliberate: two diffusion pipelines resident at once will exhaust unified memory on every Mac we target.
• Defaults are 50 diffusion steps. Lowering steps is the single biggest speed lever, at a quality cost.
• Wan rejects frame counts that are not 4n+1 and sizes above the checkpoint's pixel-area ceiling — both come back as 400 with the constraint named, rather than failing deep in the pipeline.
• Image-to-video takes a real uploaded file. Remote image URLs are refused rather than fetched — a server that dereferences caller-supplied URLs is an SSRF hole.
• Video generation is the newest lane in rapid-mlx and the least settled. Treat throughput numbers as a snapshot, not a contract.

Frequently asked questions

Can I generate video locally on a Mac?

Yes. rapid-mlx runs Wan 2.1/2.2, CogVideoX-Fun and LTX-2.3 entirely on Apple Silicon through an asynchronous OpenAI-compatible Videos API at POST /v1/videos — no cloud service and no per-clip cost. Install with pip install 'rapid-mlx[video]', which also needs ffmpeg and Python 3.11 or newer.

Which video generation models run on Apple Silicon?

Eight MLX-converted checkpoints ship in rapid-mlx 0.11.3: four Wan 2.2 checkpoints (TI2V 5B in q8 and bf16, I2V A14B q8, T2V A14B bf16), three CogVideoX-Fun 5B quants (q4, q8, bf16), and LTX-2.3 in q4. wan2.2-ti2v-5b-q8 is the recommended starting point because it is the smallest and handles both text-to-video and image-to-video.

How long does local video generation take on a Mac?

Minutes per second of footage, not real time. On an M3 Ultra, CogVideoX-Fun q4 produced a one-second 672x384 clip in about 338 seconds at the default 50 diffusion steps. Lowering the step count is the single biggest speed lever, at a cost in quality. Static and slow-panning scenes hold up best; fast subject motion is still experimental.

How much RAM do I need to generate video on a Mac?

24 GB of unified memory for the smallest checkpoints (CogVideoX-Fun q4 peaks around 14.5 GB, LTX-2.3 q4 similar), 32 GB for Wan 2.2 TI2V 5B q8, and 64-96 GB for the 14B Wan MoE checkpoints. Jobs are serialized on purpose — two diffusion pipelines resident at once would exhaust unified memory.

Does image-to-video work locally?

Yes, on checkpoints that support it. Wan 2.2 TI2V and I2V accept a conditioning first frame — pass it as an uploaded file with -F input_reference=@start.png. Remote image URLs are deliberately refused rather than fetched, because a server that dereferences caller-supplied URLs is an SSRF hole.

External reading

• Wan-Video on GitHub — upstream Wan weights and reference implementation.
• Wan-AI on Hugging Face — the open-weights releases (2.1 and 2.2; later Wan versions are API-only).

Where next