Video generation
9 video aliases · LTX-2.5 / 2.3 (2) · Wan 2.1 / 2.2 (4) · CogVideoX-Fun (3). Text-to-video and image-to-video through the asynchronous OpenAI-compatible Videos API. LTX-2.5 clips come back with synchronized audio.
rapid-mlx serves POST /v1/videos — the same job-based shape as OpenAI's Videos API. You submit a prompt, get a job id back immediately, poll it, then download the finished MP4. Generation is serialized: one clip renders at a time, because a diffusion pipeline plus its VAE will not share unified memory politely with a second one.
- family
- Video generation
- aliases
- 9
- lines
- 3
- install
- pip install 'rapid-mlx[video]'
- also needs
- ffmpeg · Python 3.11+
- OpenAI base URL
- http://localhost:8000/v1
Requirements
Two constraints that do not apply to the rest of rapid-mlx:
- Python 3.11 or newer. The upstream
mlx-video-with-audioruntime does not support 3.10. Core text and audio features still run on 3.10 — this is a video-lane-only floor. - ffmpeg reachable, for muxing frames into an MP4:
brew install ffmpeg. Since 0.11.9 it is resolved fromFFMPEG_BINARYfirst, then the processPATH, then the usual Homebrew and system locations — so a GUI-launched server with a minimal environment finds it without you exporting anything. SetFFMPEG_BINARYto an absolute path if you keep it somewhere unusual.
Usage
Submit, poll, download — three calls. The job id comes back on the first one.
pip install 'rapid-mlx[video]'
brew install ffmpeg
rapid-mlx serve ltx-2.5-mlx-q8
# Submit a text-to-video job — the MP4 comes back with synchronized audio
curl http://127.0.0.1:8000/v1/videos \
-F model=ltx-2.5-mlx-q8 \
-F 'prompt=a fox running through fresh snow, cinematic tracking shot' \
-F seconds=4 \
-F size=1280x720 \
-F seed=42
# Poll it, then download the MP4
curl http://127.0.0.1:8000/v1/videos/video_ID
curl http://127.0.0.1:8000/v1/videos/video_ID/content --output result.mp4
For image-to-video, add -F input_reference=@start.png — supported when the served checkpoint is TI2V or I2V.
Motion & conditioning controls
Beyond seconds, POST /v1/videos takes four optional controls. What each backend accepts differs, and the route rejects the combinations that would silently do nothing:
| field | range | notes |
|---|---|---|
| fps | backend-dependent | Sets the output frame rate on LTX (up to 60 fps on 2.5) and CogVideoX-Fun. Wan rejects it — it uses the checkpoint's native rate (16 fps for 2.1, 24 for 2.2). |
| frames | see rule | Overrides the count derived from seconds. LTX needs 8n+1, minimum 9. Wan and CogVideoX-Fun need 4n+1, minimum 5. |
| guidance_scale | 1–30 | Classifier-free guidance strength. Pairs with negative_prompt. LTX-2.5 takes neither — the route rejects both. |
| conditioning_strength | 0–1 | How closely LTX image-to-video follows input_reference. Rejected without a reference image, and not supported on Wan or CogVideoX-Fun. |
Rather than memorise which backend takes what, ask the running server:
curl http://localhost:8000/v1/videos/capabilities
It reports the live model's family, native fps, supported modes (text-to-video / image-to-video), the size rule — a fixed list for CogVideoX-Fun, a 64-pixel-aligned range plus a maximum pixel area for Wan — and the frame-count rule as a minimum/step/offset triple. That is the same contract the route validates against, so it cannot drift from what the server will actually accept.
Families in this lane
The three backends
Each backend is a model family in its own right, with its own page — aliases, controls, RAM requirements and examples:
Notes & caveats
- Jobs are serialized — a second request queues rather than running concurrently. This is deliberate: two diffusion pipelines resident at once will exhaust unified memory on every Mac we target.
- Defaults are 50 diffusion steps. Lowering steps is the single biggest speed lever, at a quality cost.
- Wan rejects frame counts that are not
4n+1and sizes above the checkpoint's pixel-area ceiling — both come back as400with the constraint named, rather than failing deep in the pipeline. - Image-to-video takes a real uploaded file. Remote image URLs are refused rather than fetched — a server that dereferences caller-supplied URLs is an SSRF hole.
- Video generation is the newest lane in rapid-mlx and the least settled. Treat throughput numbers as a snapshot, not a contract.
Frequently asked questions
Can I generate video locally on a Mac?
Yes. rapid-mlx runs LTX-2.5/2.3, Wan 2.1/2.2 and CogVideoX-Fun entirely on Apple Silicon through an asynchronous OpenAI-compatible Videos API at POST /v1/videos — no cloud service and no per-clip cost — and LTX-2.5 clips come back with synchronized audio. Install with pip install 'rapid-mlx[video]', which also needs ffmpeg and Python 3.11 or newer.
Which video generation models run on Apple Silicon?
Nine MLX-converted checkpoints ship today: LTX-2.5 in q8 and LTX-2.3 in q4, four Wan 2.2 checkpoints (TI2V 5B in q8 and bf16, I2V A14B q8, T2V A14B bf16), and three CogVideoX-Fun 5B quants (q4, q8, bf16). ltx-2.5-mlx-q8 is the recommended starting point — the only checkpoint that returns synchronized audio, and it serves on a 24 GB Mac via its low-RAM distilled path. wan2.2-ti2v-5b-q8 is the smallest download.
How long does local video generation take on a Mac?
Minutes per second of footage, not real time. On an M3 Ultra, CogVideoX-Fun q4 produced a one-second 672x384 clip in about 338 seconds at the default 50 diffusion steps. Lowering the step count is the single biggest speed lever, at a cost in quality. Static and slow-panning scenes hold up best; fast subject motion is still experimental.
How much RAM do I need to generate video on a Mac?
24 GB of unified memory for the smallest checkpoints (CogVideoX-Fun q4 peaks around 14.5 GB, LTX-2.3 q4 similar; LTX-2.5 q8 is a 67.7 GB download but also serves at 24 GB via its low-RAM distilled path), 32 GB for Wan 2.2 TI2V 5B q8, and 64-96 GB for the 14B Wan MoE checkpoints. Jobs are serialized on purpose — two diffusion pipelines resident at once would exhaust unified memory.
Does image-to-video work locally?
Yes, on checkpoints that support it. Wan 2.2 TI2V and I2V accept a conditioning first frame — pass it as an uploaded file with -F input_reference=@start.png. Remote image URLs are deliberately refused rather than fetched, because a server that dereferences caller-supplied URLs is an SSRF hole.