Moving from Ollama to rapid-mlx on Apple Silicon

The muscle memory transfers — run, pull, rm, ps all work the same. Point your tools at a new port, and run both servers side by side while you migrate. Here's the whole move.

Jul 21, 2026 · 4 min read

ollamamigrationlocal-llmapple-silicon

If you run local models on a Mac with Ollama and you're curious about more speed on Apple Silicon, a drop-in Anthropic API (not just OpenAI), or an Apache-2.0 engine — moving to rapid-mlx is mostly muscle memory. The commands you already type map almost one-to-one, and because the two servers listen on different ports, you can run both side by side and migrate at your own pace. Nothing to rip out.

Should you switch?

Straight, no spin:

What rapid-mlx gives you here: faster inference on Apple Silicon, a drop-in **OpenAI and Anthropic** API (so Claude Code, Cursor, Codex, and OpenAI clients all work), Apache-2.0, and newer model generations out of the box (Qwen 3.6, Gemma 4).
Where Ollama still wins: it's cross-platform (Windows/Linux too), and its model library is larger. If you need a non-Mac box, or a specific model rapid-mlx doesn't carry, stay on Ollama — or run both.

That last point is the theme: this isn't a burn-the-boats migration.

The commands are almost the same

rapid-mlx deliberately mirrors Ollama's verbs, so most of your muscle memory just works:

You typed (Ollama)	Now type (rapid-mlx)	What it does
`ollama run <model>`	`rapid-mlx run <model>`	interactive chat REPL
`ollama pull <model>`	`rapid-mlx pull <model>`	download a model
`ollama rm <model>`	`rapid-mlx rm <model>`	delete a cached model
`ollama ps`	`rapid-mlx ps`	show running models
`ollama list`	`rapid-mlx models`	list available models
`ollama serve`	`rapid-mlx serve <model>`	start the local server

One real difference: ollama serve starts a model-agnostic daemon, while rapid-mlx serve <model> boots a server for a specific model at http://localhost:8000. (run still auto-loads on demand, Ollama-style.)

Model-name cheat sheet

Model names don't carry over verbatim — and that's mostly good news, because rapid-mlx ships newer generations than what a lot of Ollama setups are pinned to. Reach for the equivalent:

On Ollama	On rapid-mlx (recommended)
`qwen2.5:3b`	`qwen3.5-4b-4bit`
`qwen2.5:7b`	`qwen3-8b-4bit` / `qwen3.6-35b-4bit`
`gemma2:9b`	`gemma-4-12b-4bit`
`gemma2:27b`	`gemma-4-26b-4bit`
`llama3.1:8b`	`qwen3.6-35b-4bit` or `gemma-4-12b-4bit` (Llama 3 ships as `llama3-3b-4bit`)
`deepseek-r1:8b`	`deepseek-r1-8b-4bit`
`deepseek-r1:32b`	`deepseek-r1-32b-4bit`
`mistral`	`mistral-24b-4bit`
`phi3`	`phi-3.5-mini-4bit`
`gpt-oss:20b`	`gpt-oss-20b`
_tiny, punches up_	`bonsai-27b-2bit` — a 27B in 7.9 GB, runs on a 16 GB Mac

Run rapid-mlx models for the full list, or browse the model mirror with a RAM filter.

Point your tools at the new port

Anything you have wired to Ollama's OpenAI-compatible endpoint just needs a new base URL:

Ollama:     http://localhost:11434/v1
rapid-mlx:  http://localhost:8000/v1

That covers most modern setups — Cursor, Continue, LangChain, and any OpenAI client. (Claude Code is even easier — it speaks the Anthropic wire; see the Claude Code recipe.)

The one real gotcha: rapid-mlx serves the OpenAI (/v1/chat/completions) and Anthropic (/v1/messages) APIs — not Ollama's native /api/generate · /api/chat · /api/tags. If a tool is hardcoded to Ollama's own API, point it at the OpenAI path instead. Most tools already default to OpenAI-compatible, so this is usually a non-issue.

Run both side by side while you migrate

Because the ports don't collide, keep Ollama on 11434 and bring rapid-mlx up on 8000:

# terminal 1 — your existing Ollama, untouched
ollama serve            # :11434

# terminal 2 — rapid-mlx alongside it
rapid-mlx serve qwen3.6-35b-4bit    # :8000

Move one tool over, use it for a day, and if anything's missing you haven't lost your old setup. When you're confident, stop Ollama — or don't. Nothing forces the choice.

Honest expectations

A local model is a local model on either engine — neither is a frontier cloud model. What you're actually buying by switching on Apple Silicon is speed, the Anthropic API surface (which unlocks Claude Code locally), Apache-2.0, and current-gen weights. What you'd give up is cross-platform support and the breadth of Ollama's library. For most Mac-based, OpenAI/Anthropic-tool workflows, the move is a few minutes and reversible — which is the whole point of doing it side by side.

Where to go next

Run Claude Code on a local model · Cursor · Codex · Aider
Browse the model mirror — RAM filter to pick the right replacement model.
Install rapid-mlx — Homebrew or one curl. Apache 2.0.

Run this yourself. rapid-mlx is an open-source, OpenAI-compatible inference server for Apple Silicon. pip install rapid-mlx, then rapid-mlx serve <alias>. Browse every supported model on the family docs or pick one by your Mac's RAM at models.rapidmlx.com.