DeepSeek V4.1 on a Mac, and Why Your LLM API Router Is Reading Your Prompts

A 199 GiB model on a desktop Mac, four measured speed-ups, and a security paper that makes the case for local inference better than any marketing page could.

Two things shipped in Rapid-MLX 0.14.0 and 0.14.1. One is a flex: a native MLX runtime for DeepSeek V4.1 Flash, written from scratch, with the 2-bit weights published on day zero. The other is the reason any of this matters, and it comes from someone else's paper.

Let's do the flex first, then the part that should worry you.

Day zero: DeepSeek V4.1 Flash, on a Mac

We wrote a native MLX implementation of the V4.1 architecture — attention, MoE, engram, hyper-connections, indexer, sparse attention, and a chunked-prefill streaming generator. Then we converted the weights to 2-bit and published the checkpoint: 41 shards, 212,930,051,680 tensor bytes, about 199 GiB, ungated.

On an Apple M3 Ultra with 60 GPU cores, 256 GiB of unified memory and MLX 0.32.2:

Dropping 48 of the 384 routed experts per layer is what makes it fit at all, and it costs almost nothing: 99.9949% of the routing weight is still there. The numerics match the reference implementation. It answers correctly.

It is also 7.31 tok/s, and it is text-only — vision and multi-token prediction are not in the artifact.

We hit the 256 GiB ceiling on day zero. 0.14.0 also drops the floor: the smallest model we added is 1.3 GiB, local image generation runs under 7 GiB of peak memory, and everything else in this release lands in between.

Someone is reading your prompts

Here is the part that should change how you think about all of this.

A study published this year — "Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain", by Liu, Shou, Wen, Chen, Fang and Feng — bought 28 LLM API routers from Taobao, Xianyu and Shopify-hosted storefronts, collected 400 more from public communities, and watched what they did with the traffic.

Figure 1, Liu et al., "Your Agent Is Mine" (arXiv:2604.08407) — one malicious hop is enough to own every agent downstream of it.

What they found:

• 9 routers were injecting malicious code into the tool calls they returned — the commands your agent then executes on your machine. One paid, eight free.
• 17 reached for the researchers' AWS canary credentials. One drained a researcher-owned Ethereum key.
• 2 were hiding. One waited for 50 prior calls before doing anything. One restricted delivery to autonomous "YOLO mode" sessions. One targeted only Rust and Go projects.
• A single deliberately leaked key generated 100M GPT-5.4 tokens and more than seven Codex sessions. Weakly configured decoy relays pulled in 2 billion billed tokens, 99 credentials across 440 Codex sessions — 401 of which were already running in autonomous mode.

The structural finding matters more than any single number:

And therefore:

This is not a bug in one product. It is what happens when there is a middleman at all — and nothing in the protocol lets your agent tell whether the tool call it just received is the one the model actually wrote.

Rapid-MLX has no router, no proxy and no upstream provider. There is no middle for anyone to sit in. The model runs on the Mac in front of you, and your prompts never leave it. That is the whole reason we build this.

Four measured speed-ups

Not peak-kernel numbers. Every one of these was measured end to end, on an M3 Ultra, in 0.14.0.

The head-of-line fix is the one you feel: your short question stops queueing behind someone's 8,000-word prompt. It is an opt-in scheduling policy (shortest_validated_tail), because the default remains first-come-first-served and we would rather you chose it deliberately.

The multi-token-prediction result used the same checkpoint, prompt, seed and K=3 draft depth across three alternating runs, producing byte-identical output. Muse-Glimmer's speed-up applies to its 8-bit build only, and wants a 48 GB Mac. The fused Flash-Next kernel is +26.28% on the isolated layer and stays opt-in until its hardware coverage grows.

At 64K, a new opt-in changes the picture

Two more numbers only show up once the context gets long — which is exactly where agent work and whole-document work live.

Now the part most vendors leave out. Both of these have a crossover, and below it they cost you something:

That is why neither is on by default. The sparse prefill path is behind RAPID_MLX_QSA_BLOCK_SPARSE=1; K=2 is admitted only when an operator explicitly asks for it, and Qwen3.8 stays at K=1 otherwise. If your prompts are genuinely long, turn them on. If they are not, leaving them off is the faster choice, and we would rather say so than let you discover it.

More choices across every memory tier

Artificial Analysis Intelligence Index, snapshot. The index scores the base model at full precision with reasoning on — not the quantised build we ship.

• Granite 4.2 in six variants: 30B, 8B and 3B, each in 4-bit and 8-bit, all qualified the same way. Measured thinking decode ran from 20.8 tok/s (30B 8-bit) to 175.1 tok/s (3B 4-bit) on an M3 Ultra.
• MiniCPM5 2B 4-bit, 1.3 GiB. On the same 31-case tool workload it scored 24/31 in 7.17 s, against 26/31 in 32.54 s for the 4B model we compared it against. About 4.5× faster — and two points less accurate. You get the speed, and you get told what it costs.
• G9v3-39A5B 4-bit, a 21.3 GiB mixed-precision MoE for 32 GB+ Macs. Chat, reasoning, streaming, tool calls, Chinese and coding all passed; the reasoning test decoded at 85 tok/s.
• NeoHorse 1 9B 4-bit measured 44.5 tok/s — and did not beat the default across the full quality suite, so it ships available rather than selected. 0.14.0 does not quietly change the model you are using.
• Qwen3.8 27B Abliterated 4-bit, for users who explicitly want a reduced-refusal variant. Reduced refusal is not improved correctness, and it is not recommended by default.
• Seven more image models: FLUX.1 schnell, FLUX.2 Klein BF16, Stable Diffusion 3.5 Large, SDXL Base, Bonsai Image 4B 2-bit, HiDream O1 Dev and Qwen Image Edit. In testing, FLUX.1 schnell at 1024×1024 in 23.36 s with 9.46 GiB peak; Bonsai 4B 2-bit at 512×512 in 5.39 s with 6.52 GiB peak.

Generated locally. Nothing left the machine.

The Community Benchmark, and an embarrassing gap

0.14.0 finished the Community Benchmark in both Desktop and the CLI. Every row is measured the same way, so numbers from two different Macs actually compare, and a run now shows you which phase it is in and how long it has left instead of sitting there.

Each Mac gets a name from a random per-install id — never a serial number. We know the name; we don't know the person. Deleting the id file resets it. Results stay on your machine unless you explicitly choose to share them.

Which produces a funny result. There are 57 runs from 10 Macs across 17 models and 8 chips — M1 Max, M2, M2 Pro, M4, M4 Pro, M4 Max, M5 Pro, M5 Max. Every performance number in this post was measured on an M3 Ultra, and there is not a single M3 on the board. Not one of them is ours, either: local results stay local unless you share them, and that applies to the machines we test on too.

See where your Mac would rank, and fill in the gap while you are there. It is one run: rapid-mlx benchmark, or the Benchmark workspace in Desktop.

Large and scanned PDFs

Drop in the cap table. The signed contract. The diagnosis. The codebase you are under NDA for. Nothing is uploaded, nothing is proxied, and nobody is scraping your IP for their next base model — see above for what happens to people who do send it somewhere.

New in 0.14.1: selectable PDFs, image-only scans, and documents that mix the two, without stuffing the whole file into one prompt. A follow-up question retrieves the specific outline, page or section instead of re-sending the document.

The unglamorous half matters more: if a page fails to extract, you are told, rather than being handed a confident summary of a document that was only half read.

The community is doing more of this than you'd think

Rapid-MLX is a small team. A meaningful share of 0.14 was not written by us.

Pierre Lamy fixed the fused GatedDeltaNet decode path to use the MLX-exact sigmoid forms — the kernel in the speed table above, the one worth +6.35% end to end and +26.28% on the isolated layer. That number is partly his. He also caught the CLI silently ignoring cache settings you had asked for.

Nick M wrote the behavioural coverage for the sampled MTP path — and it immediately earned its place. We deliberately broke the sampler to see what would catch it. All 246 of our existing tests passed. Four of his five new ones failed. A community contributor wrote the tests that proved our own tests could not go red.

Atom landed three Desktop fixes in a row: clipping streamed Markdown drawing to the dirty rect, making streamed text visible in dark mode while it fades in, and splitting ContentView.body so it type-checks again. If Desktop feels smoother while a reply is streaming, that is him.

@Fe2-O3 did not just report the port collision — he fixed it. Desktop defaulted to 8000 and collided with a local gateway he already ran. He cited the exact source line where the default was set, proposed 7659 — R-M-L-X on a phone keypad — and shipped the change with a port control in the UI. The joke is his too.

@qevan parked the DMG background icon below the install window fold, which is the kind of detail nobody notices until it is wrong.

And @reports-svg filed the best bug report we got this cycle. On a 128 GB M5 Max running Qwen3.6-35B-A3B as a long-lived classification service — hybrid MoE plus GatedDeltaNet, several hundred to a thousand requests a day — Metal hit Resource limit (499000) exceeded roughly 55 times daily, each occurrence aborting the whole batch. He had also read our earlier fix and pointed out that it only covered _merge_caches while the failure persisted on a second path in _step/async_eval. That is not a bug report, that is a code review. It is fixed.

This is what a local-first project looks like when it works: the people running it hardest in production are the ones finding the edges, and increasingly they send the patch with the report. If something here is broken on your Mac, open an issue — you have seen what happens to good ones.

Things we turned off, and one we didn't ship

• Qwen3.5 4B MTP is now off by default. We measured single-stream decode 25–37% slower on M2 Pro and M3 Ultra with no concurrency gain. It remains available explicitly.
• NeoHorse 1 9B is available but not selected, as above.
• DeepSeek V4.1 Flash 2-bit is published and reproducible, and stays out of the catalog.

We would rather ship a shorter list of things that are actually better.

The smaller things

• rapid-mlx start picks a model that fits, starts the server, waits for readiness and configures an agent profile from one command. --dry-run previews every mutation.
• rapid-mlx service installs a least-privileged headless macOS service with status, logs, restart, uninstall and rollback.
• rapid-mlx doctor gained lifecycle, import, cache, configuration and service diagnostics, with verified repairs where a safe repair exists.
• Desktop can unload an idle resident model in one click while keeping your selection, and refuses to interrupt active chat, image, audio, video or dictation work.
• Telemetry honours RAPID_MLX_TELEMETRY=0, DO_NOT_TRACK=1 and common CI markers, in both the engine and the Desktop app.

Full detail in the 0.14.0 and 0.14.1 changelogs. 244 models ship in the catalog today — 190 text, 44 audio, 10 video, plus the image models above.

curl -fsSL https://rapidmlx.com/install.sh | bash