Holo3 — small-MoE agent model

Holo3.1-35B-A3B is a 35B-parameter / 3B-active mixture-of-experts agent model from Holo Labs. It's available in rapid-mlx 0.9.9 as the holo3.1-35b-a3b alias (4-bit) and holo3.1-35b-a3b-8bit (8-bit) — it earned the mirror because a peer Apple-Silicon agent team reported it running well in production.

Why it's notable

The bar for a rapid-mlx mirror is narrow: a model has to fit a real Apple-Silicon Mac (not a Mac Studio cluster), do something specific better than its size class, and have at least one credible production user. Holo3 hits all three.

• Right size class. 35B total / 3B active parameter MoE — the active-parameter speed of a 3B model with the quality envelope of a frontier mid-tier. The 4-bit quant will fit a 32 GB Mac with headroom; the 8-bit will fit 64 GB.
• Agent-tuned. Holo Labs targets agent workflows specifically. The post-training mix and tool-call envelope are designed for multi-step tool use, not just chat.
• Production-reported. A peer Apple-Silicon agent team running their own stack reported good results — independent confirmation outside Holo Labs's marketing.
• Active project. Holo Labs ships a regular cadence on Hugging Face — useful signal that the line will keep getting maintained.

Architecture sketch

family

Holo3 (Holo Labs)

variant

Holo3.1-35B-A3B

shape

MoE · 35B total · 3B active

focus

Agent / tool-using workflows

upstream HF

huggingface.co/HoloLabs

rapid-mlx alias

holo3.1-35b-a3b · holo3.1-35b-a3b-8bit

rapid-mlx mirror

pipenetwork/Holo-3.1-35B-A3B-MLX-4bit · -8bit

tool parser

hermes · reasoning: qwen3

Running it

Both quants are one command away — rapid-mlx pulls the mirror on first use:

rapid-mlx serve holo3.1-35b-a3b        # 4-bit — fits a 32 GB Mac with headroom
rapid-mlx serve holo3.1-35b-a3b-8bit   # 8-bit — 64 GB

It serves on the standard OpenAI-compatible endpoint with a Hermes-shape tool-call envelope and a qwen3 reasoning parser, so existing agent stacks work unchanged.

Where next