# Rapid-MLX 0.15.5 re-measure of the pre-0.14 model pages — M4 Pro 48 GB (2026-10-03)

Raw data behind the `/models/` pages that, until this run, still showed
numbers from rapid-mlx 0.12.10 (Mac mini M2 Pro, 2026-08-11). Those rows were
retired; these pages now lead with this run. Same Mac, same harness and
method as the 2026-10-02 run at [/blog/assets/bench-2026-10-m4pro/](/blog/assets/bench-2026-10-m4pro/).

- Machine: Mac mini, Apple M4 Pro, 48 GB unified memory (see `env.txt`;
  macOS 26.5.1).
- Engine: `rapid-mlx==0.15.5` from PyPI in a fresh venv (server on CPython
  3.12.14; the stdlib-only harness ran on the host's system Python 3.9.6, which
  is the `python` recorded in each raw scenario file).
  Every model that ran did so on the slim base install — no `[vision]` or
  other extra was installed. No other flags.
- Harness: the published /compare harness (`bench_compare.py`, seed 20260926,
  temperature 0, streaming, `include_usage`), each model served alone on
  loopback with a fresh temporary HOME, telemetry off, the shared Hugging Face
  cache, 60 s + GPU-idle cooldown between models. Scenarios:
  - **a** — single-stream decode, median of 3 × 512-token generations.
  - **b** — cold TTFT at a ~2,000-token and a ~16,000-token prompt, median of 3
    (`ttft_2k_median_s` / `ttft_16k_median_s` in `summary.json`).
  - **d** — 4 concurrent streams × 2 rounds, aggregate decode
    (`aggregate_4stream_median_tps`).
- `summary.json` is the roll-up; each model directory holds the per-scenario
  raw request JSON (`a.json`, `b.json`, `d.json`), one-line medians
  (`*.summary`), the `/v1/models` record (`models.json`), the `/v1/status`
  snapshot after the run (`status.json`), and the server's own precheck
  (`precheck.txt`, `info.txt`). Server logs are omitted (they contain local
  paths); the relevant lines of any failure are quoted in `summary.json` and
  below. Decode rates come from engine-reported usage token counts.
- Peak memory: server-reported `/v1/status` `metal.peak_memory_gb` (with the
  MLX active figure alongside); `peak_rss_gb` is the ps sample of the server
  PID tree every 2 s. Both include the engine's in-memory prefix cache, which
  is sized to 20% of RAM by default — small models therefore peak well above
  their weight size on this 48 GB machine.

## Models that did not complete the run

- **`lfm2.5-2.6b-4bit`** (`LiquidAI/LFM2.5-2.6B-MLX`, subfolder `4bit`) — the
  server did not start. The 1.6 GB pull completed, then loading failed with:

  > Error loading model: The '4bit' subfolder of LiquidAI/LFM2.5-2.6B-MLX is
  > present but incomplete — its weight shards are missing. A previous download
  > did not finish, or the publisher reorganized the repo.

  Every file was on disk. This Mac's Hugging Face cache stores large blobs in a
  shared, de-duplicated `hub/blobs` store that the repo's own `blobs/` entries
  symlink into, and the engine's subfolder completeness check only accepts weight
  files that resolve inside the repo's own `blobs/` directory. The non-INFO server
  output is in `lfm2.5-2.6b-4bit/failure.txt`. It was not re-run with a workaround.
- `qwen3-0.6b` attempt 0 was discarded before any measurement: the driver's
  working directory contained a folder named `qwen3-0.6b` (its own output
  folder), which the CLI loaded as a local model path instead of the catalog
  alias. Every server was then started from an empty working directory.

Its page shows no speed, latency or measured-memory number and says it has
not been measured on the current version.

## The four larger models (second pass, same day)

`gemma-4-12b-4bit`, `bonsai-27b-2bit`, `qwen3.6-27b` and `qwen3-coder-30b-4bit`
were skipped in the first pass because the bench Mac's disk had 11 GiB free and
the driver keeps at least 5 GiB free. A stale prefix-cache folder from an older,
unrelated run (5.7 GB, not model weights) was then deleted, the four checkpoints
were pre-downloaded with `rapid-mlx pull`, and the same driver, venv, harness and
method measured them from an empty working directory between 15:51 and 16:16 PDT
on 2026-10-03. No model files used by any published benchmark were deleted.

| Model | Decode | TTFT · ~2K prompt | TTFT · ~16K prompt | 4-stream aggregate | Metal peak |
|---|---|---|---|---|---|
| `gemma-4-12b-4bit` | 33.5 tok/s | 8.7 s | 71.0 s | 70.8 tok/s | 13.8 GB |
| `bonsai-27b-2bit` | 22.4 tok/s | 19.4 s | 73.8 s | 35.8 tok/s | 13.5 GB |
| `qwen3.6-27b` | 23.4 tok/s | 18.9 s | 28.9 s ¹ | 25.1 tok/s | 22.8 GB |
| `qwen3-coder-30b-4bit` | 91.5 tok/s | 2.6 s | 34.7 s | 135.2 tok/s | 27.3 GB |

1. PFlash is on by default for `qwen3.6-27b` (the catalog marks it
   `pflash_tier=verified`). The ~16K-token prompts were compressed to about 20%
   of their tokens before prefill (server log: `compressed 15474 -> 3095 tokens`),
   so the 16K figure is not a full-prefill time. The ~2K prompts were not
   compressed. Pass `--pflash off` for the uncompressed baseline (not measured here).

- `gemma-4-12b-4bit` ran text-only: on the slim base install the server logs
  `vision runtime absent; serving this text-capable checkpoint text-only`.
  Install `rapid-mlx[vision]` for image input (not measured here).
- `qwen3.6-27b` ran with MTP speculative decode, its default. With four
  concurrent streams the aggregate (25.1 tok/s) is about the single-stream rate.

Every page citing this run links here and labels the chip, engine version and
date (M4 Pro 48 GB · rapid-mlx 0.15.5 · 2026-10-03).