llama.cpp build b10238 adds Multi-Token Prediction (MTP) support for Qwen3-Next, letting local deployments generate multiple tokens per forward pass instead of standard one-at-a-time autoregressive decoding.
The change shipped in PR #25589, co-authored with Hugging Face’s Sigbjørn Skjæret. What landed:
- MTP loading for Qwen3-Next — new
load_mtpflags;num_mtpcomputed directly from the model’s MTP layer definitions via the_QwenMtpMixinclass - Python type-check fixes — corrections in
gguf-py/gguf/constants.pyand the Qwen MTP mixin - Prebuilt binaries — macOS Apple Silicon, Linux (CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android arm64, iOS XCFramework, and Windows
MTP lets a model predict several tokens in a single forward pass. Accepted tokens skip the autoregressive loop entirely, cutting per-token latency without requiring a separate draft model. For Qwen3-Next running locally, that translates to faster generation on the same hardware — whether that’s an Apple Silicon Mac, an ROCm GPU, or a CPU-only Linux box.
NOTE
MTP requires enabling load_mtp at load time. The build derives num_mtp from the model’s own MTP layer definitions — no manual parameter count needed.
The macOS Intel (x64) build is listed as disabled in this release. macOS users need Apple Silicon. Linux users get the widest backend coverage: CPU, Vulkan, ROCm 7.2, OpenVINO, and SYCL in both FP32 and FP16.
Build b10238 is available now from the llama.cpp releases page.