llama.cpp build b9994 ships one functional change: Q2_0 quantization now runs on the Metal backend (PR #25419). Everything else in the release is build matrix and platform packaging.
Q2_0 is the most aggressive quantization format llama.cpp supports — 2-bit weights that cut model memory roughly 4× compared to Q8_0. A 70B model that needs ~70 GB at Q8_0 fits in ~18 GB at Q2_0. On Apple Silicon with unified memory, that is the difference between “won’t load” and “runs.”
What b9994 ships across platforms:
- macOS / iOS: Apple Silicon (arm64) with Metal Q2_0, Intel x64, iOS XCFramework
- Linux: Ubuntu x64 and arm64 CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16
- Windows: x64 CPU, CUDA 12 and CUDA 13.3, Vulkan, OpenVINO, SYCL, HIP; arm64 CPU and OpenCL Adreno
- Android: arm64 CPU
- openEuler: 310p and 910b (ACL Graph) on x86 and aarch64
KleidiAI on macOS arm64 is listed but disabled. openEuler standard builds are also disabled.
NOTE
Q2_0 trades quality for size. Perplexity degradation is measurable — expect it on coding and reasoning tasks. Use it to fit a model that otherwise won’t run, not as a default quantization.
The release is a single-feature build. If you are already on b9993 and not running Q2_0 models on Metal, there is no reason to update. If you have been blocked from running 70B+ models on a 32 GB Mac, this is the build that unblocks you — pull b9994 and grab a Q2_0 GGUF.