llama.cpp build b10234 adds F16 support for binary operations on Metal — Apple Silicon can now run element-wise math in half precision without falling back to F32.
What shipped:
- Metal F16 bin ops (PR #26465) — addition, multiplication, and other element-wise operations now execute in float16 on Apple GPUs
- Windows CUDA 13.3 DLLs available alongside CUDA 12.4 — both shipped in the same release
- ROCm 7.2 Linux builds for AMD GPUs
- SYCL FP16 builds for Intel GPUs, joining the existing FP32 variant
- OpenCL Adreno builds for Windows on ARM with Qualcomm GPUs
Prebuilt binaries span over 25 configurations across macOS, iOS, Linux, Android, Windows, and openEuler. Two are explicitly disabled: macOS Apple Silicon with KleidiAI, and the general openEuler target.
NOTE
KleidiAI builds for macOS arm64 are disabled in b10234. If you relied on KleidiAI-accelerated inference on Apple Silicon, use the standard arm64 build until it’s re-enabled.
F16 binary ops on Metal matter because element-wise operations — attention scaling, RoPE, residual connections — run constantly during transformer inference. Executing them in half precision instead of upcasting to F32 cuts memory bandwidth, which is the primary bottleneck for LLM inference on Apple Silicon’s unified memory.
Available now — grab prebuilt binaries from the release page. The standard macOS arm64 build is what most Apple Silicon users want; Windows users with NVIDIA GPUs should pick the CUDA 13.3 variant if their driver supports it.