llama.cpp build b9974 fixes a CUDA crash that killed inference before it started — if a GPU had zero free VRAM, the act of querying that memory was enough to bring down the whole process.
What changed in b9974:
cudaMemGetInfo()fix — querying memory on a starved device now returns a non-fatal error instead of crashing (PR #25157)- Zero-memory assignment — devices with no free VRAM get zero total/free memory, so the layer-fit algorithm skips them automatically
-dev noneflag — CUDA-enabled builds no longer crash when users explicitly pass this flag- Build matrix — macOS arm64/x64, iOS XCFramework, Linux CPU/Vulkan/ROCm 7.2
The bug hit hardest on multi-GPU rigs. llama.cpp queries memory on all CUDA devices at startup — even ones the user never intended to use for inference. A secondary GPU sitting at 100% VRAM would take down the entire run before a single token was generated.
NOTE
The KleidiAI-enabled macOS arm64 build is marked DISABLED in this release’s CI matrix.
Available now as b9974. Anyone running multi-GPU systems or boxes with secondary CUDA devices should update — the crash was silent and gave no useful error message.