Skip to content

llama.cpp b9974: CUDA crash fix for zero-VRAM GPUs

llama.cpp build b9974 fixes a CUDA crash where querying memory on a GPU with no free VRAM killed the entire inference process before it started.

llama.cpp build b9974 fixes a CUDA crash that killed inference before it started — if a GPU had zero free VRAM, the act of querying that memory was enough to bring down the whole process.

What changed in b9974:

  • cudaMemGetInfo() fix — querying memory on a starved device now returns a non-fatal error instead of crashing (PR #25157)
  • Zero-memory assignment — devices with no free VRAM get zero total/free memory, so the layer-fit algorithm skips them automatically
  • -dev none flag — CUDA-enabled builds no longer crash when users explicitly pass this flag
  • Build matrix — macOS arm64/x64, iOS XCFramework, Linux CPU/Vulkan/ROCm 7.2

The bug hit hardest on multi-GPU rigs. llama.cpp queries memory on all CUDA devices at startup — even ones the user never intended to use for inference. A secondary GPU sitting at 100% VRAM would take down the entire run before a single token was generated.

NOTE

The KleidiAI-enabled macOS arm64 build is marked DISABLED in this release’s CI matrix.

Available now as b9974. Anyone running multi-GPU systems or boxes with secondary CUDA devices should update — the crash was silent and gave no useful error message.