Skip to content

llama.cpp b10321: Metal NORM fix for partial simdgroup rows

llama.cpp build b10321 fixes a Metal kernel bug where partial simdgroups caused incorrect NORM/RMS_NORM outputs on Apple Silicon. No crash — silently wrong results.

llama.cpp build b10321 fixes a Metal kernel bug that silently corrupted normalization outputs on Apple Silicon whenever a row’s length wasn’t a clean multiple of the simdgroup size.

  • Bug: ggml_metal_op_norm sized threadgroups with nth = std::min(nth, args.ne00_t), which can leave nth not a multiple of the simdgroup size
  • Mechanism: the cross-simdgroup reduction has each lane of the last simdgroup read one partial sum from shared memory — a partial simdgroup has fewer lanes than there are simdgroups, so tail sums are never read
  • Example: ne00_t = 33 → nth = 33 → two simdgroups, one lane in the second → one of two partial sums dropped → row sum too small
  • Impact: mean and variance wrong for the entire row → NORM/RMS_NORM outputs incorrect
  • Fix: round ne00_t up to a whole number of simdgroups instead of clamping

Normalization runs on every transformer block. Wrong RMS_NORM output means activations drift from what the weights expect — no crash, just silently degraded generation. The fix rounds ne00_t up rather than dropping the clamp, keeping the threadgroup small while ensuring the reduction has enough lanes to collect every partial sum.

WARNING

This bug produced no error or crash. Models ran and generated text, but with incorrect normalization on affected row lengths. If you ran llama.cpp on Metal prior to b10321, outputs for affected configurations were silently wrong.

Build b10321 is live on the llama.cpp releases page. If you’re running inference on Apple Silicon, update before your next run.