llama.cpp build b10255 extends Intel GPU (SYCL) attention acceleration to quantized KV caches — and Claude co-authored the PR.
The oneDNN SDPA path previously handled only FP16 KV caches. This build adds the formats that matter for local inference, where VRAM is the bottleneck:
- Q4_0, Q4_1, Q5_0, Q5_1, Q8_0 — dequantized to dense FP16 on-device via
to_fp16_sycl/to_fp16_nc_sycl - F32 — converted via
cont_to_f16_sycl - BF16 and IQ types — excluded, no conversion kernel available
- Stream sync —
stream->wait_and_throw()now unconditional (PR #25741 by @malsbat) - V_is_K_view aliasing removed — K and V always dequantized to separate buffers
Non-F16 K/V tensors get converted to FP16 on-device before entering the SDPA graph, then the fused systolic kernel runs the same path as native FP16. The gate is K >= 1024 and Q >= 32, prefill only.
NOTE
FP16 KV caches still run at any sequence length. The K >= 1024 / Q >= 32 gate applies only to non-F16 types during prefill.
The PR lists Co-Authored-By: Claude noreply@anthropic.com — Anthropic’s model contributed to the implementation, not just the commit message. The docs cleanup (dropping GGML_SYCL_FA_DEBUG from SYCL.md) also carries Claude’s co-author tag.
Available now in build b10255. Intel GPU users running quantized models with long contexts benefit most — wider KV cache support means oneDNN acceleration kicks in where it previously fell back to slower paths.