llama.cpp build b9931 ships two OpenCL optimizations that cut wasted compute in Mixture-of-Experts prefill — by skipping padding tiles entirely.
The changes target a specific inefficiency: MoE prefill GEMM groups tokens into TILESIZE_N=32 per-expert tiles, but at low tokens-per-expert most of those tiles are padding. The release adds:
- Ragged-tile FP16 skip — when a tile’s upper 16 slots are all padding (router index
0xFFFFFFFF), the kernel skips the seconddotx16_reduce8half. Applied to all eight*_f32_nsMoE GEMMs. Default on; opt out withGGML_OPENCL_MOE_RAGGED_FP16=0. - Quarter-granularity tile-skip — replaces two
dotx16_reduce8calls with fourdotx8_reduce4(8-column) calls, skipping each empty trailing skip-group independently. Controlled byGGML_OPENCL_MOE_RAGGED_GRAN={8,16,32}.
Both paths are numerically and byte-identical to the non-skipped version — skipped lanes are padding, so no output changes. The win is pure compute reduction on OpenCL backends during prefill for MoE models like DeepSeek-V3 or Mixtral, where token distribution across experts is sparse and tiles fill unevenly.
NOTE
These optimizations are OpenCL-only. CUDA and Metal backends have their own MoE kernels and are unaffected by these env vars.
Available in llama.cpp b9931 — no migration needed, FP16 skip is on by default.