llama.cpp b9951 ships an initial ET backend for ggml — the C tensor library powering local LLM inference across llama.cpp, whisper.cpp, and related runtimes. PR #24179 adds 11 compute kernels, performance logging, and a simplified build path.
The kernels landing in this release:
- MUL_MAT (including F32xF32 variant) — core matrix multiplication for transformer inference
- MUL_MAT_ID — grouped matmul for expert routing in MoE models
- ROPE — rotary positional embeddings
- RMS_NORM — normalization layer used in LLaMA-family architectures
- GLU — gated linear unit activation
- SOFT_MAX — attention weight computation
- GET_ROWS / SET_ROWS — embedding table lookup and writeback
- CONT / CONT_F16 — continuity / type conversion ops
- RESHAPE / TRANSPOSE — tensor shape manipulation
The backend embeds kernels directly with a filesystem fallback, and the runtime can now be imported via a standard CMake find_package() call — previous versions required import hacks that the PR explicitly removes.
Two kernel fixes shipped alongside the initial implementation: GET_ROWS lost a batch dimension that’s now restored, and SET_ROWS had an incorrect broadcasting guard removed. The PR also refactors block operations out into separate functions and disables offloading in debug builds.
NOTE
The release notes do not specify what hardware or platform “ET” targets. ggml already supports CUDA, Metal, Vulkan, SYCL, and CPU backends — ET appears to be a new addition to that registry, but the acronym is not expanded in the changelog.
Available now in llama.cpp build b9951. If you’re running local models on unsupported hardware, the ET backend may be worth probing — but check the supported ops doc in the PR before expecting full model coverage.