llama.cpp build b10285 adds multi-row batching for DeepSeek OCR — processing image rows in a single pass instead of one at a time.
Two PRs landed in this build:
- PR #26154 — introduces multi-row batching support for deepseek-ocr in the mtmd (multimodal) pipeline
- PR #26615 — weaves deepseek-ocr rows in one shot rather than per-row, eliminating sequential dispatch overhead
- Co-authored by Saba Fallah
The mtmd pipeline handles vision-language model inference in llama.cpp. For OCR workloads where a document image gets split into rows, batching rows together cuts the per-row dispatch overhead that compounds with sequential processing. The “weave in one shot” optimization in #26615 builds directly on the batching foundation from #26154 — it’s a two-step change, not a single patch.
NOTE
The macOS Apple Silicon build with KleidiAI is marked DISABLED in this release. Users on that configuration should use the standard arm64 build.
Pre-built binaries cover 25+ platform configurations: CUDA 12 (12.4 DLLs), CUDA 13 (13.3 DLLs), ROCm 7.2, Vulkan, SYCL FP32/FP16, OpenVINO, and HIP on the GPU side. CPU builds span Ubuntu x64/arm64/s390x, Windows x64/arm64, Android arm64, and openEuler x86/aarch64.
Available now — full release notes and binaries on GitHub.