Skip to content

llama.cpp b9979: fixes silent prompt truncation from embedded NUL bytes

llama.cpp b9979 fixes a silent prompt truncation bug where embedded NUL bytes dropped messages with no log. PR #25548 adds text_len to mtmd_input_text.

A NUL byte embedded in message content silently truncated prompts in llama.cpp’s multimodal input path — dropping every subsequent message and the assistant marker, with no log entry.

The bug lived in mtmd_input_text, which carried the prompt as a bare const char* with no length parameter. When the tokenizer hit a NUL byte, it stopped. Everything after — later messages, the assistant marker — was gone.

  • Root cause: mtmd_input_text used const char* without a length field, so C-string semantics cut the prompt at the first embedded NUL
  • Impact: Silent truncation at the tokenizer boundary — no warning, no error, just a shorter prompt fed to the model
  • Fix: PR #25548 adds an explicit text_len threaded through the pipeline, matching llama_tokenize and the text-only path
  • Co-author: Xuan Son Nguyen (Hugging Face)

For multimodal workflows where message content might carry binary-adjacent data, this bug could produce degraded or nonsensical outputs with zero indication of why. The model would respond to a shorter, incomplete conversation and you’d have no way to tell from the logs.

NOTE

The macOS Apple Silicon KleidiAI-enabled build is disabled in b9979. All other artifacts ship normally — including CUDA 12.4, CUDA 13.3, Vulkan, ROCm 7.2, SYCL FP32/FP16, and OpenVINO variants.

Available now in the b9979 release. If you’re running multimodal inference with user-generated content, update — the truncation won’t surface in your logs.