Skip to content

llama.cpp b10328: Docker tool isolation for local LLM servers

llama.cpp b10328 adds Docker-based tool isolation to its server, letting local LLM function calls run in containers instead of on your host machine.

llama.cpp b10328 adds Docker-based tool isolation to its server — tool calls from local LLMs can now run inside containers instead of on your host machine.

The changes in PR #26507:

  • Tool isolation via Docker — tool calls from the llama.cpp server can execute inside a container
  • Naming evolved during development — sandbox → isolate, x-tool-docker → x-tool-runtime
  • get_info endpoint adapted — server info now reports isolation configuration
  • Docs included — setup documentation ships with the PR

Local LLM function calling has a trust problem. When a model calls a tool — execute code, read a file, hit an API — that tool runs with your user’s permissions by default. Docker isolation puts a container boundary between the model’s tool calls and your filesystem. The “initial” label in the PR title means this is a first pass, not a hardened sandbox.

NOTE

The rename from sandbox to isolate during development signals the maintainers aren’t claiming full sandbox guarantees. Docker containers provide process isolation, not a security boundary equivalent to gVisor or Firecracker.

Available in b10328 now. The x-tool-runtime option controls which runtime handles tool execution — Docker must be installed on the host.