llama.cpp b9938 gives AMD ROCm users the fast-math optimizations CUDA has had all along.
The release ships one change that matters for AMD GPU users:
-funsafe-math-optimizationsenabled for ggml-hip (PR #24668), matching CUDA’s existing fast-math compilation path- Co-authored by Mark Caldwell of CloudHands AI
- Prebuilt binaries include Ubuntu x64 ROCm 7.2 and Windows x64 HIP among 17+ platform targets
| Backend | Fast Math Status | NaN Risk |
|---|---|---|
| CUDA | Already enabled | None |
| HIP (before b9938) | Not enabled | — |
| HIP (b9938+) | Enabled via -funsafe-math-optimizations | Avoided |
CUDA binaries in llama.cpp compile with fast math. AMD’s HIP backend didn’t. This release closes that gap — but deliberately avoids -ffast-math, the blanket flag that implies -ffinite-math-only. That breaks ggml’s compilation because the library uses INFINITY for masking operations, and finite-math mode produces NaNs in those code paths.
-funsafe-math-optimizations targets the same performance gains without the correctness landmine.
NOTE
Building from source on AMD ROCm? The flag is now enabled automatically in CMake for HIP targets — no build command changes required.
Available now in release b9938. Ubuntu ROCm 7.2 and Windows HIP prebuilts are the relevant downloads for AMD GPU users.