The eval scores are green, and an hour ago the model still handed a confidently wrong answer to a real user. That gap — scoring responses after they’re served — is what almost no LLM observability tool closes. (source)
Five tools got compared on two questions that cut through the marketing: does it block a bad generation before the user sees it, and can you self-host without an enterprise contract.
- LangSmith — Strong tracing, real OTLP endpoint, solid datasets. Self-hosting is Enterprise-plan only. Prompts and outputs run through their cloud unless you pay for the in-house tier.
- Langfuse — MIT-licensed core. The one you can self-host without a sales call.
- Phoenix, Braintrust, Galileo — All score post-response. Each has a different catch around self-hosting and data control; the original comparison breaks down each one.
None of the five block a bad generation before it reaches the user. They’re regression-test engines, not guardrails.
NOTE
The missing layer is preventive evaluation — scoring a generation before serving it. Every tool here runs post-response. The best you get is turning today’s failure into tomorrow’s test case.
If you need to stop bad answers in the moment, none of these tools do that. If you need self-hosted evals without an enterprise contract, Langfuse is the only one with an MIT core. The rest route your data through their cloud or behind a paywall.