Elastic’s search labs team shipped a persistent agent memory architecture that hits 0.89 recall on retrieval benchmarks with zero cross-tenant data leaks. The system runs on three Elasticsearch indices and combines hybrid search with a reranker — no external vector database required.
The architecture stacks several techniques that are individually familiar but rarely combined in a single memory layer:
- Three-index design — separates raw conversation logs, extracted facts, and session metadata into distinct indices, each optimized for its access pattern
- Hybrid retrieval with RRF — combines BM25 keyword search and vector similarity using Reciprocal Rank Fusion, then passes results through a reranker for final ordering
- Supersession and decay — newer facts can override older ones, and memory relevance decays over time so stale context doesn’t pollute agent responses
- Per-user DLS isolation — Elasticsearch’s document-level security enforces strict tenant boundaries at the query layer, not the application layer
The 0.89 recall number matters because agent memory is a retrieval problem with unusually high stakes. When an agent forgets a user preference or pulls in another tenant’s data, the failure is visible and immediate — not buried in a search results page. Getting recall wrong here means the agent contradicts itself or leaks private information.
This approach sidesteps the dedicated vector database entirely. Elasticsearch already handles BM25, dense vectors, and document-level security natively, so the team avoided stitching together Pinecone or Weaviate with a separate access control layer. That’s a meaningful simplification for teams already running Elastic infrastructure.
NOTE
The post doesn’t specify which embedding model or reranker they used, or whether the 0.89 recall was measured on a public benchmark or internal evaluation set. The number is directional, not independently reproducible from the published details.
The practical signal here: if your stack already includes Elasticsearch, you can build a multi-tenant agent memory layer without adding a vector DB — and the recall is competitive with purpose-built alternatives. The full architecture walkthrough covers index mappings, RRF configuration, and DLS setup in detail.