Jev-Mem Puts System-One Control at the Center of Agent Memory

Introduction

Long-horizon AI agents need more than a context window. They must retain user preferences, past events, and relationships among pieces of information across interactions. Yet many agentic memory systems place an autoregressive language model in charge of nearly every memory operation, from writing and organizing memories to retrieving them. This makes generation part of the critical path even when the decision is routine, increasing both cost and latency.

Jev-Mem proposes a different division of labor. Inspired by the distinction between System One and System Two cognition, it assigns frequent, lightweight memory decisions to a dedicated controller and reserves deeper model reasoning for cases that genuinely require it.

A three-plane design

The architecture consists of three cooperating components:

• System-One control plane: During construction, it determines memory types and organizes relations among memories. During retrieval, it performs query routing, allocates a retrieval budget, traverses the memory graph, scores candidates, and decides when to stop.
• Multi-relational memory plane: Instead of treating memories as isolated text chunks, it stores them in a structured representation that captures multiple relations and supports associative retrieval.
• System-Two reasoning plane: It is invoked for complex reasoning and answer synthesis rather than for every routine memory-management step.

The important idea is not to remove the language model from memory. It is to change where the model is needed. A faster control layer handles repetitive decisions, while the reasoning layer focuses on interpreting retrieved information and producing an answer. In principle, this can prevent expensive generation from becoming the default mechanism for every memory action.

Reported results

On the LoCoMo benchmark, Jev-Mem achieves an overall LLM-as-a-Judge score of 0.777, an 11.0% relative improvement over the strongest baseline. Its memory-construction time is 158 seconds, which the paper summary describes as a 6.6x speedup over the fastest competing memory system. Average query latency is 0.93 seconds, a reported reduction of 36.7%.

These figures point to an important optimization direction for agent memory: efficiency can come not only from using a stronger generator, but also from limiting unnecessary generation and making retrieval control more explicit. Jev-Mem treats memory as a dynamic system with types, relations, budgets, traversal decisions, and stopping criteria rather than as a simple vector lookup layer.

The approach may be relevant to long-running assistants, research agents, and workflow systems whose memory grows over time. At the same time, the supplied material reports results primarily on LoCoMo, so it does not establish how the architecture behaves across domains, models, or substantially larger memory stores. The authors have released code and an interactive Hugging Face Space, providing a basis for further reproduction and evaluation.

Source: Hugging Face Daily Papers