Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents

LLM agents often rely on external memory systems to store and retrieve experiences across tasks. However, most existing systems use fixed, hand-designed heuristics for memory access (e.g., always retrieving the top-k items). The authors argue that this static approach is a fundamental bottleneck because optimal memory behavior is highly context-dependent—varying based on task progress, memory density, and whether the agent is stuck or making progress. They ask: can we learn an adaptive, lightweight controller that manages memory operations without the overhead of additional LLM calls?

MemCon (Memory as a Controlled Process) models memory management as a Markov Decision Process (MDP). It acts as a thin, backend-agnostic wrapper around any existing memory system. Instead of using fixed rules, MemCon uses a tabular contextual bandit with Upper Confidence Bound (UCB) exploration to learn an online policy. This policy decides on actions such as retrieving, re-retrieving, injecting distilled plans, consolidating, or forgetting. The system learns from task-by-task binary feedback (success/failure) and uses reverse-discounted credit assignment to update its policy. Crucially, MemCon requires no pretraining and performs all control decisions via millisecond-scale table lookups, avoiding the need for secondary LLM calls.

The authors evaluated MemCon across six benchmarks (including ALFWorld, PDDL, and GAIA), three agent frameworks, and three LLM backbones. MemCon consistently outperformed nine strong memory baselines, achieving up to 15.2 points in higher task success rates. Furthermore, by adaptively pruning and selecting relevant information, MemCon reduced token consumption by 5–20% compared to static baselines. The results demonstrate that a learned controller can effectively navigate the trade-off between retrieving too much (which dilutes the context) and too little (which misses critical information).