MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

How can large language models (LLMs) accumulate and reuse reasoning experience across sequential, novel problems at test time without requiring parameter updates or large-scale external training data?

The authors propose MILES, a framework that augments a frozen LLM with an external, modular memory. This memory consists of asymmetric pairs: a sub-goal embedding (acting as a retrieval key) and a sub-instruction (providing natural language guidance). MILES employs a two-layer, coarse-to-fine retrieval mechanism. First, it uses similarity-based retrieval to identify potentially relevant memory units. Second, it uses lightweight, per-item selection heads—trained on the fly using feedback from confident reasoning trajectories—to rerank these candidates based on their predicted impact on final-answer correctness. This allows the system to grow its memory incrementally as it processes new problems.

MILES consistently matches or outperforms existing test-time reasoning methods across six benchmarks and four different LLM backbones. By turning memory selection into a learned, correctness-optimized decision, the framework achieves superior accuracy-efficiency tradeoffs compared to heuristic-based retrieval methods. The study demonstrates that the system can effectively transfer learned selection patterns to uncertain samples, allowing the model to improve its reasoning performance as it accumulates more experience. Additionally, MILES learns reusable reasoning knowledge that is transferable across models and not tied to a specific LLM.

Most current LLM reasoning techniques treat each problem in isolation or rely on static, pre-computed memory. MILES offers a scalable, self-improving alternative that enables frozen models to learn from their own successes and failures in real-time. This approach is particularly valuable for practical applications where models must handle sequential, complex reasoning tasks under strict computational constraints.