Adam Santoro, Ryan Faulkner, David Raposo, Jack Rae, Mike Chrzanowski, Théophane Weber, Daan Wierstra, Oriol Vinyals, Razvan Pascanu, Timothy Lillicrap
5 min
Standard memory-based neural networks, such as LSTMs, are effective at storing and retrieving information over time but often lack an explicit mechanism to reason about the relationships between stored entities. This paper investigates whether augmenting these architectures with a bias toward explicit memory-memory interaction improves performance on tasks that require complex relational reasoning across sequential data.
The authors introduce the Relational Memory Core (RMC), a new memory module that replaces or supplements standard hidden state updates with a multi-head dot product attention mechanism. Unlike standard attention that looks back across all previous time steps, the RMC applies attention between a fixed set of memory slots at each individual time step. This allows the model to learn how to "shuttle" information between different memory compartments, effectively enabling the network to compare and contrast stored information as it processes new inputs.
The RMC demonstrates significant improvements in tasks specifically designed to stress relational reasoning. In a custom "Nth Farthest" task—which requires calculating and sorting Euclidean distances between vectors—the RMC achieved 91% accuracy, while standard LSTM and Differentiable Neural Computer (DNC) baselines failed to surpass 30%. Furthermore, the RMC achieved state-of-the-art results on several large-scale language modeling benchmarks, including WikiText-103, Project Gutenberg, and GigaWord, and showed strong performance in partially observable reinforcement learning environments like Mini PacMan.
By providing an architectural inductive bias for relational reasoning, the RMC addresses a fundamental limitation in traditional recurrent models. This approach suggests that for tasks involving symbolic manipulation, program evaluation, or long-term temporal dependencies, the ability to explicitly relate stored memories is as critical as the capacity to store them. The RMC provides a flexible, scalable framework that can be integrated into existing recurrent architectures to enhance their reasoning capabilities.
Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods. It is unclear, however, whether they also have an ability to perform complex relational reasoning with the information they remember. Here, we first confirm our intuitions that standard memory architectures may struggle at tasks that heavily involve an understanding of the ways in which entities are connected -- i.e., tasks involving relational reasoning. We then improve upon these deficits by using a new memory module -- a \textit{Relational Memory Core} (RMC) -- which employs multi-head dot product attention to allow memories to interact. Finally, we test the RMC on a suite of tasks that may profit from more capable relational reasoning across sequential information, and show large gains in RL domains (e.g. Mini PacMan), program evaluation, and language modeling, achieving state-of-the-art results on the WikiText-103, Project Gutenberg, and GigaWord datasets.
Alex: Does this design also change how the model handles information over time — like, what it decides to hold onto versus let go of?
Sam: Yes, and that's one of the more considered design choices in the paper. The RMC borrows a concept from earlier models called "gates" — think of these as filters that decide what information to keep and what to discard. The paper introduces something called "memory gating," which applies these filters to entire rows of memory at once, rather than individual values. That gives the system finer control over what it holds onto as time passes. The model stays the same size, but gets better at deciding what's worth remembering.
Alex: So it's not about building a bigger brain — it's about teaching the existing one to have a more organised conversation with itself.
Sam: That's a good way to put it. And it points to something worth noting more broadly — sometimes the bottleneck isn't how much memory a system has, but how that memory is organised and accessed.
Alex: You mentioned the paper tested this on something called the "Nth Farthest" task. What makes that particular challenge difficult for a standard model?
Sam: It's a task that requires the model to sort relationships — specifically, to figure out which object is the Nth farthest from a reference point, across a whole set of objects. The tricky part is that the model has to compute and compare distances between many things simultaneously, and the reference point may not even be given directly. Standard models struggle here because they don't have the structure to hold all those comparisons in mind at once.
Alex: And the RMC handled it meaningfully better?
Sam: Notably better, yes. Where older models struggled to reach even thirty percent accuracy on that task, the RMC reached around ninety-one percent. It's a clear demonstration that when the architecture is designed to support relational reasoning from the ground up, the model can handle tasks that were previously out of reach.
Alex: So the key insight isn't just "more memory" — it's memory that's structured to think relationally from the start.
Sam: That's the paper's central argument. The RMC provides a backbone that allows the model to learn both how to store information and how to reason about the connections between it. It's a meaningful step toward AI systems that don't just recall facts, but understand how those facts fit together.
Alex: Thanks for walking us through that, Sam. And thanks to everyone listening to ResearchPod.