Hyangsuk Min, Hwanjun Song
4 min
Users of modern platforms repeatedly need summaries of recent dialogue, but the window rarely contains enough context to be interpreted on its own. We formalize this setting as streaming dialogue summarization, where a system must summarize a current window using selective memory from an unbounded history under a fixed budget. We show that the central challenge is not how much history is accessed, but whether memory recovers the evidence that the current window presupposes. We construct a benchmark and evaluation protocol that separately assesses whether memory contains gap-resolving evidence and whether the generated summary reflects it. We propose ReMEMBER, a missing-evidence memory framework that conditions retrieval on unresolved window dependencies and refines retrieved chunks into evidence-dense memory under a fixed budget. Experiments on dialogues with histories up to 160K tokens show that ReMEMBER improves memory recall and gap-resolution completeness over memory construction baselines under the same budget.
Modern communication platforms generate continuous streams of dialogue where users frequently need summaries of recent segments. However, these segments are rarely self-contained; they often contain pronouns, entity references, or rationales that depend on earlier parts of the conversation. Existing summarization methods either compress the entire history (losing fine-grained detail) or use standard retrieval (which often fetches irrelevant context or fails to resolve specific dependencies). This paper defines this as the streaming dialogue summarization task, where the core challenge is maintaining a memory that provides the specific evidence needed to resolve contextual gaps under a strict token budget.
To address these limitations, the authors propose ReMEMBER (Retrieving Missing Evidence Memory By Elicited Recall). The framework operates in two distinct stages:
Gap-Conditioned Evidence Retrieval: Instead of using the entire current window as a query, the system identifies specific 'gap-bearing' utterances—those that are summary-worthy but contextually incomplete. It then constructs targeted queries that explicitly define the type of missing evidence required, using both sparse and dense retrieval to find relevant historical chunks.
Gap-Conditioned Chunk Refinement: Retrieved chunks often contain noise. ReMEMBER decomposes these chunks into individual utterances and scores them based on their relevance to the specific gap query. It then retains only the most evidence-dense turns, ensuring that the final memory module is optimized for resolving dependencies rather than just storing history.
By shifting the focus from similarity-based retrieval to evidence-based dependency resolution, ReMEMBER significantly improves the quality of summaries in long-context scenarios (up to 160K tokens). This approach demonstrates that for streaming applications, the effectiveness of a memory system is determined by its ability to recover the specific 'missing pieces' of a conversation, rather than the total volume of history accessed. The authors also provide a new benchmark and evaluation protocol that separates memory recall from summary generation, offering a more precise way to measure how well systems handle long-term dialogue dependencies.
Sam: Once it finds relevant chunks of earlier conversation, how does it decide what to actually keep? I'd imagine even a targeted search can return more than you need.
Alex: Right, and that's where the second mechanism comes in. The system takes whatever it retrieved and breaks it down into individual conversational turns. It then re-ranks those turns by how directly they address the specific gap—and only keeps the most essential ones. The researchers call this "gap-conditioned chunk refinement." The goal is what they describe as high "evidence density"—every piece of stored memory should be actively doing work to clarify the current conversation, not just sitting there taking up space.
Sam: So the memory is small but precise, rather than large and approximate.
Alex: Exactly. And according to the paper, that precision is what allows the system to maintain reliable recall even in very long conversations—the kind that might span the equivalent of a full novel's worth of text.
Sam: It's a meaningful shift in how we think about AI memory. Rather than brute-force storage, it's more like targeted recall—the way a good detective doesn't read every document in the archive, but knows exactly which one to pull.
Alex: That's a fair characterisation. The paper suggests this approach offers a more reliable way to connect what's being said right now with what was established much earlier—which is, when you think about it, exactly what any good summariser, human or machine, needs to do. Thanks for listening to ResearchPod.