Xuan Wei, Longbin Ji, Guan Wang, Xiangrui Liu, Zhenyu Zhang, Shuohuan Wang, Yu Sun, Qingqi Hong
4 min
Abstract
Long-form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating videos shot by shot. However, they mainly focus on optimizing plausible next-shot continuations without verifying whether the historical memory preserves identity-critical subject evidence. Consequently, as generation proceeds, recurring subjects may be diluted, overwritten, or forgotten. In this paper, we propose Memento, a subject-reconstruction-guided framework that treats subject preservation as an explicit identity grounding problem, based on the premise that a memory bank faithfully preserving a subject should support reconstructing that subject from memory alone. Specifically, Memento jointly trains autoregressive next-shot generation with memory-based subject reconstruction, recovering target appearances using historical memory and global story captions. To disentangle long-range subject evidence from short-range cues, Memento introduces a dual-query memory mechanism, where one query retrieves identity-relevant memory and the other selects short-context keyframes for coherent continuation. Additionally, a subject-aware cinematic data pipeline provides precise reconstruction supervision via consistent, pronoun-free subject descriptions. Experiments demonstrate that Memento achieves state-of-the-art performance in long-term subject consistency, cross-shot coherence, and visual quality.
Sam: So it's deliberately splitting those two jobs apart. Without that split, what was going wrong?
Alex: The model was letting the most recent, visually dominant details crowd out the identity information. If a character stands in front of a bright red wall, the redness of that wall could effectively overwrite the memory of their face. By separating the queries, the system protects the identity channel from being drowned out by whatever happens to be visually loud in the current shot.
Sam: That's a subtle but important distinction. It's not that the AI was ignoring the character — it's that the wrong information kept winning the competition for memory space.
Alex: That's a good way to frame it. And to make the training work, they also built a data pipeline using precise, pronoun-free descriptions of each character. So instead of "he walked to the door," the description names the character explicitly every time. That removes ambiguity and ensures the model always knows exactly which individual it's supposed to be tracking.
Sam: Are there limits to what this approach can handle? If the model generates a bad shot and saves it to memory, does that error carry forward?
Alex: That's a genuine limitation the paper notes. Because the model generates video sequentially — one segment at a time — a poorly generated shot can get stored in memory and then propagate forward, compounding the problem. The system also still struggles with physical plausibility: things like realistic gravity or how objects interact in space remain difficult.
Sam: So it's a meaningful step forward on the identity problem, but not a complete solution to everything that makes long-form AI video hard.
Alex: That's an accurate read. The contribution here is focused and specific: by reframing identity as something the model must actively reconstruct rather than passively guess, and by giving it a structured way to separate long-term identity from short-term context, Memento addresses one of the more persistent failure modes in this space. Whether that holds up across more varied and demanding scenarios is something further research will need to establish.
Sam: It's an interesting shift in thinking — from "make it look right" to "prove you remember." I'll be curious to see how far that principle extends.
Alex: Thanks for listening to ResearchPod.