Long-form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating videos shot by shot. However, they mainly focus on optimizing plausible next-shot continuations without verifying whether the historical memory preserves identity-critical subject evidence. Consequently, as generation proceeds, recurring subjects may be diluted, overwritten, or forgotten. In this paper, we propose Memento, a subject-reconstruction-guided framework that treats subject preservation as an explicit identity grounding problem, based on the premise that a memory bank faithfully preserving a subject should support reconstructing that subject from memory alone. Specifically, Memento jointly trains autoregressive next-shot generation with memory-based subject reconstruction, recovering target appearances using historical memory and global story captions. To disentangle long-range subject evidence from short-range cues, Memento introduces a dual-query memory mechanism, where one query retrieves identity-relevant memory and the other selects short-context keyframes for coherent continuation. Additionally, a subject-aware cinematic data pipeline provides precise reconstruction supervision via consistent, pronoun-free subject descriptions. Experiments demonstrate that Memento achieves state-of-the-art performance in long-term subject consistency, cross-shot coherence, and visual quality.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a persistent problem in AI video generation: why do AI-created characters often seem to forget what they look like?
Sam: I've actually seen this. A person walks through a door in one shot, and on the other side they look like a completely different individual. Is this paper trying to solve that?
Alex: Exactly. The researchers call it "subject drift." As a video progresses, the AI gradually loses track of a character's face, clothing, or hair — and the illusion of a continuous story falls apart.
Sam: So the AI is essentially forgetting who it's supposed to be filming, scene by scene. How does this new framework — Memento — actually prevent that?
Alex: The core idea is to treat identity as a test of memory, not just a visual guess. Think of it like a student who has to summarize a book from memory to prove they actually read it, rather than just skimming the back cover. The AI can't just produce something that looks plausible — it has to demonstrate it genuinely remembers the character.
Sam: So instead of guessing what the next shot should look like, the AI has to actively rebuild the character's appearance from its own stored memory?
Alex: That's the key mechanism. The model is forced to reconstruct the subject using only what it has stored internally, combined with a high-level description of the overall story. If it can't rebuild the character accurately, that's a signal it failed to store the identity properly in the first place.
Sam: That does sound like a stricter standard. But if the AI is spending effort reconstructing the character, does that come at the cost of handling the actual motion or the new scene it's supposed to be generating?
Alex: That's a fair concern, and the authors address it directly. They introduce what they call a dual-query memory mechanism. Here's a useful way to think about it: imagine you're at a crowded party trying to keep track of a friend. You need to hold their face clearly in your mind — their hair, their jacket — but you also need to track where they are in the room right now, whether they're near the door or by the snacks. Those are two genuinely different jobs.
Sam: So you're doing two kinds of remembering at once. One is about who the person is, and the other is about what's happening around them.
Alex: Exactly. The system runs two separate memory queries simultaneously. One pulls from the full history of the video — long-range details like the shape of someone's nose or a distinctive jacket — to keep the character's identity stable. The other focuses on recent context, like the last few camera angles or the current scene layout, to keep the action coherent.
Sam: So it's deliberately splitting those two jobs apart. Without that split, what was going wrong?
Alex: The model was letting the most recent, visually dominant details crowd out the identity information. If a character stands in front of a bright red wall, the redness of that wall could effectively overwrite the memory of their face. By separating the queries, the system protects the identity channel from being drowned out by whatever happens to be visually loud in the current shot.
Sam: That's a subtle but important distinction. It's not that the AI was ignoring the character — it's that the wrong information kept winning the competition for memory space.
Alex: That's a good way to frame it. And to make the training work, they also built a data pipeline using precise, pronoun-free descriptions of each character. So instead of "he walked to the door," the description names the character explicitly every time. That removes ambiguity and ensures the model always knows exactly which individual it's supposed to be tracking.
Sam: Are there limits to what this approach can handle? If the model generates a bad shot and saves it to memory, does that error carry forward?
Alex: That's a genuine limitation the paper notes. Because the model generates video sequentially — one segment at a time — a poorly generated shot can get stored in memory and then propagate forward, compounding the problem. The system also still struggles with physical plausibility: things like realistic gravity or how objects interact in space remain difficult.
Sam: So it's a meaningful step forward on the identity problem, but not a complete solution to everything that makes long-form AI video hard.
Alex: That's an accurate read. The contribution here is focused and specific: by reframing identity as something the model must actively reconstruct rather than passively guess, and by giving it a structured way to separate long-term identity from short-term context, Memento addresses one of the more persistent failure modes in this space. Whether that holds up across more varied and demanding scenarios is something further research will need to establish.
Sam: It's an interesting shift in thinking — from "make it look right" to "prove you remember." I'll be curious to see how far that principle extends.
Alex: Thanks for listening to ResearchPod.