Pose-driven human animation synthesizes a video of a target person from a single reference image and a driving pose stream. Real-time generation is essential for interactive applications such as live streaming, telepresence, and virtual avatars, yet diffusion-based systems require minutes to hours per clip, precluding responsive interaction. We present LiveAnimate, to our knowledge the first animation system to combine real-time streaming with stable long-form generation at billion scale, built on a 14B-parameter video Diffusion Transformer (DiT). A two-stage training pipeline first adapts a pretrained bidirectional DiT into a block-causal autoregressive generator through Reference-Anchored Teacher-Forcing Adaptation, and then reduces the sampling budget to three steps through Block-wise Self-Forcing Distillation. To preserve appearance over extended streams, we introduce Pose-Retrieval Sink Attention (PR-Sink), a bounded KV-cache mechanism combining a Static Sink that permanently anchors the first generated block, a Dynamic Sink that holds a pose-retrieved historical block, and a three-slot Rolling Window. When a pose recurs, PR-Sink restores the relevant appearance context without retaining the entire sequence, so memory and per-block latency remain constant regardless of stream duration. Together with Ulysses sequence parallelism and operator fusion, these designs enable 19.63\,FPS streaming inference on two NVIDIA H100 GPUs. On a three-minute benchmark, LiveAnimate maintains nearly constant perceptual quality and identity from the first 30 seconds to the final minute, while prior systems degrade substantially or require hours of offline computation for the same rollout. These results establish a new operating point in quality, latency, and duration for interactive full-body animation.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a paper called "LiveAnimate" — a system designed to animate a person in real time using just a single photograph.
Sam: So the goal is to take one photo of someone and generate a live, moving avatar of them — without it stuttering or slowly forgetting what they look like?
Alex: That's exactly it. And the reason that's difficult comes down to how these AI models are normally built. Most animation models are what researchers call "offline" — they need to see the entire video sequence before they can produce anything. That process can take minutes or even hours for a short clip.
Sam: Which means they're completely useless for a live stream, where the future frames simply don't exist yet.
Alex: Right. So the first thing the LiveAnimate team had to solve was the architecture itself — how the model reads and processes information. Instead of consuming the whole video at once, they shifted to what they call a "block-causal" approach. Think of it like reading a novel one chapter at a time, rather than trying to memorize the whole book before you start. The model processes a small chunk of frames, then moves on to the next, carrying only a summary of what came before.
Sam: That makes sense for speed. But here's what I'm not sure about — if it's only looking at recent chunks, how does it remember what the original person actually looks like? Couldn't the avatar slowly drift and start looking like someone else?
Alex: That is precisely the central problem, and it has a name: identity drift. If the model loses its grip on the reference image — the original photograph — the avatar's face can gradually shift, its clothing can change colour, details start to blur. Over a long stream, it can become unrecognisable.
Sam: So how do they prevent that?
Alex: They use two things working together. The first is something called a "bounded KV-cache." KV stands for "key-value," but don't worry about the technical name — the concept is simpler. Imagine a small, fixed-size notebook that the model carries with it. It can only hold a certain number of entries, so it has to be selective about what it writes down. Crucially, the size of that notebook stays the same whether the stream has been running for ten seconds or ten hours. Memory usage never grows out of control.
Sam: That's a neat constraint. But it raises an obvious question — how does the system decide what's worth writing in that notebook and what gets discarded?
Alex: That's where the second mechanism comes in, and it's the more interesting one. They call it "Pose-Retrieval Sink Attention." Here's the intuition: imagine a librarian with two jobs. The first job is to keep one permanent reference book always open on the desk — that's the original photograph, always available, never swapped out. The second job is to watch what the person is doing and, when they strike a familiar pose, go into the archive and pull the relevant file from a previous moment when they held that same pose.
Sam: Oh — so if the avatar raises its right arm, the system doesn't just guess what that should look like. It actually retrieves a memory from an earlier moment in the stream when the arm was in a similar position, and uses that as a guide.
Alex: Exactly. The pose acts as a search query. The model finds the closest historical match, retrieves the appearance information from that moment, and uses it to stay consistent. It's a way of having a long memory without actually storing everything.
Sam: That's a genuinely clever workaround. But I want to ask about speed, because this is still a very large model we're talking about. How do they get it running fast enough to feel live?
Alex: They use a two-stage training process. In the first stage, they retrain the model to work within this new block-by-block structure — essentially teaching it the new rules of the road. In the second stage, they apply a technique called distillation. Think of distillation like this: the original model takes many careful steps to produce each frame, the way a painter might layer dozens of brushstrokes. Distillation trains a leaner version of the model to reach a very similar result in far fewer steps — more like a skilled sketch artist who captures the essence quickly.
Sam: So the quality doesn't collapse, it just learns to get there more efficiently.
Alex: That's the intent. The paper reports the system can generate video at around twenty frames per second — which is fast enough to feel continuous and live to a viewer — while keeping the visual quality close to what the original, much slower model would have produced.
Sam: What strikes me about all of this is that none of these solutions are simple shortcuts. The bounded cache, the pose retrieval, the distillation — each one is solving a specific, concrete problem that would otherwise make real-time animation impossible.
Alex: That's a fair summary. What LiveAnimate represents is a careful set of engineering trade-offs: you give up the luxury of seeing the whole video at once, and in return you gain the ability to run indefinitely, in real time, from a single image. Whether that trade-off holds up across a wide range of real-world conditions is something further research will need to examine — but as a proof of concept, the approach is coherent and the reasoning behind each design choice is clear.
Sam: Something to watch as the technology develops.
Alex: Agreed. Thanks for listening to ResearchPod.