Reasoning has become a core capability for large models, especially when reliable decisions require understanding logical consequences. Recent video generation models offer a reasoning path distinct from previous Chain-of-Thought (CoT): reasoning can unfold through temporally connected frames, known as Chain-of-Frame (CoF) reasoning. However, existing video generators are primarily trained on general video corpora, still lacking diverse supervision and dedicated designs for CoF reasoning. To address this gap, we introduce OpenCoF, a framework comprising the OpenCoF-17K dataset, a reasoning video dataset spanning 11 task families, and Wan-CoF, a fine-tuned video model for studying whether diverse temporal supervision improves CoF behavior. Across four video reasoning benchmarks, Wan-CoF achieves considerable gains over the Wan2.2-I2V-A14B baseline. Building on this, we empirically explore more advanced designs for CoF capabilities, i.e., equipping the model with visual and textual reasoning tokens. This mechanism respectively captures low-level visual cues and high-level semantic priors for spatial and temporal reasoning. Through performance comparisons and attention analysis, we examine how these tokens contribute across model depth, denoising steps, space, and time. Our results suggest that stronger video reasoning requires both broad temporal supervision and explicit mechanisms for organizing intermediate reasoning state. We open-source the dataset, model, and code to facilitate future research on reasoning-oriented video generation.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a framework called OpenCoF, which is trying to change how AI models "think" while they generate videos.
Sam: That's right. Modern AI can produce remarkably realistic video footage, but it often fails at something more fundamental: following the logical steps of a physical task. Think about folding a piece of cloth. Each fold changes the shape, and the next fold has to account for what just happened. Current AI models tend to skip that chain of reasoning and just guess at a plausible-looking result. This paper argues that instead of guessing, models should work through the process frame by frame—a process the researchers call Chain-of-Frame reasoning.
Alex: So it's like the difference between a student who shows their work on a maths problem versus one who just writes down an answer and hopes for the best?
Sam: Exactly. And the problem with just guessing the answer is that the video might look fine on the surface but violate basic physical rules halfway through. To address this, the researchers built a dataset of over seventeen thousand videos, each one carefully constructed to show a task unfolding step by step. They then took a standard video model and trained it further on this data to see whether it could learn more reliable, sequential reasoning.
Alex: So the dataset itself is doing a lot of the heavy lifting here. How did they actually build it? Were they just downloading videos from the internet?
Sam: Not quite. For a lot of the dataset, they used something closer to a physics engine—the kind of simulation software you'd find in a video game. By simulating how a ball rolls down a slope, or how puzzle pieces fit together, they could guarantee that every training example had a perfectly logical, step-by-step progression. They also pulled in existing datasets from robotics and navigation research to cover messier, real-world scenarios like moving objects or navigating a maze.
Alex: So they combined clean, engineered simulation data with real, imperfect data to cover the full range of what the model might encounter.
Sam: Precisely. And that combination matters, because a model trained only on perfect simulations tends to struggle when things get untidy in the real world.
Alex: That makes sense. But building a better dataset is only half the story. How does the model actually use that data to keep its reasoning on track while it's generating a video?
Sam: This is where the core technical idea comes in. The researchers introduced what they call "reasoning tokens." Imagine you're drawing a detailed diagram. You might jot small reference notes in the corner of your page—coordinates, proportions, reminders about what goes where. These tokens work the same way. They are small, learnable pieces of information that the model inserts into its own internal workspace as it works, so it can refer back to them at any point.
Alex: So the model is building its own mental scratchpad as it goes.
Sam: Right. And there are two distinct types. Visual tokens act like a focused spotlight on specific parts of the scene—tracking, say, exactly where a robotic gripper is positioned so the next frame stays physically consistent. Textual tokens are more like a standing reminder of the overall goal. They stay active throughout the whole process, making sure the model doesn't drift away from the original instruction, like "fold the left corner."
Alex: So it's a division of labour. One type manages the fine physical details, the other keeps the big-picture plan in view.
Sam: That's a good way to put it. And the researchers found that this specialisation actually shows up in the data. When they measured how much the model relied on each type of token at different stages, the visual tokens were most active early on, when the overall structure of the video was still being established—like sketching an outline before filling in the details. The textual tokens remained more consistently active throughout, providing a steady background guide.
Alex: That's a meaningful distinction. Does the model actually get better at the reasoning itself, or does it just produce smoother-looking video?
Sam: The evidence points to genuine reasoning improvement. When they compared their model against standard ones on established benchmarks, the biggest gains appeared in categories measuring things like temporal consistency—whether the physics of a scene hold up over time—rather than in raw visual quality. The model is getting better at following the rules of the task, not just polishing the pixels.
Alex: So the improvement is in the "how," not just the "what." That's a significant distinction.
Sam: It is. And it's worth noting how they achieved this without rebuilding the model from scratch. They used a technique called Low-Rank Adaptation—essentially a method for adjusting only a small, targeted portion of a large model rather than retraining the entire thing. It keeps the approach practical, because you're not paying the enormous computational cost of starting over.
Alex: So they get the benefit of better reasoning by making a relatively small, surgical change to an existing model.
Sam: Exactly. Which is part of what makes this approach worth paying attention to. The field has spent a lot of energy on making models bigger. This work suggests that sometimes, giving a model a better way to organise its own thinking can be just as productive.
Alex: Are there clear limits to what this approach can do right now?
Sam: The researchers were candid about one specific gap. They studied the visual and textual tokens separately throughout this work. We don't yet know how they interact when both are active at the same time. It's possible that combining them creates a more robust system. It's also possible they interfere with each other in ways that are hard to predict. That remains an open question.
Alex: So it's like having two specialists who've each been tested independently, but haven't yet been asked to collaborate on the same problem.
Sam: That's a fair analogy. And resolving that question matters for the longer-term goal, which is building systems capable of what researchers call long-horizon planning—tasks that unfold over many steps, like a robot assembling a complex device or a simulation tracking how a fluid moves through a system over time. A combined token approach could, in principle, manage both the high-level strategy and the precise physical details simultaneously. But that work is still ahead.
Alex: It's a useful reminder that progress in AI isn't always about raw scale. Sometimes it's about giving a system better tools for organising what it already knows.
Sam: That's the underlying point of this paper. High visual quality and reliable reasoning are not the same thing, and for a long time the field has conflated them. OpenCoF is a clear step toward separating those two goals and addressing them deliberately. There's more work to do, but the direction is a meaningful one.
Alex: Thanks for walking us through it, Sam. And thanks to everyone listening to ResearchPod.