Physics-Grounded Fluid Video Generation with a Simulation Dataset and Dual-Stream Optical-Flow Supervision

Video diffusion models often struggle to maintain physical consistency when generating fluids, frequently producing artifacts like liquid columns that break in mid-air or water levels that fail to rise during pouring. The authors hypothesize that this failure arises because large-scale training corpora lack explicit motion supervision, causing models to prioritize visual appearance over physical dynamics. They investigate whether injecting explicit motion-level signals into a pretrained video generator can improve physical plausibility without sacrificing its existing capabilities.

The researchers make two primary contributions. First, they construct a specialized fluid dataset consisting of 1,638 procedurally generated videos (using particle-based multiphase simulation) and 2,320 curated real-world videos. This dataset covers varied container geometries and physical parameters like tilt speed and force magnitude. Second, they propose a dual-stream architecture that adds a lightweight Optical-Flow Decoder to a frozen, pretrained diffusion-transformer backbone. This branch is trained to predict pixel-level motion, with its features fused into the RGB stream via zero-initialized convolutions. This design allows the model to learn physically consistent motion fields while preserving the pretrained model's strong appearance and text-alignment priors.

Experimental results across 1.3B and 14B parameter model scales demonstrate that the dual-stream approach consistently outperforms the frozen backbone. On the iStock-Fluid benchmark, the 14B model improved Physical-Commonsense scores by 8.75 points and Video-Quality scores by 4.65 points. Human evaluations and direct optical-flow accuracy probes confirm that the model internalizes a coherent motion prior, resulting in fluid behavior that better respects gravity, momentum, and mass conservation compared to standard generative baselines.