GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models

GAUGE (Measurement-Grounded Benchmark for Physical Fidelity) is a diagnostic tool designed to evaluate how well numerical simulators (physics engines) and generative video world models replicate real-world physical behavior. While existing benchmarks often rely on human perception or visual similarity, GAUGE uses real-world experimental data—including motion-capture trajectories and calibrated physical metadata—to provide a quantitative assessment of physical fidelity.

The benchmark consists of 22 controlled task families, ranging from simple rigid-body collisions to complex volumetric deformations. For physics engines, the researchers reconstructed these tasks in Isaac Sim, Genesis, and Newton, comparing simulated outputs against real-world trajectories using metrics like Root Mean Square Error (RMSE) and Dynamic Time Warping (DTW). For video world models, the team evaluated six models (including Cosmos3 and Wan-2.2) by prompting them to generate future video rollouts from an initial frame. They then used automated tracking to extract trajectories from these videos and tested them for consistency with physical laws, such as momentum transfer and oscillation periods.

The study demonstrates that no single physics engine is uniformly faithful across all regimes. Significant discrepancies appear in tasks involving impulsive contact (like Newton's cradle), rapid textile motion, and volumetric deformation. For video world models, the findings are even more stark: while these models can generate videos that look like they follow the correct "form" of a physical equation, they frequently fail to recover accurate accelerations, momentum, or timing. This suggests that current world models often prioritize visual plausibility over underlying physical correctness, which can lead to artifacts that hinder their use in reliable embodied intelligence training.