CVSBench: A Comprehensive Benchmark for Cross-view Spatial Reasoning and Dreaming

As Vision-Language Models (VLMs) are increasingly deployed in autonomous driving and urban navigation, their ability to maintain spatial consistency across drastically different viewpoints—such as transitioning from an overhead satellite view to a ground-level street view—becomes critical. This paper investigates whether current VLMs possess this cross-view spatial intelligence and explores how to improve their performance when they inevitably fail.

The CVSBench Benchmark

The authors introduce CVSBench, a large-scale, human-verified benchmark designed to test spatial reasoning across heterogeneous viewpoints. Unlike existing benchmarks that focus on indoor scenes or limited viewpoint rotations, CVSBench utilizes 3,297 satellite-street image pairs. It includes three primary task categories:

• Cross-view VQA: Models must answer questions about a scene from a viewpoint they cannot see, forcing them to infer spatial attributes based on the provided image.
• Cross-view Grounding: Models must align objects (bounding boxes) across different perspectives.
• Viewpoint Identification: Models must determine the camera's location and orientation based on visual cues.

Findings and Strategies

Extensive testing shows that state-of-the-art VLMs struggle to maintain layout and object consistency under extreme viewpoint changes. The authors tested whether language-only reasoning (Chain-of-Thought prompting) could bridge this gap, but found that textual inference yields only marginal improvements.

In contrast, the authors demonstrate that "visual imagination" is far more effective. By employing depth estimation and image generation models to synthesize intermediate 3D representations—effectively creating a "God's-eye-view" cognitive map—the models achieved substantially higher accuracy. This suggests that for robust spatial cognition, VLMs require explicit visual-spatial representations rather than relying solely on textual reasoning.