CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) has revolutionized real-time scene rendering and open-vocabulary understanding. However, current methods are largely limited to explicit, noun-based queries (e.g., "find the chair"). Practical embodied AI applications require agents to interpret implicit, context-dependent instructions that involve spatial constraints, affordances, and causal logic (e.g., "find the object to dry my hands next to the sink"). This paper introduces the task of Reasoning 3D Gaussian Segmentation to bridge the gap between low-level visual features and high-level logical inference.

To address this challenge, the authors propose CausalSplat, which decomposes the reasoning process into three distinct modules:

1. Semantic Field Construction: This module lifts 2D segmentation masks into a 3D feature field. It uses a spatial weighting strategy to suppress boundary noise and contrastive feature optimization to ensure semantic consistency, ultimately clustering 2D masks into unified 3D entities.
2. Multimodal Semantic Scene Graph Construction: The framework organizes 3D entities into a graph where nodes represent objects and edges represent spatial relationships. This structure allows the model to move beyond simple feature matching and explicitly reason about topological connections.
3. Scene Graph-based Multimodal Reasoning: By leveraging the reasoning capabilities of Large Language Models (LLMs), the framework parses complex natural language instructions into executable structured intents, performing a topological search within the scene graph to identify the target object.

The authors introduce two new benchmarks, Causal-LERF and Causal-ScanNet, which categorize reasoning into four progressive levels: commonsense, spatial, affordance, and predictive/counterfactual reasoning. Experiments demonstrate that CausalSplat significantly outperforms existing state-of-the-art methods across these benchmarks, proving its effectiveness in handling complex, multi-level logical queries in real-world 3D indoor scenes.