On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting

Dynamic 3D Gaussian Splatting (3DGS) methods typically rely on a single deformation model to represent scene motion. However, these models often fail to generalize across diverse scenes, spatial regions, and temporal frames because each deformation formulation acts as a rigid inductive bias. This paper investigates whether combining multiple specialized deformation models—a Mixture-of-Experts (MoE) approach—can improve reconstruction robustness and fidelity.

The authors propose two strategies for multi-deformation modeling, differentiated by their integration constraints:

1. Mixture of Deformation Experts (MoDE): This approach integrates multiple experts directly into the standard 3DGS pipeline. All experts operate on a shared canonical Gaussian representation and are optimized jointly. This allows for direct 3D reconstruction without modifying the original training schedule.

2. Mixture of Experts for Dynamic Gaussian Splatting (MoE-GS): This approach decouples expert optimization. Each deformation model is trained independently in separate runs. These experts are then combined in the image-space domain using a volume-aware pixel routing mechanism, which allows for greater flexibility in expert specialization at the cost of requiring multiple training stages.

The study demonstrates that the choice of integration constraint fundamentally shapes the behavior of the dynamic representation. MoDE provides an efficient, minimally invasive way to handle diverse motions within a unified 3D structure, while MoE-GS offers superior flexibility by allowing experts to specialize in their preferred motion regimes without the constraints of a shared canonical space. By comparing these methods, the authors clarify the design space for dynamic scene reconstruction, showing that there is no single universally optimal solution, but rather a set of trade-offs between reconstruction fidelity, training stability, and computational efficiency.