ERF-GS: Reconstructing Fast Motion from Disjoint Event-RGB Viewpoints

Dynamic 3D scene reconstruction using methods like 3D Gaussian Splatting (3DGS) often struggles with fast-moving objects, leading to motion blur and poor temporal fidelity when using conventional frame-based RGB cameras. This paper addresses the challenge of reconstructing high-speed dynamics by fusing sparse, high-temporal-resolution event data with standard RGB video, specifically targeting scenarios where inputs are blurry and camera viewpoints are disjoint.

The authors introduce the Event-RGB Fusion Gaussian Splatting (ERF-GS) framework. Unlike previous methods that require strict alignment between RGB and event sensors, ERF-GS is designed to function with disjoint viewpoints and does not require ground-truth RGB frames for its event-based learning components. The framework introduces two primary innovations:

1. Event-Assisted Regularized Loss (EARL): This component uses a confidence-weighted event loss to supervise the Gaussian representation between RGB frames. It incorporates trajectory and color regularization to prevent the Gaussians from "zigzagging" or experiencing color shifts when supervised by noisy, monochrome event data.
2. Event-Guided Densification (EDS): This strategy uses event activations—which naturally highlight motion—to identify regions in 3D space that lack sufficient Gaussian representation, allowing the model to dynamically add Gaussians to moving objects that would otherwise be missed by standard gradient-based densification.

ERF-GS demonstrates significant improvements over existing dynamic 3DGS baselines, such as 4DGS and E-D3DGS, when tested on modified versions of the Neu3D and Nvidia datasets. By simulating realistic motion blur and using disjoint viewpoints, the authors show that their framework achieves a higher dynamic Peak Signal-to-Noise Ratio (PSNR) of over 0.9 dB compared to RGB-only baselines. The framework successfully mitigates motion-induced artifacts, proving that event-based fusion is a robust strategy for challenging, real-world dynamic scenes.