RainDancer: RGB–Event Video Deraining with Rain-Oriented Spiking Dynamics

Video deraining is essential for reliable visual perception in adverse weather, yet traditional RGB-only methods struggle to distinguish between rain streaks and fine background textures, especially in dynamic scenes. While event cameras provide high-temporal-resolution motion cues that can help identify rain, directly fusing RGB and event data often introduces cross-modal interference. This paper asks: how can we effectively leverage event-based motion cues for deraining without corrupting the background structure with event-based noise?

The authors propose RainDancer, a progressive framework based on a "decompose-before-interact" paradigm. Instead of fusing raw RGB and event features, the model first separates each modality into distinct rain and background representations.

• RGB Branch: Frame features are progressively decomposed into background and rain components using attention mechanisms.
• Event Branch: A rain-oriented Spiking Neural Network (SNN) module, termed SNN-RoED, captures sparse, bursty event dynamics associated with rain motion while separating them from background-triggered events.
• Component-Level Fusion: The model performs semantically aligned fusion, where background-to-background fusion preserves scene structure, and rain-to-rain fusion enhances rain suppression.

Finally, the model uses event-domain supervision to regularize the reconstruction of sparse events, ensuring structural consistency and accurate gradient orientation.

RainDancer demonstrates superior performance in both synthetic and real-world rainy video datasets. By explicitly separating rain and background components before interaction, the model significantly reduces the "entanglement" problem where rain artifacts are mistaken for background or vice versa. Quantitative results show improved visual quality and enhanced robustness for downstream tasks like object detection compared to existing early-fusion or late-fusion RGB-Event methods.