M2C-EvDet: Multi-Domain Multi-Order Cross-Modal Knowledge Distillation for Event-based Object Detection

Event-based cameras offer high temporal resolution and dynamic range, but event data is inherently sparse and lacks the rich visual semantics found in traditional RGB frames. This creates a significant performance gap between event-based object detection (EvDet) and frame-based detection. The authors investigate how to bridge this gap using cross-modal knowledge distillation without requiring RGB data during inference.

The authors propose M2C-EvDet, a framework that transfers knowledge from a teacher (RGB-based) to a student (event-based) model. The framework introduces two primary innovations:

1. Adaptive Frequency-Decoupled Feature Distillation (AF2D2): This module uses multi-scale wavelet decomposition to separate low-frequency features (which contain shared, modality-agnostic information) from high-frequency features (which capture modality-specific details and noise). By distilling these components separately, the student model learns more robust global and local object semantics.

2. Multi-Order Relational Distillation (MORD): While traditional methods focus on pair-wise relations, MORD uses hypergraph computation to model high-order correlations. It employs a hyper-attention mechanism that acts as an intermediate bridge, allowing information to propagate among multiple feature nodes simultaneously.

Experiments on three RGB-Event object detection datasets show that M2C-EvDet outperforms existing distillation methods. Specifically, the model achieved a 3.6 mAP improvement on the DSEC-Detection dataset, establishing a new state-of-the-art benchmark. The results suggest that decoupling frequency information and modeling high-order relations are critical for effectively transferring knowledge from RGB to event-based modalities.

This work provides a robust way to enhance event-based perception systems by leveraging the abundance of existing RGB data. By enabling the student model to learn both fine-grained frequency-decoupled features and complex high-order relationships, the framework significantly improves detection accuracy in challenging, dynamic environments where event cameras are typically deployed.