PQFA: Parallel Quantum Feature Augmentation of Fused Representations for Multimodal Classification

Multimodal learning typically focuses on improving the fusion of heterogeneous data (like text and images). However, the potential for enhancing these fused representations through lightweight, post-fusion transformations remains under-explored. This paper investigates whether shallow, parallel variational quantum circuits can serve as efficient feature augmenters for fused multimodal representations, and whether such quantum components provide benefits beyond what can be achieved by classical neural network layers of similar capacity.

The authors propose Parallel Quantum Feature Augmentation (PQFA), a hybrid quantum-classical framework. The pipeline begins with frozen RoBERTa and ViT encoders extracting unimodal features, which are then refined through bidirectional cross-attention, attentive pooling, and adaptive gated fusion. The resulting fused classical representation is passed to multiple parallel parameterized quantum circuits (PQCs). These circuits use amplitude encoding to map classical data into quantum states, apply trainable unitary transformations, and output measurement readouts that are concatenated with the original classical features for the final classification task. The authors rigorously compare PQFA against a no-quantum backbone and a width-matched classical MLP augmentation baseline to ensure that performance gains are attributable to the quantum transformation rather than simple parameter expansion.

PQFA consistently outperforms both the unaugmented fusion backbone and classical MLP-based augmentation on the MM-IMDb and N24News datasets. Notably, PQFA achieves these gains using approximately 2.2K parameters, compared to 24.0K for the width-matched MLP baseline, demonstrating high parameter efficiency. The framework also exhibits improved robustness in missing-modality scenarios, particularly when the textual modality is degraded. Extensive ablations and feature-space analyses confirm that the improvements are not merely artifacts of random mappings or increased classical capacity, but represent task-aligned feature transformations.