Evaluating and Enhancing Negation Comprehension in Remote Sensing MLLMs

Multimodal Large Language Models (MLLMs) have shown great promise in remote sensing (RS) tasks, yet their ability to handle negation—identifying what is absent or false—remains largely untested. This is a critical gap for real-world applications like disaster response, where distinguishing between 'flooded' and 'non-flooded' areas is essential. The authors investigate how well current RS-focused MLLMs understand negation and how to improve their performance without requiring massive labeled datasets.

The authors introduce RS-Neg, a new benchmark containing over 22,000 samples across four tasks: visual question answering, multiple-choice questions, visual grounding, and scene classification. To build this dataset, they developed an automated pipeline that uses LLMs to synthesize negation queries and a 'dynamic visual focus' module to verify that the negated concepts are indeed absent from the imagery.

To address the identified performance gaps, they propose NeFo (Negation-focused optimization), a test-time adaptation (TTA) method. NeFo uses a small subset of unlabeled test data to fine-tune the model using two objectives: a 'truth-value inversion' loss that forces the model to distinguish between negated and affirmative queries, and a 'knowledge retaining' loss that prevents the model from losing its original capabilities during adaptation.

The evaluation reveals that even advanced MLLMs exhibit significant performance degradation and hallucinations when faced with negation in remote sensing contexts. For instance, some models performed worse than random guessing on multiple-choice tasks involving negation. The authors demonstrate that existing reasoning-augmented models do not inherently solve this issue. However, applying the NeFo method significantly improves negation understanding across various base models, showing strong generalization to unseen tasks while using only a small fraction of unlabeled data.