Assessing the Benefits of Combining Advanced Deep Learning Techniques for Post-Disaster Building Damage Assessment from UAV Imagery

Research Approach

Post-disaster building damage assessment is critical for emergency response but is hindered by the need for large, labeled datasets and the difficulty of generalizing across different regions and damage definitions. While Computer Vision (CV) models excel at object detection, they often lack the contextual reasoning required for complex damage classification. Conversely, Large Vision-Language Models (LVLMs) possess strong reasoning capabilities but struggle with precise spatial localization and object detection. This paper proposes a hybrid framework that decouples these tasks: a CV model (Grounding DINO) identifies buildings and generates bounding boxes, which are then passed to an LVLM (such as GPT-4o or QwenVL) to classify the damage level of each detected building.

The authors evaluated their framework on two real-world benchmarks, RescueNet and FloodNet. By combining the spatial precision of a fine-tuned CV detector with the semantic reasoning of an LVLM, the framework achieved superior performance in counting intact, damaged, and destroyed buildings compared to isolated baselines. Notably, the framework improved performance by up to 2.1 R^2 points while requiring only limited annotated data for the detection stage. The authors deliberately avoided fine-tuning the LVLMs to prevent catastrophic forgetting and to maintain the models' broad reasoning capabilities, demonstrating that this modular approach is effective even when using frozen, pre-trained LVLMs.

This research provides a practical, data-efficient solution for disaster management, where high-quality labeled data is often scarce or unavailable in the immediate aftermath of an event. By decoupling detection from classification, the framework allows practitioners to leverage the best of both worlds—spatial accuracy and high-level reasoning—without the prohibitive computational costs of training large models from scratch. The inclusion of open-source and lightweight models in the study also suggests a viable path toward deploying these tools directly on UAV platforms for real-time reconnaissance.