Development of Vision-Language Model–based GNSS Spoofing Detection for Autonomous Vehicle Navigation

Autonomous vehicles (AVs) rely heavily on Global Navigation Satellite Systems (GNSS) for navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. While traditional signal-level and geometric integrity checks exist, sophisticated attackers can bypass these by generating spoofed signals that remain mathematically consistent with onboard sensor fusion. This paper investigates whether Vision-Language Models (VLMs) can provide a robust, behavior-level defense by validating whether the maneuvers implied by GNSS data are semantically consistent with the vehicle's actual visual and physical environment.

The researchers developed a VLM-based framework that fuses front-camera visual data with in-vehicle telemetry (speed, acceleration, and yaw rate). The core innovation is a three-stage fine-tuning process that grounds visual cues and calibrates sensor data within a shared semantic space. To address the high computational cost of VLMs, the authors implemented an adaptive inference policy that triggers model analysis only when sensor readings indicate a change in vehicle maneuver, reducing computational overhead by approximately 86%. The model was validated using a custom dataset collected in Tuscaloosa, Alabama, featuring three types of intelligent spoofing attacks: wrong-turn attacks (trajectory mirroring), overshoot attacks (position freezing), and stop attacks (drift generation).

The fine-tuned VLM framework significantly outperformed zero-shot baselines, achieving F1-scores between 94% and 95% on the validation dataset. The system correctly classified all instances of wrong-turn and stop attacks and maintained 88%–93% accuracy for overshoot attacks. By using an adaptive inference policy, the system achieved a processing latency of 65ms–73ms per 4-second window, demonstrating that VLM-based semantic validation is a viable, real-time layer of defense for autonomous navigation.