LLM-Assisted Dynamic Threat Analysis for Attacker-Reachable Software Weaknesses in Autonomous Vehicles

As autonomous vehicles move from controlled environments to public roads, their software becomes safety-critical. A defect reachable from adversarial sensor or network inputs can alter steering, braking, or other control decisions, leading to safety hazards. While static analysis can identify candidate vulnerability sites across an entire code repository, confirming whether a weakness is dynamically exploitable requires executable test artifacts. Constructing these artifacts manually is extremely difficult due to complex target types, message formats, middleware dependencies, and build configurations.

This paper investigates whether large language models (LLMs) can automate the creation of executable test artifacts for Autoware, an open-source, production-scale autonomous-driving stack based on ROS 2. The study aims to evaluate the full pipeline from static candidate identification to dynamic confirmation, exposing the intermediate failure modes that determine whether automated testing can succeed at scale.

The authors perform a compiler-precise static analysis across 185 Autoware packages and 1,857 source files to map the safety-relevant attack surface, extracting decision rules, validation checks, and input-to-safety-output flows. From this static inventory, they derive a weakness taxonomy and sample 740 reachable sites.

For each sampled site, two local open-weight LLMs, a no-static-context ablation, and a naive-template baseline generate a total of 3,700 artifact sets. These generated artifacts are then compiled against the native Autoware build environment under sanitizers. Artifacts that fail compilation enter an automated, compiler-in-the-loop repair loop. Successfully compiled harnesses are subsequently executed under a fixed fuzzing budget to determine if any candidate weaknesses can be dynamically confirmed.

The central finding is a build-integration failure taxonomy showing that 80% of first-shot compilation failures arise from dependency wiring rather than program logic. Model capability drastically affects initial compilation rates: the reasoning model compiled 64% of harnesses on the first attempt, compared to only 6% for the code-specialized model.

However, closer inspection reveals severe limitations beneath these nominal compilation numbers. Repairing the code achieved full object-compileability for the reasoning model only by introducing extensive stubbing and replacement implementations that bypassed the real target code. Consequently, fewer than half of its generated harnesses actually reached the fuzzer. Furthermore, all 37 observed crashes originated entirely inside the stubbed helper code rather than within Autoware itself. Within the allocated budget, zero candidate weaknesses were dynamically confirmed.