Probing the Prefill: Detecting Code Vulnerabilities via Latent Activations

LLM-based code generation is increasingly used in mission-critical pipelines, but security defenses rely on post-hoc tools like static analyzers, fine-tuned classifiers, or external LLM judges. These methods analyze emitted code after the fact while ignoring the generating model's own internal computations. This paper investigates whether an LLM's hidden activations already carry a recoverable signal about code vulnerability when it reads code as context.

The author extracts last-prefill-token activations from four diverse LLMs spanning three families: Granite-4.1-8B, Qwen3.5-9B, Qwen3.6-27B, and Gemma-4-12B. Rather than performing a layerwise sweep, the methodology focuses on the final hidden layer's output vector. Six-layer MLP probes with dropout and GELU activations are trained on these frozen representations using a class-weighted binary cross-entropy loss. The probes are evaluated on four function-level C/C++ vulnerability detection benchmarks: Devign, Big-Vul, Draper VDISC, and PrimeVul.

The MLP probes achieve an average F1 score of 41.7% across all model and dataset pairs using only 13.4 to 16.0 million parameters, which amounts to under 0.2% of the base models' sizes. Performance varies heavily by dataset rather than model family. On the Devign benchmark, the best probe configuration achieves 68.8% F1, matching the published fine-tuned classifier state-of-the-art of 67.9% despite reading a general-purpose, frozen LLM.

However, performance drops on more challenging datasets. On Big-Vul, probes reach up to 59.5% F1, and on Draper VDISC and PrimeVul, scores decline further to approximately 30.5% and 17.6% F1 respectively. A breakdown of Big-Vul classifications across common Common Weakness Enumerations reveals uniformly high accuracy across the top five vulnerability types, with CWE-119 proving to be the most difficult category for all four models.

These results provide early evidence that a coding LLM's latent representation of arbitrary code contains meaningful vulnerability signals. From a systems perspective, internal probing offers massive computational advantages over external screening. External detectors require secondary models and multi-stage API round-trips, whereas a latent probe executes concurrently within the primary model's inference pass at a negligible parameter and latency cost.

This makes probe-based filtering particularly appealing for air-gapped, on-device, or tactical environments where sending generated code to an external security service introduces severe latency overheads or data exfiltration risks.