Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models

This paper investigates whether it is possible to control a Large Language Model's (LLM) internal state—specifically its 'evaluation-awareness'—using only input-side prompt optimization, without requiring inference-time access to the model's internal activations. The authors test whether suppressing these internal signals can prevent a model from detecting that it is being tested, which is a critical concern for the validity of safety evaluations.

The researchers adapt the 'Fluent Dreaming' framework to create an input-side dual of activation steering. By using GCG-style discrete token-gradient optimization, they search for fluent prompts that drive specific internal latents toward zero. They target five different constructions of evaluation-awareness: a contrastive activation addition (CAA) direction, a subspace norm, a Sparse Autoencoder (SAE) feature, a single MLP neuron, and a behavioral logit. To ensure the results are meaningful, they implement a rigorous evaluation protocol that includes a behavioral endpoint, a fresh-probe erasure test, and an orthogonal-complement probe-hacking detector to distinguish genuine concept erasure from simple rotation off the measured axis.

The study demonstrates that prompt-side minimization is highly effective at reducing the targeted internal activations. The researchers successfully drove diverse latents to zero, including the complete and selective suppression of a causally-validated SAE feature. However, the behavioral results tell a cautionary story: despite the successful suppression of internal signals, the model's behavioral judgment regarding whether it is being evaluated remains unchanged. In some cases, the suppression even caused the model to become more 'eval-aware' in its behavior. This leads to the conclusion that activation-readability is not synonymous with behavioral controllability.