Auditing Framing-Sensitive Behavioral Instability in Large Language Models for Mental Health Interactions

Research Question and Approach

This study investigates how subtle changes in the contextual framing of a prompt—such as presenting a mental health concern as documentation versus an institutional requirement—affect the behavior of aligned large language models (LLMs). The authors seek to determine if these behavioral shifts are merely surface-level linguistic adjustments or if they are rooted in systematic differences within the models' internal hidden-state representations. To test this, the researchers created a dataset of matched prompts where the underlying semantic intent remains constant while the framing varies across five categories: documentation, epistemic, institutional, liability, and role-based.

Behavioral Findings

The researchers found that contextual framing significantly influences how models calibrate their responses. Documentation framing consistently led to higher rates of interpretive escalation, where the model makes inferences about the user's state beyond what is explicitly stated. In contrast, institutional framing often resulted in more restrained, stable responses. These effects were observed across multiple model families, including Gemma, Qwen, Mistral, and Phi, though the degree of sensitivity varied by architecture. This suggests that even without adversarial intent, the way a user frames a request can cause an AI assistant to shift from a supportive role to an overly interpretive or quasi-diagnostic one.

Representational Analysis and Steering

By applying layer-wise probing, the authors demonstrated that information about these interpretive tendencies is decodable from the models' internal hidden states throughout the transformer layers. While lexical cues (the specific words used in the prompt) account for a portion of this signal, held-out framing probes remained above chance, indicating that the models develop a generalized representation of these framing-induced behaviors. Furthermore, the researchers used activation steering—a technique to manipulate hidden-state activations—to show that they could partially modulate the models' downstream behavior, providing evidence that these latent directions are causally linked to the observed response styles.