Could Inference-Time Interventions Preserve Alignment? Safety Cost of Steering Vectors Is Separable and Reducible

Steering vectors are a popular, lightweight method for controlling LLM behavior by adding specific directions to activation space. However, recent research has shown that these vectors often inadvertently compromise safety, making models more susceptible to jailbreak attacks. This paper investigates whether this safety degradation is an inherent cost of steering or if it arises from a separable, removable component within the steering vector.

The authors introduce CAST (Constrained Ablation for Safe STeering), a post-hoc method to sanitize steering vectors. Instead of simply ablating known refusal directions—which often fails to restore safety—the authors treat the identification of the safety-degrading component as a constrained optimization problem. They use a Lagrangian primal-dual framework to learn a direction to remove from the steering vector, subject to three explicit constraints: maintaining the original steering effect, preventing an increase in false refusals on benign prompts, and maximizing the recovery of the model's baseline refusal behavior on harmful prompts.

The study demonstrates that the safety-degrading component of a steering vector is geometrically separable from the component responsible for the steering effect. Across three different LLMs (Llama-3.1-8B and Qwen2.5-7B/14B) and various jailbreak attack types, the CAST-sanitized vectors significantly reduced the Attack Success Rate (ASR) compared to naive ablation methods. Notably, the method remains effective even against unseen, adaptive jailbreak attacks, suggesting that the learned sanitization generalizes well beyond the training distribution. The authors show that it is possible to achieve "safe steering" without retraining the model or paying a significant tax in utility or false refusal rates.

This work provides a practical, surgical solution to a critical bottleneck in model interpretability and control. By showing that safety costs are not necessarily coupled with behavioral steering, the authors offer a general recipe for applying activation-level interventions safely. This allows researchers and practitioners to deploy steering vectors for personalization or bias mitigation with greater confidence that they are not inadvertently creating new vulnerabilities in the model's alignment.