Distributed Sparse Interventions in Language Models

Modern language models exhibit flexible, task-solving capabilities, but the internal mechanisms governing these behaviors remain poorly understood. Traditional model steering techniques typically assume that tasks are represented as linear directions in activation space, allowing for additive interventions. The authors challenge these assumptions, arguing that task-relevant computations are often distributed across specific neurons and exhibit complex, nonlinear interactions that global vector-based approaches fail to capture.

To address this, the authors introduce Distributed Sparse Interventions (DSI). This method identifies sparse subsets of neurons that jointly activate a specific task. The DSI procedure involves four steps: computing activation differences between few-shot and zero-shot prompts, selecting an initial set of neurons using robustified gradients, iteratively refining the intervention to account for nonlinearities, and applying the final sparse intervention during a forward pass.

Empirical evaluations across three instruction-tuned models (Qwen3, Gemma 3, and Llama 3.2) and 12 distinct tasks demonstrate that DSI is highly effective. By intervening on as few as 8 to 64 neurons (0.01% to 0.04% of total neurons), the authors achieve task performance that often matches or exceeds 10-shot in-context learning. The study highlights that neuron-level interventions are not merely additive; they involve significant nonlinear dependencies. The iterative refinement component of DSI is crucial, as it allows the model to navigate these nonlinearities, leading to superior performance compared to methods that rely solely on first-order Taylor approximations.

This research provides a more granular understanding of how language models represent tasks. By moving away from global activation vectors toward sparse, neuron-level interventions, the authors offer a more precise tool for model steering and interpretability. This approach not only enables fine-grained control over model behavior but also helps localize task-relevant circuits, providing a pathway to better understand how models compose and execute multiple tasks simultaneously.