Localizing RL-Induced Tool Use to a Single Crosscoder Feature

This paper investigates the mechanistic basis of agentic tool-use capabilities in large language models (LLMs) following reinforcement learning (RL) fine-tuning. Specifically, the authors seek to understand how RL alters internal model representations and whether these changes can be isolated into interpretable, steerable features for behavioral control without requiring further fine-tuning.

The researchers employ Dedicated Feature Crosscoders (DFC), a technique that partitions a joint dictionary into model-exclusive and shared sub-dictionaries using gradient masking. They perform a systematic hyperparameter sweep across 48 crosscoder variants applied to a Qwen2.5-3B model pair (a base model and an RL-fine-tuned tool-use model). The study evaluates the models through reconstruction fidelity, behavioral performance, and targeted activation steering, where they manually intervene on specific neurons to observe changes in tool-calling accuracy.

The authors find that DFC partitioning effectively concentrates RL-induced tool-calling capabilities into a compact, steerable set of A-exclusive features. Remarkably, steering just one of these A-exclusive neurons can increase tool-correctness by 65 percentage points. Furthermore, the study identifies a phenomenon called "capability spillover," where the shared dictionary weights allow the base model to passively acquire some tool-calling ability without any fine-tuning. While the DFC successfully isolates these features, the exclusive partition acts more like a filter than a perfect sink; penalizing these features pushes tool-relevant signal back into the shared partition, suggesting that RL-induced capabilities are not entirely orthogonal to the model's original structure.