Tunable Tool-Call Rates in LLM Agents via Representation Steering

LLM agents often struggle to balance the costs of tool use—such as latency and potential side effects—against the risk of providing incorrect answers due to missing information. Existing methods to manage this, such as fine-tuning or prompt engineering, are often expensive or difficult to adjust dynamically. This paper investigates whether an agent's decision to call a tool is encoded in a readable, steerable linear direction within the model's internal representations, allowing for lightweight, inference-time control.

The researchers identify a single linear direction in the model's residual stream that governs the propensity to call a tool. They extract this direction using a difference-of-means (DIM) approach, comparing the activations of high-propensity and low-propensity tool-use queries in a multi-tool environment. By adding a scalar multiple of this direction (the steering coefficient α) to the residual stream at a specific mid-to-late decoder layer during inference, they can modulate the model's tool-calling behavior without any retraining or prompt modifications.

• Continuous Control: The steering coefficient α allows for monotonic control of tool-call rates, ranging from near 0% to over 90%.
• Knowledge Selectivity: The intervention is intelligent; it preferentially induces tool calls for long-tail factual questions that the model is unlikely to know, while suppressing calls for common knowledge questions.
• Generalization: The steering direction is tool-general, meaning a direction extracted from a few tools can successfully steer unseen tools. It also transfers across diverse model architectures, including dense, Mixture-of-Experts (MoE), and multimodal models.
• Performance Gains: By sweeping the steering coefficient, the researchers trace a cost-accuracy Pareto frontier, demonstrating that they can nearly double open-domain QA accuracy by optimizing the frequency of tool calls.