Agent-as-a-Router: Agentic Model Routing for Coding Tasks

As users increasingly access multiple Large Language Models (LLMs) for coding tasks, a critical challenge is determining which model is best suited for a specific task to optimize for both performance and cost. The authors investigate why existing routing methods—which typically treat model selection as a static, one-off classification problem—often fail to match the performance of an ideal per-task oracle.

The authors first perform an ablation study to diagnose the routing bottleneck. They find that providing a standard LLM router with per-dimension performance statistics yields a 15.3% relative gain, suggesting that the limitation is information access rather than reasoning. Based on this, they propose the Agent-as-a-Router framework, which formalizes routing as a continuous Context-Action-Feedback (C-A-F) loop. In this loop, the router observes a task context, selects a model, receives execution-grounded feedback (verified scores and costs), and stores this experience in memory to inform future decisions. They instantiate this as ACRouter, which includes an Orchestrator for decision-making, a Verifier for sandbox-based evaluation, and a Memory module for storing historical outcomes.

ACRouter consistently achieves the lowest cumulative regret—the gap between the router's performance and the optimal per-task choice—across in-distribution coding tasks. Furthermore, the framework demonstrates strong generalization capabilities on out-of-distribution agentic-programming tasks. The results confirm that by actively closing the information gap through iterative, execution-grounded learning, the router evolves to become more efficient and accurate over time, outperforming static heuristic or purely discriminative routing policies.