Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

Agents-A1 is a 35B parameter agentic model designed to address the limitations of scaling model parameters for long-horizon tasks. Instead of increasing model size, the authors focus on scaling the agent's horizon—the length and complexity of the decision-making process. By building a specialized knowledge-action infrastructure and employing a multi-stage training recipe, Agents-A1 matches or outperforms 1T-parameter models on benchmarks involving multi-step search, scientific reasoning, and complex instruction following.

The authors introduce a three-stage training pipeline to unify heterogeneous agentic abilities:

1. Full-domain Supervised Fine-Tuning (SFT): The base model is aligned with broad agentic behaviors using a dataset of 100K long-horizon trajectories (averaging 45K tokens).
2. Domain-level Teacher Training: Specialized teachers are trained for specific domains (e.g., search, scientific reasoning, instruction following) using targeted SFT or reinforcement learning (RL).
3. Multi-teacher On-Policy Distillation (OPD): The specialized teachers are consolidated into a single student model using a domain-routed objective. This includes Salient Vocabulary Alignment (SVA) to ensure the student learns from the teacher's preferred token distributions and domain-normalized aggregation to prevent high-frequency domains from dominating the updates.

Central to this work is the Knowledge-Action Graph (KAG), which records evidence, actions, observations, and verifier outcomes. This infrastructure allows the model to learn from grounded feedback and process-level supervision rather than isolated text, enabling more robust long-horizon planning.

This paper provides a practical, reproducible path for achieving high-level agentic performance without the massive compute requirements of trillion-parameter models. By demonstrating that scaling the agent's horizon—via better infrastructure and specialized distillation—can bridge the gap to frontier model performance, the authors offer a scalable alternative for researchers and practitioners working on complex, interactive AI tasks.