Agentic Neural Architecture Search

Neural Architecture Search (NAS) has traditionally relied on manually engineered search spaces, which are labor-intensive and task-specific. While Large Language Models (LLMs) can generate architectures in open-ended spaces, they often struggle to systematically explore combinatorial design choices. This paper investigates how to effectively combine LLM-driven design with NAS-driven search, specifically asking: what is the optimal division of labor between these two paradigms?

The authors propose AgentNAS, a three-phase pipeline that separates the roles of design and optimization. In Phase 1, an LLM acts as a designer, proposing and refining a high-quality 'seed' architecture and training recipe. In Phase 2, the LLM decomposes this seed into a 'slotted architecture'—a scaffold containing named, interchangeable module slots. This process automatically defines a bounded, task-specific search space. Finally, in Phase 3, a conventional NAS algorithm (such as regularized evolution) explores this space by recombining modules within the slots. This modular structure allows the authors to independently measure the contribution of the LLM (the seed) and the NAS (the combinatorial refinement).

AgentNAS demonstrates that LLM-driven design and NAS-driven search are complementary. The LLM-generated seed often outperforms existing baselines on its own, but the addition of NAS-driven combinatorial search consistently yields further improvements. Across 17 tasks—including classification, regression, and segmentation—AgentNAS achieved state-of-the-art results on 11 tasks. Notably, the system performed well even in 'blind' settings where the LLM had no access to domain-specific metadata, suggesting that the pipeline effectively adapts to diverse data modalities without requiring manual engineering of the search space.