TMAX: A simple recipe for terminal agents

Research Overview

Terminal-using agents are increasingly critical for complex, long-horizon coding and system administration tasks. However, academic progress has been hindered by a lack of diverse, high-quality training data and simple, reproducible reinforcement learning (RL) recipes. This paper introduces TMAX, a comprehensive framework for generating terminal-based training environments and a robust RL training recipe for open-weight models.

Data Generation Pipeline

The authors address the scarcity of terminal data by creating TMAX-15K, a dataset of 14,600 RL environment instances. Unlike previous approaches that rely on complex, multi-stage validation or focus on narrow domains like software engineering, the TMAX pipeline uses a compositional approach. By sampling across nine structured axes—including domain, skill type, task complexity, and persona—the authors generate a diverse set of tasks that are balanced across multiple domains. A key innovation is the use of 'soft filtering' during RL training, which eliminates the need for expensive, pre-training teacher-based validation of task correctness.

RL Training and Performance

To train terminal agents, the authors employ Divergence Proximal Policy Optimization (DPPO), a variant of Group Relative Policy Optimization (GRPO) designed to improve stability in long-horizon agentic scenarios. Their 9B parameter model achieves 27% on Terminal-Bench 2.0, outperforming significantly larger models from prior work. The authors show that this RL training does not merely 'harness-fit' to specific evaluation prompts; rather, it teaches the model transferable skills that improve performance on non-agentic evaluations and across different terminal harnesses.

This work provides a strong, open-source baseline for the academic community to study terminal agents. By releasing the TMAX-15K dataset, the training code, and the model checkpoints, the authors lower the barrier to entry for research into agentic RL. The findings suggest that simple, well-calibrated synthetic data and standard RL algorithms are sufficient to achieve frontier-level performance in terminal-based tasks, provided the data is sufficiently diverse and difficult.