T1 Teaches Terminal Agents to Persist Through Long-Horizon Tasks

Why terminal agents need a different training recipe

AI agents are increasingly expected to do more than produce a plausible answer. They may need to inspect a codebase, install dependencies, edit several files, run tests, diagnose failures, and repeat the process until an external checker accepts the result. These workflows are difficult because a single mistake can surface many turns later, while the agent must keep its original objective in context throughout the trajectory.

T1, introduced by researchers from Tencent Hunyuan, is designed for this setting. It is a mixture-of-experts model with 122 billion total parameters. Instead of learning only from static text, it operates a real shell inside a cloud sandbox. A task may involve more than 300 tool-call turns, allowing the training environment to resemble the iterative nature of coding and system-operation work.

Rewarding progress through verifiers

The model is trained with reinforcement learning. Each task comes with its own verifier, and the outcome of executing that verifier supplies the main signal for whether the task was completed. T1 also uses a denser process reward: a trajectory is scored by the absolute number of verifiers it passes. This gives the agent feedback about partial progress rather than forcing every useful intermediate action into a simple success-or-failure outcome.

The paper emphasizes that reward design alone is not enough. Long trajectories create a mismatch between the rollout system that samples actions and the training system that later computes the update. In a mixture-of-experts model, even a small change in tokenization or expert selection can alter subsequent probabilities and make the learning signal less faithful to the sampled behavior.

Aligning rollouts with optimization

T1’s recipe addresses this problem in three related ways:

• Aggressive warm-starting helps stabilize actor-critic training before long-horizon reinforcement learning begins.
• TITO construction trains on the exact token identifiers sampled during rollout and repairs drift at turn boundaries.
• Rollout routing replay, or R3, records the expert selected for every token at every MoE layer and replays those routing decisions during training.

Together, TITO and R3 reduce the reported training-to-inference log-probability gap from 0.021 to 0.013. The authors also report zero token drift in the region used for the loss. These details point to a broader lesson: for tool-using agents, reproducibility of the trajectory may be as important as the choice of reinforcement-learning objective.

Results and broader significance

The researchers use isolated seeds and synthesized tasks that are disjoint from Terminal-Bench 2.1 for training. This is intended to reduce the possibility that improvements come simply from memorizing benchmark-specific tasks. In the reported evaluation, the post-training pipeline lifts the base model’s resolved rate on Terminal-Bench 2.1 from 43.8% to 64.0%. On Long-Horizon Terminal Bench, T1 reaches 27.9% and is reported to surpass GPT-5.4 and GLM-5.1.

T1’s contribution is therefore both a model result and an engineering recipe. Real shell execution connects learning to observable outcomes; verifiers turn task completion into a measurable reward; and token and routing replay make optimization better match the behavior that was actually sampled. The results do not imply that long-horizon agents are solved, but they suggest that reliable interaction infrastructure and trajectory fidelity will be central to improving agents for software engineering and other multi-step workflows.

Source: Hugging Face Daily Papers