TerminalHorizon Builds More Realistic Data for Long-Horizon Terminal Agents

Introduction

Terminal agents are increasingly expected to do more than execute isolated shell commands. A real professional task may require an agent to inspect an unfamiliar environment, collect evidence, form a judgment, implement a solution, test the result, respond to feedback, and turn intermediate work into a final deliverable. Short training trajectories can demonstrate individual commands or tool calls, but they often fail to capture how these behaviors connect across a sustained problem-solving process.

The paper presents TerminalHorizon, an automated data synthesis engine designed to address this gap. Developed by an Alibaba research team, the system focuses on whether a synthesized environment still requires the agent to perform the important work rather than merely replaying a task whose difficult parts were completed in advance.

Key ideas

• Detecting omitted work. Environment synthesis can unintentionally finish or simplify the central steps that should have required exploration, implementation, or verification. TerminalHorizon uses independent execution to identify these missing elements. The evidence collected during execution then guides the reconstruction of the environment.
• Expanding tasks across stages. Instead of generating unrelated short exercises, the system keeps a common goal and gradually adds new requirements. Later stages build on earlier outputs, requiring the agent to explore further, improve its approach, and react to new feedback.
• Preserving the full work process. The resulting trajectories are intended to include more than a final answer. They connect evidence gathering, judgment, strategy adjustment, implementation, validation, and delivery within one evolving task.
• Scaling the dataset. The researchers construct 1.5K task environments and use the resulting data for supervised fine-tuning of Qwen3.5-27B, producing TerminalHorizon-27B.
• Evaluating different capabilities. The authors report substantial gains across 12 benchmarks covering coding-agent tasks, agentic tasks in real-world production settings, and general reasoning. Several RSI case studies, each with a 12-hour research budget, are also used to examine whether the model can continue experimenting after an initial solution, learn from feedback, and retain effective improvements in its final outputs.

Why it matters

The contribution is not simply to make trajectories longer. Long context is useful only when it contains meaningful work. If a synthesized environment has already performed the crucial implementation or validation, extending the interaction does not necessarily teach an agent to act autonomously. TerminalHorizon’s independent-execution step is intended to expose those shortcuts and restore the operations that the agent must actually carry out.

The staged construction process is also closer to how professional projects evolve. A first result may create a new requirement; a test may reveal that the original method is inadequate; and an intermediate artifact may become the input to the next phase. Training data that preserves these dependencies can better test whether an agent remembers useful changes, revises its strategy, and converts partial results into a stronger deliverable.

The available material, however, gives only aggregate claims rather than detailed scores for each benchmark, task-category breakdowns, or complete comparisons with alternative data-generation methods. TerminalHorizon is therefore best understood as a data-engineering approach for long-horizon agent training. Its reported results are promising, while the size of its advantage across specific workloads will require the full experimental details for closer assessment.

Source: Hugging Face Daily Papers