NexForge Scales LLM Agents with Requirement-Driven Task Synthesis

Lead

Training stronger LLM agents increasingly depends on the availability of executable, diverse, and realistic task data. NexForge focuses on that data bottleneck. Many existing approaches synthesize agent tasks by anchoring them to predefined tools, repositories, or skill graphs. This makes evaluation and generation manageable, but it also creates friction: every new domain can require a new pipeline, broader coverage demands manual substrate engineering, and the resulting distribution may reflect the chosen environment more than real user demand.

NexForge proposes a different starting point. Rather than asking what can be generated from an existing substrate, it asks what capabilities the agent should acquire. Given high-level capability requirements, the framework synthesizes executable tasks and expert trajectories that can be used for supervised fine-tuning.

Key points

• Requirement-driven generation: NexForge is designed around capability requirements instead of fixed repositories, tools, or handcrafted skill graphs.
• Real-world demand modeling: The framework first investigates demand to build representative scenarios and task profiles, aiming for a task mix that better matches practical use cases.
• Distribution-aware compilation: It converts those profiles into concrete task directives while considering distributional coverage, rather than relying on narrow templates.
• Automatic environment construction: For each directive, NexForge retrieves or constructs the necessary files, dependencies, and runtime configurations, turning prompts into executable environments.
• Expert rollouts for SFT: The system then synthesizes expert trajectories, producing training data that captures multi-step interaction and tool use.
• Reported scaling results: Without domain-specific infrastructure, the authors report generating 3.6K terminal tasks and 2K office tasks. These data improve Qwen3.5-35B-A3B Base from 22.5% to 52.0% on Terminal-Bench 2.0 and from 813 to 1338 Elo on GDPval. Scaling to 43.2K terminal tasks raises Terminal-Bench 2.0 performance to 58.4%.

Why it matters

The main contribution is not simply another synthetic dataset, but a more general recipe for producing agent training environments from capability needs. For terminal agents, office agents, and broader tool-using systems, the hard part is not only solving isolated tasks. It is covering long-tail workflows while keeping tasks executable, verifiable, and scalable.

The paper also states that NexForge is the core data engine behind the publicly available Nex-N2 model family, including Nex-N2-Pro and Nex-N2-Mini. With further scaling, NexForge-generated data contributes to training models that lift Qwen3.5-35B-A3B to 75.3% on Terminal-Bench 2.1 and 1585 Elo on GDPval. That suggests data synthesis may be a major lever for improving open agent models.

Important details still need scrutiny in the full paper: how task quality is filtered, how reliable the expert rollouts are, how real-world demand is operationalized, and whether the generated distribution aligns too closely with specific benchmarks. Still, NexForge points toward a useful shift: from hand-engineered agent playgrounds to automatically synthesized environments driven by capability requirements.

Source: Hugging Face Daily Papers