DeepSearch-World: Self-Distillation for Search Agents in a Verifiable Setting

Introduction

This paper tackles a practical bottleneck in tool-using agents: how can a web agent actually improve from its own experience? Supervised fine-tuning usually depends on fixed teacher-generated trajectories, while sparse-reward reinforcement learning often gives too little guidance for long-horizon search and reading tasks. DeepSearch-World answers this by creating a deterministic, verifiable environment for web search agents, so that learning signals can be grounded in reproducible interactions rather than guesswork.

Key points

• A verifiable environment for web agents: DeepSearch-World uses reproducible search and page-reading tools, making it possible to inspect whether the agent is moving in the right direction.
• Scale matters: the environment includes 420K multi-hop QA tasks, built from entity-level random walks.
• Supports self-evolving behaviors: the setup is designed to encourage progress verification, grounded reflection, and failure recovery.
• DeepSearch-Evolve is the training loop: instead of one-shot imitation, it repeatedly generates trajectories, filters them, mixes data, and fine-tunes the agent.
• Strong baseline performance: without distillation from more capable models, DeepSearch-World-9B reaches 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA.

Why it matters

The main contribution here is not just a better benchmark, but a different way to think about agent training. Long-horizon web tasks are hard because success depends on more than final answers: the agent must notice when a path is unproductive, verify intermediate progress, and recover from failure. A deterministic environment makes those behaviors measurable, which in turn makes self-improvement more realistic.

This also bridges two common training paradigms. Supervised learning offers clean trajectories but limited adaptability. Reinforcement learning offers adaptation but often weak supervision in sparse-reward settings. DeepSearch-World sits between them: it preserves interaction-based learning while making trajectory quality verifiable. The planned release of the environment, training pool, validation set, model, and code should make it easier for others to study self-improving deep search agents.

Hugging Face Daily Papers