From Web Search to Web Research: OpenDS Tests AI Agents in the Wild

Introduction

AI search agents can already call search engines, open pages, and produce answers with citations. Yet locating a relevant page is very different from completing a dependable research task. On the live Web, evidence is scattered across independent sources, hidden behind different interaction flows, and presented in text, images, documents, or other formats. An agent must decide what to search next, how to access the evidence, whether separate clues support one another, and when it has enough information to produce a unique factual answer.

A paper titled Deep Search in Real-World Web Environments proposes OpenDS and the OpenDS-World benchmark to examine this gap.

Key points

• The setting is live and open-ended. OpenDS moves beyond static retrieval. Agents must use Web search, browser interaction, and evidence-processing tools while exploring the real Web.
• The benchmark combines heterogeneous challenges. OpenDS-World contains 200 tasks spanning different information modalities, file formats, and interactive access requirements. Independent human verification was used to check evidence validity and answer uniqueness.
• Strong models are still inconsistent. Twelve frontier models were evaluated under a unified tool environment and execution protocol. The best model reached 39.0% accuracy, showing that language competence and tool access do not automatically translate into successful end-to-end research.
• The human comparison is difficult but revealing. Human evaluation across 1,000 independent attempts achieved 21.1% accuracy. This should not be read as a simple ranking of humans against models, since their workflows and tool familiarity may differ. It does, however, underline the difficulty of the tasks.
• The bottleneck is broader than retrieval. Analyses of tool use, search cost, termination behavior, and failure recovery found substantial differences in research strategies. Coordinating tools and managing long search trajectories remain persistent problems.

Why it matters

OpenDS broadens the definition of deep search. Instead of checking only whether a final answer is correct, it evaluates the chain of decisions that leads to that answer. An agent may need to widen a query, follow a promising source, inspect a nonstandard file, connect evidence from separate websites, or backtrack after reaching a dead end. A failure at any point can lead to premature termination, redundant browsing, or unsupported conclusions.

The benchmark therefore points to several practical research priorities: better task decomposition, stronger search planning, reliable evidence-chain management, more precise browser control, and explicit recovery strategies. It also makes uncertainty and provenance more important. A useful agent should not merely state an answer; it should show which evidence supports it and recognize when the evidence is insufficient.

For users, the message is straightforward: current deep-search systems are promising research assistants, but they are not yet autonomous investigators that can be trusted without supervision. Progress will require agents that manage the research process as carefully as they manage language generation.

Source: Hugging Face Daily Papers