HyperBrowseComp Turns Web Research into a Multilingual, Multimodal Stress Test

Introduction

Many web question-answering benchmarks reward a model for locating a relatively explicit fact in a search result. Real-world research is less tidy. Relevant clues may be scattered across websites in different languages, videos, scanned documents, images, or maps. The final answer may be short, but reaching it can require a long chain of discovery and verification. HyperBrowseComp, introduced by researchers at MBZUAI, is designed to test precisely this kind of work.

Key points

• Broad language coverage: The benchmark contains 423 manually authored and human-validated questions in 13 languages. The questions were written by native speakers or highly proficient users.
• Open-web evidence discovery: Each question has a concise, publicly verifiable answer, but finding it may require locating obscure evidence and linking several clues across sources.
• Heterogeneous evidence: Some tasks require examining videos, scanned documents, images, or maps rather than relying on text-only web pages.
• Reduced reliance on memorization: The authors use models without internet access to filter out easier questions, lowering the chance that a task can be solved from parametric knowledge alone.
• Comparable evaluation settings: Models are tested with provider-native search and with a shared external retrieval harness under a common agent protocol. A sample is also evaluated by humans to put model performance and effort into context.

Why it matters

HyperBrowseComp frames browsing as a combination of capabilities rather than a single search action. An agent must understand the question, formulate a useful search path, navigate multiple languages, interpret different media, assess evidence quality, and maintain a coherent chain of reasoning until it reaches a verifiable answer. A successful keyword match is therefore not enough; the system must sustain information seeking while handling incomplete, noisy, or conflicting sources.

The benchmark also suggests that future agent evaluations should look beyond answer accuracy. Search calls, exploration paths, evidence coverage, and the effort required by human researchers can all help describe how a system arrives at an answer. The supplied material does not include model score tables, so HyperBrowseComp should currently be understood as a benchmark and research direction, not as a performance ranking.

Source: Hugging Face Daily Papers