Open-o3-Video-v2 Turns Video Reasoning into Active Evidence Search

Introduction

Reliable video reasoning requires more than producing a plausible answer. A model should also be able to show which moments and objects support its conclusion. Many video systems begin with a sparse, fixed set of frames. This is efficient, but it creates a blind spot: if an important action occurs between sampled frames, the model has no direct evidence to inspect, regardless of how strong its language reasoning may be.

Open-o3-Video-v2, introduced by a Peking University team, reframes the task as an active and evidence-grounded process. The model can identify missing information, request additional observations, localize relevant moments and objects, and then continue reasoning with the new evidence. The project releases its code and models publicly.

Key points

• Active observation instead of fixed sampling. The model first assesses whether the available visual information is sufficient. When a clue is missing, it seeks additional observations rather than relying exclusively on the initial sparse sampler.
• Evidence is written into the reasoning trajectory. Timestamps indicate when an event occurs, while object-level bounding boxes identify the visual regions that support a claim. This creates a clearer link between a final answer and the underlying video.
• Training data targets multi-turn behavior. STGR-v2-CoT135K contains 52,000 multi-turn agentic grounded reasoning trajectories. STGR-v2-RL-13K is used to support reinforcement learning.
• Retrieval efficiency becomes part of optimization. The proposed Clue-Seek GRPO tracks how tool calls change accumulated evidence. It rewards clue coverage and step-wise gains, while penalizing excessive, repetitive or uninformative retrieval.
• Reported gains extend beyond one benchmark. On V-STaR, Open-o3-Video-v2-4B obtains 40.4% mAM and 56.3% mLGM. Compared with its Qwen3-VL-4B backbone, the reported improvements are 8.5 and 12.6 percentage points, respectively. The paper also reports consistent gains on general video understanding, long-video reasoning and temporal grounding benchmarks.

Why it matters

The main contribution is a closed loop: identify an evidence gap, acquire a clue, ground the supporting content, and then continue reasoning. This is a meaningful shift from one-shot interpretation of preselected frames toward a video agent that decides what it needs to inspect next. Such behavior is particularly relevant to long videos, fine-grained actions and questions that require comparing events across time.

Grounding also improves inspectability. A timestamp answers when the supporting event occurs, while a bounding box indicates which object or region matters. Together, they can make it easier to check whether the conclusion is actually supported, rather than being generated mainly from linguistic priors. The trade-off is that active retrieval introduces more tool calls and potentially higher inference cost. The usefulness of each additional observation therefore becomes an important systems issue.

Open-o3-Video-v2 presents active evidence acquisition and object-level spatio-temporal grounding as complementary capabilities. Its broader significance lies not only in the reported benchmark gains, but also in offering a training design for video agents that need to form, preserve and use verifiable evidence.

Source: Hugging Face Daily Papers