Orbia Tests Whether Generated Videos Build a Consistent 3D World

Introduction

Video generators are increasingly being used as interactive environments rather than simple tools for producing attractive clips. A user may explore a scene through text, actions, or camera movement, but realistic individual frames do not guarantee a stable 3D world underneath. After the camera moves away and returns, objects may change shape, shift position, or disappear. The model may even fail to preserve content that it created during the exploration itself.

Orbia is designed to examine this problem systematically. Instead of asking a vision-language model whether two frames “look consistent,” it uses controlled exploration and revisit procedures to measure appearance, geometry, and spatial relationships in a more explicit way.

Key points

• Input preservation: Registered RGB-D references and annotated spatial anchors are used to test whether a model retains the appearance, geometry, and relative placement of elements from the original scene.
• Generated-content persistence: Some regions are absent from the initial input and become visible only as the model generates them during exploration. Orbia later revisits those regions and compares them with their first observations, testing whether the model preserves what it created.
• 3D self-consistency: The benchmark measures cross-view depth and feature correspondence, agreement in regional scale, and rendering quality from held-out Gaussian representations. Together, these tests ask whether different views actually describe the same 3D scene.
• Long-term stability: In addition to short camera changes, Orbia includes minute-long exploration and revisit settings intended to reveal accumulated spatial-memory errors and structural drift.

The construction pipeline combines public datasets, self-captured scenes, and simulation. It contains 720 instances across indoor and outdoor environments and defines six exploration-and-revisit protocols. Native-interface adapters extend the evaluation to 27 models conditioned on text, actions, or cameras. The supplied material emphasizes the benchmark and its methodology rather than detailed model rankings, so Orbia is best understood as a diagnostic framework instead of a simple leaderboard.

Why it matters

The main contribution of Orbia is to turn the vague idea of “temporal consistency” into a set of observable 3D questions. A useful world model for robotics, embodied AI, virtual interaction, or simulation must do more than predict a plausible next frame. It must keep track of where objects are, how regions connect, and whether an established world state can still be accessed later.

The benchmark also separates visual quality from world-modeling ability. Better textures and more realistic frames do not automatically produce reliable spatial memory. Future systems may need to jointly improve appearance generation, geometric representation, long-term memory, and viewpoint consistency. By distinguishing preservation of the input, persistence of generated content, and 3D self-consistency, Orbia provides a clearer way to diagnose failures and a foundation for developing more persistent interactive video worlds.

Source: Hugging Face Daily Papers