From One Image to an Executable World: RCWM Rebuilds 3D Scenes Recursively

Introduction

Turning a single image into a navigable, editable, and potentially interactive 3D world requires more than recognizing objects. A system must also preserve scale, occlusion, boundaries, camera geometry, and the spatial relationships that connect individual elements into a coherent scene. Recursive Code World Models (RCWM) addresses this construction problem by representing the reconstructed world as executable code and building it through repeated recursive refinement.

The central idea: global, local, then global again

RCWM has two closely related components. The first is the Recursive Scene Program (RSP), a compositional representation of the world in scene code. Instead of producing only a final static reconstruction, RSP keeps the structure of the scene explicit and editable. Its hierarchy also gives the system a place to continue reasoning when a coarse reconstruction is not sufficient.

The second component is a construction solver that recursively invokes the same procedure. Each call follows a three-stage pattern:

• Establish the whole: set up the broad layout, camera projection, and major spatial relationships;
• Reconstruct unresolved parts: descend into regions or objects that still need work, giving them their own perception and code-editing loop;
• Revisit the parent: bring the refined part back into the larger scene and correct boundaries, relative placement, and errors shared across components.

This is more than a longer sequence of edits. The global-local-global structure is intended to prevent local improvements from drifting away from the scene as a whole. Reference-aligned views propagate a shared camera projection across recursion levels, so local modules remain tied to the same visual evidence rather than independently inventing their viewing geometry.

The role of the vision-language coding agent

A vision-language coding agent compares the reference image with rendered views of the current scene. That comparison guides whether the agent should modify code, descend into a smaller subproblem, or return to a higher-level scene. The agent therefore participates throughout construction instead of generating a program only once at the beginning.

Parent revisitation is particularly important because some problems become visible only after local refinement. Objects may meet incorrectly, relative positions may shift, or several parts may inherit a common error. Returning to the parent scene gives the solver an opportunity to revise the composition after those effects emerge.

Why it matters

RCWM presents world modeling as recursive program construction rather than one-shot scene generation. For applications that require editable assets, simulation, or later interaction, an executable and hierarchical representation can offer more flexibility than a purely static reconstruction.

According to the supplied material, RCWM outperforms previous code-based image-to-scene reconstruction approaches on complex scenes. Ablation studies also support the value of recursive construction and indicate that deeper calls may help recover finer structures. However, the available material does not include numerical results, benchmark details, or compute costs. The contribution is therefore best viewed as a supported construction principle, not as evidence that single-image 3D reconstruction has become a solved general problem.

Source: Hugging Face Daily Papers