Code World Model Turns a Coding Agent into the Brain of a Simulated World

Introduction

Video world models generally learn how environments change from sequences of visual observations. This provides evidence about outcomes, but not necessarily the rules, knowledge, or mechanisms that produced them. A pixel sequence may show that an object moved without preserving why it moved or how the event should affect later states. Code World Model (CWM) proposes to separate these responsibilities: code governs the evolution of the world, while a video model produces its visual appearance.

Core ideas

• A coding agent acts as the world brain. A language model reasons over actions and events, then generates executable code to maintain world state and apply consequences. Objects, attributes, and event effects can therefore be updated through explicit state and rules rather than inferred only from the next frame.
• Dynamics and rendering are decoupled. CWM does not ask one video model to both reason about long-term logic and render high-fidelity footage. The coding side first determines how the world changes; the visual side then realizes the resulting state.
• A proxy representation bridges the two sides. The proposed proxy representation describes frame-wise spatiotemporal constraints, including information about entities and their movement. These specifications are compiled into a proxy video that conditions a video model to generate visual observations with richer appearance and dynamics.
• Training depends on aligned data. The authors develop pipelines for creating proxy–observation pairs from gameplay and real-world videos. After fine-tuning on paired gameplay data, MiniMax-H3 follows proxy-based specifications from simple interactive worlds built by the coding agent while preserving detailed visual output.

Why it matters

CWM’s main contribution is an architectural division of labor. Executable code provides an explicit and potentially inspectable mechanism for persistent state updates, while generative video models provide flexible visual realization. For interactive environments that require long-horizon consequences, consistent rules, and open-ended evolution, this separation offers a clearer control interface than relying on visual prediction alone. It may also make it easier to alter a rule or inspect how a state changed.

The current evidence is centered on simple interactive worlds and paired gameplay data. It remains unclear how well the proxy can represent more complex physical mechanisms, social rules, or real-world settings, and whether executable updates and generated observations will remain aligned as complexity grows. Still, CWM presents a useful direction: combining programmatic world state with generative visual observation rather than treating pixels as the entire world model.

Source: Hugging Face Daily Papers