H3-World Turns a Video Generator into an Interactive World Model

Introduction

Video generators are beginning to move beyond one-shot prompt completion. A more ambitious direction is to make them respond continuously to instructions, behaving less like passive renderers and more like interactive worlds. H3-World explores this direction by turning the language understanding already present in MiniMax-H3 into a control interface.

The key idea is not to build a keyboard-action encoder from scratch. Instead, keyboard-style controls are expressed as language that the pretrained video model can already process. This lets the system reuse semantic capabilities acquired during large-scale video pretraining.

Making language control temporally precise

MiniMax-H3 can already respond to natural-language descriptions of character behavior and camera motion. However, a general prompt does not necessarily specify when an action should begin or end. When several actions occur in sequence, one instruction can affect the wrong part of a clip or bleed into another action.

H3-World addresses this problem by representing an action as a structured combination of character and camera instructions. Each instruction is then aligned with the corresponding temporal video latent interval. The framework also introduces temporal attention routing, which restricts an instruction’s influence to its intended time span. This provides a more explicit link between what should happen and when it should happen.

Core design points

• Language-native control: Keyboard actions are rewritten as textual instructions and injected through MiniMax-H3’s pretrained text pathway.
• Joint character and camera control: An action can specify both the behavior of the subject and the movement of the camera observing it.
• Temporal grounding: Different instructions are assigned to different latent intervals, reducing cross-action control leakage.
• Lightweight adaptation: Training uses 8,000 gameplay samples and 10,000 LoRA optimization steps, with only 0.199% of parameters trainable.
• Generalization: The reported system handles unseen action compositions and visual scenarios while retaining strong generation quality.

Why it matters

H3-World suggests a practical route toward video-based world models. Rather than attaching a complete action-learning stack to a generative model, researchers can exploit the model’s existing language and visual semantics, then use a small adaptation layer to connect them to interaction. This may reduce training requirements and make language a flexible interface for composing new behaviors.

The work should nevertheless be interpreted within the evidence provided. The supplied summary does not include the full benchmark metrics, interaction latency, or tests of long-horizon state consistency. H3-World therefore represents an important proof of how emergent language control in video generators can be organized into interactive behavior, not a complete solution to real-time world simulation.

Hugging Face Daily Papers