ForgeWM Turns Action-Conditioned Video Models into Few-Step World Models

Introduction

An interactive video world model must do more than produce plausible frames. It must respond quickly to player input and keep later frames consistent with the actions and state that came before. This is particularly difficult in games, where keyboard controls are discrete while mouse movement is continuous. Once a video model compresses several frames into a latent chunk, the control signal and the visual transition can easily become misaligned. ForgeWM addresses this problem with a progressive recipe for training few-step causal world models.

The core training recipe

ForgeWM starts from a bidirectional action-conditioned video generator. Rather than distilling it directly into a single-step system, the method converts it in stages:

• Domain adaptation makes the generator better suited to interactive game videos and their control signals.
• Teacher-forced causal training exposes the model to real historical states while teaching it to operate in an autoregressive manner.
• Causal consistency distillation encourages the student to preserve the teacher’s predictions across compressed temporal states.
• On-policy distribution matching uses the bidirectional teacher to reduce the gap between training states and the states encountered during rollout.

The result is a set of budget-specialized students operating with one, two, or four denoising steps. Specializing models for different budgets gives deployment a clearer latency-quality trade-off instead of asking one model to serve every operating condition.

A two-path deployment design

ForgeWM also separates urgent interaction from later visual refinement. During gameplay, the one-step student produces a draft with minimal latency, prioritizing responsiveness. For replay or post-processing, that saved draft can be re-noised and refined. This is different from regenerating a new result from pure noise: refinement starts from the trajectory already experienced by the player.

According to the paper, replay-time refinement reaches the quality of the four-step reference while staying roughly three times closer to the experienced trajectory than regeneration from noise. The design therefore treats a fast prediction not as a disposable approximation, but as a useful intermediate that can be improved when latency is no longer the main constraint.

Results and implications

On paired Minecraft trajectories, ForgeWM leads the evaluated systems across the reported image-quality, reference-aligned motion-profile, action-sign, and mouse-control measures, and obtains the lowest reference LPIPS. The same four-stage recipe is also transferred to a gamepad-controlled FPS setting, suggesting that the approach is not limited to keyboard-and-mouse interfaces.

The broader contribution is a practical connection between few-step video distillation and interactive world modeling. Speed, control alignment, and autoregressive stability are handled as parts of one training pipeline rather than as independent fixes. ForgeWM does not eliminate the broader challenges of long-horizon world simulation, but its open and reproducible implementation, together with keyboard, mouse, and gamepad support, offers a concrete route toward playable real-time video world models.

Source: Hugging Face Daily Papers