The Physics Exam for Video World Models

Introduction

Video world models are increasingly expected to do more than produce realistic-looking clips. They are being considered as tools for predicting what happens next and for supporting planning in embodied AI. That makes physical consistency essential. A generated sequence may have convincing textures, lighting, and motion while still violating basic rules of collision, fluid behavior, heat, or material change. Such failures are not merely cosmetic: they can undermine the reliability of a model used to anticipate actions in the world.

A benchmark built around measurable relationships

The paper introduces World Models' Last Exam in Physics, a benchmark designed to turn physical plausibility into a more interpretable measurement problem. It contains 40 controlled tasks covering:

• mechanics;
• optics;
• fluids;
• thermal and phase-change phenomena;
• electromagnetism; and
• surface tension.

Each task pairs an initial image and a generation prompt with predefined physical criteria. The evaluator does not require a model to reproduce a reference video frame by frame. Instead, it checks whether the generated sequence exhibits a specified observable relationship. This makes it possible to compare different visual realizations against the same physical requirement.

From visual impression to physical measurement

The evaluation pipeline first screens whether the relevant phenomenon is observable in the generated clip. This matters because a video in which an object disappears or becomes unclear should not necessarily receive a confident physical judgment. If the task is observable, the evaluator applies measurements tailored to that task and its expected relationship.

The authors tested eight video generation models on 1,280 videos. The results point to two persistent issues: physical inconsistencies remain common, and performance varies substantially from one task to another. The best model achieved an overall score of 57.76 out of 100. In other words, stronger visual generation does not automatically produce a model that behaves consistently across multiple areas of physics.

The study also evaluates the measurement module on synthetic videos with known physical relationships, providing evidence that it can work under controlled conditions. In comparisons with a direct vision-language-model baseline, the proposed evaluator showed higher agreement with human judgments for both within-task rankings and pairwise comparisons.

Why this matters

The main contribution is not simply another leaderboard. By tying scores to observable evidence, the benchmark can help diagnose where a model fails: a system may perform well on motion but poorly on fluids, or produce plausible appearance while missing a thermal relationship. This is more actionable than a single visual-quality score for researchers working on simulation, robotics, and action planning.

The benchmark should still be interpreted carefully. It measures selected observable relationships, not a complete understanding of physics. Visibility, task design, and the limitations of each measurement can all affect the result. The next step is likely to combine such tests with action-conditioned generation, long-horizon prediction, and interaction in real or simulated environments. The central challenge remains clear: world models must move from generating scenes that look right to predicting outcomes that are physically right.

Source: Hugging Face Daily Papers