PhyRubric Turns Physical Common Sense in Text-to-Video into a Diagnostic Test

Introduction

Text-to-video generation is no longer judged only by whether a scene looks sharp or whether motion appears smooth. If these systems are to function as world simulators, they must also produce events that respect basic physical commonsense. Existing evaluations often ask annotators to assign one overall score based on intuition. Such a score may summarize a reaction, but it does not reliably explain which physical rule was broken or where the failure occurred.

PhyRubric proposes a more diagnostic alternative. Rather than evaluating a video against a vague global impression, it first derives explicit semantic and physical requirements from the text prompt, and then checks those requirements against the generated result.

The central design: write the rubric first

The framework creates a caption-specific rubric before the video is viewed. This ordering matters: the evaluator is less likely to adjust its expectations after seeing what the model happened to generate. The rubric covers both whether the requested entities and actions are present and whether the situation obeys relevant physical principles.

Each rubric item receives an individual verdict supported by evidence linked to a timestamp. The final score is derived from these item-level judgments rather than from a single intuitive rating. For model developers, that output is substantially more actionable. It can identify the failed requirement and the part of the video where the violation is visible.

The authors instantiate the approach in PhyRubric-Bench, which spans 34 categories across five physical domains. They also introduce PhyRubric-Train, a collection of 3,788 videos used to fine-tune the open-source PhyRubric-Judge.

Findings: the difficult cases are not limited to motion

The study evaluates ten text-to-video models and concludes that physical commonsense remains far from solved. Heat and acoustics are the hardest domains for most of the evaluated systems.

Acoustics is tested without relying on an audible soundtrack. Because generated videos are silent, the benchmark looks for visible consequences of sound—for example, a sheet of paper responding to a speaker or an alarm clock visibly shivering as it rings. Heat presents a similar challenge: the model must depict consequences of temperature, not merely place objects in a plausible-looking setting.

According to the supplied material, PhyRubric-Judge correlates more closely with human judgments than existing open-source judges and matches the reported performance of the frontier proprietary Gemini 4 Argon. The available information does not provide domain-by-domain correlation figures, so it would be premature to claim that agreement with humans is specifically lower for heat or acoustics.

Why it matters

PhyRubric is more than another leaderboard score. Timestamped, item-level evidence can help researchers distinguish failures in motion generation, scene understanding, causal consistency, or other parts of the pipeline. Caption-specific rubrics also make the evaluation more sensitive to what a particular prompt actually requires than a universal checklist would be.

The framework still depends on a vision-language evaluator interpreting evidence correctly, and the supplied abstract does not describe every detail of rubric construction or human annotation. Even so, it points to a meaningful shift in evaluation priorities. For systems presented as world models, asking whether a video looks convincing is not enough. Benchmarks must also determine whether visible events follow testable physical rules—and provide enough evidence to guide the next model iteration.

Source: Hugging Face Daily Papers